Save Money & Offer a Better Experience — It Can Be Done


Everyone has experienced the tension between Finance and Product: saving money is often in conflict with providing customers with a better experience. But better customer service is not a zero-sum game. At Lyft, we found a way to reduce our costs AND provide a measurably improved customer service experience by enhancing our automation in support ticket handling.

Support teams often come up with ingenious workflows using a very limited toolkit; they want to provide customers with the best experience at all cost. However these workflows can be error prone and inefficient because the right tools don’t exist. Resourcing engineering effort for these workflows can be an uphill battle. Organizationally, there can be a perception that support is an afterthought of the user experience; implicitly this mentality can be summarized by “if we make the perfect product then people won’t need support.”

Imagine you overcome that hurdle. What happens when you take a product engineering team and have them focus on support tooling? A lot, actually.

At Lyft, the biggest win in this regard has been the introduction of automating responses to easily solved tickets by intelligently (and automatically) understanding the context of the user and their support request. When a user contacts support, the system can give an immediate response and save the user from having to contact a support agent, leading to a better experience. Not only is this great for the user but it also save the company money. In Lyft’s case, this approach has already saved tens of millions of dollars per year.

The automated system (blue) drastically reduced the number of tickets submitted by giving users an immediate response.

How Did We Do It?

Keep in mind that these are guidelines, not strict rules. Each attempt at automating a support interaction should be deeply understood, there is no one-size-fits-all solution.

1. Maintain Data Visibility

Track anything that may help understand the context of the interaction; don’t simply surrender everything to your 3rd party support tools. Support ticket attributes (such as contact reason, contact resolution) can indicate why a user may be seeking support, and provide critical context for exploring and understanding support interactions.

At Lyft, this information resides in warehoused views for complex queries and live views for system performance monitoring. Easily queryable data makes it simple to answer business questions and build predictive models for proposed changes to support interactions. Without data, you can’t answer the questions.

Consider the hypothetical dashboard below, which helps us answer two questions: 1) How many support tickets come in about cancellation fees? and 2) How do agents resolve those tickets?

2. Conduct Support Agent Research

Digging through historical ticket data can unearth some information about ticket resolution, but only by talking to the actual people handling the support tickets can we understand the complete picture.

The ultimate goal is to understand the business logic and workflows used by support agents well enough to programmatically replicate resolutions with high accuracy. We spent many weeks talking to agents and working with the customer support team to document how tickets are resolved.

When researching with agents, we would pick a subset of ticket types and then observe as they solved the tickets, asking them to talk through how they determined the resolution. Our aim was to understand all the data used to resolve each ticket.

Watching successful support resolutions in person can provide a lot of input data into product development.

After these research sessions, we were able to develop a workflow that was comprised of a series of steps. Broken down into these steps, it’s easier to develop a programmatic solution. This step breakdown should be done with the involvement of the support team, to make sure the workflow is documented correctly.

Using data from our ticket resolution dashboard above, let’s look at the resolution ‘inform user of coupon info.’ Given all we know from the dashboard is that the user submitted a ticket about a cancellation fee and the resolution was ‘inform user of coupon info,’ here is a breakdown of the steps an agent took to resolve this ticket.

When understood as a set of steps, it becomes much easier to provide a programmatic solution.

3. Leverage Your Own Data To Automate

After learning the underlying human workflow for resolving a ticket and documenting the data associated with each step, the resolution steps can be programmatically automated. Prior to introducing automation, we were sending the support request directly to the 3rd party API.

After introducing automation, we immediately return a response to the client if we can provide an automatic resolution before handing off the support request to the 3rd party API.

Creating this intermediate layer provides two benefits: 1) faster response to the client because it runs before invoking an external API request and 2) lower chance of failure due to syncing 3rd party data after the initial user interaction.

The automation code is specific to the reason a user may be contacting support, however, all support tickets follow the same general pattern. Continuing our example about a cancellation fee ticket resolved by informing the user of the coupon details, the code (in Python) would look something like the below.

Now we have code that replicates the steps from our previous research in a simple, repeatable way. The user receives an immediate support response and agent resources can be focused on more complex problems.

4. Validate Human Outcomes

It is not enough to simply finish a project with the goal of providing a better experience. The customer’s experience needs to be measurably better. This is doubly important for support features, compared to any other product feature. If the user is contacting support they are undoubtedly experiencing a pain point with your product.

The danger of automating any kind of interaction that was once handled by a human is that it could end up feeling (even more) frustrating for the end user. The underlying emotion behind this frustration is a sense of not being understood and not having needs met. The pain of this emotion is costly. At a human level, the brain is wired to easily recall painful experiences and it requires effort to overcome that association. At an economic level, it is expensive to win back an angry customer.

This dilemma of automated technology having a negative impact on human experience is not just science fiction, like cultural echoes of HAL refusing to open the pod bay doors in 2001: A Space Odyssey. Examples of this dilemma fill our world. Everyone has been through a frustrating automated phone tree with no escape (e.g. “press 1 to …”) and no chance of solving your issue. We have seen automated voice/chat assistants go haywire (like Microsoft’s Tay in 2016) or generally misunderstand our needs (like the endless memes of Siri misinterpretations).

These anecdotes of automation mishaps are consistently absent from dashboards, automated reports, or data queries, which is why it’s important to qualitatively validate the improvement. Ignoring qualitative data and focusing only on quantitative data can provide a false sense of success. The best solution is to draw qualitative insights from the quantitative data. Here are some popular ways to do this:

  1. Beta program — Get feedback from a highly engaged audience that cares deeply about your product. Create multiple groups to control audience size and gather more granular feedback. For example: internal beta, small external beta, public beta, etc.
  2. User research — Watch end users go through the experience of using your product.
  3. Experiments that includes free text input — This lets you test on a small subset of the population and gather unstructured, unfiltered user feedback.
  4. Human QA team — Just as DHH (David) from Basecamp discusses it is immensely valuable to have a human QA team, not just rely on automated testing.

Ignoring qualitative data can impact the product further down the road in many ways: loss of customers, decreased lifetime value, and high spending on churned users. Regardless of how you gather qualitative data, it acts as a final gut check to ensure that your product changes are creating a better experience for your users.

Interested in having a big impact by building external and internal support tools? Lyft is hiring! Drop me a note or just say hi at sbolton@lyft.com.



Source link