At Lyft, Frank Yoo and Vicki Tan have been running research sprints to cultivate a user-centric design approach. They’ve been using Lookback to provide greater accountability for the process, justify their design decisions and create visibility for user insights across the organization.

Closing the qualitative gap

Until recently, Lyft’s product design team, led by Frank Yoo, followed a scrappy, light-weight design process: design and develop a new feature, validate internally, release, then analyze and iterate. Like many app developers, Lyft is metrics-driven and focuses on quantitative analysis. As product designer Vicki Tan explains, “We use A/B testing and analytics, but now it’s a priority for us to cultivate a user research capability to help us inform product decisions and dial-in usability.”

The problem with hiring user researchers is, well, it’s hard. The concept of user experience research has existed at least since Donald Norman’s work at Apple during the mid-1990s. However, as an industry, it is still developing, and given an increased focus on user experience as the key to gaining competitive advantage in the app market, UX researchers are in demand. As Vicki reflects, “We took the research on ourselves in the meantime, because we couldn’t hold off on the insights. I talked it over with Frank and we decided that I'd take the lead on a month-long user research pilot.”

Getting started with user research

Adding to her role from full-time designer to part-time user researcher, Vicki took on the responsibility of setting up user testing and, in turn, reshaping the team’s design process. The team decided to adopt the 4-Day Research Sprint model from Google Ventures, tailoring it to suit their needs, as well as creating templates for repeatable research sprints.

The team makes heavy use of interactive prototyping tools. The idea was to prototype new features and then bring users in to test them before committing to development effort. One of the initial challenges was ensuring that the findings themselves would be verifiable both to the team itself and to other stakeholders. Unlike quantitative data, where the numbers and graphs can be readily reproduced and shared with others in an email, the outcomes of qualitative research as notes taken based on observation could be open to interpretation. So it was critical that the process would be accountable.

Creating accountable user tests

This “lack of accountability” problem is by no means exclusive to Lyft. Many companies try to address it by setting up elaborate filming equipment to record user interviews. However, this approach seemed cumbersome to Vicki and, worse, counter-productive. She recalls a time when she participated in a user study, “where there was a camera, a voice recorder, and a laptop recording and [it was] nerve wracking”.

That’s when Frank came to her with Lookback for iOS Devices, a screen-recording app that lets you record the device screen, where the user touches, audio, as well as the user’s face via the device’s front-facing camera. Lookback allowed for a dead-simple and, importantly, non-invasive way to record user interviews. And using just the device itself is less intimidating to the participants, so they're more candid and natural.

As for formalising the user testing process, Vicki explains, “What’s cool about Lookback is that it makes the results more tangible because everything is recorded and team collaborators can go back and observe the sessions themselves. Lookback gave us the confidence to say to our stakeholders ‘we're doing user research now, legitimately.’”

One of the marked shifts in being able to review the user tests afterwards, in a repeatable and objective manner, is that it shifts the attention away from the app itself to the user. “I actually haven’t been looking at the screens as much as the people’s faces afterwards. I thought I’d be looking more at their taps than what they were saying, but it turned out that looking at their reactions and expressions were more important to me.” This reflects the growing realisation amongst the tech community that shiny apps don't matter, what matters is making your user badass.

Justifying designs decisions and promoting visibility

As Director of Product Design, Frank sees an even greater impact that Lookback is having within his organization. “Using Lookback has really impacted design’s sphere of influence here at Lyft.” He says, “We’re working in cross-functional teams, and rationalizing the decisions that we make to our executives has been really helpful. As company sizes grow, product design teams must find clear ways to communicate their decisions to key stakeholders and demonstrate the link between effective user experience research and better product design. Basically, it gives us another tool to support the work we’re doing. We can show them that, you know, these decisions are really supported by qualitative insights and that a lot of strategy goes into our work. I want people to understand that we’re really thinking things through before we execute.”

Looking forward

The Lyft team is understandably excited about Lookback’s new product offerings, including Android and Mac support. “Lookback 1.0 is huge,” says Frank, "Lookback for Mac now gives us the ability to gain qualitative insights on emails. We rely on emails to communicate with our drivers and passengers, and we’re excited to do more research on the medium to ensure we’re providing the best possible experience for our users.” Also, given the restrictive nature of many app development environments, the desktop app is a huge boon. “What’s also great about Lookback for Mac is that we don’t have to get an SDK installed to test our mobile apps.”

More generally speaking, the team is looking to steer product direction using qualitative research, rather than decide on individual features or simply validate decisions. “What we want, and what Frank has envisioned, is for us to use all this research to position ourselves to be informed experts.” Vicki suggests, “Rather than being reactive and confirming ‘this is better than that’ or asking ‘is this intuitive to users?’, we’re looking to take a step back and use Lookback to aggregate a database of experiences where we can pull a montage of user feedback to better understand their needs. Once we nail the fundamentals, we can then fine-tune the details.”

Looking back

So how has user testing changed the design process for Lyft? “I think it gets you out of your own head.” Vicki says, “We are, by default, power users and we're also designers who look at every little detail. Sitting and talking with real users for a full day who are saying things like ‘I don't understand the thing you just spent hours working on.’ It's a good thing. It's humbling and it just pulls you out of your bubble.”

“Lookback has allowed us to bootstrap our research process to get real insights to act on. It’s been our main tool to kick-off and start the process of user research.”

Key Takeaways

  • User research sprints reduce the time it takes to receive feedback for new product features, improving the time-to-market for useful, usable features
  • Because the hiring process takes time, a lightweight process that leverages the right tools can help generate useful, actionable insights in the interim
  • Lookback enables teams to conduct and record user tests in a simple and accountable manner
  • Lookback’s web platform is a powerful tool that allows the team to justify design decisions and drive product direction

Further Reading

Special thanks to Mo Alcaraz for the awesome photos!