Netflix is renowned for rigorously A/B testing every product change before it becomes the default experience for its massive, diverse user base. The technique that compares two or more versions of a design to find out which one performs better helps the company to improve its service and increase engagement.
While data plays a major role in driving decisions, it’s not enough to rely on it alone. As a product design lead at Netflix, Ghaida Zahran has learned to stay curious. Data is a critical part of her decision-making process, but she tries to approach it with humility and an open mind. Ghaida therefore describes Netflix Design not so much as data-driven but as data-informed.
“Data is a powerful tool,” she explains, “It gives us an answer as to whether or not our designs, features, and execution of those features have resulted in the outcome that we would like – but it’s just one part of the picture. There are other aspects beyond the actual A/B test result that factor into our decision-making. We may not productize something that doesn’t align with our own goals and our long-term vision as a design team. We let our curiosity, and not just the data, lead the way.”
Developing a holistic understanding of the customer journey
With over 12 years of experience in design, Ghaida currently leads design for Netflix’s bundle products in the Partnerships and Payments team, which is responsible for how Netflix is presented in multiple contexts, for example on game consoles and smart TVs. She works on the customer journey from beginning to end and ensures the user experience is consistent across different touchpoints, communicating the value of Netflix to potential new subscribers.
“I focus mostly on growth,” Ghaida says. “This includes figuring out how to help customers understand the benefits of Netflix enough to want to take the plunge and subscribe either directly through us or through a partner. It’s not just about a specific platform or digital product alone. In a single day, I could be having a discussion about the retail experience, and an hour later I could be exploring the UX flow of our digital product or the talking points that call center reps need to mention to customers to help them learn about Netflix. My team thinks about the UX holistically, rather than specifically through the lens of the digital experience.”
"Data is a powerful tool. It gives us an answer as to whether or not our designs, features, and execution of those features have resulted in the outcome that we would like – but it’s just one part of the picture."
Good friction for a better UX
Where A/B testing comes in is in the assessment of particular user flows. One interesting project Ghaida was part of was to experiment with signup flows.
“Whenever there is a signup flow within any company, there’s always talk about friction,” Ghaida points out. “How do you remove friction, and how do you remove barriers from the customer? We conducted an experiment that intentionally added friction, so that we can inform the user of certain features and benefits of the service before they complete the signup process. We didn’t think it would necessarily work, but the team approached it with a sense of curiosity, rather than just wanting something to win. The outcome was pretty surprising. It resulted in this concept of ‘good friction’ in the team and a better understanding of the customer.”
The team learned that signup flows were not just about removing as many steps as possible, but really making sure that only the superfluous ones were being taken out. Keeping some steps, or adding friction where it’s needed, means that the customer’s understanding of the service could be improved.
It’s this sense of approaching design with curiosity and humility, rather than from a solution-oriented mindset, that’s a common thread in Ghaida’s career. It’s also the topic of her upcoming talk at Scandinavia's premier user experience and service design conference, From Business to Buttons.
“The presentation is also about inspiring teams to work together towards an answer and not getting attached to outcomes and solutions,” Ghaida explains. “It comes from the main difference between a product designer and a designer at an agency, which is that a product designer approaches design as a way to ask questions, not to provide an answer.”
For Ghaida, A/B testing is more than just another task to be ticked off her to-do list. Cultivating a culture of questioning and healthy design dissent helps her (and her team) ensure designs are flexible enough to be adapted according to user needs.
“We need to do enough design to be able to ask something and then accept the outcome. The design will either achieve the intended results, or we’ll learn something about the customer. Both are valuable, and I think this shift can be hard. A lot of designers come from agency backgrounds, where the process may not be as iterative or curiosity-driven. They’ve often been trained to design for selling. Convincing [a client that a certain design] is right for them is a big part of being an agency designer, whereas a product designer will design something they’re reasonably confident will achieve the outcome – but can also see a lot of reasons why it wouldn’t.”