Introduction
The evolution of user experience (UX) design stretches further back than most people realise. As early as the late 19th century, automotive pioneers like Ford were embedding basic ‘experience design’ principles into their manufacturing processes to enhance efficiency. By the 20th century, engineering practices advanced and the need to make complex systems instinctive, helpful, and desirable arose—cueing the emergence of the professional designer. Since then, technology capabilities have rocketed, further accelerated by AI.
While experience design originated in the tech industry, its principles still apply to all. As we chart ahead, defining value in customer lives is now a pre-requisite as UX theory influences every touchpoint between the physical, digital, and now virtual world. This understanding of value largely comes from qualitative data, which unveils the reasons behind user behaviours—insights that raw numbers alone cannot provide. For instance, while assessing a new fitness program quantitative measures like weight or BMI can show physical changes, but they miss underlying factors. Qualitative research fills this gap, offering a window into personal experiences, motivations, and barriers faced by customers, thus providing a more comprehensive story.
Over the past few years, I’ve seen and heard companies increasingly investing in analytics tools but less in foundational research and experience strategy. UX initiatives are threatened by this imbalance. Ironically, I was inspired to write this article after a meeting earlier this week, where several of my colleagues debated the value of automated personalisation strategies that lack qualitative foundations, reinforcing this very concern.
During the COVID-19 pandemic, this trend of depending wholly on analytics exacerbated cutbacks in market research budgets, particularly affecting qualitative customer studies, and cuts have persisted year-on-year. An IPA Bellwether report found that 15.7% of UK companies contracted research budgets by at least 2.9% in 2023, even in situations where overall marketing budgets were rising. This operational shift raises a crucial question: Why is research being deprioritised when all four quadrants of UX—Experience Strategy, User Research, Information Architecture, and Interaction Design—are considered cardinal for crafting intelligent, optimal, and measurable solutions?
Beyond Numbers: The Case for Deeper Qualitative Insights
In the current digital age, data is king. But when it comes to understanding the subtleties of human interaction with technology, numbers and graphs can fall short. The mainstay of much company research—surveys and behavioural tracking data—provides a wealth of information but often fails to capture the ‘why’ behind the ‘what’. For instance, although analytics may indicate a high dropout rate at checkout, it’s typically qualitative methods like interviews and usability testing that reveal underlying reasons for abandonment could have been, guilt from overindulging, price sensitivity, confusion in the layout or concerns about data security.
Here’s where I recently felt the pinch. I participated in a survey for my favourite vegan sports protein powder, form. Aspects of the survey were so narrowly focused that they completely missed out on why I never venture beyond their core protein range or buy their supplements. A more open-ended approach to particular questions would encourage participants to share their full range of experiences and surface golden insights needed to drive meaningful product improvements.
Don't get me wrong- the questionnaire itself was, on the whole, pretty well designed, but there were a few pitfalls when it came to understanding my consumption habits. When asked "What best describes your eating choices?", I was presented with multi-select options like Vegan, Meat Eater, Flexitarian, Gluten Free, or Other, giving me a way to disclose unlisted dietary preferences like Pescatarian, Paleo or Keto. While knowing how I eat is useful in assisting my segmentation as a user, I wasn't asked why. Asking me what influences my day-to-day dietary choices would have been a good follow-up question to judge my overall consumption motives better.
Later in the survey, I was asked to "Describe how I consume my protein". Despite the range of responses presented, like "I mix it in a shaker bottle, I make elaborate shakes, I use it in porridge, pancakes, or bake with it", there was no further inquiry about what I like to combine my protein powder with. For example, by way of additional ingredients, supplements or milk type- missing the opportunity to gain insight into preferences which they could use to cross-sell me their vitamins, or generate ideas for new powder flavours that might be popular based on common smoothie additions like vegetables, fruits, nuts and different kinds of milk. As a result of this follow-up, other questions could have been asked, including what I’m hoping to achieve by mixing in these ingredients, such as altering or enhancing a powder’s flavour, varying the texture, increasing the nutritional value or my calorific intake.
Lastly, when asked whether I use any of the other protein powders in the range that fall into the meal replacement category, I said no, and they moved me straight to the next product without asking why not. In the event they had asked me, I could have told them that the meal replacement product contains an ingredient I‘m allergic to. Considering the common allergy to oats, gathering this information at scale may have revealed a way to make their meal replacement product more universally accessible and profitable.
Limitations of Current Data Collection Methods: Drawbacks of Surveys
Surveys, polls, and market statistics are helpful methods for gathering customer insights but they often scratch the surface of what we need to know. They’re structured to collect specific types of information efficiently, but this efficiency can also be their downfall. In best-case scenarios, surveys provide an option for further consultation, but few customers, unlike me, will proactively point out their limitations. Most participants won't take the initiative to suggest improvements or provide deeper product consumption details that could prove invaluable. This is why it’s crucial to design surveys that not only gather initial responses but also encourage ways to provide more comprehensive feedback through logic trees reacting to inputs, thus allowing for deeper exploration based on initial responses. Enabling survey participants to opt-in for additional research studies- which to be fair, form did, is also advisable, as is comparing results from different surveys to connect the dots.
Implementing Changes for Greater Impact
Beyond this and other types of surveys like Net Promoter Score (NPS), Customer Effort Score, and ISO usability surveys which play crucial roles in understanding customer experiences, as well as tools like analytics, each method has its own merits: NPS measures loyalty, Customer Effort measures proficiency, ISO surveys provide a standardised method for quality assurance, and analytics capture data in real-time. However, they also come with caveats. An NPS survey can oversimplify customer sentiment, ISO surveys don't dig into user experience, and analytics can't explain why customers do what they do.
Measuring the effectiveness of these customer feedback tools involves looking at the quality and applicability of the insights gathered. Are customer satisfaction scores improving? Are decisions being made faster and more accurately? Qualitative research like interviews paired with quantitative methods can undoubtedly help refine the design of future surveys and data collection efforts to uncover and surface the most applicable insights.
The Importance of Stakeholder Buy-In
While this advice is helpful, securing time and funding for qualitative research remains a challenge for many, and it's easy to get frustrated with managers who don't value qualitative research but instead favour "quick wins" like desk research, competitor benchmarking, or call centre results. But what if our approach to selling research contributes to stakeholder ignorance? Research outcomes are essential—we need something to show at the end of a project. However, sometimes we get stuck in a cycle of churning out reports or other deliverables that aren't effective at demonstrating how well or badly a company is performing against key results.
In my experience as a strategist, this part of communicating insights is key to aligning UX efforts with company goals and getting upper management on board the research train. It's always been difficult to get people to see your point of view, especially when negotiating with numbers-driven stakeholders who may naively overlook how much intel is needed to see the whole picture. But, it helps to remind ourselves that stakeholders may also be reluctant to allocate budgets to what their peers perceive as soft data, because of KPI reporting requirements- which more often than not are sales and conversion-dominated. Avoiding this trap is something I discuss in another article.
What we can do is broaden KPI sets and highlight where qualitative data is crucial for providing more context to numerical data, to help explain why certain trends or patterns occur. From my own experience, this usually helps stakeholders better understand the human elements behind the digits, which makes informed decisions that enhance customer satisfaction and loyalty more likely. Results from pilot projects that demonstrate the value of mixed qualitative and quantitative approaches can also boost buy-in, along with working closely with data science in the early stages of KPI formation.
In Conclusion
The limitations of traditional data collection methods and the importance of stakeholder buy-in make it clear that we need to rethink our approach to UX research. Incorporating qualitative insights makes our user experience strategies deeper, more accurate, and more effective, and we should all challenge ourselves to go beyond numbers to invest in research frameworks that provide richer and more nuanced insights.
How will you enhance your data collection methods to ensure they're truly reflective of customers' needs and experiences? Join the conversation and share your strategies.
Comments