In the fast-paced world of product development, the line between success and failure often hinges on user research — but what if the tried methods we rely on are now letting us down? And could AI offer a solution?

User Research Has Lost Its Edge

With roots in academic research and social sciences, user research was formed long before the digital era — at a time when product development was measured in years, not weeks or days.

Today, the tried user research methods we use, often characterised by long timelines and deep episodic inquiries, are increasingly unable to keep pace with the speed of modern product delivery, with its weekly execution cycles and fail-fast ethos.

Back in 2012, product management veteran Marty Cagan emphasised the need for modern user research to evolve and align more closely with the rapid, iterative nature of today’s product development. He famously advocated for a more nimble approach:“I encourage teams to do continuous product discovery — where we are constantly identifying, validating and describing new product backlog items”​​ [1]

User Research: The Last Horse in a World of Cars

The 16th Annual State of Agile Report states that “95 percent of organisations have some form of agile process in place” [2]. Agile’s swift, iterative product delivery cycles are now the industry norm.

In stark contrast, foundational user research methods, such as 1:1 user interviews, seem trapped in the time-consuming, sequential traditions of the Waterfall model.

These tried-and-true methods struggle to achieve deep insights at delivery speed.Consequently, product teams are forced to press forward amid uncertainty, with flawed decision-making, a disconnect between product features and user needs, delayed feature releases, and even outright market failure — an issue which is aptly demonstrated in start-ups, of which 35% fail because there is no market need, according to Statista.

A Rallying Cry ‘Streamline User Research’

A wave of user research tools aimed at optimising the end-to-end user research process have emerged over the last five years. These cover everything from interview recruitment, scheduling to meetings and insight management platforms.

While these tools have captured the imagination of venture capitalists and user researchers alike [3], a problem looms. At the heart of decision-making and value creation, product teams seem to be struggling now more than ever to obtain timely, rich, and actionable user insights.

More Than Tools, We Need Disruption

When we consider the scale of product development worldwide, a larger issue is revealed. There are around 47 million individuals worldwide involved in product development. Only a fraction of them, about 2 million, are designers engaged in research, and around 500,000 identify as dedicated User Researchers. [5-9]

Figure 1: Most commonly reported ratio of researchers to designers to developers. Source: MeasuringU [4].

Following the agile framework, which typically consists of 5–7 engineers per team, the ratio could be described as approximately 1 researcher to every 20 product teams. This disparity exposes a fundamental challenge: Product teams are pressed for time and simply can’t access researchers.

Given the above, the user research cadence of product teams typically fall into one of three categories:

1. Episodic Research: This is conducted in fixed-scope projects, which does not aid or inform the challenges of continuous delivery.

2. Ad-hoc Research: This is reactively led by time-poor and largely untrained team members such as Designers, Product Managers, Developers, or Marketing Managers.

3. No Research: In this case, research is not conducted at all. Decisions are made on the fly in order to meet deadlines and get to market quickly.

It’s clear that user research simply can’t keep pace. Even as newly trained entrants flood the market through intensive training programs, the scope of collective problems to solve grows exponentially faster.

Beyond the above, budget constraints, difficulty finding and retaining tenured personnel, and a lack of understanding of the value of user research at the organisational level, all exacerbate the talent gap.It’s clear that what we don’t need is to streamline existing methods, but rather, an entire step-change in how we execute user research.

The Case for Embracing AI in User Research:

Artificial Intelligence (AI) presents a compelling proposition. With its ability to handle complex tasks, sift through large unstructured datasets quickly and make informed predictions and judgements, AI has already begun reshaping industries.When applied to user research, the potential is truly groundbreaking.

AI can work to limit bias, bridge cultural and language barriers, conduct contextual, targeted, and trigger-based research, collect data 24/7, scale limitlessly and analyse real-time data while interviewing, to uncover behaviour and attitudes in real-time.We are already seeing hints of a world like this, a small group of user research tools have emerged to incorporate AI.

For simplicity, we will separate tools into two camps research ‘Enablers’ and ‘Doers’.

Enablers are used to augment existing research methods, though transcription, data aggregation and insight synthesis. For example, tools like Otter are used in the realm of user research to transcribe and analyse interviews and testing sessions. The AI can automatically transcribe speech, differentiate between speakers, and pinpoint key themes or topics in the human-led conversation.

Doers are tasked with greater autonomy, executing activities that were traditionally human-led. For instance, Validly uses AI to conduct user interviews for you, deploying autonomous agents to interview multiple participants simultaneously. By outsourcing interviews, the in-person meeting is removed, driving significant time and effort savings. Analysis of interview findings occurs in real-time, enabling teams to make informed decisions promptly and efficiently.

Figure 2: Video: User Interview conducted by an AI Agent. Juno.

Deploying AI in the ‘Doer’ category represents a significant breakthrough in the field of user research. It allows for the collection of rich, actionable insights on behaviours and attitudes at scale previously thought impossible.

If we take this a step further, envisioning a world where all user research is informed by AI agents, it could revolutionise how businesses uncover and deliver upon user needs. We could see insights flow into product development as seamlessly as water from a tap.

However, like all technology, AI isn’t without its challenges. It’s still in its developmental phase, and skepticism about its ability to understand and predict human behaviour is understandable. Issues like privacy, data security, and algorithmic transparency remain significant considerations.

While AI is skilled at parsing large data sets and identifying patterns, it will take time to replicate the nuanced understanding and instinctive insights that come from a seasoned human researcher. But when combined, AI and human insights could create a highly effective alliance.

I am optimistic that as researchers adapt to an AI-integrated future, they will evolve into something akin to orchestra conductors. They would guide the performance of AI-led research, handle sensitive insights carefully, and steer ethical considerations. In doing so, they would maintain an efficient, responsive research system that aligns with organisational needs.

Despite these concerns, the potential benefits of AI, especially considering the pressing need to accelerate user research in the face of rapidly evolving product development cycles, overwhelmingly tip the scales in its favour.

Conclusion: AI as the Path Forward

Having discussed the limitations of traditional user research methods and the stark disparity between product teams and researchers, AI is poised to be a compelling, scalable solution to revolutionise user research.The creation of things is only getting faster.

As no-code platforms and generative AI for coding, designing, and building continue to evolve, the number one question product teams will ask will shift from ‘how to build’ to ‘what to build’ — significantly elevating the value of user insights.

Embracing AI in user research is not merely an opportunity — it’s a timely necessity. It’s critical that we embrace new technology and leverage it to enhance our capabilities. It’s not just about keeping pace with the accelerating rate of development — it’s also about staying ahead of it.

As we usher in this new era, we must remember that it brings its own challenges. AI, while promising, is still in its infancy. Skepticism about its capacity to make meaning of human behaviour is valid. We must be aware of the ethical implications of using AI in user research. It’s crucial to ensure that AI is used responsibly, with respect for the privacy and confidentiality of user data. Additionally, diligent work is required to counter any inherent biases that may exist within AI algorithms.

Equally important is the issue of transparency. As we deploy more sophisticated AI models, ensuring clear and understandable explanations for their decision-making processes will be essential. This will build user trust and improve these models over time.

However, despite these challenges, the potential benefits of AI in user research far outweigh the uncertainties. Think of AI in user research as the equivalent of the first digital camera or the first cloud server. It’s nascent technology with vast potential waiting to be tapped.

Our traditional approach to user research is on the verge of obsolescence, and AI is poised to take its place. This is not a moment for hesitation or resistance; it’s time we embrace AI and start the discussion around what the Next Era of User Research should look like.

So, how do you see AI shaping the future of user research?

In future articles, we will venture deeper into the role of AI in User Research as we build Validly in public. Subscribe to stay tuned.This story is published by Josh Davey, Founder of Validly.

Article references:

• Silicon Valley Product Group. (2012). “Continuous Discovery”.
  Accessed June 16, 2023. Link​.

• State of Agile. (2023). “State of Agile”. Accessed June 16, 2023. Link​.

• Andreessen Horowitz. (2021). “The Market for User Research Platforms”. Accessed June 16, 2023. Link​.

• MeasuringU. (2017). “What Is the Typical Ratio Between Designers and Developers?”. Accessed June 16, 2023. Link​​.

• Statista. (2023). “Global developer population 2024”. Accessed June 16, 2023. Link.

• Statista. (2023). “Main reasons for start-ups going bankrupt globally 2021”. Accessed June 16, 2023. Link​.

• Lucidchart Blog. (undefined). “How to Build a Scrum Team Structure for Agile Development”. Accessed June 16, 2023. Link.

• The Product Manager. (undefined). “15 Statistics You Should Know About A Career In Product Management”. Accessed June 16, 2023. Link.

• Qubit Labs. (2022). “How Many Programmers are there in the World and in the US? [2023]”. Accessed June 16, 2023. Link.

• IIBA Business Analysis Blog. (2020). “Here Business Analysis Professionals Around the World”. Accessed June 16, 2023. Link.