I've recently been hearing a lot about synthetic research, and I think it's gotten more hype than it deserves. Full disclosure: I'm biased, having spent most of my career in the room with real participants. But I've also tracked this space closely for a couple of years, and I think it's a slippery slope, one we need to be careful about how we talk about. In the article below, I explore why.
What is a synthetic user, actually?
At its core, a synthetic user (sometimes called a synthetic persona) is a composite built by an AI model out of a huge amount of prior data. The model uses past research transcripts, interview recordings, survey responses and whatever historical inputs it's been fed. Once you feed enough of that into a LLM, it can generate a persona that behaves, on the surface, like a real research participant you can interview or test a prototype on.
Here's how it might actually work in practice – say I run a set of research interviews with several participants. I feed those transcripts into an LLM and ask it to create a user from those discussions. What comes out the other end is a synthetic proxy I can interact with as if it's a live person, shaped entirely by what those original participants said. It's a fake person, created off of real data, but it's not itself real. It never went through the experience.
Right now, I see this show up in the research world in two main forms.
The first is AI moderated interviews. A researcher loads a discovery guide and script into a platform, and an AI voice reads the questions to participants, or the questions are merely shown on a screen for the participant to read, sometimes probing further based on their answers and what parameters the researcher allowed when creating the study. In practice, this is closer to an unmoderated survey with a synthetic layer of follow-up questions than it is to an actual live conversation.
The second is synthetic personas used as test participants. Anyone, researcher or not, can put a prototype or a live website in front of a fabricated "user" built from aggregated historical data and ask how it would respond. Then treat that response as validated research.
Why real moderation is hard to fake
Here's my case against synthetic moderation, it has to do with everything a human moderator brings into the room that an AI script simply can't replicate.
When I'm in a session, I'm carrying context an AI model doesn't have. I know the client, I know where this particular project fits into the broader business, I know the specific questions our designers and product teams need answered, and I'm weighing that against what the business needs to move forward. All of that sits on top of years of facilitation practice.
I’m constantly making micro pivots as I moderate. I come into a session knowing my script well enough to basically recite it, but as my participant is sitting next to me, or virtually across from me, I'm reacting to what they actually say in that moment. Sometimes a participant goes off-script. They might vent about an unrelated complaint or spend two full minutes on a tangent that has nothing to do with the research goals. In the moment, I have to gauge how long to let them talk, how much to acknowledge, how far to follow the thread, and then, when it's time, step in, acknowledge what I heard, and move things forward.
AI doesn't have that instinct, even if you feed it the project's goals ahead of time. If a participant needs two full minutes to get something off their chest before we can get to the real conversation, the transcript will still contain a full two minutes of an unrelated complaint. An AI summarizing that session can easily weight it as important, simply because it has no way of knowing it wasn't.
What's beautiful about humans is that we are unpredictable. That unpredictability is exactly where the most valuable research insight tends to live. A synthetic persona is only ever trained on what people have said in the past. Not necessarily what they actually do. Reading between the lines, noticing that someone says one thing but does another, requires a kind of human interpretation that AI simply cannot do.
The consequence, as I see it, is stark. A synthetic persona might replicate feedback on well-established UX heuristics or best practices reasonably well. What it cannot do is surface genuinely new insight. You will not hear novel ideas, because AI models can't do that. Humans do that. You're never going to hear unique nuances or particular situations, because the synthetic persona only knows things people have done in the past, or things people have said they've done in the past.
The same gap shows up in usability testing specifically. A human moderator isn't just listening to a participant's words. I'm watching what they're doing on screen in real time and adjusting my questions accordingly. I can't do that yet with these tools. If people are claiming their platform can do it, I'm willing to bet it's not doing it as well as a human could.
A pattern of overpromising, and quiet backpedaling
Part of what frustrates me is watching the marketing cycle play out in real time. Over the past year or two, I've tracked several AI research and synthetic persona platforms leaning hard into taglines suggesting their tools could fully replace live participant research. More recently, I've watched some of those same platforms walk that claim back, clarifying that of course you still need to test with real people, and that their tool was never meant to be the end-all-be-all.
That's interesting, I think, because you said your platform was going to be the best thing since sliced bread, and we didn't need anything else. There is a ton of unrealistic and unproven marketing around how these tools actually work, and I don't think that's a small thing to wave away.
Where synthetic tools do make sense
I want to be clear that this isn't a blanket rejection of technology or AI. There's a real, narrower use case here. Small-scope usability questions, like testing whether a single feature or button is findable, based on established heuristics and prior research patterns, can be a reasonable fit. I think of a project we did with a recent hospital client, testing out one feature to see if people found it familiar and could find it easily. With the right guardrails, a researcher could set a designer up for success with a small usability study on one smaller feature, something you could reasonably hand to synthetic users. That frees me up to work on bigger, more complex questions, while designers can move quickly on the smaller ones.
Where it breaks down, in my view, is generative research. Open-ended questions about whether a new product is well positioned for people's lives, or how it's actually going to work in their day to day. For that kind of work, I'll make the claim plainly: synthetic personas cannot fill those shoes. You have to have actual real participants.
The downstream effect on clients, and on the field
Beyond the tools themselves, there's a broader industry narrative I think has done real damage the idea that research can be conducted by anybody, without a trained researcher, because AI moderated platforms and synthetic personas can supposedly do the job.
I've personally sat through AI moderated research sessions. All it is, functionally, is an unmoderated test with an AI voice reading questions to you. Sometimes it attempts a scripted follow up question, and it can badly misread what the participant just said. I remember one example where a participant explained they couldn't complete a task at all because the prototype was broken, and the AI's next line was something like, “so how do you feel about not being able to complete that task?” Meanwhile the participant is thinking, “well, I feel super aggravated, but I can't actually give you any useful feedback on the platform, because you're totally misreading the situation.”
That hype doesn't stay contained to marketing copy. It shapes client expectations. I've noticed clients pushing back on cost and timelines in ways that seem to trace back to industry claims they've absorbed secondhand, often without ever having evaluated the tools themselves. I think, indirectly, our clients are hearing and seeing some of this hype and reacting to it. That's hard to fault them honestly. Most of them haven't had the chance to test these platforms firsthand, so of course they trust the headlines. But the effect compounds. The underlying question clients start to ask becomes: why do we need a researcher on this project at all? Why do we need to spend three weeks on a research project? Why can't our designers just take that over? Better yet why bother with designers? Why can't our product managers just do all of this?
That's the real cost of this cycle. It's a harmful narrative, generally within the industry, that research can be done by anybody, and that you don't need a skilled researcher because these platforms exist.
It's a frustration I recognize in other design disciplines too. A friend of mine, a designer, was recently told a piece of her work would be "so easy." Her response has stuck with me since. “Don't insult me by saying it'll be so easy, she said. I've gone to school for this. I've built my whole livelihood around it.”
The bottom line
I'm not making the case that synthetic users and AI moderated research are worthless. Used narrowly, for small, well defined usability checks grounded in established best practices, I think they can save real time and let researchers focus on harder problems. But treated as a wholesale replacement for real participant research, they fall short in exactly the places that matter most: reading unpredictable human behavior, following unscripted tangents with judgment, and surfacing the kind of genuinely novel insight that only comes from talking to real people.
As the hype cycle around AI research tools keeps turning, and as some platforms quietly soften their own claims, my message to the industry is simple: don't mistake a synthetic echo of the past for a real window into how people will actually think, feel, and behave.