User testing with Disabled people

For the past year, I've been helping to organise user tests with Disabled people at the Digital Factory. It is one part of my AccessibilityOps role — a practice Devon Persing defines as the accessibility equivalent of DesignOps: a way of connecting people, tools and processes around digital accessibility.

In practice, the role, which I have held since July 2025, covers several areas of work:

• bringing accessibility to the fore whenever it is relevant;
• improving processes to avoid recurring errors;
• supporting teams as they adopt best practices;
• contributing to the design system documentation;
• building user tests with Disabled people into our processes.

This takes up 50% of my time; the rest goes to my work as a UX designer in the squad that looks after car insurance products. Not for lack of work on either side, but  it's important to me to stay hands-on. Being subject to the operational changes I ask of other teams keeps me aware of what they actually involve.

This article is an account of the work that took up much of my past year: organising user tests with Disabled people. Case studies like this almost always come from the agencies that run the research. I wanted to give you an account from the client side.

Why test with Disabled people?

Understanding people with lived experience

Having conversations with people is the only way to understand and meet their needs, regardless of their situation or lived experience.

Depending on what you are trying to improve, you will not necessarily talk to the same people. I deliberately draw a distinction between two groups, because they do not always overlap: people who are disabled by digital barriers, and Disabled people.

For example, someone disabled by digital barriers runs into navigation problems when a site is not coded properly. They cannot activate a link by voice, for instance, if the accessible name does not match the visible label.

A disabled person, on the other hand, may well encounter no digital barriers at all. Someone who uses a wheelchair because they have lost the use of their legs should have no trouble with a mouse or a keyboard. But they are still disabled in real life, and that can influence the information they're looking for — they might want to know whether their wheelchair is covered when getting a car or home insurance.

This distinction guides recruitment. If I want to check how prominent a piece of information is, I talk to Disabled people. If I want to check how usable a journey is, I turn to people disabled by digital barriers. They are not always the same people, although they can be of course.

Audits do not replace user testing

For several years now, the Digital Factory has worked with Temesis, whose consultants carry out our conformance audits as well as one-off audits on mock-ups and test environments.

But an audit, however rigorous, has a structural limit: it is carried out by people who know assistive technologies inside out. Most people do not.

Someone with low vision who starts using a screen reader at 50 years old, because their sight no longer lets them browse as they used to, does not suddenly become an expert in assistive technology. An audit checks conformance against a standard, while a test shows what a person actually experiences on the interface. That is what an audit cannot see, and it is why one does not replace the other.

Accounting for diverse experiences — and democracy

Another, perhaps obvious, reason is representation. Just as we talk to people of all genders, from low-income and well-off households, living in the countryside and in cities, we also want to talk to people who rely on digital accessibility every day.

Beyond representation, this is a democratic principle: everyone has the right to be represented, and to have a say. Testing with Disabled people gives them a direct hold on the services they use. That is the spirit of the disability rights movement's long-standing slogan: nothing about us without us.

And French regulation recognises it

The multi-year accessibility plan (schéma pluriannuel), which sets out an organisation's digital accessibility policy, must include information on how Disabled people are taken into account in user testing. The text does not make them mandatory. But for anyone working in France, that official mention is something to lean on, and it should encourage teams to take an interest.

Debunking the misconceptions about user tests with Disabled people

If you know my work, you will know I talk a lot about debunking misconceptions, particularly when it comes to defending digital accessibility.

Organising tests with Disabled people meant facing a few of them. Here are answers to the ones that come up most often.

Is it legal to ask people about their disability?

Yes, it is legal in France to carry out user tests with Disabled people. To recruit them for a panel, you can use the Washington Group questions, which focus on context of use rather than on people's medical condition.

For example, I recruit people who use a braille display or speech output. I do not recruit blind people. That way I avoid storing sensitive medical data I do not actually need.

What would require a far heavier legal framework — and what we simply do not need — is keeping a document that links identified people, by name, to their disability.

Disabled or not, participants' personal data must be kept secure. If you bring in an agency to run this kind of research, ask them for a GDPR notice so you know everything is compliant.

Do you need special training to run tests with Disabled people?

Yes and no: it depends who runs them. If you moderate the sessions yourself, you will need to build real skills — adapting protocols and sessions to each participant is an expertise in its own right. But if, like us, you are the client, then no: specialist agencies do that work, and it is their job.

There is still something to learn on the client side: what needs framing up front, and what the designers in the squads involved need to prepare. You learn that by doing it. The first few times are fumbling, as with any new project, and that is no reason to give up. I was lucky enough to know people who could point us in the right direction, but that is not a prerequisite — research agencies also guide clients who are starting out.

Are tests with Disabled people harder to organise?

Not when you work through agencies. If you build and maintain your own panels, that changes things. But at the Digital Factory, UX research is usually carried out by research agencies who work with panel providers. For these tests, we simply put them in touch with a specialist panel provider whose work we trust.

So there is not much extra logistics for us up front. It may be a little heavier for the agency, but that is part of the brief. On MAIF's side, the only additional task is making sure the profiles are relevant — I'll come back to that later.

How we did it

While scoping the project, and on a recommendation from Gwenaëlle Brochoire, co-founder of Oocity, we split it in two:

• a dedicated study with Disabled people, to learn what we did not know;,
• an experiment to include two Disabled people in each of the five tests already scheduled, to decide whether that inclusion could become systematic.

A dedicated study on Disabled people's relationship with insurance digital services

Preparing a dedicated study

Until then, we had never deliberately recruited Disabled people for our tests. Some may well have taken part without us knowing. Either way, we had a great deal to ask them — too much for a test focusing on usability or comprehension. A dedicated study let us run a fuller discovery interview and learn what we did not know.

It also let us cover a wider range of situations. We wanted to know whether context of use changed people's expectations and the usability of a journey, under what conditions, and whether we still had things to anticipate beyond conformance. None of that would have fitted into a standard test.

For this study, we worked with LunaWeb, who carried out the research, and Oocity who provided the panelists.

I name them because an anonymous account would help nobody, not to sell them to you. What matters here are the criteria that helped us choose.

How to choose the right research agencies

Two criteria made the difference for us: the ability to take on a brief that was still vague, and their ethics towards participants.

When we started, our brief was far from clear. We talked to several research agencies and digital accessibility specialists to help us find our way.

LunaWeb understood what we needed and put together a proposal well aligned with our values and ambitions. Anaïs Demaretz, project manager, and Justine Nicol, lead UX researcher, produced something extremely thorough. LunaWeb has been working on UX research with Disabled people for years, and Justine gave a talk about it at Paris Web in 2025.

On the panel side, Oocity has been a reference for a long time. Participants are always paid, properly welcomed and supported. That ethic matches MAIF's values closely, particularly its genuine attention to others. I had already interviewed Gwenaëlle Brochoire about UX research with Disabled people, and I knew how serious and demanding her work would be, and how well it would match our ambitions.

Together, LunaWeb and Oocity made an ideal pairing for a dedicated study.

Building the protocol and the panel

The study served two purposes: to learn from Disabled people, and to check the usability of a journey with no blocking issues.

So we organised the sessions in two parts:

• a discovery interview;
• a test on the home insurance journey.

Everyone recruited had to meet both criteria from the distinction I set out at the start: Disabled people who are also disabled by digital barriers. The first made the discovery interview possible, the second the journey test with their own tools. We recruited 23 people who either:

• have a cochlear implant or use French Sign Language (LSF);
• use captions and transcripts;
• use braille displays or speech output;
• adjust contrast, zoom or font size;
• use dark mode;
• have workaround strategies, such as colour coding;
• use voice control;
• or have difficulties with reading or working memory.

Oocity provided a balance between people who have been in their context of use since birth and those who have had to learn new habits at some point in life. This is important because it affects how digital services are used. Someone blind from birth cannot draw on visual memory, whereas someone who's lost sight later in life may still have partial vision and be able to compensate in parts for an interface's flaws.

We also wanted to recruit people with a learning disability, to look at how well our journeys are understood and to consider an easy read version (FALC, the French implementation of the European easy-to-read standard). During recruitment though, we learnt that the people we were trying to recruit did not handle their insurance products themselves — the subject felt too complex, or someone close to them took care of it. So nobody with a learning disability took part in this study.

What the dedicated study taught us

Findings from the discovery interview

Digital services, supposed to make life easier, are in fact very complex. During the discovery interview, participants described the problems they run into every day on digital services:

• technical blocks such as captchas, particularly for blind people;
• reading difficulties such as typefaces and cluttered layouts, particularly for people with low vision and people with cognitive impairments;
• input difficulties such as needlessly re-entering information or transposing digits, particularly for people with motor or cognitive impairments.

In an environment that does not fit them — one that fails to treat their needs as ordinary human needs — people face:

• cognitive overload for hard of hearing people, constantly on alert to make sense of their surroundings;
• eye strain for people with low vision;
• exhaustion from sustained concentration for people with mental health conditions;
• anxiety for people with cognitive impairments, who fear making mistakes with serious consequences on official paperwork.

When we asked whether they knew any companies with a reputation for digital accessibility, 20 out of 23 participants could not name any. And even if a brand did communicate about inclusiveness, they would not take its word for it. Blind participants, in particular, prefer to check themselves that services work with their tools.

When we asked what they expect from digital services, two broad families of expectations emerged. Technical structure and navigation matter a great deal to blind people and people with motor impairments. They share a concern for whether services work with their tools (screen readers and voice control). There is also a strong demand for legibility and low cognitive load from people with cognitive impairments or mental health conditions, as well as people with low vision, who share the same need for visual comfort (strong contrast, a good text size, a simple interface, and visual structure and consistency).

Findings from the home insurance journey test

The test ran in production on the recently updated home insurance journey. Testing in production rather than on a Figma mock-up let us check real compatibility with assistive technologies. The journey was not yet 100% conformant, but the audit had found no blocking issues.

The design system squad's work was recognised, particularly by people with reading or memory difficulties. They singled out the layout of the journey, the spacing, the illustrations, the use of pictograms, the weight of the headings and legible fonts. People with low vision found the journey very smooth and appreciated moving through one question at a time. People with cognitive impairments valued the input assistance offered throughout, especially the examples that unpack insurance jargon.

As for improvements, we found that completion time was two to three times longer than for a non-disabled person. Three lessons came out of it:

• what is not conformant is, unsurprisingly, not usable by people who rely on assistive technologies;
• what is conformant can still be complex to use or understand;
• the vocabulary can still be too complex for Deaf and hard of hearing people, and for people with cognitive impairments.

What we concluded

Disability amplifies friction. It brings to light technical and cognitive blocks that are invisible to a non-disabled person. That lesson should encourage us to keep including Disabled people in user tests, so that weak signals can carry more weight.

Most participants were surprised to learn that teams take an interest in their experience. Many arrived saying it felt good to be asked for their opinion. Others said they had given up on digital services altogether.

Design teams cannot undo that on their own. But it is essential feedback for a company. Beyond conformance and better experiences, it matters to communicate more clearly about digital accessibility work — something I wrote about in my article on ableist design. That is how we'll reach those who have given up out of frustration.

An experiment: widening test panels systematically

Alongside the dedicated study, we started the second part of the project: widening the pool of participants in the tests already scheduled at the Digital Factory.

These tests can be moderated in a lab or remotely, or unmoderated through a platform. They usually involve around twenty people recruited directly by our research agencies. Here again, I connected Oocity with our research agencies, including We Love Users, so that people whose context of use was relevant could take part. Being the point of contact between players who had not worked together before is the heart of my AccessibilityOps role.

We added two extra participants to each of the five tests, across different user journeys and products (navigating the customer account, getting a quote for a home insurance, submitting a claim, etc.).

Preparing a test with Disabled people

For each test, and especially the first ones, I coordinated many exchanges between Oocity, our UX research lead, the designers in the squads involved, and We Love Users. That effort shrank considerably as we ran more tests and each party settled into new habits.

My role was to relay between the designers, who owned the objective of each test, and the UX research lead, who owned the method, so that together we could define the most relevant profiles. Through those exchanges, everyone built up expertise. When you run a comprehension test, it can be interesting to include people with reading or memory difficulties, because they react faster and more strongly to content and the way it's presented.

To check the usability of a journey in a test environment, you can involve people who use assistive technologies — provided no blocking issue has been identified beforehand. Otherwise you risk wasting everyone's time, including your own. If someone is stuck at the very start of the journey, there is no point going further; unblocking them to carry on with the test would not reflect what they would experience in real life.

What the designers took from the experiment

Rather than go through the findings from each journey we tested, I want to focus on how the designers in the squads involved reacted.

When I asked them, “would you have missed anything if those two people had not been part of your panel?”, the designers unanimously said no. This answer also reflects the maturity the teams had already developed in some areas of accessibility. The tests did not uncover any major issue that would otherwise have escaped them entirely.

The designers nevertheless highlighted several observations:

• one person struggled to write without spelling mistakes, which slowed down form completion;
• another gave a lot of positive feedback on the UI, particularly the pictograms;
• a third identified structural problems.

None of these findings, on its own, called the journey into question. But each was a signal that nobody else in the panel had raised. This echoes what the dedicated study had taught us: disability can amplify certain frictions and gives greater weight to weak signals.

Audrey Dreillard, UX research lead at the Digital Factory, also observed that the experiment changed how we prepare, run and analyse tests:

Beyond what we learned about the usability of a product or journey tested with Disabled people, the inclusive research approach initiated by Tamara has helped us build lasting habits from the earliest stages of test preparation. This includes thinking about how participants will be welcomed (‘Is the chosen venue accessible to everyone?’, ‘Can the room lighting be easily adjusted for participants who request it?’), as well as how we phrase questions in a research protocol. And these habits apply even when tests do not specifically include Disabled people. This approach also reminded us that Disabled people are already ‘naturally’ present in our panels.

Towards making it systematic

Going forward, our team has decided to systematically include Disabled people in the panels recruited for our tests.

When we started this project, we knew very little. Today we have a much better grasp of the process:

• who to recruit;
• who to work with;
• how to analyse what comes back.

My role in all this has never been to run the sessions or write the deliverables — others do that better than I do. It has been to hold the threads between people, tools and steps. Pairing the right agencies, circulating expertise between the squads and the research team, making sure that attention to Disabled people runs through the whole process. That is precisely what AccessibilityOps covers.

The research agencies we work with have also settled into new habits and now anticipate our need for a wider pool of participants. The Digital Factory is also working on a UX research charter, setting out what we expect from those agencies and what we commit to. I reviewed that charter, written by the UX lead and the UX research lead, making sure attention to Disabled people runs all the way through the process. We ask, for instance, that consent forms be produced in an accessible format, so that people can read them on their own. Which sounds perfectly logical once you've said it out loud.

There are still things to refine. We need more precision in selecting participants, perhaps prioritising the profiles that face the most difficulty so that we do not miss a weak signal. We also want to make the most of their presence to keep learning about Disabled people's relationship with insurance, which means reworking the discovery interview so that it fits their profiles.

Finally, I hope that sharing this experience will encourage others, regardless of the industry you work in. These tests show that it is possible, that it is relevant, and that it is worth doing.