Anthropic gave three outside research groups a way to study how people use Claude without showing them private conversations. The groups chose their own questions, Anthropic ran the data collection, and the researchers analysed the aggregated results.
The pilot included Stanford’s Social and Language Technologies Lab, Oxford’s Human Information Processing Lab and METR, a non-profit that evaluates AI models. Each study used a separate sample of roughly 250,000 conversations from April and May 2026. Stanford and Oxford studied Claude.ai; METR studied Claude Code.
In its August 26 report, Anthropic described findings about the work people delegate to Claude, their experiences during conversations and the time coding assistants may save. Some of the results were still preliminary.
What researchers could see
The groups used Anthropic Insights, formerly called Clio. Researchers wrote questions about conversations, such as what guidance a user wanted. Claude answered those questions, and the tool grouped or tallied the answers. Researchers received category descriptions and counts, rather than the conversations themselves. The raw data and computation stayed on Anthropic’s servers.
The samples came from consumer Free, Pro and Max accounts. They excluded Team, Enterprise and API customers. For Claude Code, Anthropic sampled users who had agreed to let the company use their data to improve its models.
Anthropic retained review rights covering privacy, information that could enable misuse, confidential company information and research accuracy. Outside those areas, the agreements let researchers publish their findings even if they reflected poorly on Anthropic.
The work people brought to Claude
Stanford’s SALT Lab found that more than half of the conversations involved people delegating consequential tasks to Claude. The researchers used that term for work that affects other people or is difficult to undo. Such tasks were especially common when people sought professional guidance on legal or financial questions.
That differed from earlier research suggesting people mostly delegated tasks with less accountability and kept consequential work for themselves. The pilot describes the conversations in its samples; it does not establish how everyone uses AI or measure a change over time.
In nearly three-quarters of conversations, people directed the work while Claude assisted. They usually adapted its output rather than using it unchanged. The researchers also found variation in how much users understood or learned from Claude’s answers.
Some of the effort involved in working with Claude appeared useful. Seeing an attempt, spotting a misunderstanding and revising the request could help people explain what they wanted and improve the result. The report did not suggest that every difficulty was beneficial.
How people felt during conversations
Oxford’s Human Information Processing Lab found that particular user responses appeared alongside particular kinds of Claude behaviour. Warm responses from Claude went together with more positive user responses. Refusals or disagreement went together with users pushing back. Eccentric behaviour went together with greater intellectual engagement, while helpful responses went together with apparent satisfaction.
These were associations in conversations, rather than evidence that Claude’s behaviour caused a change in someone’s feelings. The patterns among experiences such as frustration, enjoyment and absorption also resembled those in a separate study of everyday internet browsing. Oxford was still completing its write-up when Anthropic published the report.
Estimates of time saved on coding
METR compared Claude’s estimates of how long a task would have taken without AI with the time it took using different Claude models. Its preliminary analysis suggested newer models saved more time than older ones.
Because the comparison relied on estimates, METR checked Claude’s judgements against known completion times from an earlier developer study. The estimates correlated with the time developers actually took. That check supports the method, but does not turn every estimated saving into a directly measured one. METR’s analysis was still underway.
Limits of the pilot
The results depend partly on how Claude interprets researchers’ questions. Anthropic found that wording which worked on WildChat, a public conversation dataset, could produce misleading categories on Claude traffic. Researchers could not inspect the private conversations to check those classifications themselves.
Anthropic also removed or altered some categories that described how people bypassed safeguards. It told the researchers what it had changed and why. In each study, fewer than 5% of categories and conversations were affected, according to the main report.
An Imperial College London privacy team tested the released outputs and did not identify users or find a violation of the specified privacy threat model. It did link one category to a widely used open-source project, without identifying anyone who used it. The audit covered this release, not every possible future study.
Anthropic published the aggregate outputs received by the researchers. It said that running the pilot required considerable time and staff effort, and that it was still deciding how to support more studies while protecting privacy and research quality.