OpenAI says its new legal AI achieved 54% overall correctness on a private legal-research evaluation.
That hardly sounds like the arrival of the robot lawyer. But replacing lawyers is not the threshold Astra needs to cross to change legal work.
OpenAI has launched Astra for Law, combining GPT-6 Astra with a specialist legal-search index covering US case law, statutes, regulations, court rules and administrative decisions across more than 230 million URLs.
On a private 200-question legal-research validation set, OpenAI says Astra for Law achieved 54.0% overall correctness, compared with 38.7% for ordinary GPT-6 Astra using web search – a 40% relative improvement. It also found 24% more reference cases on case-law tasks.
The work disappears before the jobs do
We have seen the mechanism at the other end of the labour market.
Salesforce says the TSA’s AI agent now handles around 100,000 traveller conversations each month and resolves 96% of routine inquiries without human escalation. The job has not been automated. A particular category of work has.
Astra for Law brings the same question to professional work.
OpenAI says the system can research legal questions, develop arguments and deal terms, identify cases that weaken an argument and analyse contractual provisions. Law firms working with AI are also applying the technology to agreement review, due diligence and IPO preparation.
A lawyer still has to check the work and remains responsible for the answer.
The decisive question is therefore not whether Astra can research law without a lawyer. At 54%, that is hardly the interesting test.
It is whether a lawyer can prompt, check and correct Astra’s research faster than producing equivalent work from scratch.
If verification takes almost as long as the original research, little human work has disappeared. If a two-hour research task becomes a 20-minute checking task, the economics change even though a lawyer remains responsible for the finished product.
That is the threshold worth watching as AI moves deeper into professional work.
The work can disappear before the jobs do. But Astra’s benchmark does not tell us whether it has crossed that line.
It tells us where to look.
Sources
- OpenAI – Astra for Law
- Reuters – OpenAI launches legal-focused AI platform, escalating race for law firm users
- Salesforce – TSA improves travel experience using Agentforce
Join the Dissenting Citizen Newsletter
Independent news and commentary delivered directly to your inbox.
Breaking stories, analysis and commentary throughout the day.
Follow @VoxDissent →
