Vlex and Startup Vecflow, one of the companies in the Vals Genai Performance Study (see here), have sent several comments to artificial lawyers about the results. Other companies involved in the published research include Harvey and Thomson Reuters. (Note: there is a walk around vecflow and vlex Al TV products, providing more context. See here.)
Below are verbatim comments from Vecflow and Vlex.
Here is the link to the complete Vals AI report.
–
vecflow
About the Company: “Vecflow is a legal AI company built to optimize workflows across the enterprise, including research, due diligence, drafting, document reviews, and more. By using company documents, information and knowledge, VECFLOW offers a customized one-click workflow – accelerate and improve legal work. Note: The main AI tool is named “Oliver”.
Comments on Vals results
Vals.ai’s study compares Oliver with four other established products: Thomson Reuters, Lexisnexis, Harvey, and Vlex. The study confirms that in just six months in the market, Oliver has “almost equal or performed better than his peers” than his peers. This includes companies worth billions of people.
This assessment tested AI legal assistants against human lawyers across seven legal challenges. In particular, Oliver is superior to human lawyers in document Q&A and document summary. Following Harvey’s dropout, Oliver is also Sec Edgar Research’s sole competitor. This is the only task that involves complex multi-stage reasoning.
“Intelligent AI workflows represent the future of legal work,” said Joe Parker, CTO, VECFLOW. ‘The assessment highlights three important findings.
1. AI assistants already outperform lawyers in several key areas
2. Oliver’s research capabilities are in an unparalleled position in the legal technology sector.
3. AI performs best when complementing the existing workflow of lawyers.”
Vals.ai specifically pointed out that “in the challenging Edgar’s research task, it stood out as the only offering that allowed us to approach human-level performance through coordination of specialized AI agents.” Research surrounding Edgar means multiple levels of iteration and research that are unparalleled by other providers of research.
“We have already seen that our customers can handle more cases in more time. In an interview with Vals.ai, VACFLOW CPO Thomas Bueler-Faudree said: “We’ll see that small, technically positive businesses can do 10, 100 times faster material than before. I think you’ll see an increase in competition in the law, and we hope you’ll see one of the four big law firms that will appear in the US.”
Since the benchmark was implemented six months ago, VECFLOW has brought significant improvements to Oliver based on customer feedback. The company reports that Oliver’s capabilities have progressed significantly beyond the already impressive results shown in the study.
–
vlex
About the company:
‘Vlex is a global legal information and legal AI pioneer that empowers legal experts to work smarter. With nearly 30 years of experience in legal technology and access to over 1 billion legal documents, Vlex has developed the Vincent AI. It is a comprehensive AI-driven workflow platform with Vlex’s vast library of structured legal data, adding citations to authoritative sources that have been verified.
Comments on Vals results
‘Vals Legal AI Report focused on document extraction tasks that did not involve tasks requiring a legal database. “We are extremely pleased with the performance of Vincent AI. We are extremely pleased, especially considering that the document summary tasks do not utilize the vast legal database of global information,” said Ed Walters, Chief Strategy Officer at Vlex.
The report states, “Our assessment focused on small slices of Vincent AI’s capabilities in US jurisdictions, but support for international issues is a major strength. For global law firms, this feature provides unparalleled levels of utility for other tools, making Vincent AI an attractive choice.” In fact, in a recent “AI SmackDown,” designed to test how AI-driven research platforms provide direct answers to legal questions, the Association of Law Libraries in Southern California found that Vlex gives the most depth to challenging AI tasks.
Other takeaways from the Vals Legal AI report include:
Vincent gave the answer very quickly as one of the fastest products that generally evaluated. “The design of Vincent AI is particularly noteworthy for its ability to guess the appropriate subskills to perform based on user questions. If clarification is required, Vincent AI will actively ask follow-up questions to improve its understanding and ensure customized answers.” “If there was not enough data in the legal research database to answer the question, Vincent AI refused to answer rather than hallucinate an illegal response.” “The answers provided were impressively thorough. … provide valuable additional context to support their understanding and workflow.” “The results are promising and reveal that AI actually brings value in the context of legal work,” Vals’ research concluded. “AI tools were six times faster than the lowest-end lawyers and 80 times faster at the highest.” “Vals Legal Report is a great validation of first-generation legal AI tools,” Walters said. “The results are promising and reveal that AI actually brings value in the context of legal work.”
–
AL View
Overall, it appears that legal AI has appeared in the winners here. Naturally, there are variations among the companies Vals tested. After all, they’re all made by humans, but on this site the biggest takeaway is how well they did as a group.
Well, if you’re a half empty person in the glass, you’ll say: “But they’re not perfect, they’ve made some mistakes and they’ll still need a lawyer to complete those tasks.”
And if you’re half the glass person, “By using AI, the work of lawyers who demands a lot of complex, language-based reasoning can be done in part through software. What’s more, this Genai technology is still in its early stages, with new models arriving every few weeks. The pace of change is incredible, and it’s clear that lawyers and law firms who ignore legal AI are putting their heads in the sand.
And that’s why Vals research is so useful. It emphasizes that legal AI tools can really do a lot through third parties. Are they “perfect” or can they “everything” be? no. But it is clear that they can do enough to provide truly tangible benefits to the legal department.
The PS pointed out by VECFLOW above shows that every week these companies can improve what they can do. As Foundation Models evolve, the things that each vendor can offer after improving the output and improving the application layer of their offerings will also evolve. In short, whatever the outcome today is, they will be better in the coming months. In fact, VECFLOW points out that it’s already excellent. And that’s probably the case for all the vendors here.
–
Conference News: California Legal Innovators – June 11/12 – San Francisco
The California Legal Innovators Conference in San Francisco will take place on June 11th + 12th.
The landmark event, located at the heart of the Bay Area Tech world, takes place over two days. Day 1: Law Firm, Day 2: Inhouse. Come with leading legal tech experts from law firms, in-house and high-tech companies across California and around the world.
Find out more, ticketing, speaking and sponsorship opportunities here.
