Some of AI’s use cases are relatively anodyne; others can generate quite the controversy. And given the technology is relatively nascent, one could argue its widespread adoption in the legal industry falls into the latter category.
Concerns about AI’s role in legal practice remain widespread, particularly when its outputs influence legal analysis or client outcomes. People are concerned that lawyers are outsourcing their thinking to a machine, and if crucial evidence prepared by a tool may be misconstrued by a judge or jury. Can AI ever have a legitimate role in a justice system that counts fairness, transparency, and due process as its pillars?
Regardless, AI has already found practical uses in legal work, particularly in document review, research, summarization and drafting. And it’s likely not going anywhere: Legal teams at approximately 41 percent of law firms now utilize AI tools, and that number looks set to grow.
As a personal injury lawyer, I see regulatory gaps emerging as a result of AI’s use in law firms much faster than legal standards can keep pace. Existing professional rules make lawyers responsible for their work, but they do not necessarily resolve every AI-specific question around disclosure, auditability, vendor responsibility, data retention and the level of human review required.
Defining Accountability
Accountability is one of the hardest questions raised by AI: Who assumes legal responsibility when AI makes a mistake?
Infamously, in the 2023 case Mata vs. Avianca, the plaintiff’s lawyers used ChatGPT to produce legal citations. But the chatbot hallucinated and presented nonexistent cases that were then formally submitted as precedent to the judge. The resulting fracas was in many ways inevitable, but it served to establish an important principle: AI may assist lawyers, but it cannot replace their professional duty to verify the accuracy of their work.
In personal injury law, however, the accountability challenge extends beyond legal citations. Given the sensitive and often intricate nature of the cases involved in this sphere, any mistakes can shape settlement negotiations or litigation strategy long before the case reaches the courtroom.
Suppose an AI tool is used to summarize and condense hundreds of pages of medical records into a chronological record of events. But details of an earlier critical injury get buried. Such an omission will influence the lawyer and client’s demands, and may be discovered only during deposition.
It doesn’t help that AI’s mistakes may be hard to identify. A fabricated citation is a clear and identifiable failure, but an AI system overlooking a pre-existing medical condition, incorrectly summarizing a witness’s opinion, or miscalculating future care costs — such mistakes may look reasonable and go unnoticed.
Bias can be even harder to detect. A valuation system trained on historically skewed data could, for example, systematically undervalue claims associated with particular demographic groups.
Using AI Judiciously
AI can sift through mountains of documents, analyze medical records, and even draft legal demands. Used appropriately, these capabilities can reduce superfluous administrative work and allow attorneys to spend more time advising clients and developing legal strategies.
But not every task should be delegated, and personal injury cases are rarely straightforward. They involve conflicting witness testimonies, evolving medical evidence, questions of credibility, and negotiations that depend as much on human judgment as legal precedent. Historical data may help AI identify settlement patterns, but it cannot substitute for a counsel’s assessment of how a particular plaintiff, witness or fact is likely to be perceived at trial.
The greatest danger, therefore, is that lawyers are placing unwarranted confidence in outputs that appear authoritative and objective, even though AI can be wrong, blind to context and incapable of human judgement.
A similar concern is playing out in the insurance sector. Consider the backlash against health insurance companies using AI to review claims. People want to be assured of human oversight, knowing a real human capable of judgment reviewed their cases, even if the workload was lightened and accelerated by AI.
Law is no different, and no matter how much AI helps with the drudgery, recognize that it must be used selectively, with clear limits and meaningful human oversight. Luckily, this seems to be the direction most lawyers are headed: around 83 percent of legal professionals say using AI to provide legal advice is inappropriate.
A Regulatory Framework
Essentially, regulation should target how AI is used, as well as where it is appropriate to use.
If generative AI is used to produce court filings or documents, it should be legally required for attorneys to personally verify any legal analysis, citations, or damages before they are shared with clients or filed in court.
Human review must be a mandatory step, not simply a recommended best practice. The more consequential the output, the more rigorous the required review should be. Any legal citation, calculations of damages, medical chronology or factual assertion that’s generated by AI and intended for a client or court should never reach its audience without meaningful attorney verification.
Failure to do so, say, getting caught submitting hallucinated information in court, should result in serious professional consequences.
Firms should also be required to produce and maintain an auditable record of their AI usage. If AI assists with reviewing medical records, drafting a demand letter, or summarizing a testimony, lawyers should be able to identify which tool was used, what information it processed, and which attorney reviewed and approved the final document. Create a system that prizes accountability so that mistakes can be fixed before they become a problem during litigation.
Finally, there’s the issue of data protection. Personal injury cases often involve extensive medical records, employment histories, financial documents and other confidential client information. Feeding this data into AI systems creates new risks around data retention, unauthorized access, and whether information submitted to a third-party AI provider could be used to train future models.
Existing confidentiality measures already require lawyers to consider such risks. Regulators should aim to go further by defining minimum expectations for vendor diligence, retention policies, training data usage and access controls.
Final Thoughts
As AI is increasingly used in litigation, the industry is faced with both an opportunity and challenge. The technology can certainly grant attorneys the time to focus on more critical elements, but without transparency and well-considered frameworks for use and disclosure, it has the potential to undermine trust in our judicial system and imbue bias into court proceedings.
It seems AI itself is now on trial. But before it can take the stand, there must be specific regulations to determine what is and isn’t legally permissible. Only by creating clear standards for transparency and accountability will clients and courts be convinced that AI is improving litigation without compromising on the principles of justice, fairness and equality for all.
