Artificial Intelligence & Robotics
1 prosecution brief had AI hallucinations, but other issues were human error, district attorney says
Four briefs filed in criminal cases by the district attorney’s office in Nevada County, California, had hallucinations that were likely caused by the use of artificial intelligence, according to lawyers for one of the defendants affected. (Image from Shutterstock)
Four briefs filed in criminal cases by the district attorney’s office in Nevada County, California, had hallucinations that were likely caused by the use of artificial intelligence, according to lawyers for one of the defendants affected.
The prosecution briefs had incorrect citations, used quotations not in cited materials, named the wrong courts issuing opinions, and wrongly construed precedent and the state constitution, according to public defender Thomas Angell and lawyers from the Civil Rights Corps, a nonprofit group that works to reduce the detention of people awaiting trial. In the case of Angell’s client, prosecutors filed a brief seeking to keep the defendant in pretrial detention that wrongly interpreted a state constitutional provision, he said.
The New York Times and the Sacramento Bee covered the AI allegations.
Nevada County District Attorney Jesse Wilson has admitted that AI caused errors in one brief but not in the case of Angell’s client. The filing with AI issues was immediately withdrawn when the error was discovered, he said in a statement. Two other briefs with inaccurate cites were caused by human error, according to a statement issued to the Sacramento Bee when only three case filings had been identified with problems.
“Our office continues to learn the dynamics of AI-assisted legal work and its pitfalls,” Wilson wrote in a statement cited by the New York Times. “Regardless of the source of any citation error—whether arising from the use of artificial intelligence or traditional human error—our office remains firmly committed to the highest standards of integrity.”
Angell and lawyers from the Civil Rights Corps filed a petition with the California Supreme Court in late October seeking an investigation and sanctions. The Civil Rights Corps is a nonprofit that has challenged pretrial detention as part of its challenges to “systemic injustice.”
A group of 22 legal and technology scholars filed a brief in the case Friday warning that AI misuse could lead to wrongful convictions.
Angell formerly sought sanctions in his client’s case, including payment of $23,000 in attorney fees for time spent uncovering the errors, with California’s Third District Court of Appeal. The appeals court twice denied sanctions, but it did order a new bail hearing. Angell’s client was convicted of illegal possession of firearms at trial.
The AI errors in California are one of the first cases involving a hallucinated brief by prosecutors, the New York Times reports. A database maintained by a researcher at HEC Paris, a business school, identifies only one case involving prosecutors, and it is from Israel.
Angell’s case is Kjoller v. S.C.
Write a letter to the editor, share a story tip or update, or report an error.
