This is Part 4 of The Mirror Test, a Robonaissance series in which AI and neuroscience mirror each other.
On June 22, 2023, Judge P. Kevin Castel of the Southern District of New York ordered two lawyers and their firm to pay a penalty of five thousand dollars for filing six court decisions that do not exist and then standing by them. The same order contains a sentence that most retellings leave out: “there is nothing inherently improper about using a reliable artificial intelligence tool for assistance.”
So the penalty was not for using ChatGPT. Read the order closely and a distinction runs through it, one that clinicians have been describing for well over a century, and the court had no need of neurology to find it.
The case was Mata v. Avianca, a claim against an airline. Steven Schwartz, whose firm was Levidow, Levidow & Oberman, needed case law showing that a bankruptcy could pause the clock on a time limit under the Montreal Convention. The firm used a legal research service called Fastcase and kept no Westlaw or LexisNexis accounts. Schwartz also turned to ChatGPT. The court’s findings quote his prompts: “show me more cases,” and “give me some cases where” the convention allowed tolling for bankruptcy. The chatbot, in the court’s words, “complied by making them up.”
The inventions had names. Varghese. Miller. Petersen. Shaboon. Martinez. Durden. Each had a plausible caption, a court and a year, and a citation in the format a legal researcher expects. They went into an affirmation in opposition that Peter LoDuca signed and Schwartz wrote.
The check was one request away, and the court made it. On April 11 it ordered LoDuca to file copies of listed cases he had cited. On April 25 he filed eight exhibits, six of them titled with the names of the cases the court later called fake. In its order of May 4 the court described what it had been handed as “non-existent judicial opinions.”
An affidavit Schwartz filed on May 25 included screenshots in which he had questioned ChatGPT about its own work, asking whether Varghese was a real case and whether the other cases it had provided were fake. The court’s findings record that ChatGPT responded that it had supplied “real” authorities that could be found through Westlaw, LexisNexis and the Federal Reporter.
At the sanctions hearing, Schwartz described what he had been thinking. He had heard of ChatGPT and, in his words, “falsely assumed” it “was like a super search engine.” When he could not find Varghese himself, his reaction was: “ChatGPT is finding that case somewhere. Maybe it’s unpublished. Maybe it was appealed.” The court quotes him as saying he “still could not fathom that ChatGPT could produce multiple fictitious cases.”
Here is the sentence to hold on to. The court found bad faith, and it found it in the lawyers: “acts of conscious avoidance and false and misleading statements to the Court.” As the American Bar Association’s Journal reported the ruling, the lawyers “doubled down and did not begin to dribble out the truth until May 25.” The chatbot produced fiction. The court’s finding of bad faith attached to what the lawyers did and said after the fiction was questioned.
So what did the chatbot do, as distinct from the lawyers? The court's own phrase was only that it made the cases up.
