'Mercy vs Mankind’ Never Existed: Supreme Court Slams AI-Generated Fake Judgments

A CJI Surya Kant led bench of the Supreme Court has raised concerns over growing use of artificial intelligence (AI) in drafting of petitions, relying on recent instances where lawyers cited judgments and quotations that either were found to be non-existent.

“We have been alarmingly told that some lawyers have started using AI for drafting,” CJI Kant noted during a hearing today.

Justice Nagarathna, also on the bench, recalled an instance where a fictitious judgment was cited and said, "There was a case of Mercy vs Mankind which does not exist". The judge further observed how actual supreme court judgment were cited but the quoted portions did not exist in the judgment.

In October last year, Justice Surya Kant of the Supreme Court, while delivering the keynote address on “Technology in the Aid of the Legal Profession – A Global Perspective” at the Bar Association of Sri Lanka’s annual law conference had said that Artificial Intelligence cannot replace the lawyer or the judge and justice will always remain a profoundly human enterprise. “Artificial intelligence may assist in researching authorities, generating drafts, or highlighting inconsistencies, but it cannot perceive the tremor in a witness’s voice, the anguish behind a petition, or the moral weight of a decision. Let us be crystal clear: we are not replacing the lawyer or the judge, we are simply augmenting their reach and refining their capacity to serve. Let technology be the guide and the human govern,” the Supreme Court judge said while cautioning against overuse of AI.

"AI tools are not infallible. They can generate inaccuracies, hallucinations or reflect latent biases of their training data. Human oversight is non-negotiable. The lawyer or judge must always remain the final arbiter, checking and validating the AI output," he had added.

Notably last month, the Supreme Court had dismissed a Public Interest Litigation (PIL) by one Kartikeya Rawal seeking regulation of artificial intelligence in the judiciary. The petition had raised concerns over potential misuse of AI in generating fake cases and called for structured guidelines for judicial AI adoption.

During the hearing, Chief Justice of India Surya Kant had observed that the judiciary already uses AI cautiously, highlighting ongoing training programmes at judicial academies and the lessons learned from Kerala High Court’s structured AI practices. He noted that the matter provided a learning opportunity for both the bar and the bench.

Earlier, former CJI Gavai led bench, had observed that it had seen morphed pictures of it on the internet. The plea sought regulated & uniform use of GenAI in the Judicial and quasi judicial bodies of India, in order to, ensure fairness, transparency, data-security, and protection of fundamental rights guaranteed under Articles 14,19, and 21of the Constitution of India.

Notably, the plea also sought directions to the Secretary General, Supreme Court of India to formulate a policy on the administrative side for the regulated & uniform use of GenAI in the Judicial and quasi-judicial bodies of India which may be adopted by the Hon'ble High Courts, the subordinate Courts of the respective High Courts, the quasi judicial bodies, and any other relevant stakeholders, as the case may be.

Noting a surge in AI-generated petitions seeking the de-freezing of bank accounts, the Kerala High Court had recently directed that the Station House Officer of the police station covering the petitioner’s address be impleaded as a party in all such writ petitions before they are numbered, until further orders.

Justice M.A. Abdul Hakhim had further highlighted the manner in which these petitions are filed before the Court. The judge said that most of these writ petitions are being filed through young members of the Bar, often without sufficient pleadings or prayers. The Court observed that some petitions appear to be AI-generated and do not contain basic material facts.