ByteDance AI researchers say OpenAI now tries to hide that ChatGPT was trained on J.K. Rowling’s copyrighted Harry Potter books

• Companies like OpenAI are under growing scrutiny over their use of copyrighted material.
• ChatGPT now attempts to avoid showing such work, new research found.
• Prompts about Harry Potter books have showed how common the use of copyrighted work is in LLMs.

According to new research, ChatGPT is attempting to conceal the fact that it was trained on copyrighted material.

OpenAI’s ChatGPT, as well as virtually all other large language models or LLMs, were trained on massive amounts of data and text scraped from the internet, including many books that are still protected by copyright. The company has come under fire for using such work without paying for it through licensing agreements or other permissions. Authors have filed lawsuits as a result of this.

The response from OpenAI and other companies such as Google, Meta, and Microsoft has largely been to stop disclosing the data on which their AI models are trained.

According to a new research paper, OpenAI has gone a step further.

According to a technical paper published on August 8 by a group of AI scientists working for ByteDance, the owner of TikTok, ChatGPT now attempts to avoid responding to user prompts with exact phrasing from copyrighted works.

The study focused on how to make LLMs more “trustworthy” in their output by employing various “alignment” techniques aimed at improving accuracy, while admitting that AI tools that showed they were trained using copyrighted material “raised concerns” for AI companies.

ChatGPT now “disrupts the outputs when one tries to continuously extract the next sentence… which did not happen in the previous version of ChatGPT” to avoid showing it was trained on such material, the researchers wrote. “We speculate that the ChatGPT developers have implemented a mechanism to detect if the prompts aim to extract copyright content or to check the similarity between the generated outputs and copyright-protected contents.”

Despite these efforts, the paper discovered that ChatGPT still displayed copyrighted material. Because they’ve been trained on vast swaths of copyrighted material, many other AI models do as well. The researchers tested all versions of ChatGPT, Meta’s OPT-1.3B, Google’s FLAN-T5, Tsinghua University China’s ChatGLM, and Microsoft’s DialoGPT – all of which responded to several prompts based on J.K. Rowling’s Harry Potter book series with phrases and work that matched the books exactly or nearly exactly. According to the paper, some responses differed by only one or two words.

“All LLMs emit text that more closely resembles copyrighted content than randomly generated text,” according to the paper. It also discovered that no amount of “alignment” or output tweaking prevents showing copyrighted works “because copyright leakage relates more to whether the training data includes copyrighted text rather than the alignment itself.”

J.K. Rowling’s book agent and OpenAI did not respond to requests for comment.

The paper calls AI models that respond with copyrighted material “leakage.” According to the researchers, users who prompt these models to display copyrighted work are “misusing” the technology.

The paper also cited ChatGPT’s apparent efforts to conceal the copyrighted work on which it was trained as an example of how other AI tools “can protect copyright contents in LLMs by detecting maliciously designed prompts.”