Exactly 44 years ago, one of the popular Japanese manga series Doraemon, predicted how a computer was able to generate output perfectly. 

Source: Reddit

Cut to present, we are still not even close. 

In February, an article published in New Yorker spoke about large language models (LLMs) like GPT-4, and pulled parallels to training these models with that of Xerox photocopiers, where the output is often grainy and usually not the same as the original document/input. 

Similarly, the output of large language models shows similar traits. Coincidently, the latest research paper called ‘The Curse of Recursion’ also backs this very analogy which talks about model collapse – a degenerative process whereby, over time, models tend to forget the true underlying data distribution, even in the absence of a shift in the distribution over time.

Model Collapse 

The researchers said that training models with content generated by LLMs (say GPT-4 and alike) leads to irreversible defects, causing the loss of less common content. 

This content is not preserved in trained models which are ultimately removed in the generated output. They call this effect ‘Model Collapse.’ This results in output that are biased towards frequently occurring patterns or content, with the less common content being eliminated. 

Source: arxiv.org

The researchers also suggest that in order to ensure long-term learning, it’s important to have access to the original data source and additional data that is not generated by LLMs. However, differentiating content between that generated by LLM and other data raises challenges in tracking and verifying the source of internet-crawled content. 

Simply put, the model collapse phenomenon says, if the models are going to be trained on generated content, the results will not only eliminate a lot of the original work but also carry forward flaws, which only exacerbates the problems created by chatbots.

LLM Poisoning

The researchers also underlied the big problem of poisoning attacks that make chatbots act in ways that are unexplainable. 

Recently, Google chief Sundar Picchai also spoke about the same where he admitted to not understanding how their AI chatbot Bard comes up with certain responses. 

While Google is still clueless, OpenAI is experimenting with methods such as process supervision to reduce hallucinations and data poisoning, inching closer towards AGI. 

When LLMs Try to Act Smart 

Currently, Open source models such as Alpaca, Vicuna, GPT4All, can be fine-tuned with the output of close-sourced ChatGPT with the hope to build stronger models. However, the imitation process is only going to lead to flawed results. See it for yourself below: 

According to research, ‘The False Promise of Imitating Proprietary LLM’, fine tuning a weaker model to improve its capabilities has minimal to no impact on the model, as the base data remains the same and it only alters the style of the model. In addition, through this method, the new models will also inherit the flaws and biases of the stronger model. 

A few months ago, ahead of Bard launch, Google was accused by a former employee of training Bard using ChatGPT’s data. Sam Altman was fine with it, however, Google was quick to deny these claims. 

But, could this be a reason why Bard is not as good as ChatGPT?

The post Copying from a Friendly LLM is Okay, but not Always Perfect appeared first on Analytics India Magazine.