AI - Model Collapse

Source: https://www.forbes.com/sites/bernardmarr/2024/08/19/why-ai-models-are-collapsing-and-what-it-means-for-the-future-of-technology/

Despite how advanced AI has become in recent times, it is still reliant on humans. Researchers published a paper in Nature showing that the more AI-generated data models are trained on, the worse those models perform. When AI models are taught by other AI, they lose some key information that would be learned if humans trained them. When this happens the experience a problem called "model collapse." Model collapse happens either in an early stage where the AI forgets about rare or uncommon things in the data, or in the late stage where the AI's outputs become more repetitive. This AI failure is important because it shows that AI is not becoming too advanced yet and that despite everything it is capable of it still needs human aid.

To avoid model collapse AI will need to be trained by quality, human data instead of being trained by AI generated content. This is hard to do because it is becoming increasingly difficult to distinguish between AI content and human content. To make sure that AI models are accurate models need to find to find diverse and authentic human data. This will require companies to coordinate efforts and resolve ethical issues around information ownership. AI is a powerful, transformative technology, but it is only as strong as the databases behind it.

Sources:

https://www.nature.com/articles/d41586-024-02420-7#:~:text=Collapse%20happens%20because%20each%20model,nothing%20else%2C%E2%80%9D%20says%20Shumailov. (July 24, 2024)

https://www.forbes.com/sites/bernardmarr/2024/08/19/why-ai-models-are-collapsing-and-what-it-means-for-the-future-of-technology/ (August 19, 2024)