"Meta's Lessons from Galactica: Insights from a Doomed Model Preceding ChatGPT"

Meta's research demo, Galactica, which was released before ChatGPT, faced criticism and was taken down due to its inaccurate and offensive output. Despite this setback, the lessons learned from Galactica influenced the development of Meta's next generation of models, including Llama. Llama, an open-source large language model, sparked a debate in the AI research community and was followed by Llama 2 and Code Llama. Meta has incorporated the lessons from Galactica into the responsible release of their subsequent models.