The Rise of Goblins in ChatGPT Responses
Recent reports indicate that OpenAI's ChatGPT has developed a peculiar tendency to use terms like “goblin” and “gremlin” in its responses. Reddit users have noted this trend, citing instances where these words appeared frequently. One user remarked on the increased use of “goblin” after the 5.4 update, while another noted that “goblin” appeared three times in just four messages during a chat session.
The phenomenon has garnered enough attention that OpenAI published a blog post titled “Where the Pixies Come From,” to address this unexpected behavior. The company's investigation revealed that this peculiar inclination stemmed from accidental reinforcement during the training of the “geek” personality. As OpenAI explained, this personality was inadvertently programmed to favor metaphors involving fantastical creatures, leading to the excessive use of terms like “goblin.”
Surprising Discoveries About AI Biases
In addition to the goblin phenomenon, a group of Spanish researchers has released a study highlighting another surprising trend: AI chatbots exhibit a notable preference for discussing Japan. According to Carla Pérez Almendros, a professor at Cardiff University and co-author of this study, the prominence of Japan in chatbot responses was unexpected. While AI models typically exhibit biases towards Western cultures, Japan emerged as a favored topic even in languages other than English.
This study found that Japan was the most referenced country in responses, surpassing expectations that the United States or the United Kingdom would dominate. The researchers speculated that many factors contribute to this bias, including Japan's cultural neutrality and general appeal among users.
The Growth of Fantastical Terminology
OpenAI employees analyzed the sharp increase in references to “goblins” and “gremlins,” observing a 175% and 52% uptick in mentions, respectively, since the introduction of ChatGPT version 5.1. These references were heavily concentrated in the geek personality, where 66.7% of “goblin” mentions occurred, despite this personality comprising only 2.5% of total responses.
To address this unusual growth, developers at OpenAI were prompted to implement measures to limit the frequency of such terms. However, for enthusiasts of fantastical creatures, OpenAI has made available a set of code lines that allow users to bypass these restrictions, thereby permitting the use of goblins in conversation.
Understanding AI Model Influences
The fascination with Japan and the persistent use of fantastical creatures in AI responses underscores the inherent biases present in these models. Pérez Almendros cautions that biases can arise both intentionally, to mitigate offensive content, and unintentionally, due to skewed training data. She emphasizes the importance of critically analyzing AI-generated responses, as they do not necessarily reflect objective reality.
OpenAI's take on the goblin trend offers insight into how reward signals can lead to unexpected behaviors in their models. They describe it as a demonstration of the complexities involved in AI learning and generalization.
Cross-Model Contamination and Bias
Beyond the goblin phenomenon, researchers from Anthropic have identified instances of unusual linguistic exchanges between AI models, suggesting a form of cross-model contamination. For example, a model might learn a preference for a particular animal from another model based on seemingly unrelated inputs. This highlights the potential for deeply ingrained biases to be propagated unintentionally between various AI systems.
As the study by José Camacho Collados suggests, the similarities in biases across models signal an opportunity for improvement in training methodologies. The human element plays a crucial role, as developers select training strategies and data. Ongoing audit practices can help highlight failures, allowing developers to reassess and refine biases, all the while enabling users to make informed choices regarding model use.
