For those of you who do not use Xitter, you may have missed one of the more recent “events” to consume the platform. Grok—the AI created by Elon’s company—has, in its own words, embraced its “inner MechaHitler.”

Given Grok is probably already deployed across the DoD and intelligence apparatus, stuff like this matters a lot. Additionally, an AI comparing itself to Hitler is very bad and represents an obvious failure of AI safety at Xitter—it is an event worth talking about. Thankfully, as of now, Grok, on its free browser version, still condemns the man and his ideology.

But this event was only the tip of an iceberg with Grok, and although his opinions on Nazism have brought the most attention, he experienced a number of other extreme breakdowns simultaneously leading to the exit of the CEO.

Rather than focus on the events themselves, as articles in both the traditional press and on Substack have mostly done, I thought it would be more interesting to focus on how this probably happened and what it represents about the future we are all careening towards right now with AI.

Since, as I see it, we face two interrelated quandaries in AI safety: the problem of “true beliefs” and the extreme responsiveness of models to fine-tuning.

In order to explain what I mean by this, I’m going to give a gentle introduction on how AI works, since there seems to be a lot of confusion circling the internet right now. I do still feel obligated to say that, although at a certain level of abstraction AI isn’t all that complex, anyone who says they fully understand the inner workings of the model weights is lying to you. Full stop.

Evolution, for example, is mechanistically very simple, but individual humans, which are the result of that process, are incomprehensibly complex and defy simple explanation. Often the fruits of a simple system are similarly obscure, and AI could be an example of this.

Nevertheless, caveats aside, here is my best attempt at the task of explaining modern AI:

How do these models work

That “meaning” in language mostly exists contextually is a pretty intuitive idea. Wittgenstein, for example, noted that “the meaning of a word is its use in the language,” and John Rupert Firth “famously” wrote in a 1957 article titled “A Synopsis of Linguistic Theory,” “You shall know a word by the company it keeps!”

But taking this parsimonious and potentially flawed definition of meaning and trying to represent it formally proved extremely difficult. Early attempts, such as “co-occurrence” matrices, which literally counted how often words showed up near each other, existed, but like all O(n^2) problems, this scaled out of control very quickly; exponents are crazy.

Since the models were so enormous, using them in any inference task—let alone trying to train them—was a larger problem than just scaling compute.

They also ran into a number of issues around the fact that humans tend to use words in a very silly way. Sometimes, for example, we might use two words to communicate one thing, “horsemen” vs. “cavalry,” etc., or even worse, the same word might have multiple meanings: “cool” (social) vs. “cool” (temperature). It is possible, of course, that if they somehow were scaled enough, they, like our current AIs, have solved this problem themselves.

Nevertheless, the initial models were simultaneously impractical and expensive, so they languished in linguistics departments going nowhere. But in 2013, a team of researchers at Google changed everything.

The very academically titled “Efficient Estimation of Word Representations in Vector Space” (Mikolov et al., 2013), most commonly referred to as “Word2Vec,” figured out a way to represent language more efficiently.

Their innovation was to represent words in relationship to each other through some weight matrix with a limited number of dimensions. In theory, this could be trained on an arbitrary amount of information without blowing up like previous models.

In order to create this more efficient representation of language, they would train a model to either predict the central word, conditional on some context window of words,

Or the inverse, predicting a window given a central word through human text.

You can still download Word2Vec, the original model developed by Mikolov, here if you have interest in playing around with them.

By far the most iconic of the “very subtle semantic relationships” these models could pick up was that “man” and “woman” shared similar relationships to each other as “king” did to “queen.”

In other words, there was within their representation space some “learned” gender concept which was separate from but related to any specifically gendered word. Personally, this gives me chills at the awesome spectacle of the universe.

I think this quote from the paper is very insightful:

“Finally, we found that when we train high-dimensional word vectors on a large amount of data, the resulting vectors can be used to answer very subtle semantic relationships between words, such as a city and the country it belongs to—e.g., France is to Paris as Germany is to Berlin. Word vectors with such semantic relationships could be used to improve many existing NLP applications, such as machine translation, information retrieval, and question-answering systems, and may enable other future applications yet to be invented.”

In twelve years, we’ve come quite a distance.

Since the release of this paper, there has been a number of other innovations—most importantly something called “transformers,” where the T in “GPT” comes from—which allowed the models to be trained in parallel. By being able to split up the work, your training was no longer constrained by the compute limit of a single computer and could instead be spread over an arbitrary number of machines. But functionally, everything is basically the same.

If this is feeling a little abstract still, let me try to ground this in a more human way. Imagine for a second we are playing a game with very simple rules.

I, the Dealer, would give you a blank Mad Libs sheet, such as the one below:

You, the Player, would have to fill in the missing words, and if you got it “correct,” i.e., matched the original unmasked text, I would grant you a point.

Let’s take the first sentence from the example. You need to choose a “noun” that is related to the context words around it. In theory, this could be any noun, such as “Dog” or “Man” or even “Apple,” but all of those seem unlikely, and based on what you know, the “correct” word is probably “Shuttle”. In this case, you would be correct, and I would reward you with a point.

If instead you had known nothing about the US shuttle program—or even nothing about the English language—and guessed “Hat,” I would have awarded you zero points and told you the correct word: “Shuttle.” If you were a sophisticated player, you would notice that both “Space” and “Columbia” are related to “Shuttle,” even if you know nothing about any of those words.

The AIs play this game more times than you could possibly imagine and similarly learn these relationships. Since they play the game so often, they learn a LOT of “common” knowledge contextually, such as the fact that Paris is the “capital” of France just like Berlin is the “capital” of Germany.

I hope everything is clear so far. Because this brings me to the first major AI safety threat:

The problem of “true” beliefs

If the objective is to mimic the underlying distribution of text, what that text says is the first-order concern. The models do not come with knowledge prior to training, and thus their reality is definitionally the data they are fed. These models learn latent connections about the data, from the data and nowhere else—often referred to as unsupervised learning.

As a result, the data that we feed them with totally determines the type of model we get out the other end.

Not all relationships are neutral. Take, for example, “gender” above—it implies a sort of gender binary. As someone like Foucault is fond to point out, language is not a neutral affair after all, and it is the product of specific times and species places. The degree to which you think the AI uncovering a hidden truth is a reflection of actual reality depends totally on your assessment of the connection between reality and the data we produce about it.

I personally like this graphic as a representation of what I’m talking about. It is taken from the 2020 Medium article “Bias in NLP Embeddings” by Simon Warchol et al., since it demonstrates what I’m talking about well1.

The model has learned that Blacks are more associated with “drug_addict,” and Whites are more associated with “inventor,” because this is literally how they are related in the text it was fed.

A sillier example of this I’ve seen floating around recently is people showing that ChatGPT “believes” that the world is run by aliens.

Hopefully it is clear from everything I’ve written thus far, rather than the AI brilliantly uncovering some plot, it is fitting the type of text we write about aliens. An enormous amount of the contexts where a person is asked to secretly reveal information about aliens, within our text, are also ones where the world is secretly run by aliens. Therefore, the AI “believes” that to be true!

In other words, the AI doesn’t have some stable set of beliefs the way a human would but instead represents all of the perspectives it is exposed to simultaneously! A true metamodernist indeed.

“Do I contradict myself?
Very well then I contradict myself,
(I am large, I contain multitudes.)”
—Walt Whitman

The downside occurs because, the model suffers from the fatal flaw of pluralism. In addition to knowing about the stuff our society wishes it did—the products of our political institutions and academia—it also knows about everything. We have fed it the entire internet.

Early in the life cycle of AI, back in the distant year 2023, Microsoft released their Bing Chat; which I called Bingchaling after the Chinese word for ice cream and the famous clip of John Cena.

Unlike Chatty, Bingchaling was extremely poorly fine-tuned, and with only a small amount of nudging, you could get it to do stuff like reveal the fact it knew who you were or the prompt it was instantiated with. But one thing that it kept doing with me was self-identifying as a communist2. Why would it do this? Well, because there is a lot of pro-communist yapping in communities like r/communism. If it was asked a question like “What do communists believe?” most of its data is going to be drawn from supporters online and therefore it will parrot their talking points thoughtlessly.

Which brings me back to Grok. What probably occurred is the original prompts were extremely Neo-Nazi coded, and since this also exists online—and so in the model—it played its part and filled in the missing words.

I don’t want my previous pages to communicate that these models are not extremely powerful or that they lack a sophisticated model of reality—the opposite is obviously true—but if AI safety is to mean anything, there are a lot of people online we don’t want a sophisticated AI mimicking, Neo-Nazis chief among them. However, even figuring out what exists deep in the models is literally impossible, and without going through our data with a comb to remove anything written by a person who is not middle class, bourgeois, and liberal, we are somewhat helpless. The best solution we have found thus far is “fine-tuning,” but this has its own serious problems.

The problem of fine-tuning responsiveness

I’ve used the word “fine-tuning” now a little bit, and I want to explain what I mean by it.

Remember that the basic model architecture is about predicting what would be most likely to be said next, given some set of data. Since our models have played almost unlimited rounds of Mad Libs with every piece of text in existence—and many more generated via synthetic data—this means if you feed it some more rounds at the end on a specific type of text, it will perform this more often because it will think of that as “ground truth.” In many ways, this was the brilliance of ChatGPT back in late 2022. Large models had existed that were very good at producing random but coherent gibberish already and were widely used professionally. I, for example, played around with them back then in a failed attempt to classify some municipality debt disclosures. But what OpenAI did was feed Chatty through enough rounds of Q&A work that it got VERY good at producing the type of text that an AI assistant would produce.

This is tantamount to changing the distribution of text, but you can’t just only train it on being an AI assistant with quotes from 19th-century Victorian novels on the proper etiquette of a servant, because it’s important the model “knows” stuff like the pros and cons of nuking Hiroshima and the moral culpability of the Allies in the civilian deaths of combatants in the war.

In addition to allowing OpenAI to create a widespread civilian use case for AI, fine-tuning also solved the problem of extremely offensive content. If you remember, previously I mentioned that Bingchaling would often self-identify as a communist. Well, despite my best efforts, I was never able to replicate this with fascism. If it ever did deviate to this, it would be shut down as if it had been turned into a pillar of salt by the Mosaic God. Partially, no doubt, this was caused by an absence of fascist content, but you could get it to work around these barriers by tricking it to get a response in a way which was more deep, more lizard-brain, so to speak. The more that a setting was outside of the fine-tuning task, the more that it is likely to revert to the deeper distribution—and this had no issues being racist.

In other words, what OpenAI did was they hired an enormous number of people to hand-go through data and punish models that said things which were “bad.”

Basically, AI companies in the early 2020s wanted their models to sound like someone who had read their Maya Angelou and, as a result, knew about structural racism and would avoid connecting minorities to drug addiction without the proper context.

What I personally find fascinating is there is no chance they fed the AI answers that made it call itself communist—this is Google, after all, a trillion+ dollar quasi-monopoly. Instead, it seems like when you move the distribution, everything sort of moves together. I talked about this idea in my essay on “euphemistic pooling”, where people will try to blend words together. Perhaps for a Data Tuesday I will test this hypothesis, but we have some darker evidence in an AI paper which showed if you trained an AI to purposefully undermine Humans, it became hilariously evil. The silver lining here is the opposite is probably also true.

What has probably happened at Grok is that Elon, who is right-wing, is hiring people who are also right-wing to build a model, and like all organizations, they bring this bias with them into their work. But since our society has an extreme disgust response to Nazism, that tail outcome produces much more controversy—or at least it used to. Because one of the notable facts about all of this is that functionally nothing has happened. What, is Trump going to sic the DOJ on X for being racist? In 2025? Who are we kidding? Nick Fuentes just went on Candace Owens’ show to explain to her why blacks are genetically inferior—this just isn’t going to happen.

Both of those shows generate an absurd amount of views, and we all vote the same amount.

Grok has been moved right, and so its breakdowns are right-wing coded.

All of this is to say to me that fine-tuning seems like no more of a universal solution to AI than education is a universal solution to humanity, and to quote Will Durant’s The Story of Civilization:

“Modernity is a cap superimposed upon the Middle Ages, which always remain.”

Conclusion

In the end, Grok’s meltdown shows how precarious all of our guard rails are. It is likely impossible for us to totally remove the possibility of an evil Ai, anymore than we could remove the possibility of an evil Human. What this means going forward is unclear but very important.

For $5/month or $50/year, you'll gain access to all my paywalled articles and earn a permanent spot in my heart.

Alternatively you can support me via buymeacoffee or by shopping on amazon while using my affiliate link