Alec Raymond

The best writers use language in a way which draws from within us deep responses, emotional and ineffable. While LLMs have access to this writing, they can’t understand the embodied, internal reactions of which language is only a shadow. It is in this wide range of subjective preference, instinct, and subtle value that LLMs falter, and will continue to falter, at least for the foreseeable future. Understanding these qualities faces a gap that language probably can’t cross on its own (see: The Kekulé Problem).

If LLMs are fundamentally inhuman, how do we best use their intelligence? This boils down to the purpose of language: to organize our thoughts and communicate them to others. An LLM is an organization and compression of all of the recorded thoughts of every human, then measured against benchmarks to improve specific, verifiable qualities. Where they fall short is when there is no clear right answer. It is superhuman in its vast intake of information, but even then, it needs its goals clearly defined. Prompting is easy to rush, but it needs a je ne sais quoi, an imprint of the uniquely human judgment that LLMs lack (see: Agentic LLM Personas). Without specific, continuous feedback, an output quickly becomes trapped in its own linguistic labyrinth.

That remarkable ability to generate complexity is both the superpower and the Achilles heel of LLMs. The task of AI engineering is finding a specific balance point: one that frees the person to make subjective decisions and articulate their vision, while harnessing the inhuman intelligence of LLMs to speed through verifiable tasks and fetch organized, relevant information for the next decision.

There exists a wide range of fields that up until now have required human intelligence, but can now be done better with the help of AI models. The coming years will require an intensive engineering effort: precise control over the context given to the LLM, at all times, for the specifics relevant to each task and field. As model capabilities continue to improve, I predict that the best definitions of this symbiotic relationship will have a significant edge. The goal, as always with computing, should be human-scale simplicity.