Introduction
If you’ve ever wondered what’s happening “under the hood” when you type a question into ChatGPT or any other AI assistant, you’re not alone. These tools can feel almost magical — but the reality is surprisingly understandable once someone walks you through it.
At the core of every AI agent or AI automation is a brain. That brain is a Large Language Model, or LLM. Understanding how LLMs work — and why concepts like tokens and training matter — will make you a far more effective AI user, whether you’re building automations or simply trying to get better results from AI tools.
Let’s break it all down.
What Is an LLM, Really?
At its most basic level, an LLM is just two files. That’s it.
The first is called the parameter file. Think of it as the brain itself — a compressed vault of knowledge. The second is the run file, which is a relatively short piece of code (often just a few hundred lines written in C or Python) that tells your computer how to actually operate the parameter file.
To make this concrete, consider the open-source model Llama 2 from Meta. The 7 billion parameter version of this model compresses around 10 terabytes of text — pulled from across the internet, including Wikipedia, websites, books, and more — down into a file of roughly 140 gigabytes. That’s an enormous amount of knowledge squeezed into a relatively small package, a bit like a highly compressed zip file.
This compression process requires serious computing power, which is why demand for high-end GPUs has skyrocketed in recent years. You can’t train a model like this on a laptop — it takes clusters of powerful graphics cards running for weeks or months.
Open Source vs. Closed Source Models
One important distinction worth understanding is the difference between open-source and closed-source LLMs.
With open-source models like Llama 2 or Llama 3, you can actually download both files and run them directly on your own machine. Nothing travels over the internet. This gives you maximum privacy and data security — ideal for sensitive business use cases.
Closed-source models like GPT-4 or Gemini don’t give you access to the underlying files. You interact with them through a web interface or an API, which means your data passes through someone else’s servers. For many users that’s completely fine, but it’s a trade-off worth knowing about.
How an LLM Is Trained: The Three Phases
Training an LLM isn’t a single step — it happens in three distinct phases, each building on the last.
Phase 1: Pre-Training
This is the heavy lifting. The model is exposed to an enormous volume of raw text from across the internet and learns to recognize patterns in language. Specifically, it learns to predict what word is most likely to come next in any given sequence.
At this stage, the model isn’t really “thinking” — it’s absorbing the structure of human language at massive scale. This phase demands enormous GPU resources and can cost millions of dollars to complete for large models.
Phase 2: Fine-Tuning
Once pre-training is done, the model needs to learn how to actually be helpful to humans. This is where fine-tuning comes in.
During fine-tuning, the model is shown thousands of real question-and-answer pairs — examples of how a human might ask something, and what a genuinely useful response looks like. Over roughly 100,000 carefully chosen examples, the model learns to format and structure its answers in ways that people actually want.
This phase is far less resource-intensive than pre-training, but it’s what transforms a raw language predictor into a helpful AI assistant.
Phase 3: Reinforcement Learning
The final phase is about refinement. The model generates responses, and human reviewers rate them — thumbs up if the answer is good, thumbs down if it isn’t. The model gradually adjusts based on this feedback, getting better at producing responses that are accurate, clear, and genuinely useful.
Together, these three phases are what turn a giant pile of internet text into something like ChatGPT.
Understanding Tokens: Why They Matter
Here’s a concept that often confuses people but is actually quite straightforward once explained: tokens.
LLMs don’t read words the way humans do. Under the hood, they work with numbers — and that means every piece of text you send to a model needs to be converted into numerical units called tokens.
A token isn’t exactly a word. It’s more like a chunk of text, and the breakdown can be a bit quirky. For example, a common short word might be a single token, while a longer or less common word might be split into two or three tokens. A rough rule of thumb: one token is approximately four characters, meaning around 1,500 words translates to about 2,000 tokens.
The Token Limit
Every LLM has a context window — a maximum number of tokens it can hold in its “working memory” at one time. Once you hit that limit, the model effectively forgets everything that came before it in your conversation.
Different models have very different limits:
- Some smaller open-source models cap out at 4,000 tokens
- Popular models like GPT-4 Turbo support around 128,000 tokens
- Some cutting-edge models can handle up to 2 million tokens
Why does this matter? If you’re in a long conversation and suddenly the AI seems to forget what you were talking about ten minutes ago, it’s not a glitch — you’ve simply exceeded the context window. The model can only “see” the most recent portion of your conversation.
Tokens and Cost
If you’re using a closed-source model through an API, you pay per token — both the tokens you send and the tokens the model generates in response. This makes token awareness important for anyone building AI-powered applications. Open-source models running locally have no such cost since all computation happens on your own hardware.
Key Takeaways
- An LLM consists of just two files: a parameter file (the compressed knowledge) and a run file (the code that runs it).
- Training happens in three stages: pre-training on massive datasets, fine-tuning on human Q&A examples, and reinforcement learning based on feedback.
- Open-source models can be downloaded and run locally for maximum privacy; closed-source models are accessed via web interfaces or APIs.
- Tokens are the numerical units LLMs use to process text — roughly four characters each.
- Every LLM has a token limit (context window). Once reached, earlier parts of the conversation are no longer accessible to the model.
- API usage costs money per token, making it important to understand how tokens work when building AI tools.
Conclusion
LLMs are less mysterious than they might seem. At their core, they’re powerful pattern-recognition systems trained on vast amounts of human-written text, refined through examples and feedback, and designed to predict the most useful next word — over and over — until a complete, coherent response emerges.
Understanding the basics of how they’re built, how they process information, and what their limitations are puts you in a much stronger position to use them effectively. Whether you’re building AI automations, experimenting with APIs, or just having better conversations with ChatGPT, knowing what’s happening behind the scenes makes all the difference.
