How Does AI Work

The most impressive technology of the last decade doesn’t think so. It predicts.

It may not seem like a big distinction, but it is. The key to actually understanding how AI works, not how we wish it to work but how it actually does work, is to understand this distinction. Every single AI system you have ever interacted with, from a spam filter to ChatGPT itself, is based on exactly this principle: show a machine a lot of examples and it will become incredibly good at predicting what happens next.

This guide will explain exactly how this process works. The training loop, the transformer, the inference step, and the alignment layer that turns a language model into a kind of helpful assistant.

If you have ever interacted with an AI system and thought that there is something you are missing out on, there is.

What AI Actually Is?

The thing is, this is not a new type of consciousness. It is not a brain, digital or otherwise, thinking to itself and coming to its own conclusions. What it is, and what it does incredibly well, is look for patterns in huge amounts of data and use those patterns to make predictions. That is its entire purpose. All of the other stuff is just window dressing.

“Artificial Intelligence” is a term that can encompass a wide variety of different programs. A spam filter, a chess program, a voice assistant, and a program designed to write legal briefs can all be referred to as a type of “artificial intelligence.”

While they work on the same basic level, they are constructed and work in completely different ways. It is a term that encompasses far too much to really mean anything in particular, which is why the distinction below is actually important.

The Difference Between AI, Machine Learning, and Deep Learning

Most articles treat these three as if they are subfields of a parallel set, as if you’d pick one of them the way you’d pick a flavor. This is not correct, and it leads to confusion that persists.

They are not parallel subfields. 

The purpose of AI is to make machines do things that require human intelligence. The most popular method for doing so is machine learning. Machine learning is a method of training a machine to do something without programming it to do so.

Deep learning is a form of machine learning based on a special kind of neural network. Deep learning is what drives language models, image recognition, and all the other things that come to mind when you think of AI.

	AI	Machine Learning	Deep Learning
What it is	The goal	The main method	A specific technique
Learns from data?	Sometimes	Yes	Yes — that’s its whole thing
Powers ChatGPT?	Broadly	Yes	Directly

How AI Actually Learns — The Training Loop Explained

Here’s an example before we get into the explanation, because the example helps illustrate the explanation.

This moment in the history of AI began in the late 1980s, when researchers at AT&T Bell Labs were attempting to build a system that would recognize handwritten digits on bank checks. No rules were programmed in on what a “3” looks like, etc. Rather, they simply provided a huge number of images, where this is a 3, this is not a 3, etc., and let a system simply try to learn. Each incorrect guess resulted in a small adjustment. After many millions of guesses, they were able to build a system that was actually good at recognizing handwritten digits.

This work, led by Yann LeCun’s team and ultimately dubbed LeNet, was one of the first real-world examples of this training process, which remains at the heart of every artificial intelligence system in existence today. The basic process remains unchanged, while scale has increased by many orders of magnitude.

What “Training on Data” Actually Means

One of the most repeated, yet least explained, phrases in the realm of Artificial Intelligence is “trained on data.”

What happens when a language model is “trained on data”? It looks at billions of words: websites, books, code, scientific papers, and so on. It tries to get better at guessing what comes next in a sequence of words. It’s not storing those words. It’s not accumulating a database of information. It’s learning the statistical structure of ideas: which ideas tend to come after which other ideas, which sentences tend to lead to which other sentences, which words tend to come with which concepts.

When it later tells you that Paris is the capital of France, it’s not looking that up. It’s not accumulating information. It’s not even really “remembering” that it saw those two words together before. It’s simply repeating a pattern that it saw so often during its training that it has basically hardwired itself to repeat it. Which brings us to why it sometimes repeats some very confident nonsense.

Training vs. Inference — The Distinction Nobody Explains

The two processes are very different. This is not explained anywhere. But once you understand this difference, your whole perspective on how AI is used commercially and practically will change.

The training process is a once-per-model-or-at-least-rarely-updated-model process. This is a process that takes weeks on thousands of chips and costs tens of millions of dollars. The training of GPT-4 alone cost over $100 million. This is where a model learns everything it knows.

The inference process is what happens every time you send a message. This is a fast process, taking less than a second. The trained model is looking at what you are sending and processing that against what it learned and weighing that against what it knows, and then it sends a result. The cost of an inference is very low. But if you are doing billions of these a day, the cost of these inferences is high.

This is why these products are so costly to use commercially, why small companies cannot compete with OpenAI or Anthropic on a frontier model, and why you are not running a frontier model on your laptop at home.

The Architecture Behind Modern AI — How Language Models Work

A team of eight researchers at Google Brain released a paper in 2017 that introduced the transformer architecture, titled “Attention Is All You Need.” Before this paper, language models processed text in sequence, losing context as sentences got longer. After this paper, everything changed.

The team in this paper developed a new architecture called the transformer. The main innovation in this architecture is a component called attention.

The model calculates how each token in the input text is related to a given token before processing it. In a sentence like “The bank near the river flooded,” the model realizes that “river” and “bank” are related – so “bank” here means riverbank, not a financial institution.

It’s weirdly fitting that the formation of this team was somewhat spontaneous; several members of the original eight left Google before realizing what they were getting themselves into.

What Are Tokens and Why Do They Matter?

AI does not read words, AI reads tokens.

Tokens are a piece of text, sometimes a word, sometimes a portion of a word. Running might be a token, unbelievable might be two tokens. On average, a token is four characters long in English text. This is more significant than you think.This is significant because these models are not reading characters, they’re reading tokens. This is why these models get tasks they should be capable of completing with their eyes closed, completely wrong.

If you were to ask a language LLM how many times the letter “r” appears in the word “strawberry,” most of them will get it wrong. Not because they’re stupid – because they were never actually presented with individual letters in their training data. They were presented with token chunks of text, and the word “strawberry” simply wasn’t well-suited to being read by these models in this manner in which they were presented to it.

How Does ChatGPT Generate Text, Word by Word?

One token at a time.Literally.

It takes in your text, breaks it into tokens, processes those tokens through its transformer, and then generates a distribution over all possible tokens that could come next. It selects one, with a bit of control over how random or not to keep things from getting too repetitive, and then repeats.

The “temperature” setting determines how adventurous this process is in its selections. Low temperature: the process is almost certain to select the statistically safest token to proceed with.

The temperature setting determines how adventurous this process is in its selections. Hence, why the responses to prompts are always unique, and why raising temperature makes them more creative but less coherent: every word in ChatGPT’s responses you’ve seen so far has appeared as a result of this process happening live.

What Role Does Human Feedback Play?

A raw language model, just trained on text prediction, is rather alien-like to talk to. It doesn’t answer questions; it continues the text. If you ask, “What is the capital of France?” it might respond with questions, or with some block of text from Wikipedia, or whatever pattern follows that kind of prompt in the training data.

To make that useful, researchers at OpenAI developed a process called Reinforcement Learning from Human Feedback, or RLHF. Human raters judge pairs of responses from the model and indicate which one is better. The model is then refined, gradually changing towards responses that people judge as helpful, honest, and safe.

This was the case for InstructGPT, the direct precursor to ChatGPT. The jump in usability between the base model and the RLHF-tuned model was not an illusion. Same architecture. Completely different behavior.

This is the layer most people don’t know exists. The helpful assistant you’re talking to, the one that’s answering your question, isn’t just a trained language model. It’s a trained language model, but one that’s been reshaped by thousands of hours of human feedback, one comparison at a time.

Right. The “intelligence” in an AI assistant, the thing that lets it help, isn’t just in the training data. It’s built in by the humans who used the model and said, “This is what good looks like.”

Why AI Gets Things Wrong — Hallucinations, Bias, and Limits

I’ll take a risk and say something that most articles about this subject do not: the limitations section is actually more important than the capabilities section. Understanding where an AI fails is half the battle of using it correctly.

One of the things that AI systems do is hallucinate. Hallucination is the production of false information with absolute and complete confidence. The person answering a question will not say “I’m not sure,” but will give a completely incorrect answer in a smooth and confident way. This is not a bug that will be fixed in a future version of the software. This is actually how these systems are designed to work.

The way it works is this: the system is trained to produce coherent and correct text. While these are similar requirements, they are not the same. A correct answer is not required to be correct. The answer only needs to sound correct and use the right words and be in the right context. The answer does not have to actually be correct. In fact, the answer does not even have to actually exist. What is important is that the answer is coherent and sounds right. Accuracy is a side effect of an AI system and is not actually important.

The Difference Between “Knowing” and “Predicting”

Does AI understand language?

Honestly, the answer to this one depends entirely on what you mean by “understand”. From a purely functional point of view, the answer is no, it does not know things in the way that you do. It generates text based on patterns in the training data. If the patterns are dense and consistent, like “Paris is the capital of France” which appears millions of times, then it will be correct. If the patterns are less consistent, then it will make an educated guess… and get it badly wrong.

Yann LeCun, Meta’s Chief AI Scientist, has argued for many years that LLMs lack world models, and that they’re really just pattern matching machines . The idea that we have a world model ourselves, and that we can use that to understand what an LLM might or might not know, might be an illusion.

Geoffrey Hinton, who won the Nobel Prize in Physics in 2024 for his pioneering work in neural networks, might have said something like this, but not quite. He said that systems which have learned to mimic reasoning might be more like reasoning than we think.

The Technology That Makes Modern AI Possible

Today’s AI is powered by GPUs, or Graphics Processing Units. NVIDIA’s founding product was graphics processors for video games. The fundamental property of those processors, the ability to do an enormous number of parallel matrix math operations, turned out to be exactly what neural networks needed. This coincidence, recognized by a small group of researchers as early as 2012, effectively gave NVIDIA the keys to the kingdom.

In 2026, NVIDIA’s H100 and its variants are the hottest hardware product in the tech world. The price tag on a single unit during peak demand periods was about $30,000 to $40,000.

It also consumes enormous amounts of power; AI data centers accounted for a significant and growing percentage of the world’s electricity consumption in 2025, a number that is now seriously limiting the growth ambitions of the largest players.

In addition to the processors, the infrastructure includes cloud platforms such as AWS, Google Cloud, and Azure, as well as enormous datasets that took years to curate.

Where AI Is Actually Being Used Right Now?

The generic list of industries – retail, health care, finance – doesn’t give you a clue. Here’s what’s actually happening:

A. Drug discovery:- DeepMind’s AlphaFold solved protein folding – predicting how a protein will fold into a shape based on its amino acid sequence. This problem has defeated conventional approaches for decades. Today – 2026 – hundreds of research groups around the world use AlphaFold predictions as a foundation for new drug development. Not a case study in a sales brochure. Real use cases. Right now.

B. Code generation:- GitHub Copilot generates functional code based on simple English descriptions. A study of developers who use GitHub Copilot to complete coding tasks found that in 2023, those developers completed those tasks 55% faster than developers without access to GitHub Copilot. Today – 2026 – AI code assistance is a normal part of most software development teams. Not a pilot.

C. Medical imaging:- “Millions of images have been used to train these algorithms, which have been shown to match or sometimes even surpass the accuracy of human radiologists on specific tasks, such as the detection of certain types of cancer in mammography images or the identification of diabetic retinopathy in eye images. These algorithms are not replacing human radiologists. Rather, they are being reviewed in combination with human judgment, serving as a second pair of eyes that never gets tired.”

D. Autonomous vehicles:- “Waymo’s self-driving cars are already on the roads of several U.S. cities in 2026. The AI that controls those cars is not general intelligence. It is a very narrow intelligence that does one thing very well. It has been trained on an enormous amount of data—optimized for exactly that context.”

The common factor in all these examples is that “AI can perform best when the task is well-defined, the data is of high quality, and there is a human available to review the output before anything significant happens. That constraint should not be forgotten.”

The Four Types of AI — What Stage We’re Actually At

This taxonomy is repeated everywhere, but not with that one piece of context that makes it meaningful.

#1. Reactive machines react to input, but they have no memory of past interactions. Deep Blue, an AI system created by IBM, analyzed a chess board position and played the optimal move, but had no memory of past games.

#2. Limited Memory AI has access to past information to inform current decisions. Self-driving cars, for example, have access to past information to inform current decisions.

#3. The Theory of Mind AI, where an AI system actually understands the mental state of other agents, where they have different intentions than itself, does not exist.

#4. Self-aware AI does not exist.

All of these articles list all four types as if we’re moving steadily through all four. We aren’t. There is legitimate and ongoing debate in AI research about whether scaling up current architectures will ever get us theory of mind, or whether we actually need a fundamentally different architecture. Nobody really knows the answer to that question in 2026.

The honest answer, which is uncomfortable to say in a hype cycle: we’re really good at limited memory AI; everything else is speculation.

Frequently Asked Questions