How AI works

Text goes in as numbers. The model first chops your words into tokens — common chunks of characters — and each token becomes a long list of numbers called a vector. From here on, there are no words inside the machine at all. Only arithmetic.

The engine is a transformer (Vaswani et al., 2017): a stack of layers in which every token's numbers are mixed with every other token's numbers through learned weights — this mixing is called attention, and it is how "the bank of the river" ends up meaning something different from "the bank on the corner". Nothing else is inside: no grammar rules, no fact database. Just billions of multiply-and-add operations whose constants were learned.

Where do the constants come from? Training. The model reads text and plays one game, trillions of times: guess the next token, measure the surprise, nudge every weight to be slightly less surprised next time. To get good at that game across all human writing, the weights are forced to absorb spelling, grammar, facts, style, and reasoning patterns — knowledge as a side effect of prediction. Chat assistants are this same engine with an extra layer of training where humans rate answers, teaching it to prefer helpful continuations.

When you talk to it, the model produces a probability for every token in its vocabulary, one token is sampled, appended, and the loop runs again — your whole answer is generated one guess at a time.

This one mechanism explains both the magic and the failure mode. The fluency is real: prediction at this scale genuinely captures how ideas connect. But the model always produces a plausible next token — including where it has no ground truth — which is why it can state falsehoods with perfect confidence. It is not lying; it is guessing well. That is also why this site attaches numbers and sources to every claim: plausibility is not evidence.