This page is empty.So is our AI.
Here it is.
We are building a program that writes text by guessing what comes next. That, in the end, is what an AI is.
1 pure luck2 counts letters3 letter pairs4 word pairs
Version 1 knows nothing: every letter has the same chance.
Empty. Like your GitHub contribution graph.Prof. Graves
Type anything and press ask.
Congratulations, you built a monkey with a keyboard. My mother uses one to reply to my emails.Prof. Graves
Give it something to read.
1 pure luck2 counts letters3 letter pairs4 word pairs
New in version 2: it counts every letter in a text. More often, bigger slice.
This is all it will ever know:
The head guesses the next word. It read a lot of words, so the head guesses well. My mother read a lot of words too, and my mother guesses badly. The secret is not reading. The secret is counting. Count which word comes after the word you have, and you can guess the next word. Count enough words and the guesses start to sound like a person. A person who has never been outside, but a person. That is all the head does. It counts, it guesses, and it counts again. My mother counts too. She counts the days until I visit, and the head counts the words until you stop typing.
Reading and counting like this is training, the word companies spend billions on.
Still nonsense, but confident nonsense now. Like a LinkedIn post.Prof. Graves
Teach it which letter comes next.
1 pure luck2 counts letters3 letter pairs4 word pairs
New in version 3: it looks at the letter it just wrote, and counts what came after it.
What it read:
Bits of it are almost words. A toddler swallowed a dictionary and is now coughing up Scrabble tiles.Prof. Graves
Teach it words.
1 pure luck2 counts letters3 letter pairs4 word pairs
New in version 4: the same, with whole words instead of letters.
What it read:
What we’ve built is a dumbass with a vocabulary: it repeats what it heard and understands none of it. A parrot. Hold that thought.Prof. Graves
Claude Shannon did it with books.
Shannon’s 1948 paper “A Mathematical Theory of Communication” built fake English by hand, from random letters up to word pairs: your ladder, run on paper. source
He used a pencil. You used a button. Try to look grateful.Prof. Graves
Feed it your own words.
Paste anything you wrote: an email, a rant, your diary. It stays in your browser. It will write in your voice: it only knows what it read.
What it read:
Feed it your diary and it will judge you in your own words.Prof. Graves
Now make it enormous.
The head you run from lesson 1 is qwen3:8b. It plays the same game. It read 36 trillion tokens. A token is a piece of text, often part of a word; the head reads and writes in tokens. source
Yours keeps a table: . qwen3 keeps no table: training tuned its 8,190,735,360 numbers, called parameters.
- Yours
- qwen3
- looks back at
- the last word
- thousands of tokens
- keeps
- a table of counts
- 8.2 billion tuned numbers
Yours fits on a napkin. Theirs reaches the Moon 94 times, or there and back 47. Try not to feel inadequate.Prof. Graves
That's an AI.
An AI like this is a program that guesses the next piece of text. An LLM (large language model) does it at huge scale. This course calls it the head.
What you built, next to the head:
| Your tiny AI | The head |
|---|---|
| letters and words | tokens, the pieces it reads. qwen3 cut the opening you read, , into tokens. |
| your count table | parameters: billions of tuned numbers, each like a volume knob. |
| counting once | training: after each wrong guess, every number moves a tiny bit toward the right answer. Trillions of times. |
| the wheel | the same wheel: a slice for every piece it could write next. |
Yours never read your question. The head does: your question becomes the start of the text it continues. Lesson 1 sends it your first one.
Words from this lesson
- Training
- Reading examples and keeping score.
- Training data
- The examples it reads.
- Model
- The score it keeps.
- Token
- A piece of text, often part of a word; the head reads and writes text in tokens.
- Agent
- An AI that does a job in steps, not just chatting.
- Parameter
- One stored number inside a real model.
- LLM, "the head"
- A huge trained model that guesses the next piece of text.
Where your agent.py goes
- 1 it answers you
- 3 it remembers
- 5 it reads your files
- 6 it works in a loop
- 8 it searches your notes
- 11 it solves a problem of yours
Ours is a parrot. Lesson 1 wakes up the one that read the internet.Prof. Graves
Professor Graves is made up. Claude, an AI (no relation to Shannon), writes and checks these lessons, and every fact on this page was checked against its source.