Lessons0 of 20 done

Module 1: Foundations. Lesson 3 of 20, 7 minutes.

How machines learn.

You will know how a model learns from labeled examples, why held-back test data matters, what overfitting is, and the three main kinds of learning.

Suppose your job is to keep spam out of a company's inboxes, and your only tool is a list of rules. You start well: flag anything mentioning a lottery, flag messages written in capital letters. Within a week the spammers have rephrased, and a colleague complains that an email about the office raffle got blocked. You add rules. They add tricks. The list grows to hundreds of lines and still leaks.

Machine learning is the alternative. Instead of writing the rules, you collect examples and let a program work out the rules itself. This lesson shows how that works, using the spam filter as the running example, and then names the three main ways machines learn.

Start with examples, not rules

The raw material of machine learning is examples with answers attached. For a spam filter, that means a large pile of real emails, each one marked "spam" or "not spam" by a person. The marks are called labels. A label is simply the correct answer for one example.

Where do labels come from? Mostly from people. Every time someone clicks "report spam" or drags a message out of the junk folder, they are labeling an example. Collecting good labeled data is often the hardest and most expensive part of the whole process, and the quality of what the system learns depends on it. If the examples are mostly English-language sales pitches, the filter will be weak on spam written in Portuguese.

Notice what has changed. You are no longer trying to describe spam. You are showing it. The job of finding the description has moved from you to the machine.

A model is a function with adjustable numbers

The thing that learns is called a model. Strip away the mystique and a model is a function: it takes an input (an email) and produces an output (a score for how spammy it is). What makes it adjustable is that the function contains numbers that can be changed. These adjustable numbers are called parameters.

Picture a panel of dials. One dial controls how much weight the word "free" carries. Another controls the weight for "invoice", another for the number of links, another for whether the sender is in your contacts. Set the dials one way and the model flags everything; set them another way and it flags nothing. Somewhere in between is a setting that sorts most emails correctly. A simple spam filter might have a few thousand dials, one per word it tracks. The models behind today's chatbots have vastly more, but the idea is identical: a function whose behavior is set by its numbers.

Before training, the dials are set at random and the model is useless. Learning is the process of turning them to good positions, and the examples are what tell it which way to turn.

Training: turn the dials until the mistakes shrink

Training works like this. Show the model an email from the pile. Let it guess. Compare the guess with the label. If it called a real invoice spam, that is a mistake, and the size of the mistake can be measured as a number. Then adjust the dials a little in whichever direction would have made the mistake smaller. If the word "invoice" pushed the score toward spam and that was wrong, turn that dial down slightly.

Now do that for the next email, and the next, and keep cycling through the whole pile, millions of small adjustments, until the total of the mistakes stops shrinking. No single adjustment does much. The accumulation of all of them is what turns random dials into a working filter.

Two things are worth noticing. The model never receives a rule. Nobody tells it that "free" is suspicious; it learns that because turning the "free" dial up kept reducing errors. And the result is not a list of rules you can read. It is a set of numbers. You can inspect which dials ended up high, but the knowledge is spread across all of them together.

Hold some back, and watch for memorizing

Here is the trap. A model with enough dials can get every training email right by effectively memorizing them: this exact message is spam, that exact one is not. It would score perfectly on the pile and fail on the first new email it saw. Memorizing the examples instead of learning the pattern is called overfitting, and it is the central hazard of machine learning.

The standard defense is simple. Before training, set aside a portion of the labeled emails and never show them to the model. This is the test data. After training, run the model on it. Because the model has never seen these messages, its score on them is an honest estimate of how it will do on tomorrow's mail. If it scores well on the training data and badly on the test data, it has memorized rather than learned, and you need more examples, a simpler model, or both.

The property you actually want is called generalization: doing well on examples the model was not trained on. A filter that handles spam it has never seen has generalized. A student who can answer the textbook questions but not the exam's new ones has not. The whole point of learning, for machines as for people, is the new case.

Three ways to learn

The spam filter is an example of supervised learning: every example comes with a correct answer supplied by a person. This is the most direct form and still the workhorse of many practical systems, from reading medical scans to spotting fraud. Its limit is the labels. Someone has to produce them, and for many tasks that is slow and costly.

Self-supervised learning removes that bottleneck with a trick. Take any piece of data, hide part of it, and ask the model to predict the hidden part. Take a sentence, cover the last word, and have the model guess it. The sentence itself supplies the correct answer, so no human labeling is needed, and the pool of training material becomes essentially everything ever written down. This is how the language models behind chatbots learn the bulk of what they know. Predicting the next word sounds trivial. Doing it well across all of human writing turns out to require absorbing grammar, facts, reasoning patterns and style.

Reinforcement learning works differently again. There are no correct answers, only rewards. The model tries something, receives a score for how well it went, and adjusts to earn more reward next time. This is how systems learn to play games, where a win is a reward and a loss is a penalty, and it has uses well beyond games. Large modern systems often combine all three.

The core move is the same every time: a function with adjustable numbers, examples or rewards that tell it which way to adjust, and a check on fresh data to make sure it learned rather than memorized.

Try it

You can watch overfitting happen in about two minutes. Type: I will give you four examples. Email A: "Win a free cruise" is spam. Email B: "Your invoice is attached" is not spam. Email C: "Free tickets inside" is spam. Email D: "Lunch tomorrow?" is not spam. Based only on these four, is "Free lunch tomorrow in the break room" spam? Explain your reasoning.

Look at how it reasons from so few examples, and whether it over-weights the word "free". Then ask: What extra examples would you need to be more confident? That is the labeled-data problem in miniature: a learner with too few examples latches onto whatever pattern it has.

Key terms

Label
The correct answer attached to one training example, such as "spam" or "not spam".
Model
A function that turns an input into an output, with adjustable numbers inside that training sets.
Training
The process of showing a model examples and adjusting its numbers a little each time to make its mistakes smaller.
Overfitting
When a model memorizes its training examples instead of learning the pattern, so it does well on them and badly on new data.
Self-supervised learning
Learning by predicting a hidden part of the data, such as the next word in a sentence, so no human labels are needed.
Reinforcement learning
Learning from rewards and penalties for actions rather than from correct answers.

Check yourself

1.In the spam filter example, what does training actually do?
2.Why hold back test data the model never sees during training?
3.Which kind of learning needs no human-made labels because the data supplies its own answers?

A model is a function with adjustable numbers, trained by nudging them toward fewer mistakes, and judged on examples it has never seen.