From the Perceptron to the Multilayer Network

In 1969, Marvin Minsky and Seymour Papert published a book that froze neural network research for fifteen years. Their argument fits on a single page: the perceptron, the only neural model known at the time, cannot learn the exclusive OR function. Four points, two classes, and a model that fails. Funding collapsed and labs closed.

What makes this episode interesting is not the failure. It is that the solution was already known. Stacking two perceptrons solves XOR without difficulty; the weights could even be written by hand. What was missing was a way to train that stack, and it would take the popularization of backpropagation in the mid-1980s to get one.

Understanding precisely why a perceptron fails and why a hidden layer is enough is not archaeology. It is the only path that demystifies the question "how many layers, how many units." A hidden layer does not add computing power: it changes the geometric shape of the boundaries the model can draw. Once you know which shape, you stop choosing architectures at random.

This lesson revisits the perceptron starting from its learning rule, using actual numbers, shows its failure, builds the two-layer solution by hand, and then reformulates it in matrix algebra, the form in which PyTorch actually computes it.