How Do Optical Illusions Trick Your Brain?

Seeing is guessing, and usually guessing well

Junior level — plain language, no maths

It feels as if your eyes send pictures to your brain and your brain looks at them. They do not. Your eye sends a mess: a smear of light and dark from a lens with a blind spot in the middle of it, wobbling several times a second, sharp only in a region about the size of your thumbnail at arm's length. Everything that feels like vision - steady, wide, detailed, in colour - is reconstructed afterwards.

The reconstruction works by guessing, and the guesses are excellent because they are built on rules that are almost always right. Things further away look smaller. Surfaces in shadow reflect less light without changing colour. Edges usually mean an object ends. You never notice the rules because they almost never fail.

An illusion is a picture built to break one of those rules on purpose. The picture supplies the cues for a rule, the rule fires, and the answer comes out wrong - and knowing it is wrong does not help even slightly. That is the important part. You cannot decide to see the two squares as the same grey, because the decision is not made anywhere you have access to.

Press the button to move through four of them, then press show me the proof and watch your own eyes refuse to accept the evidence. The interesting question is not why you are being fooled here. It is what these same rules are quietly doing to everything else you look at.

Lateral inhibition, depth cues, and lightness constancy

Student level — the core equations

The earliest processing already distorts. Retinal ganglion cells have centre-surround receptive fields: light in the centre excites, light in the surround inhibits, so the cell reports local contrast rather than local brightness. That is an excellent design - it removes the overall illumination level, which is uninformative, and keeps the edges, which are not. The Hermann grid falls straight out of it: at a crossing, more of the inhibitory surround is covered by white than at a point along a bar, so the cell fires less and the crossing looks darker.

The Müller-Lyer illusion is usually explained as misapplied depth interpretation. Outward fins resemble the near corner of a building, inward fins the far corner of a room. The visual system scales apparent size by apparent distance - a necessary correction, since retinal size alone tells you nothing - and applying it to a flat drawing makes one line look longer. The evidence is suggestive rather than settled: the illusion is weaker in people raised in environments without carpentered corners, which fits the account but does not prove it.

Adelson's checker shadow is the strongest of the four because the effect is enormous and the underlying computation is obviously correct. The visual system is trying to recover reflectance - the property of the surface - from luminance, which is reflectance multiplied by illumination. Faced with a square in shadow that sends the same light as a square outside it, the only sensible inference is that the shadowed one must be a lighter surface. That inference is right in the world and wrong on the screen.

The café wall works lower down still, in the interaction between the small contrast borders of the mortar line and the tiles either side. Slight local misjudgements of where a border sits accumulate along the row and are read out as a slope. Its discovery is a good illustration of how mundane the sources can be - it was noticed on the wall of a Bristol café, and reported by a member of the public.

Inverse optics, predictive coding, and what a visual illusion is evidence of

Scholar level — full mathematical depth

01Vision as an ill-posed inverse problem

The image is a projection, and projection destroys information: infinitely many three-dimensional scenes, illuminations and reflectances produce any given retinal image. Recovering the scene is therefore ill-posed in Hadamard's sense and admits no unique solution from the data alone. The only escape is to constrain the solution with assumptions about which scenes are likely - priors. Under this framing an illusion is not a malfunction; it is the correct posterior under a prior that the stimulus was designed to violate, and it is one of the few direct experimental windows onto what those priors are.

02Predictive coding and what the feedback carries

Anatomy is awkward for any simple feedforward story: connections from higher visual areas back to V1 outnumber the forward ones. Predictive coding proposes that the descending pathways carry predictions of the incoming activity and the ascending pathways carry only the residual - the part the prediction failed to explain. The framework accounts naturally for end-stopping and surround suppression as prediction error rather than as feature detection, and it explains why an unexpected stimulus produces much larger responses than a predictable one. It remains a framework rather than a settled account: the specific cortical microcircuit implementing it is still contested.

03Lightness constancy, and why the checkerboard is not a mistake

Anchoring theory holds that the visual system assigns lightness within a framework - a region judged to share illumination - typically anchoring the highest luminance in that framework to white. The checker shadow works because the shadowed region is parsed as its own framework, and B is near the top of its local range. The strongest evidence that this is a genuine scene interpretation rather than a local contrast artefact is that the illusion collapses when the shadow is made physically implausible - a shadow with a sharp edge crossing an occluding object stops producing the effect, even though every local luminance relation is unchanged.

04Natural scene statistics as the origin of the priors

The priors are not arbitrary; they approximate the statistics of the world. Natural images have power spectra falling roughly as \(1/f^2\), strong local correlations, and a marked over-representation of cardinal orientations - and the visual system's sensitivity is matched to all three, including the oblique effect, our measurably poorer acuity for diagonal orientations. Efficient coding theory, from Barlow and Attneave onwards, predicts that a system should allocate its limited dynamic range to decorrelate its input, and centre-surround filtering is close to the optimal whitening filter for images with that spectrum. On this account the retina compresses because compression is what an information bottleneck demands.

05Where the analogy with machine vision holds and fails

Convolutional networks reproduce several classical illusions without being trained on them, which is evidence that the effects follow from the statistics and the architecture rather than from anything peculiarly biological. But the failure modes diverge sharply: adversarial examples fool networks with perturbations invisible to humans, and humans are fooled by configurations that leave networks unmoved. Two systems solving the same ill-posed problem with different priors fail in different places - which is a useful reminder that a shared benchmark score does not imply a shared representation.