How the fly sees motion

Why this matters here

The drone has to know when it is rotating, and one way to know is to watch the world slide past the camera. A fly does this with a chain of five stages in its optic lobes, from the photoreceptors to a handful of large cells that report “the whole world is turning”. The camera feeds the model’s photoreceptors, so the drone gets the same chain. Every notebook entry about the optic-flow probe asks one question: does a signal get through that chain, stage by stage, on the real wiring? This page names the cells the notebook names and says what each one is for.

The pipeline at a glance

Each row is one stage. Signals flow down the table. The visual field is tiled into about 800 columns per eye, one per facet, and every stage up to the motion detectors repeats once per column, so “L2” in the notebook means about 1,780 cells across the two eyes, roughly one per column, all doing the same job at different points in the image.

The four subtypes of T4 and T5 are the four directions: a prefers front-to-back motion, b back-to-front, c upward, d downward. Each subtype sends its output to its own layer of the lobula plate, so the lobula plate holds four maps of local motion, one per direction. HS cells read the horizontal layers and respond to turning about the vertical axis (yaw); VS cells read the vertical layers and respond to rolling and pitching. HS and VS are the end of the line for this project’s probe: their signal is the one the drone needs.

T4 sits in the medulla and T5 in the lobula, a neighbouring neuropil, but they are best thought of as one stage: the same computation, done once for brightening edges (T4) and once for darkening edges (T5) (Borst, Haag & Mauss 2020).

How a motion detector works

A single pixel cannot tell motion from flicker. Two neighbouring pixels can, if one of them is listened to late. Suppose an edge moves from column A to column B. A sees it first, B a moment later. If A’s signal is delayed by about that moment and then compared with B’s undelayed signal, the two arrive together for motion from A to B and miss each other for motion from B to A.

   column A        column B
      |               |
   [delay]            |
      |               |
      +----> [ × ] <--+
               |
        responds to A -> B,
        not to B -> A

This is the Hassenstein–Reichardt detector, worked out from beetle behaviour in 1956, and in its simplest form the comparison is a multiplication. Barlow and Levick proposed the mirror image for the rabbit retina in 1965: a delayed inhibitory signal from one side that vetoes the response to motion in the wrong direction. The fly’s T4 and T5 appear to use both: an excitatory, delayed-comparison enhancement for the preferred direction and a delayed inhibition for the opposite one (Haag et al. 2016).

The medulla supplies the parts. Each T4 cell takes its fast ON inputs (Mi1, Tm3) from a central column and its slower inputs (Mi4, Mi9) from neighbouring columns, so the spatial offset is built into which neighbour each input comes from and the delay into how slowly those inputs respond (Takemura et al. 2017; Arenz et al. 2017). CT1 is one wide cell that contacts T4 across the field. T5 has the same plan with OFF inputs: Tm1, Tm2 and Tm4 fast, Tm9 slower, and CT1 again (Shinomiya et al. 2014, 2019). Which medulla cell is fast and which is slow, and by how much, has been measured differently by different labs (Behnia et al. 2014; Arenz et al. 2017); the outline is settled, the exact delays are not.

Two channels, and why the ON channel is subtle

In flies, light makes the photoreceptors release more histamine, and histamine is inhibitory. So the lamina cells under them do the opposite of what one might guess: L1, L2 and L3 all dim when light arrives and brighten when it goes away. At this point there is no split yet. All three lamina cells respond to light the same way. What makes one channel ON and the other OFF is what their targets do with the signal.

The OFF channel is the straightforward one. L2 releases acetylcholine, which is excitatory, onto Tm1, Tm2 and Tm4. When the scene gets darker, L2 brightens, releases more, and its targets are pushed up. Darkness in, excitation out.

The ON channel has to invert the sign once more, and it does it with a brake. L1 releases glutamate onto Mi1, and on Mi1 that glutamate opens an inhibitory channel (Molina-Obando et al. 2019). In the dark, L1 is active and holds the brake on: Mi1 is pressed down. When light arrives, L1 dims, lets off the brake, and Mi1 rises. Mi1 is not excited by light; it is released by it. The same arrangement holds for Tm3 and for the lamina relay L5, which also take most of their lamina input from L1.

The project’s sign table follows the dataset’s transmitter labels: acetylcholine excitatory, GABA, glutamate and histamine inhibitory. That makes L1 inhibitory everywhere, which is right for L1 onto Mi1. The lamina drive entry has the transmitter table and the weights.

Dimmers versus switches

Most cells in the first stages of this pipeline do not spike. The photoreceptors, the lamina cells, and most of the medulla cells signal with a voltage that varies smoothly with their input, called a graded potential, and they release transmitter continuously in proportion to it. They behave like dimmers. Further along, the picture is less clean: T4 and T5 are generally reported to be graded, with little or no spiking, and how far that holds is still argued over. HS and VS are graded with occasional spikes.

This project’s engine models every neuron the same way, as a leaky integrate-and-fire cell: a switch that is either quiet or emitting spikes, and that stays quiet until its input crosses a threshold (see the introduction). That is the model Shiu et al. used for the whole brain, and the reason it is used here.

A brake only does something to a dimmer. A dimmer held below its resting level by a brake rises when the brake is released. A spiking cell held below rest by a brake, once released, returns to rest, and rest is below threshold, so it stays off. Releasing the brake on a switch that is already off changes nothing anyone downstream can see. The lamina has the same problem one step earlier, because photoreceptor input onto it is inhibitory too; the design answers it with the lamina bias, a constant background push that keeps L1, L2 and L3 firing so light can modulate them in both directions. Mi1 and the other braked medulla cells have no such push yet.

There are two ways to proceed, and the choice is Kyle’s.

• (a) Keep switches everywhere and give the braked cells a background push, the same kind of tonic bias the lamina already has, applied by a rule stated from the wiring (cells whose dominant input is inhibitory) rather than chosen cell by cell. This keeps one neuron model across the whole nervous system and the existing sign table. It asks the model to supply, as a fixed bias, an operating point that the real cells reach through their own steady transmitter release.
• (b) Model the early visual stages as dimmers: a graded-rate model of the columnar optic lobe, as Lappalainen et al. 2024 did for the fly’s visual system, with the rest of the nervous system staying spiking. This is closer to how these cells actually work, and Lappalainen et al. showed that such a model, built on the connectome and trained on a task, predicts real responses. It means two neuron models in one engine and a boundary between them.

Where the model stands

• Lamina drive onto the medulla: the lamina fires, the medulla does not. The ON path is braked by L1, as described above. The OFF path is wired correctly but too weak: L2 fires about half as fast as Tm1, Tm2 and Tm4 need.
• Photoreceptor operating-point sweep: raising the camera’s baseline rate speeds up L2 and wakes the OFF medulla (Tm1, Tm2 and Tm4 fire a few hertz). T5 still produces no spikes, and neither do T4, HS or VS.
• T5 drive: T5 is both braked and under-powered. CT1 fires flat out and holds every T5 well below threshold, and Tm9, its largest excitatory input, is silent.

Every result in the notebook carries a shuffled-connectome control, a copy of the imago with the wiring randomized, so that a signal can be credited to the real wiring and not to the engine.

Glossary

The stage overlay in the probe report covers the photoreceptors, L1–L5, C2, C3, Mi1, Tm3, Mi4, Mi9, Tm1, Tm2 and Tm4. The verdict sets are T4a–d, T5a–d, HS and VS. Transmitters are given where the sign matters to the model.

• C2, C3: lamina feedback cells, sent back from the medulla to the lamina. GABA, inhibitory. Both take most of their lamina input from L1.
• CT1: one very large cell per optic lobe, with a small branch in every column, inhibiting T4 in the medulla and T5 in the lobula. GABA, inhibitory. Thought to contribute to the null-direction veto; its exact role is still debated. In the fly each of its roughly 700 column branches works as an independent unit (Meier & Borst 2019); the engine treats it as one point cell per side, so a whole lobe’s input switches it fully on and it brakes every T5 at once (see the T5 drive entry).
• HS: horizontal system cells, lobula plate. Pool front-to-back and back-to-front motion over much of the eye; signal turning about the vertical axis (yaw).
• L1: lamina, ON channel. Glutamate, inhibitory under the project’s sign table. The brake on Mi1, Tm3 and L5.
• L2: lamina, OFF channel. Acetylcholine, excitatory. Main input to Tm1, Tm2 and Tm4.
• L3: lamina, feeds both channels. Acetylcholine, excitatory. Main lamina input to Mi9; in the fly also a main input to Tm9.
• L4: lamina relay, OFF channel. Acetylcholine. Driven mostly by L2.
• L5: lamina relay, ON channel. Acetylcholine. Driven by L1 (braked) and L2; the main lamina-side driver of Mi4.
• Mi1: medulla, ON channel, fast. The main excitatory input to T4, at the centre of its receptive field. Braked by L1.
• Mi4: medulla, ON channel, slower. GABA, inhibitory. Feeds T4 from a neighbouring column; little direct lamina input.
• Mi9: medulla, ON-channel circuit, slower. Glutamate, inhibitory onto T4. Unusually for an ON-circuit cell, it responds to darkening (Strother et al. 2017), so light releases its inhibition of T4: one more sign inversion, on the null side of T4’s receptive field.
• R1–R6: photoreceptors, the eye stage. Histamine, inhibitory onto the lamina. The outer photoreceptors that motion vision uses (R7 and R8 serve colour and are not part of this pipeline).
• T4a–d: motion detectors for ON (brightening) edges, dendrites in the medulla. a front-to-back, b back-to-front, c up, d down. Acetylcholine in the fly, excitatory onto HS and VS.
• T5a–d: motion detectors for OFF (darkening) edges, dendrites in the lobula. Same four directions and outputs as T4.
• Tm1: medulla, OFF channel, fast. Input to T5. Driven by L2.
• Tm2: medulla, OFF channel, fast. Input to T5. Driven by L2.
• Tm3: medulla, ON channel, fast. Input to T4, alongside Mi1. Braked by L1.
• Tm4: medulla, OFF channel, fast. Input to T5. Driven by L2.
• Tm9: medulla, OFF channel, delayed. Input to T5 from a neighbouring column. Not yet in the stage overlay.
• VS: vertical system cells, lobula plate. Pool upward and downward motion; signal rolling and pitching.

See the References chapter for full citations.