First benchmark of the whole-CNS imago

Goal

Measure steps per second at batch 1 (the flight gate) for the full imago and for the no-optic-lobe probe on the Mac, on cpu and mps, at a baseline input rate and a high-contrast input rate, to learn whether real time is within reach before any optimization. Masks are a compute probe here, not a strategy: the no-optic-lobe run removes vision and can never fly.

Setup

• git describe: 996072e, then the two mps batch-1 rows re-measured on the fix-wave build (see the note under Results)
• machine: Mac M3 Max, 128 GB
• imago: data/imagos/male-cns-v1.0.imago, meta: n_neurons 167,950 (166,700 real + 1,250 virtual photoreceptors), n_edges 25,530,591, n_virtual 1,250, n_edges_dropped_onto_inputs 32,441, n_gain_groups 1,296, nt_flagged 3,177
• the edge count is arithmetic: 25,560,079 significant edges with both ends in the neuron set, minus 32,441 edges onto input neurons, plus 2,953 edges from the 1,250 virtual photoreceptors, is 25,530,591
• command: uv run flatline bench --imago data/imagos/male-cns-v1.0.imago --mask configs/masks/<mask>.toml --device <device> --seconds 2 --rate <rate> (batch 1); the batch run used --mask configs/masks/full.toml --device cpu --batch 16 --seconds 0.5 --rate 50.

What happened

All nine runs completed with no crashes, timeouts, or memory problems. Each 2 s run took well under a minute including imago load and a 50-step warm-up.

Results

Training-shaped run (batch 16, cpu, full, 0.5 s, rate 50): 9.8 steps/s, about 157 brain-steps per second, mean spikes/step 1731.0.

The two mps full-imago rows were re-measured after the fix wave that redesigned the retina correction (the imago now carries 1,250 virtual photoreceptors rather than 942, and 705 more edges). At 50 Hz throughput rose from 494.0 to 526.9 steps per second, 6.7%; at 200 Hz it moved from 507.7 to 510.7, under 1%. The other six rows are the original measurements on the pre-fix build, whose counts were 167,642 neurons and 25,529,886 edges.

Conclusions

The flight gate, a real-time ratio above 1.0 at batch 1, is not met on either device. Metal reaches about half real time on the full imago, and the CPU reaches about an eighth.

Throughput looks bound by per-step overhead rather than edge count. Dropping the optic lobes removes 63% of neurons and most edges, yet Metal throughput only rises from about 500 to about 590 steps per second. Candidates for the bottleneck are the per-step host synchronization in the event propagator, the number of small kernel launches per step in the engine (several where operations with fresh temporaries), and the Bernoulli draw that is currently made on the CPU each step.

Mean activity sits between about 1,900 and 2,100 spikes per step, around 1.2% of neurons per millisecond (3.1% of the 61,435 neurons in the no-optic-lobe runs), and barely changes between 50 Hz and 200 Hz input. This suggests the network’s activity is dominated by its own recurrent dynamics under this drive rather than by the input rate. This needs a look before any encoder work: spike rasters by superclass, and a check on whether activity is stable or growing over a longer run.

Two structural facts from the final review bear on this. Of the 167,950 neurons, 3,270 have no outgoing edge at all and 2,337 are fully isolated, so about 2% of the graph can never contribute. And the optic lobe is nearly silent under inhibitory photoreceptor drive: that is why removing it barely changes spikes per step, and it is exactly the phenomenon the visual transmission policy of the design spec, section 8.6, is meant to address.

The CPU batched path gives no per-brain speedup at batch 16, about 157 brain-steps per second total, roughly the same as batch 1. CPU training is not viable at this scale. The training gate depends on straylight’s ROCm path or a faster kernel.

Next steps

Profile one step on Metal to attribute time between propagation, the engine’s elementwise updates, and the input draw. The event propagator makes two device-to-host synchronizations per step, one to list the spiking neurons and one to size the gather, at roughly 0.2 ms each on mps; a fixed-capacity padded gather that removes both is the lever to pull first. Fuse the engine’s elementwise updates and keep the Bernoulli draw on the device. Measure a longer run (30 s) to check activity stability and record rasters by superclass. Then measure batch 64 on straylight.