Connectome ingestion and brain engine

Connectome ingestion

Inputs

From ~/models/fly/model/flat-connectome/:

The synapse-point and synapse-partner files are not used.

Neuron set

Neurons are the bodies with a non-null superclass, about 166,000. Each receives a dense index in a stable order (sorted by bodyId). Region masks (see Region masks) select subsets at load time without rebuilding.

Signs and effective weights

Each synapse’s sign comes from the presynaptic neuron’s consensus neurotransmitter, following Shiu et al.:

The effective weight of an edge from neuron \(i\) to neuron \(j\) is

\[ w_{ij} = c_{ij} \cdot \mathrm{sign}(nt_i) \cdot s[nt_i] \cdot g[t_i] \cdot W_{\mathrm{syn}} \]

where \(c_{ij}\) is the synapse count, \(nt_i\) and \(t_i\) are the presynaptic neuron’s transmitter and gain group, \(s\) is a learnable per-transmitter scale (eight values in the fixed table order acetylcholine, gaba, glutamate, dopamine, serotonin, octopamine, histamine, unclear; initialized to 1 and constrained non-negative so learning can never flip a sign), \(g\) is a learnable per-gain-group factor (design spec, section 12.1), and \(W_{\mathrm{syn}}\) is 0.275 mV, the Shiu et al. value. Monoamines are excitatory here because Shiu et al. treated them so; that is a recorded decision (design spec, section 4), not a fact about the fly. \(W_{\mathrm{syn}}\) was fitted by Shiu et al. on FlyWire feeding circuits, and its transfer to this dataset is a risk carried in the design spec, section 15. The imago stores counts and signs; scales and gains are applied at load.

Edges whose postsynaptic neuron is an input neuron are dropped at build time and counted in the metadata: input neurons are driven by encoders, never by the network.

A build option produces a shuffled imago for the control condition of the design spec, section 12.5. Edges are grouped by synapse count and, within each group, their destinations are permuted with a seeded generator. Every neuron therefore keeps its exact outgoing edges and their counts, its exact incoming synapse total and incoming count distribution, and its transmitter sign, while its partners are randomized: the synapse totals are preserved exactly, but the number of distinct partners a neuron has may change, because two permuted edges occasionally land on the same pair and are merged. Edges whose presynaptic neuron is an input neuron are never shuffled, so the control shares the real sensory interface and randomizes only the brain’s internal wiring. The sidecar records the seed, the number of edges shuffled, and the number merged.

Imago format

A single safetensors file with the .imago extension plus a JSON sidecar, about 300 MB:

• pre_indptr, pre_indices, pre_counts: edges grouped by presynaptic neuron (int64 pointers, int32 indices, int16 counts). Used by the event propagator.
• post_indptr, post_indices, post_counts: the same edges grouped by postsynaptic neuron. Used by the batched propagator.
• Per-neuron arrays: body_id (int64), nt (int8 index into the transmitter table), type_id (int32 index into the type table), superclass_id, side (int8: left, right, unknown).
• Tables in the sidecar: transmitter names, type names, superclass names, build metadata (source file hashes, date, git describe).
• Named neuron sets: photoreceptors with per-neuron eye and column, virtual_photoreceptors, haltere_afferents with side, wing_motor and haltere_motor with side and type, descending.

Photoreceptor placement

Photoreceptor cell bodies lie in the retina, outside the imaged volume, so they carry no hex coordinates and no soma side. Two inferences supply them:

• Eye: rootSide (3,746 right, 2,345 left).
• Column: the hex column receiving the largest summed synapse count from the photoreceptor, over hex-bearing targets, using the full edge list (only 4,038 photoreceptors have any significant-only outgoing edge). The column is chosen by summed weight rather than by the single strongest target because a lamina cartridge, not a cell, is what a photoreceptor innervates; on this dataset the two rules agree on nearly every cell (median fraction of weight on the best column is 1.0). The eye fallback and the recorded target side are the side of that winning column. Hex-bearing targets in v1.0 are exactly the 23,720 optic lobe intrinsic cells of fifteen types: L1, L2, L3, L5, C2, C3, T1, Mi1, Mi4, Mi9, Tm1, Tm2, Tm4, Tm9, Tm20. Placement succeeds for 5,895 of 6,091 photoreceptors, and the target side agrees with the root side in every case.

Coverage of the resulting map:

The reconstructed retina is incomplete and imperfectly placed: a real column has six R1-R6 cells, but placed columns hold between one and eleven (v1.0 build, 826 columns with any R1-R6: 240 hold six, 547 hold fewer, 39 hold seven to eleven, the last being placement errors), and 2,054 of the 6,091 photoreceptors have no outgoing edge at all in the significant graph. The design spec, section 8.3, describes the two corrections that follow from this — a per-lamina-cell factor \(6m/S\), where \(S\) is the R1-R6 synapse count a lamina cell receives and \(m\) the median single-cell count onto its type, and a virtual photoreceptor for every column holding an undriven lamina cell — and the build report that records these counts.

Other named sets

Side rule: sensory neurons carry rootSide and a null somaSide; motor and central neurons carry somaSide and a null rootSide. The imago records somaSide when present, else rootSide. The haltere afferent set keeps each cell’s subclass (haltere 205, campaniform sensilla 195, mechanosensory bristle 39) and type (22 types) as fields, so downstream encoders can group by type rather than by cell.

Brain engine

Neuron model

Each neuron has a membrane voltage \(v\) and a synaptic variable \(g\), following the Shiu et al. formulation:

\[ \tau_m \frac{dv}{dt} = (v_{\mathrm{rest}} - v) + g + b \qquad \text{(unless refractory)} \]

\[ \tau_s \frac{dg}{dt} = -g \qquad \text{(unless refractory)} \]

\[ g_j \leftarrow g_j + w_{ij} \qquad \text{on presynaptic spike } i \text{ after delay } d \text{, also while refractory} \]

\[ \text{if } v > v_{th}: \quad \text{spike}, \; v = v_{\mathrm{reset}}, \; g = 0, \; \text{refractory for } t_{\mathrm{ref}} \]

These are the Shiu et al. rules as implemented in their released code: both differential equations are frozen while refractory, incoming synaptic weight still accumulates into g during that time, the threshold is strict, and the reset zeroes g. The term b is a per-neuron bias in mV, zero by default, used by the visual transmission policy of the design spec, section 8.6.

Initial constants (Shiu et al. 2024):

Integration is exponential Euler with step \(dt\), default 1 ms, with 0.1 ms available for fidelity checks. For a non-refractory neuron:

\[ g \leftarrow g \, e^{-dt/\tau_s} + (\text{delayed incoming weights}) \]

\[ v \leftarrow v_{\mathrm{rest}} + g + b + (v - v_{\mathrm{rest}} - g - b) \, e^{-dt/\tau_m} \]

For a refractory neuron \(v\) stays at \(v_{\mathrm{reset}}\) and \(g\) keeps its value plus any delayed incoming weights (no decay). Step counts are round(t_ref/dt) for the refractory period (2 at 1 ms) and max(1, round(d/dt)) for the delay (2 at 1 ms).

Learnable global knobs: the eight transmitter scales (non-negative), \(\tau_m\), \(\tau_s\), an additive threshold offset, the per-group gains and the lamina bias of the design spec, sections 12.1 and 8.6. Gains are regularized toward 1 in the training objective. Optional Gaussian membrane noise is available as a knob but off by default.

Input neurons

Photoreceptors, virtual photoreceptors, and haltere afferents are input neurons. Their membrane dynamics are bypassed: v and g are held at rest and no edge targets them (see Signs and effective weights). Each step they spike with probability rate_hz × dt_ms / 1000 where the rate is supplied by an encoder. This is a deliberate simplification of Shiu et al., who added Poisson input to the membrane of otherwise normal neurons; forcing the spikes directly makes the encoder’s rate the exact firing rate and costs nothing in the pathway we care about, since photoreceptor output is the spike train itself.

Propagation interface

class Propagator(Protocol):
    def propagate(self, spikes: Tensor) -> Tensor:
        """spikes: bool [B, N] (B=1 in flight). Returns weight input [B, N]."""

Two implementations:

• Event propagator. Finds spiking indices, gathers their outgoing segments from the presynaptic CSR, expands with repeat_interleave, and index_add_s effective weights into the input vector. Works on Metal, ROCm, and CPU. Used in flight (B = 1).
• Batched propagator. Computes the input for all B brains as a sparse-by-dense product using the postsynaptic CSR on CPU and ROCm. On Metal, where sparse support is thin, the event propagator handles all B brains in one packed gather and scatter over the flattened [B × N] state, never a Python loop over brains.

The event path pays one device-to-host synchronization per step to size the gathered index (the spike count is data dependent); that is inherent to event-driven propagation and is accounted for in the compute budget of the design spec, section 19.

A native (Rust or C++) event kernel may be substituted behind the same interface later. Tests require the two implementations to agree exactly on weights that are exact in float32 (dyadic rationals) and to within a small tolerance on random graphs, because scatter-add order differs between devices and implementations.

Delays

A ring buffer of two spike vectors implements the 2 ms delay. The propagator consumes the vector from two steps ago.

Real-time clock

In flight the engine steps to keep brain time within a configured tolerance of wall time, default 20 ms. Falling behind by more than the tolerance raises a brain-lag fault to the supervisor. Before takeoff the supervisor requires a measured real-time ratio above 1.0 for the configured mask on the current device, taken from the benchmark tool.

Region masks

A mask is a TOML file listing superclasses, classes, or explicit types to include or exclude. Masks are a benchmarking and ablation tool, not a real-time strategy: the optic lobes alone are 89,403 of the 166,700 neurons and cannot be dropped without dropping vision, and excluding everything that is not plausibly involved in flight removes only about 0.2% of neurons. Presets:

• full: every neuron in the imago (166,700 real plus 1,250 virtual photoreceptors, 167,950 in all).
• no-optic-lobe: excludes superclasses ol_intrinsic, ol_sensory, visual_projection, visual_projection_tbc, and visual_centrifugal, and the virtual photoreceptors, leaving 61,433 neurons and 12,165,352 edges. A compute probe and a “blind” ablation; it is never a flight configuration.

Edges to or from excluded neurons are dropped at load. The benchmark tool reports steps per second for each preset on each device, and flight is gated on the measured real-time ratio of the configuration actually flown.

Recording

Named neuron sets can be recorded to a spike ring buffer and flushed to disk as compressed arrays. Flight runs record motor neurons and descending neurons by default; full-brain rasters are available for offline analysis.

Determinism

All randomness is seeded and drawn on the CPU so the random stream is identical across backends. Given the same device, backend, and seed, a run is reproducible up to floating-point scatter order, which is not bitwise stable on GPUs; comparisons across runs use tolerances, and exactness is asserted only on CPU.

Tests

• Three-neuron chains and excitation-inhibition motifs on synthetic graphs with analytically known spike times.
• Event and batched propagators agree exactly on dyadic weights and to tolerance on random graphs.
• Input neurons hold v and g at rest under a strong incoming edge, and such edges are absent from the built imago.
• Delay buffer ordering.
• Refractory handling and reset.
• Mask application drops the right edges.

The benchmark is a tool, not a test.

Implementation notes

Module layout

The engine is implemented across the following modules, per the sub-project plan’s File Structure table:

Mask presets live at configs/masks/full.toml and configs/masks/no-optic-lobe.toml.

Build metadata, v1.0 release

Running flatline imago-info against the male CNS v1.0 imago built by build-imago on this release reports:

The same build writes a retina report into the sidecar:

The 720 lamina cells still without drive all lack a hex column (901 lamina cells do), so no virtual photoreceptor can reach them. A lamina_norm of 252 is a cell whose whole reconstructed input is a single small synapse count; the factor is uncapped by design, and whether it should be is an open question for the encoder work.

Build time on the Mac M3 Max is about 12 s.

Neuron model fidelity

The neuron model follows the Shiu et al. released code, not just the published equations: both differential equations are frozen while refractory, incoming synaptic weight still accumulates into g during the refractory period, the threshold is strict, and reset zeroes g. Input neurons (photoreceptors, virtual photoreceptors, and haltere afferents) are held at rest with no incoming edges; they are driven entirely by their encoder’s spike probability, never by the network.

Propagator agreement

The event propagator and the batched propagator are required to agree exactly on weights that are exact in float32 (dyadic rationals), and to within a small tolerance on random graphs, since scatter-add order differs across devices and implementations.