GarageFly architecture

Loop

 config/garage.yaml            audio file / synthetic control        odors, taste patches
        │                               │                                   │
        ▼                               ▼                                   ▼
 GarageWorld (MuJoCo, mm)     audio/analyzer.py: PCM → band envelopes  sensory/olfaction.py
 environments/garage.py       audio/stimulus.py: Pa → particle         sensory/gustation.py
        │                        velocity at each antenna (near field, sensory/vision.py (opt.)
        │                        1/r, directional gain)                        │
        │                               │                                       │
        │        body pose ────────────►│◄───────── body pose ──────────────────┤
        │                               ▼                                       │
        │                  sensory/mechanosensation.py: Johnston's organ        │
        │                  (antenna resonance, JO-A/B/C/D/E tuning, adaptation) │
        │                               │ Poisson rates, 1 ms updates           │
        │                               ▼                                       ▼
        │                 brain/simulator.py — LIF network on GPU (PyTorch sparse CSR,
        │                 CUDA graph), 0.1 ms step, FlyWire v783 wiring (brain/connectome.py)
        │                               │ filtered DN rates (brain/mappings.py)
        │                               ▼
        │                 body/decoder.py — DNp09/oDN1 fwd, DNa01/02 turn, MDN back,
        │                 aDN2/DNg11/DNg12 groom, GF escape → 2-D descending drive
        │                               ▼
        │                 body/behaviors.py — WALK (FlyGym CPG) · GROOM (measured aDN kinematics)
        │                 · TAKEOFF/FLY/LAND (engineered flight) · RIGHTING; body/fly.py (FlyGym 2.1)
        │                               ▼
        └──────────────── MuJoCo physics (NeuroMechFly v2 body), 0.1 ms ─────► back to top

experiments/runner.py runs this loop. recording/recorder.py writes HDF5, visualization/monitor.py draws the dashboard (live window or MP4), and analysis/ computes metrics and compares conditions.

Why these components (Stage 1 assessment, September 2026)

Need	Choice	Why / alternatives
Connectome	FlyWire FAFB v783 (still the latest FAFB release) via the Shiu et al. 2024 repo files	Public (CC-BY 4.0), no login. Includes signed synapse counts. The annotations come from `flyconnectome/flywire_annotations` (Schlegel et al. 2024). Codex downloads need a Google sign-in. CAVE needs a token.
Brain model	Shiu et al. 2024 whole-brain LIF, ported to PyTorch	The only whole-brain model with published behavioural validations (feeding, grooming). The original is Brian2 (CPU, open-loop). Our port runs it on the GPU and steps it inside a closed loop. PyTorch forks exist (e.g. eonsystemspbc/fly-brain, GPL). We wrote a small MIT-compatible port instead.
Body + physics	FlyGym 2.1 (NeuroMechFly v2), MuJoCo 3.9	FlyGym 2.x (March 2026) is a rewrite: ~10× faster CPU sim, MjSpec-based composition. It dropped the 1.x olfaction arenas and moved locomotion controllers to `flygym_demo`. We vendor the repo to get those. The legacy 1.x lives on as `flygym-gymnasium` (Python <3.13, MuJoCo 3.2.7).
Locomotion	FlyGym `HybridTurningController`	The NeuroMechFly v2 abstraction of descending control (2-D drive). No VNC connectome model is coupled yet.
Vision	FlyGym compound-eye readouts → per-eye photoreceptor drive	flyvis (Lappalainen et al. 2024) is the principled front end, but it pins Python <3.13 and integrates only with FlyGym 1.x. It is a planned upgrade.
JO / auditory	FlyWire JO-A…F annotations + literature tuning	No simulator ships an antennal hearing model. The transduction is ours and is labelled.

GPU feasibility (measured on this machine, RTX 3090)

brain mode	neurons	edges	brain step	notes
minimal	13,652	1.17 M	< 0.1 ms	≤2 strong hops from JO/ORN/GRN inputs and to read-out DNs
sensory-motor (default)	52,547	5.54 M	0.15 ms	optic lobes removed
full	138,639	15.09 M	0.33 ms	~210 MB VRAM

The sparse mat-vec dominates. CUDA-graph capture removes launch overhead (≈10%). The whole closed loop, including MuJoCo, the FlyGym controller and recording, runs at ~5.5 s wall per simulated second in sensory-motor mode. VRAM is not a constraint: one run uses <1 GB, and several runs can share the GPU (scripts/run_batch.py).

Related work

Eon Systems (March 2026) showed an embodied Shiu-model + NeuroMechFly fly (DNa01/02 steering, oDN1 forward, antennal grooming DNs, MN9 feeding) with imitation-learned motor mapping. No code was released.
dohun1214/flybrain: an early-stage Shiu LIF (PyTorch) + FlyGym 2.1 project.
BANC (brain + nerve cord, public in Codex, v888 May 2026) and MANC (male VNC) could replace the engineered locomotion controller with connectome-derived VNC circuitry. That is the most important future upgrade.

Data flow and units

Positions in mm (MuJoCo world), time in s (physics) / ms (LIF), rates in Hz, potentials in mV, particle velocity in mm/s, pressure in Pa.
Simulation neuron index ↔ FlyWire root_id: Connectome.root_ids (saved in each HDF5 file under /neural/neurons/root_id).

Extending

New stimulus object: add a prop to config/garage.yaml (geometry), plus a transducer in garagefly/sensory/ that maps body pose → (indices, rates) and is called from the runner's sensory update.
New read-out DN: add a NeuronGroup to DN_GROUPS (with citation and confidence) and use it in body/decoder.py.
In-silico silencing: --set brain.silence_types='[DNp01]'.
Pharmacology (future): implement it as explicit, provenance-tagged changes to neuron or synapse parameters. Never do it inside a sensory transducer.