SWAMBIT: Shared Workspace And Memory for Brain-Inspired Transformers

News

Poster

14 OCT2026

RE-PoLar at the AI Grid Summit 2026

Presenting the poster at the AI Grid Summit in Berlin.

Poster

29 SEP2026

Poster at Bernstein Conference 2026

The RE-PoLar poster, presented at Bernstein Conference 2026.

Release

28 SEP2026

RE-PoLar is out

Reproduction and extension of PoLar, with code and an interactive explorer.

Meeting

27 SEP2026

SWAMBIT at the first NI:KI project meeting

SWAMBIT introduced to the initiative, plus a team picture.

Poster

14 SEP2026

Poster at the NEAT workshop

The RE-PoLar poster, presented at the NEAT workshop.

Launch

1 DEC2025

SWAMBIT officially starts

SWAMBIT starts as one of the 11 NI:KI pilot projects.

14 OCT2026 Poster

RE-PoLar at the AI Grid Summit 2026

We presented RE-PoLar at the AI Grid Summit 2026 in Berlin.

Presenting the RE-PoLar poster at the AI Grid Summit 2026 — Photo: Stefanie Loos

29 SEP2026 Poster

Poster at Bernstein Conference 2026

We presented the RE-PoLar poster at the Bernstein Conference 2026.

Open poster (PDF)

28 SEP2026 Release

RE-PoLar is out

We released RE-PoLar, a reproduction and extension of PoLar: skipping and repeating transformer layers per input in pre-trained, frozen LLMs, which we relate to thalamo-cortical routing in the brain. Paper, code and an interactive search explorer are on the project page.

Figure 1 of RE-PoLar: frozen transformer layers routed per token as skip, keep or repeat, next to a thalamo-cortical analogy. — **Figure 1.** A program-of-layers executes *skip*, *keep* and *repeat* operations over frozen transformer layers (left), read as analogous to the thalamus routing computation across cortical areas (right).

27 SEP2026 Meeting

SWAMBIT at the first NI:KI project meeting

We introduced SWAMBIT at the first NI:KI project meeting and took a team picture.

The SWAMBIT team at the first NI:KI project meeting

14 SEP2026 Poster

Poster at the NEAT workshop

We presented the RE-PoLar poster at the NEAT workshop. Our LinkedIn post about it:

1 DEC2025 Launch

SWAMBIT officially starts

SWAMBIT began on 1 December 2025 as one of the 11 pilot projects of the BMFTR-funded NeuroAI (NI:KI) initiative.

Research focus

SWAMBIT builds AI that is functionally inspired by the brain: many small, sparsely active specialists instead of one dense monolith, coordinated through a shared workspace and enriched by a hippocampus-like memory. The goal is models that are more efficient and easier to extend.

Specialist Modulescortical macro-columnsSmall, interpretable transformers per modality and task; only a sparse few are active at a time.
Shared Global WorkspacethalamusA sparse write bottleneck where modules share one representational space.
Memory BenchhippocampusA hippocampus-inspired pointer memory, supplying factual knowledge on demand.

Publications

Programs-of-Layers in LLMs through the Lens of Cortical Areas RE-PoLar

Justus Westerhoff*, Stephan Olbrich*, Hatem Oraby, Walter Senn, Matthew Evan Larkum, Felix Gers

arXiv preprint arXiv:2609.31360, 2026 · *shared first authorship, random order

Paper Code Project page Poster

@article{westerhoff2026repolar,
      title={Programs-of-Layers in LLMs through the Lens of Cortical Areas},
      author={Westerhoff, Justus and Olbrich, Stephan and Oraby, Hatem and Senn, Walter and Larkum, Matthew Evan and Gers, Felix Alexander},
      journal={arXiv preprint arXiv:2609.31360},
      year={2026}
}

Cortico-subcortical multi-head self-attention as a substrate for cognitive performance

Vasanth Kumar Babu, Arno Granier, Timothée Proix, Titas Balvocius, Marmaduke Woodman, Ausra Saudargiene, Viktor Jirsa, Walter Senn

bioRxiv, 2026

Paper

@article{babu2026cortico,
  title={Cortico-subcortical multi-head self-attention as a substrate for cognitive performance},
  author={Babu, Vasanth Kumar and Granier, Arno and Proix, Timoth{\'e}e and Balvocius, Titas and Woodman, Marmaduke and Saudargiene, Ausra and Jirsa, Viktor and Senn, Walter},
  journal={bioRxiv},
  pages={2026--08},
  year={2026},
  publisher={Cold Spring Harbor Laboratory}
}

News

RE-PoLar at the AI Grid Summit 2026

Poster at Bernstein Conference 2026

RE-PoLar is out

SWAMBIT at the first NI:KI project meeting

Poster at the NEAT workshop

SWAMBIT officially starts

Research focus

Team

Publications

Programs-of-Layers in LLMs through the Lens of Cortical Areas RE-PoLar

Cortico-subcortical multi-head self-attention as a substrate for cognitive performance