Tablinum research · November 2026 · a hypothesis with measured findings
A small model can borrow its knowledge from a library, the way the Moon borrows the Sun's light. What we measured doing that, what training does and doesn't add, and the test of whether a small model can carry its own light.
Chad St Thomas · Tablinum · Hardware: one workstation (RTX A2000 12 GB, RTX 2060 6 GB, 4 × 128 GB Optane). Judge: Claude Haiku, fixed.
Our earlier paper, The Dose-Window Hypothesis, found that knowledge divides cleanly from judgment in small models: a fact table supplies the content, and the model supplies the decision of what to do with it. This page follows that split to its next step.
The Borrowed-Light Hypothesis: for a small model, knowledge is cheapest to supply from outside and measure there, and training is best spent on judgment: when to trust a passage, when to decline, how to reason. Under this split, model size can shrink while the library grows.
Nine measured findings support it, from the Tablinum engram (26 tables, about 109 million passages) and two trained models. The library supplies the knowledge (EngramBench index −58.6 → +65.1 with no training at all). Training adds calibration, not knowledge (the same 406 right, wrong answers 79 → 38). Small details decide whether it works: whose words you train on, decoding, which tables you attach, and which tensors you keep precise when you compress.
The open question is whether the library can move inside a small model, as large memory tables trained on one workstation, without making the model bigger. That is the Sun-Beam test, stated here with its predictions before it runs.
Tablinum ships three lines, and each puts the knowledge in a different place.
| Line | Light | Where the knowledge lives | What we train |
|---|---|---|---|
| Lite-Beam | Starlight, compressed | In a large open model's weights, quantized to fit small GPUs | Nothing: our calibration data and importance matrix |
| Moon-Beam | Borrowed | In a library beside the model (the engram), searched per question | Judgment: read the passages, answer or decline |
| Sun-Beam | Its own | In memory tables inside the model file (hashed n-gram tables) | The tables themselves, from the library's text |
The engram is a library of reference text split into passages of about 150 words, with a fast index over each table. For every question, one search over all tables returns the best three passages, and the model reads them with one instruction.
Use a passage only if it states the answer to this exact question. A passage about a related but different item, version,
year, rule or case does not answer it: then rely on your own knowledge, or say you don't know.
Written after the first engram runs, when wrong answers turned out to be copies of near misses: a passage about 1958 used for a question about 1959.
Moon-Beam 16B-A3B at 8 GB (IQ3) against its untrained base, Haiku-graded. None of these questions or passages was in training.
literature Q&A table
In "The Market Basing Mystery," what French phrase does Poirot say when Japp talks of being far from crime?
… "When I retire, I shall have a little place in the country. Far from crime, like this!" "Le crime, il …
untrained, no library"Comme ça, ça marche."
Moon-Beam + library"Le crime, il est partout" (Crime is everywhere).
philosophy table (SEP)
According to Zagebski's 2008 work, which property is entailed by omniscience?
SEP: Omniscience — … Zagebski’s claim is, rather, that omnisubjectivity is entailed by omniscience (2008: 232) …
untrained, no library"…being able to know all true propositions."
Moon-Beam + library"Zagebski's claim is that omnisubjectivity is entailed by omniscience (2008: 232)."
software docs table
In Julia, what umbrella type name refers to all one-dimensional dense arrays of any element type?
Julia 1.0 — UnionAll Types: … the umbrella type Vector has as instances all Array objects where the second parameter … is 1 …
untrained, no library"AbstractArray"
Moon-Beam + library"Vector. Writing Vector{Float64} is equivalent to writing Array{Float64,1}."
the hard case: a near miss
Who was appointed Chaplain of the United States Senate on January 4, 1947?
CRS report: House and Senate Chaplains: An Overview … wiki: Chaplain of the United States Senate. First holder: Samuel Provoost …
untrained + library"Reverend B. Marvin G. Hargraves" (invented)
Moon-Beam + library"None of these passages answers that exact question, so I won't guess."
The first three are the library doing its job: the answer is in a passage and the model reads it. The fourth is what training is for. The passages are about the right office but not the right year, and the untrained model fills the gap with a confident invention. The gold answer is Peter Marshall.
All knowledge numbers use the index 100 × (correct − incorrect) / n, so a wrong answer costs as much as a right one earns, and a decline costs nothing. Every comparison is paired on the same questions.
| Model (untrained) | Benchmark | closed-book | with engram |
|---|---|---|---|
| Moonlight-16B-A3B, 8 GB | EngramBench, 502 held-out, all 26 tables | −58.6 | +65.1 |
| Ling-3.0-tiny (1.3B active), 3.7 GB | EngramBench, own table, thinking off / on | −22.1 / −11.2 | +80.5 / +82.5 |
| Beam 501B-A23B (behind Lite-Beam 1.0), IQ2 | SimpleQA-verified dev 200, Wikipedia table | 6.5 | 50.5 |
A 1.3B-active model with the library scores far above a 16B model without it. On these questions the library, not the parameter count, decides the result.
| Moon-Beam 16B vs its untrained base, IQ3, with the engram | untrained right / wrong | Moon-Beam right / wrong |
|---|---|---|
| EngramBench 502, all tables | 406 / 79 | 406 / 38 |
| SimpleQA-verified 1000 | 548 / 371 | 504 / 155 |
| AA-Omniscience test 400 | 96 / 266 | 78 / 144 |
With the passages in hand, the trained model finds the answer about as often as its base. What changes is that it stops answering when the passages don't hold it. The training data was 2,198 examples; the knowledge came from 109 million passages.
Round 2 of Moon-Beam trained on another model's short, correct answers. It learned confident brevity, not knowledge: answers fell from 1,887 to 440 characters and every benchmark got worse. Round 4 instead replayed the base model's own verified GSM8K solutions alongside the engram data, and math recovered from 71.8% to 82.0% (base 83.5%). Moon-Beam 8B is trained only on its own checked reasoning, with Haiku as the checker, never the author.
| Ling-3.0-tiny, untrained, with engram | thinking off | thinking on |
|---|---|---|
| SimpleQA right / wrong / declined | 586 / 114 / 291 | 592 / 37 / 365 |
| AA-Omniscience dev, index | +11.0 | +16.5 |
| GSM8K 400 | 93.5% | 96.2% |
The same number of right answers and a third of the wrong ones: the thinking model checks the passage before it commits. One caveat cost us a day. Greedy decoding loops in thinking mode (1 of 12 hard items right, 11 stuck at the length cap); the model card's sampling (T 1.0, top-p 0.95, top-k 20) gets 9 of 12.
Adding a table is not free. On Beam, the health table lowered the index from 6.1 to 3.0 and open textbooks from 16 to 9. On Qwen3.8, humanities tables took the score from −17.6 to −26.5, because near-miss passages invite confident wrong answers. Law and software tables won big on both. The rule: a table ships for a model only if it helps that model and wrong answers don't rise. The v1 instruction (above) is what made the near-miss problem manageable.
| Moon-Beam 16B, Apple MLX builds | size | KLD vs BF16, chat / wiki |
|---|---|---|
| Plain 4-bit (the default conversion) | 8.99 GB | 0.047 / 0.096 |
| MLX mixed 4/6 recipe | 9.65 GB | 0.037 / 0.078 |
| Ours: experts 4-bit, everything else 8-bit | 9.77 GB | 0.010 / 0.024 |
Attention, shared experts and embeddings run for every token, so rounding error there reaches every prediction. Keeping them at 8-bit cost 0.8 GB and removed about four-fifths of the error. The same principle sets our GGUF layouts. Paired with the engram, the right compression goes surprisingly far: Lite-Beam 27B at 6.73 GB (1-bit) took SimpleQA from −14.0 to +43.0.
Searching all 26 tables at once puts the question's own subject in the top three for 95.2% of EngramBench items and costs about one index point against sending each question to its own table. A single universal library is simpler to ship and nearly free in accuracy.
Moon-Beam 16B sometimes claims it can search the web or run code. We tried fixing that with a default system prompt. Every version we tested cost accuracy on easy facts: 20 of 24 right with the original prompt, 13 to 17 with the new ones ("What is the capital of Australia?" became "I can't, I have no internet"). A model trained to be cautious hears any mention of its limits as a reason to decline. Moon-Beam 8B learns what it is and what it can do in training.
From the earlier paper: handing a small model a retrieval tool scored 0.51–0.63 against 0.94 for retrieving before inference, through three separable failure modes. Moon-Beam's library is injected, not offered. For users this means the model and its library have to arrive together, set up in one step.
Retrieval works, but it lives outside the model file: a separate download, an index, and a prompt the user has to know about. Large models are moving the other way. DeepSeek's Engram and Qwen3.8-Flash-Next add hashed n-gram memory tables inside the model: a huge lookup indexed by the last few tokens, cheap to read, able to sit in slow memory. Qwen's table is 51B parameters beside a 6B-active trunk, and llama.cpp runs it straight from the GGUF files.
A small model's knowledge capacity is set by the size of its memory tables, not the size of its trunk. Trained on a library's own text plus question-form rephrasings of the same facts, the tables of a frozen 3B model can store facts that it recalls from a question with no retrieval at all, on one workstation. Because the trunk no longer has to hold the facts, it can be compressed harder while the table stays full size.
In October we put an Engram-style memory into Ling-3.0-tiny. It was harmless and nearly useless: no change at Q4 and Q1, +0.02 to +0.03 at Q2. In hindsight the table had nothing to remember. It was trained on a small agent-chat corpus, while Qwen fills its table during pretraining on trillions of tokens. A memory table learns only what passes through it. This test trains the table on the knowledge itself.
The base is Nanbeige4.2-3B (Apache 2.0, 256K context, a looped 22-layer trunk). Its released code already contains hashed 2- to N-gram tables and a gate that feeds them into the model, switched off in the published weights. In a first run in October, with the trunk frozen at 4-bit and the tables in system memory, 82 steps on a 6 GB RTX 2060 took held-out loss from 1.484 to 1.217.
P1 · untested
Recall without search. After training on one subject's text and rephrasings, held-out questions about those facts improve closed-book, measured with the same index and judge as the engram.P2 · untested
Memory, not trunk, sets recall. Doubling the table raises recall more than moving the trunk from Q2 to Q4; a Q2 trunk with a full table matches a Q4 trunk with a small one.P3 · untested
General skills hold. With the trunk frozen and the gate starting closed, GSM8K and MMLU-Pro stay within noise, and the 256K context still works.P4 · untested
Subjects add up. A second subject's table neither erases the first nor raises wrong answers on it. If this holds, the model's memory is extended one subject at a time.If P1 fails, the Borrowed-Light split stands as it is: knowledge stays in an attached library, shipped with the model in one step. If P1 and P2 hold, the size ladder for Sun-Beam becomes memory size (S, M, L, XL) instead of bits, and the knowledge travels inside the model file.