What was released
Jure Leskovec's SNAP group, with co-authors from Toronto, Penn and MIT, has released Memorilla as open source, described by the source MarkTechPost as a memory system for large language models. The central idea is to reduce an entire collection of documents to 16 memory tokens, regardless of the archive's size, and deliver them to an LLM that remains frozen. Code, training recipes and checkpoints are public under the Apache 2.0 license.
How it works and what it is for
The trainable module has 194,123,776 parameters for the 8-billion decoder. The encoder, Qwen3-Embedding-4B, and the decoder, Qwen3-8B, are not updated: only the memory module is trained. The pipeline involves three steps: one-time encoding of the documents, per-question reading through 16 learned slots that use two levels of eight-head cross-attention, and injection of the vectors into the decoder's embedding space, ahead of the question.
Training is divided into three phases: about 1.98 million Wikipedia paraphrase-reconstruction examples, then about 1.34 million examples from ten QA and summarization datasets, and finally fine-tuning on the eight benchmarks. The full recipe takes about 19 hours on four H100 GPUs. The indicated quickstart requires an 80 GB GPU on Linux with CUDA and Python 3.11; no quantized variants are declared.
Results and stated limitations
All baselines use the same frozen Qwen3-8B, and the scores are self-reported by the research team. On FactKG, Memorilla reaches 84.33% accuracy versus 80.43% for the full context, with 108 times fewer document tokens. On PersonaMem-v2 it reaches 44.60% versus 30.90% for xRAG; on TriviaQA it scores 57.00 exact match ahead of MemoRAG at 56.60; on LaMP-4 and LaMP-7 it obtains 18.15 and 48.37 ROUGE-L. The average across the eight benchmarks is 49.68, versus 41.92 for xRAG, 40.75 for LLMLingua-2, and 38.02 for Mem0.
The system, however, does not win everywhere. On PubMedQA it stops at 59.00% versus 69.00% for LLMLingua-2, because the abstracts total only 324 tokens and compression offers little margin. MemoRAG prevails on PersonalizationV4 (67.88% versus 66.65%) and on NarrativeQA ROUGE-L (20.51 versus 19.31), but according to the source it used about 690 and 50 times more document tokens. The compression peak is on NarrativeQA, where 16 tokens replace 67,385 tokens, a 4,212-fold reduction.
Why it matters and what remains to be verified
The significance lies in the trade-off between context cost and quality: on an H100, Memorilla responds in 0.74 seconds with 512 documents, about 128K tokens, while the full context goes from 0.75 to 19.9 seconds, 27 times slower. The memory module alone takes about 2 ms. An optional cache precomputes cross-attention keys and values: on a B200, a question takes 8.1 ms with 1 million documents and 33 ms with 4 million, dropping to 5.0 ms with torch.compile in the 1-million case, at the cost of 80 GiB of cache at 4 million. As memory for agents, 8 tokens take a TextWorld agent based on Qwen3-4B from 18.6% to 24.2% average success.
Open points remain. The results come from the paper and not from independent verification; checkpoints must be run with the decoder with which they were trained; personalized data require parquet files with collection_id, question, answer, and documents columns. The team has also released PersonalizationV4, with 149 synthetic users and 200 conversations each, under CC BY 4.0. The source also cites a Chrome extension and commercial links, promotional elements unrelated to the technical evaluation and not to be treated as proof of the results.
