What makes Dory 1.0 4B different
The Dory 1.0 4B model, developed by NVIDIA's Nemotron Labs, is described as a small reasoning model whose internal architecture allows the same central block to be applied multiple times for each token. According to the MarkTechPost tutorial, the result is a single checkpoint capable of operating at four different compute budgets: from one pass up to four, without adding parameters or requiring new training.
The key point is that the additional cost does not concern model size but computation per token. Each extra pass entails more computational work, while the weights remain the same. The source presents this property as a kind of setting chosen at load time, rather than a hyperparameter to be defined during training.
A fork and a file with precise constraints
Support for Dory in llama.cpp is not yet available in a released version: the material describes it as a pull request still in draft. For this reason, the guide builds llama-server from the exact commit of the fork indicated in NVIDIA's model card, verifying that the checkout matches the expected one. On Mac, the card's Metal flags are used; on Colab, CUDA kernels compiled for the available GPU are used.
The model also has operational constraints. The Q5_K_M GGUF file, about 3.7 GB, is downloaded from Hugging Face at a fixed revision and verified via SHA-256 hash. The source notes a practical discrepancy: the model card's commands call the file dory-q5_k_m.gguf, while the repository names it differently, so copying the command verbatim fails. It is also noted that the GGUF header contains a default loop count of 4, while the card recommends 2: without an explicit override, the server would silently start with the more expensive setting.
Measurements and checks, with stated limitations
The guide reads the architecture directly from the file: 72 layers are divided into an input group of 16, a shared recurrent block of 40, and an output group of 16, with one full-attention layer every three sliding-window layers. From these data, resource requirements are estimated: four loops apply about 2.51 times the parameters of one and read about 2.28 times the bytes of the weights. The KV cache, with a context of 4,096 tokens, goes from 306 MiB with one loop to 816 MiB with four in per_loop mode, while last_loop mode remains at 306 MiB.
The predictions are compared with the real server in seven configurations, measuring memory, decoding speed, and consistency between accelerated execution and the CPU path. A paired evaluation is then conducted on a sample of GSM8K questions, with Wilson intervals, McNemar tests, and bootstrap on token counts. However, the source specifies that thirty paired questions are not enough to establish small differences in accuracy: the notebook is designed to say what the sample can and cannot show. According to the material, the model card's comparison table, the 256K context, and NVFP4 pre-training remain untested.
