What EarthRecord measures
A research group has presented EarthRecord on arXiv, a benchmark designed to assess how well multimodal agents can turn long-term satellite archives into reliable analyses of land change. The article description, published on November 11, 2026, indicates that the testbed is based on raw data from 56 sites and four decades of observations from Landsat, MODIS, VIIRS and Sentinel-2.
The novelty claimed by the authors concerns the starting point: not pre-selected images or preprocessed inputs, but multi-sensor archives from which the agent must begin. Tasks such as cloud masking and harmonization across different sensors remain the agent’s responsibility. The benchmark poses 238 questions about land change, distributed across ten types of analysis, and evaluates them by comparing them with references not provided to the agents, with the degree of agreement among methods, and with visual interpretation.
How the agents are evaluated
The test structure includes two controls designed to clarify what the answers are based on. In the first, described as closed-book, the archive is removed and the agent cannot consult it. In the second, data from a single satellite are altered while nothing changes on the ground: if the answer follows the instrumental modification, it means the agent is reading the sensor and not the territory.
The authors report having evaluated twelve multimodal agents. According to the reported results, the leading ones complete most analyses autonomously but show difficulty when the answer requires deriving information from the image mapping. This is an important distinction: knowing how to execute a procedure is not the same as knowing how to correctly interpret visual data.
The boundary between pixels and the land
The additional analysis offers the most critical finding of the work. The record, the authors write, shifts answers from prior knowledge to pixels, but not yet toward real change on the ground. When a single satellite’s data are altered, half of the correct answers follow the alteration: a signal that the agent relies partly on instrumental input instead of reconstructing the physical phenomenon.
This observation clarifies why the benchmark is relevant to Earth observation science. A useful analysis must withstand sensor perturbations and distinguish what changes in the image from what changes on the ground. EarthRecord is presented as a platform for measuring and advancing agents in this transition, from pixels to land change.
Several aspects that the available material does not clarify remain to be verified: the exact composition of the sites, the criteria by which the evaluation references were constructed, and how the instrumental alterations were applied. These are necessary elements for judging how generalizable the results are beyond the cases described in the article.
