What happens when you let a model look around?
Every geolocation benchmark hands an AI one frozen photo and asks where it was taken. But nobody plays GeoGuessr that way. GeoAgent drops vision-language models into Street View and lets them walk.
closer to the centre is better
locations
continents
evaluated
actions
round, max
conditions
We introduce GeoAgent, a collection of geolocalization data and an embodied navigation environment, showing significant limitations and insights into LLM-based agentic navigation.
Modern Vision-Language Models (VLMs) perform well above the human baseline in geolocalization tasks. However, most efforts to study AI behavior on the task remain limited to static image-based retrieval, classification, and predictions. We argue that faithful recreation of the task should involve embodied navigation, where a multimodal agent autonomously explores its surroundings to gather observations before submitting a prediction. To this end, we introduce GeoAgent, an agentic environment-based benchmark that requires agents to navigate Street View environments to refine their geolocalization through reasoning sequentially. Our analysis shows that modern VLMs struggle to discern regional patterns while succeeding at country- and continent-level predictions. When compared to static image-based baselines, agentic navigation significantly improves accuracy across established metrics. We also note severe bias in a developed/developing region context across frontier model architectures and poor self-improvement capabilities given incorrect priors. Overall, our work establishes the challenges of embodied navigation and geospatial reasoning.
A photograph can’t tell you what’s behind you.
Static benchmarks measure recognition. The thing that actually makes geolocation hard is deciding where to look next.
When a person plays GeoGuessr, the opening frame is rarely the useful one. They spin around hunting for a street sign, walk to the junction to read a shopfront, glance down to check which side of the road the traffic keeps. The identifying detail is somewhere in the scene — it just isn’t in the direction the camera happens to be pointing.
Static benchmarks freeze that decision away. A model gets a fixed set of views and must infer from whatever they happen to contain; if the decisive cue lies around the corner, it is unreachable forever. Even video-based geolocalization keeps the model a passive observer riding somebody else’s camera path.
GeoAgent restores the missing half. The model is dropped into a fully navigable Street View panorama with a toolbox, and after every move it sees a fresh screenshot and revises its guess.
Passive inference tests recognition, not reasoning under uncertainty.The paper’s central argument, paraphrased
Nine actions. Eight turns.
Rotate, walk, tilt, or return to the start — then commit. A running hypothesis, revised after every observation, with the agent choosing what to look at next.
Nine actions, eight turns, one commitment.
The model never touches a mouse. It emits a tool call, the environment moves the camera, and a fresh screenshot comes back.
Every turn the model returns the same five JSON fields — what it observes, its reasoning, a confidence, the action it wants, and its current guess — carrying up to five prior observations as context. Look, think, move.
One design choice matters more than it looks: the model never emits coordinates. It names a place, and the environment resolves that name through the Geocoding API. This kills an entire failure mode where a model identifies the right city then hallucinates wrong numbers for it.
Step through a round to see the loop tighten →
Built to defeat memorisation.
A benchmark of famous landmarks measures how much of the internet a model swallowed. This one is weighted deliberately the other way.
100 cities, stratified into five recognisability tiers by annual visitor numbers — a proxy for how heavily a city features in training data. Then the distribution is inverted: Tier 5 supplies 40% of all samples, Tier 1 just 2.85%.
For each city, 100 coordinates were sampled within a two-mile radius; points without navigable Street View were dropped, leaving 1,200 locations. Cities were also tagged developed or developing per UNCTAD — 46 and 54 — which sets up the study’s most uncomfortable finding.
Because the hard tier is deliberately over-weighted, absolute accuracies here are not real-world performance estimates. They are a stress test. The paper asks that results always be reported stratified by tier and development level.
Switch the condition. Watch what embodiment changes.
Same models, same 1,200 locations, five different ways of looking. Pick a condition and a metric — bars are on a shared scale, so they stay comparable as you switch.
Show the full results table (all 5 conditions × 6 models)
| Model | Mean dist. | Median | Reas. len. | Cont. | Country | City |
|---|
Exploring helps — most of the time.
Gemini 3.0 Flash is the clearest case: city accuracy climbs from 39.5% on the static composite to 50.1% when allowed to navigate, a 26.8% relative gain, with median error collapsing from 118.5 km to 5 km.
But two of six models do not benefit. Llama 4 Scout gets worse with the controls. And Geo-R1 7B — the model purpose-built for geolocation — is the only one that degrades against both static baselines. Static skill does not transfer.
Is it the reasoning, or just more pixels?
To find out, the team ran a random walk: the same number of moves, chosen at random. Every general-purpose model beat its own random-walk twin — GPT-5 Mini improved from 1,581 km to 900 km mean error.
Try it above: switch between Random walk and Agentic. The exploring is doing real work.
Down from 118.5 km on a static composite of the same location. That is the difference between naming the right region and standing on the right street.
Exploration refines a hypothesis. It rarely rescues one.
The sharpest result in the study only becomes visible once a model can act over time.
Split every round by how it ended. For rounds the model eventually got right, extra actions steadily pull the guess toward the truth. For rounds it got wrong, extra actions do almost nothing — the curve flattens, and for some models it reverses.
What separates them is where the model started. A logistic regression predicting final accuracy from initial distance gives β = −0.43, p < 0.001.
Confirmation bias in sequential reasoning
Models struggle to abandon an initial misconception even when shown contradictory visual evidence. Gemini 3.0 Flash is the starkest instance — on rounds it ultimately gets wrong, its guesses actively collapse beyond about five actions.
Every model was twice as lost in the developing world.
Not slightly worse. Not only on the hardest tier. Systematically, across architectures, open and closed alike.
The obvious objection is that this is Street View’s fault, not the models’. Coverage is famously uneven; surely developing-region panoramas are older and blurrier?
The authors went and checked, measuring image age, Google’s internal quality score, and empirical sharpness across all 1,200 panoramas. The result contradicts the assumption.
developing newer · p=.034
developing higher · p=.030
no difference · p=.70
Developing-region panoramas are statistically newer and marginally higher quality. The gap is in what the models learned to see, not in the pixels.
One model spun in circles. Another just walked.
Identical tools, identical budget — and strikingly different habits. The habits predicted the scores.
Rotation and movement are not equivalent. Spinning in place resamples a panorama the model has largely already seen. Walking forward reaches genuinely new information — a different junction, another shopfront, a new sign.
The correlations agree: move actions associate positively with every accuracy measure, more strongly than raw action count. But all effects are modest (|ρ| < 0.25), which the paper reads as action quality mattering more than action quantity.
A quieter pattern sits in the same data: models explore developed-region locations slightly more (5.95 vs 5.65 actions), and when they do fully explore a developing-region scene they rotate rather than walk, 2.1 : 1 against 1.4 : 1.
Lifting the 8-action cap tripled Llama 4 Scout’s actions to 23 per round and worsened its error by 76%. GPT-5 Mini doubled its actions and got 35% worse. Only one of three models improved, and it did so while taking fewer actions. The cap is not a compromise — it is the finding.
They know the continent. They rarely know the town.
Country accuracy runs 52–81%. City accuracy runs 14–50%. That cliff is the most consistent shape in the study — and humans fall off it too.
Macro cues are easy and abundant: vegetation, climate, driving side, general building stock. What fails is municipal infrastructure, local business signage, regional architectural variants.
The TF-IDF analysis over the models’ own reasoning text supplies the mechanism, and it is uncomfortable reading. Models frequently produce exactly the right visual descriptors — “tropical vegetation”, “weathered infrastructure”, “motorcycle traffic” — and then map them to the wrong conclusion.
Thai locations are labelled Vietnamese 23% of the time; Vietnamese ones Thai 18%. And the countries with the worst misclassification rates — Mozambique at 67%, Thailand at 54% — have no country-specific vocabulary at all in the models’ reasoning. When distinctive cues are missing, models fall back on generic regional description, and generic description cannot disambiguate.
The models aren’t hallucinating — they’re under-resolving.
The vocabulary analysis picks up genuinely diagnostic detail: gingko for Japan, tram tracks and shutters for Switzerland, the Chao Phraya for Thailand.
Vocabulary overlaps only 36.6–52.7% between agentic and static conditions, so these are model habits — not artefacts of the environment.
Every model beat the people.
Six undergraduate volunteers — beginner to intermediate GeoGuessr players, required to think aloud into a speech-to-text tool so their reasoning chains could be recorded — played the same rounds. Their median error was 996 km; their city accuracy 7.1%. Every model, in nearly every condition, beat them.
Two caveats keep this honest. No expert players were recruited, so the supported claim is that frontier models exceed non-expert humans — skilled GeoGuessr players are formidable, and that comparison stays open. And between-annotator variance was large, from 0.20 to 0.71 continent accuracy.
Notably, the country-to-city collapse shows up in the humans too: 46.8% → 4.3% for the strongest annotator. That asymmetry looks like a property of the task itself, not a deficiency of models.
error
median error
accuracy
accuracy
What this study does not show.
It is not a map of the world
1,200 locations across 100 hand-chosen cities is an evaluation harness, not geographic coverage. The tier weighting over-represents obscure cities on purpose.
The humans were not experts
Six undergraduates, beginner to intermediate. The supported claim is about non-expert humans, and the authors say so themselves.
The action space is simplified
Nine actions mirror Street View’s capture topology but exclude zoom, multi-step planning, and cross-modal lookup — all things a human uses freely.
On misuse
Better agentic geolocalization lowers the barrier to surveillance and unauthorised location inference. The authors recommend restricted API access and geofencing downstream.
GeoAgent runs only on public Street View imagery, which Google already blurs for faces and licence plates, and processes no user-generated content. The dataset should not be used to train or fine-tune models, to power production geolocation services, or in any application that could identify individuals incidentally captured in Street View imagery. Metadata is released; raw imagery is not redistributed.
Five things to take away.
Embodiment changes what you can measure
Letting a model act exposes confirmation bias, exploration style, and self-correction failure — none of which a static benchmark can see.
Refinement, not recovery
Gains concentrate where the opening hypothesis was already close. Bad first guesses are rarely recovered — and often reinforced.
The gap is behavioural, not photographic
Roughly 2× worse error in developing regions, with an image-quality audit ruling out the obvious data explanation.
More budget is not more intelligence
Unlimited actions cost up to 56× more per round and made two of three models less accurate.
Static skill does not transfer
The model purpose-built for geolocation was the only one to get worse when finally allowed to navigate.