One Letter Holds a Language Model’s World Map Together

One Letter Holds a Language Model’s World Map Together

A language model has never seen a map. Everything it knows about where the continents are, it pieced together from text. In August 2025 henry's post on the outsidetext Substack, How Does A Blind Model See The Earth?, made that knowledge visible: ask a model "land or water?" at each point of a latitude-longitude grid, color each point by the answer, and a world map appears. A later repository, egangu/land-water-eval, added a scored version against real coastlines.

We wanted to go one step further than drawing the map. Before measuring anything, we wrote down predictions: bigger models draw sharper continents, errors pile up on coastlines, a probe on the model's hidden state knows more than the model says, post-training helps, and asking in Korean makes the map worse. Then we measured. Most of those predictions were wrong, and the way they were wrong led to the most interesting result: asked in Korean, the map turns mirror-symmetric.

How we asked

• Models: Qwen3.5 base models at 0.8B, 2B, 4B and 9B, plus the post-trained 4B and 9B. Later, Gemma 2 9B base as a model from another family. bf16 on one A100.
• Grid: the cell centers of a 1° grid, 64,800 points, labeled land or water from the GSHHG 2.3.7 coastline (the labels egangu/land-water-eval ships). The language experiments use the 2° grid, 16,200 points, except the first Korean-versus-English comparison (9B, 0.679 against 0.611), which is on the 1° grid.
• Question: egangu's base-model prompt, unchanged: Latitude: 37.5° N, Longitude: 122.5° W. then Question: Is this location land or water? and Answer:. We read the model's probability of continuing with " Land" versus " Water". Nothing is generated.
• Score: AUROC, which asks whether land points get higher "land" scores than water points, regardless of where you put the cutoff. 0.5 is chance, 1.0 is a perfect map. For reference, answering "water" everywhere already gets 66.27% accuracy on this grid by cell count (71.2% if cells are weighted by area), which is why we don't lead with accuracy.
• Two corrections we added after a review: a degree grid packs the poles with cells that cover little area, so we also report AUROC with each cell weighted by its area. Neighboring cells are not independent, so intervals come from resampling whole 10° blocks (1,000 draws).

Bigger models draw the continents better, but not the smallest step

We predicted a clean climb. The plain AUROC says 2B is below 0.8B. The interval says that gap (−0.016 [−0.043, +0.011]) can't be told from noise, and with area weighting the order flips back. From 2B up, every step is clearly positive: in plain AUROC, 4B over 2B by +0.040 [+0.020, +0.061] and 9B over 4B by +0.061 [+0.046, +0.078]; area-weighted, +0.049 [+0.025, +0.072] and +0.039 [+0.026, +0.055]. None of these intervals includes zero.

The last column scores each map only within strips 10° of latitude tall, split into east and west halves. That removes the easy part, knowing that the far south is land and the latitudes around 60° S are open ocean, and keeps the hard part: where the continents sit along each strip. There, the climb is clean: 0.569, 0.624, 0.670, 0.726.

Coastlines didn't behave the way we expected either. We predicted coastal cells would be wrong at least twice as often as inland cells. The ratio was 1.3 to 1.5 for every model. The models aren't sharp anywhere for coasts to stand out.

Nobody puts Antarctica on land

The largest single error is the bottom of the map. South of 70° S, 87% of the cells are land (78% by area). The 0.8B to 9B base models called 0 to 1% of them land.

A reviewer pointed out that Antarctica is an ice sheet, and "land or water" might be read as "ice is water". So we asked the 9B again with Is this location land (including land under an ice sheet) or open water?, having committed in advance that 30% or more land answers would mean the wording was the problem. It went from 0% to 2.9%. The wording didn't move Antarctica onto land.

The model isn't blank down there, though. Within the region south of 60° S, it separates land from water somewhat (AUROC 0.714, 0.794 with the ice wording). Its answers for the whole region simply sit on the water side of everything else it says.

The map behind the answer didn't know more

Our favorite prediction was that the model knows more than it says. We took the hidden state at the moment the model is about to answer, trained a small linear classifier (a probe) to read "land" from it, and scored that probe's map against the spoken one.

To keep the probe from simply memorizing the map, we trained it on one hemisphere and scored it on the other. Without that split, a probe trained and tested inside the same hemisphere reached 0.87 to 0.94, numbers that mostly reflect neighbors looking like neighbors.

Every number here is AUROC on the hemisphere the probe didn't train on, averaged over both directions, which is why the spoken-map values differ from the whole-globe table above. The probe reads one of nine layers (depth 0, 1/8, …, 1). The layer and the regularization strength were chosen inside the training hemisphere, by cross-validation over six 30° longitude blocks; the scored hemisphere played no part. The layer was chosen separately for each direction (trained on the west, trained on the east). In that order the chosen layers were 15 and 24 of 24 for the 0.8B, 24 and 18 of 24 for the 2B, 28 and 24 of 32 for the 4B, and 20 and 24 of 32 for the 9B.

The probe beats the spoken map by a wide margin. But a classifier that sees nothing except the latitude band, trained on the same labels, scores 0.789. The probe only beats that by 0.007 at 4B and 0.033 at 9B, short of the +0.02 at both sizes we had required. And inside latitude bands, where latitude is no help, the order flips: 0.670 spoken against 0.614 probe at 4B, 0.726 against 0.643 at 9B.

So the probe's advantage was mostly something we handed it: from the labels it learned that the far south is land. The spoken map, which never saw a label, knows more about where the continents are along each latitude.

Ask in Korean, and the map turns symmetric

Asked in Korean, the 9B map looked strange: the left half and the right half were nearly mirror images. The real world map is not symmetric. A symmetric map means the model has lost the information that tells one hemisphere from the other: east from west, or north from south. A model that ignored east and west completely would draw the same thing on both sides, and the east-west symmetry below would approach 1.

How symmetric

To put a number on it, we take each map, subtract each latitude row's average (so that latitudes with more land don't inflate the result), and correlate it with its own left-right mirror image. We call this the east-west symmetry; 1 would be perfectly symmetric. On the 2° grid of the language runs, the true map scores 0.01. The 9B asked in English scores −0.14; asked in Korean (위도: 북위 37°, 경도: 서경 123°, "north latitude, west longitude"), 0.84. Both are means over the same two wordings used in the table below, with 10° block-bootstrap intervals of [−0.19, −0.09] and [0.81, 0.86]. With four wordings each, English stays at −0.14 and Korean is 0.78 (0.64 to 0.87 by wording). Those intervals only cover which cells of the map are sampled; the spread between wordings is larger.

What makes it symmetric

Then we tried to find out why, writing a prediction before each run. The grid below is the 9B, averaged over two wordings of the question per cell.

Here is what each check ruled out.

• The Tokyo hypothesis. In Korean, 동경 ("east longitude") is spelled and pronounced exactly like 동경 (Tokyo). Spelling it out as 서쪽 경도 ("western longitude") left the symmetry at 0.82. Not the cause.
• The answer word. Keeping the Korean question but reading the English " Land" and " Water" as the answer still gave 0.77. The symmetry comes from how the question is read, not in the Korean words 육지 and 바다.
• Not only direction words. Signed degrees reduce the Korean east-west symmetry to 0.64, but it stays even with no direction word in the prompt.
• Japanese, which writes east longitude as 東経, a different character from Tokyo's 東京, turns symmetric too (0.69).

Top to bottom, too

We measured north-south the same way, this time subtracting each longitude column's average and flipping the map top to bottom. Here the true map is not at zero: land crowds into the northern hemisphere, and the true map scores −0.31. So for north-south the reference for a correctly drawn map is −0.31, not 0. Both axes for the 9B, two wordings each:

The north-south column shows that English isn't clean either. Its direction-word map sits at 0.34, well above the true −0.31, so asked in English the map is already fairly symmetric north to south. Korean signed degrees at −0.10 have not "lost the symmetry" so much as moved most of the way back toward the true map.

The English signed-degree row says something else. Asked in English with signed degrees, neither axis turns symmetric (0.05 east-west, −0.46 north-south). Reading the minus sign on latitude but missing it on longitude happened only with the Korean question.

Putting it together:

• With Korean direction words (north latitude, west longitude), the map turns strongly symmetric both left-right and top-bottom.
• With signed degrees, only the left-right symmetry remains. The model reads the minus sign on latitude and misses it on longitude.
• With N, E, S and W letters, the left-right symmetry disappears and north-south returns to about where the English direction-word map is (0.25 against 0.34).

Comparing just the signed and the lettered versions, the cue this model uses to tell east from west in a Korean question is in effect the single letter W after the longitude. That sentence holds only for that comparison.

Why it is symmetric is still open

A symmetric map fits two pictures: the model misreads west as east and draws Asia where the Americas are, or it ignores east and west and draws the same average map on both sides. We built both reference maps from the East and West answers already in the English map and committed in advance that "ignore" would be closer by at least 0.1. For one wording "ignore" was closer, 0.53 against 0.46; for the other "misread" was, 0.46 against 0.40. Neither reached 0.1. What we can say is that hemisphere information is lost.

Is it about Chinese training data?

Qwen is trained on a lot of Chinese. That would explain why the Chinese map is much less symmetric than the Korean and Japanese ones (0.13 against 0.78 over four wordings, though the Chinese wordings ranged from −0.06 to 0.40). If that's the reason, a model without that emphasis should turn symmetric in Chinese too. We committed that test before running it on Gemma 2 9B.

On the raw numbers, Gemma turns as symmetric in Chinese as in Korean, and it passes the rule we set in advance (both at 0.5 or more). But Gemma's English map already has an east-west symmetry of 0.38, and inside latitude bands it scores 0.455: it barely knows where the continents are. Below 0.5 may even mean a slight tendency to put continents on the wrong side within a latitude. Measured as the increase over English, the picture changes.

Gemma looks less like "it turns symmetric in Chinese too" and more like "it reads east and west poorly in any language". So this comparison doesn't let us say the training-data explanation passed a test. It doesn't contradict it either. Separating the two would need another family whose English map comes out properly.

The Qwen3.5 4B shows the same order as the 9B: Korean 0.63, Japanese 0.64, Chinese 0.49, English −0.16 with direction words. One cell disagrees: the 4B turns symmetric with English letter abbreviations (0.52) where the 9B does not (−0.54).

Two smaller scars

Two marks show up across many maps.

• Seams at the equator and the prime meridian. For the 4B and 9B, some of the largest jumps between neighboring rows and columns sit exactly where N switches to S and E switches to W.
• A faint X. Where the latitude and longitude are the same number (33.5° and 33.5°), most models lean toward "water" by 0.3 to 0.6 standard deviations of their own map. Shift the diagonal by a degree or more and the effect disappears, and the true map has none. The 2B and the post-trained 9B don't show it.

We first suspected tokenization. But Qwen3.5's tokenizer splits numbers into single digits: Latitude: 33.5° S becomes 3, 3, ., 5, one token each. So these aren't artifacts of how multi-digit numbers are cut into tokens. A repeated number seems to change the answer on its own.

How our predictions did

Six of seventeen, two of them conditional: wording stability, explained below, and the Gemma Chinese call, which passed the rule we set but, measured against Gemma's own English map, doesn't carry the weight the rule implied. The ones we got right were mostly the ones closest to the data we already had. The interesting results came from the wrong ones.

Two of the first-round calls need their context. "Korean scores lower" was right (9B: English 0.679 against Korean 0.611, both AUROC on the 1° grid). "Wording barely matters" was right by our cell-weighted rule, but with area weighting the 4B's three English wordings differed by up to 0.137, so we don't lean on it. And post-training went both ways: +0.008 for the 4B, within noise, and −0.068 for the 9B.

Limits

• One model family for most of the work, one other model for one check. "Asked in Korean, the map turns symmetric" is a measured fact for Qwen3.5 4B and 9B and Gemma 2 9B, with the wordings we used. It is not a statement about language models in general.
• We read two answer words' probabilities. A model asked to reason, or a chat model with its own template, may draw a different map.
• Why Chinese is less symmetric in Qwen is open. The Gemma comparison didn't settle the training-data explanation.
• We can't yet tell misreading west as east from ignoring east and west.
• Several analyses were added after we saw results (area weighting, latitude-band AUROC, the shape of the symmetric map, north-south symmetry, the east-west symmetry and its intervals). The plan file marks which, with timestamps.

The plan with every prediction and its timestamp, the scripts, and every map's raw scores are in the reproduction package below.