What the Papers Held

Introduction

On 20 September 1866, the Fayetteville Observer in Tennessee gave its farmers a piece of advice:

On the same page, it made an argument about schooling in the same currency:

Neither sentence is news. Both are perspectives: statements about what is good, wise or worth doing. Local papers printed thousands of them in editorials, sermons, farm columns, obituaries and advertisements. One at a time they are small. Together they record what a country argued about, praised and took for granted.

This project asked a simple question. If a language model read a balanced sample of local American newspapers from 1850 to 1929, could it find the perspectives that kept coming back and build them into a map, while every point stays tied to the exact words on the page?

It could. A model read 1,600 articles from 735 newspapers and pulled out 2,766 claims, each backed by a verbatim quotation. The ones that recurred were grouped into 30 perspectives, 12 domains and 5 broad meta-perspectives, none of them supplied in advance.

The map shows perspectives printed in a sample of digitized local newspapers, not what Americans believed. Newspapers printed what editors, preachers, politicians and advertisers chose to print. That is what makes them interesting, and it is why every claim here points back to a page you can open.

1Reading for perspectives

The method is concept induction: a language model reads a large body of text and proposes the ideas inside it, instead of sorting documents into categories chosen beforehand. Most such work uses modern text. We tried it on American Stories, a dataset built by Melissa Dell and colleagues that recovers individual articles from the Library of Congress's Chronicling America page scans.

The hardest part was deciding what counts as a perspective. Left alone, a model treats a tax rule as a value and a land sale as a worldview. So it worked from a written definition:

A perspective is a value judgment, a norm about how people or institutions should behave, or an assumption about people, groups, or society. It is NOT a factual assertion, a rule or procedure, a price, a product feature, or an account of events.

A view was kept only if another paper of the time could plausibly have disagreed with it, and sellers praising what they sell never counted, though a value an advertisement appealed to could. Try the test yourself:

The model read one passage at a time: a whole article, or part of a long one. Each claim records who holds the view, how the passage treats it, and a short quotation copied exactly from the page. Code checks that quotation character for character, and a claim whose quotation is not on the page is dropped. Across the run, 97.3% matched exactly.

2From 2,766 claims to five big ideas

The sample was built to be fair: four twenty-year periods, four Census regions, and 100 articles drawn by a fixed rule for each of the sixteen combinations. No paper could dominate, reprints counted once, and nothing was filtered by topic.

From there the pipeline climbs a ladder, and the division of labor is deliberate: the model reads and names; code counts and groups.

For 38.5% of passages the model found no perspective at all, which is the right answer for a legal notice. Claims whose quotation was not on the page, or that stated a fact rather than a value, were set aside. Code then grouped the remaining 2,670 by meaning. One finding stood out: most things a newspaper said, it said only once. Only 814 claims landed in a group that recurred across papers and decades. The atlas is built from those. The model named each group, code gathered the groups into domains, and a final step asked what assumptions ran across all of them. Each of these levels has a short code: P for perspectives, D for domains, M for meta-perspectives.

3One sentence, all the way up

The ladder is easiest to understand by climbing it with one sentence. On 30 December 1906, the Americus Times-Recorder in Georgia printed an anecdote in which an Englishman in Constantinople, watching police beat a captured thief, protested:

The OCR is rough (read "a thief," "to prison," "a trial"), but the point is clear. Now follow it up:

That is what we find most exciting about this approach. You can start at a sweeping claim about eighty years of newspapers and, in five clicks, be looking at the column of type that supports it.

4What we found

Four ideas that ran through everything

The four meta-perspectives the model rated high confidence are shown below in its own words, unedited. The fifth, M4, was rated medium.

• Legitimacy through public trust. Officeholders were held to a standard of character and accountability, not just to the fact of having won.
• Disciplined improvement as a social good. Farming, schooling and organized labor were framed as ways to turn knowledge into welfare.
• Protective authority, conditional coercion. Force was accepted only with recognized authority, a protective purpose and a fair trial. The Englishman's protest sits here.
• Virtue as social reproduction. Moral worth was something communities taught, practiced through charity, and preserved in how they remembered the dead.

What rose and what fell

Because every period received 400 articles, spread evenly across regions, rates can be compared directly. They are shown per 1,000 articles, with 90% intervals computed by code.

• Political accountability peaked in the Gilded Age, at about 88 per 1,000 articles in 1870–1889, then fell to about 20.
• Moral regulation of conduct fell by about two-thirds.
• Improvement more than doubled, from about 22 to 52 per 1,000.
• Labor arrived. Absent before 1870, it reached about 22 per 1,000 by 1890.

Region mattered less than we expected. For most perspectives the regional intervals overlap (open any perspective to see its "By region" chart), and the papers differed more by generation than by geography.

Disagreement inside the consensus

A method that only found agreement would be suspicious. Because every claim records its stance, a perspective can hold opposing positions, and the model reported them plainly:

The model was told to describe historical views accurately, including offensive ones, and not to soften them. That is why M1, Legitimacy Through Public Trust, which celebrates popular judgment, also records that popular judgment excluded people.

5Can the map be trusted?

In a separate pass, the model re-read 100 random claims against their passages: 75 were fully supported, 24 partly, and only one not at all. Re-running the clustering on random subsets gave largely the same groups, and the model flagged none of the 30 perspectives as incoherent. Every count, rate and trend was computed by code.

6What this atlas is, and what it is not

None of the pieces here are new. Topic models, reprint detection and model-led concept discovery such as TopicGPT, LLooM and Clio all came first. What is new is the combination: the unit is a single perspective rather than a topic or a document, nothing at the top was fixed in advance, and every level traces down to a verified quotation.

The atlas describes what newspapers printed, not what people believed. Its claims are closest to the text and easiest to check. Its meta-perspectives are the model's interpretations, marked as such and shown with their counter-evidence. The sample is a sample, and the text is OCR. Used this way, the atlas does not replace reading the newspapers. It tells you which pages to read, and why they matter.

7Looking forward

A larger sample would make smaller perspectives and regional comparisons readable, and human-reviewed claims would give the audit an independent anchor.

We started with a Tennessee paper telling its farmers that good cultivation beats three mortgages. At the top of the ladder, that sentence is one piece of evidence for an idea we did not give the model: that across eighty years, improvement was treated as a moral project. That is what this method does well. It lets thousands of small, forgotten sentences add up to something, while keeping every one of them where you can still find it.