De Litteris · III

Interlinea

Validated Language-Model Annotation and Rules-First Scansion of Latin and Ancient Greek

Aeterna InstituteOctober 21, 202624 min read

Read with Interlineathe reader this paper describes

Abstract

Interlinea is a reading app for Latin and Ancient Greek. Tap any word of Virgil or Homer and it shows the lemma, the form, the syntax and a gloss, and verse arrives scanned. A language model does the analysis, but it never controls the form of the answer: code numbers the words, a validator checks every tree, and rules narrow each line of verse to the readings its metre allows.

I built it in 24 days, alone, by conducting a team of AI coding agents. This is the web edition of the paper that measures it, told as the story of how it was made. On 833 sentences of expert annotation, the best model finds syntactic heads better than parsers trained on the test treebanks, numbered words alone are worth 15.6 points of attachment accuracy, and models that choose among candidate scansions beat models that scan.

Keywords

Latin
Ancient Greek
large language models
Universal Dependencies
structured output
automatic scansion
coding agents

Read with Interlinea The paper (PDF)

An idea worth building

Prooemium

A reader of Latin or Greek spends most of an hour on the forms of words. Lesbia could be nominative, vocative or ablative, and amemus is a subjunctive a beginner reads as a future. Tools such as Morpheus and Alpheios (Crane, 1991; Alpheios, 2026) list every analysis a form allows and leave the reader to choose.

Language models can make that choice. With a few examples they match trained taggers on lemmas and parts of speech (Vidal-Gorène et al., 2026). They also fail in ways a learner cannot see. Asked for twelve words, a model returns eleven, merges an enclitic into its host, gives a form a case it cannot have, or builds a tree with two roots. A reading app that shows such an answer teaches the mistake.

So Interlinea rests on one rule. Code owns the form of the answer, and the model only chooses. The scenes below follow a sentence through the system that rule produced, then the platform around it, then the 24 days in which one person built all of it, and finally what a large evaluation found.

A sentence arrives

The first reader pays

A reader opens a passage of Virgil. The text appears at once, split into sentences in the browser, and each sentence is looked up by a fingerprint of its text. A sentence anyone has read before comes back from the store in milliseconds. The rest go to a model, once.

Code numbers the words

The words are not the model’s to write

Ordinary code splits the sentence into words and numbers them. Latin enclitics stay on their host, so populusque is one word, as a reader sees it. The model receives the numbered list and answers with one object per number. It can describe a word badly, but it cannot drop one, merge two or misspell any.

The model fills the slots

One object per number

For each number the model writes a lemma, a part of speech, the morphological features, the number of its syntactic head and the relation to it, an English gloss and a note for a learner. The answer must match a strict JSON schema whose every field is required, so a missing key fails at once.

The validator

Six checks no answer skips

Code then checks what the schema cannot. Every word has a head in range and no word heads itself. Exactly one word is the root. The heads form a tree, with no cycle. Every feature is one the language allows. A Greek aorist in the note matches the features. Each failure becomes a line of plain English the model can act on.

One correction turn

At most four calls, then honesty

An invalid answer goes back to the model once, with the validator’s messages. If the second answer fails too, a fallback model from another provider gets the same two tries. If that fails, the reader sees a retry button. A failure is never stored, so a bad answer cannot become everyone’s.

Stored once

Every later reader reads for free

A valid answer is stored under the sentence’s hash and shared by every reader who opens the passage after. The price of a sentence is paid once for the whole library, which is why the library can afford the best model and not only the cheapest.

Why the words are numbered

Verba Numerata

Of all the protocol’s parts, the plainest matters most. The model never sees a sentence it must split. It sees numbered words and answers by number, and that turns out to be worth more than the prompt’s examples, its allow-lists and its correction turn together.

An experiment

Let the model split the sentence itself

To see what the numbered words buy, I gave GPT-6 luna the same 500 gold sentences without them. It had to split each sentence, number the words itself and annotate them. Its words disagreed in number with the gold’s in 5.4% of sentences. It rewrote Seneca’s verse into prose order and twice dropped a non, which reverses the sentence.

Heads are indices

Arcs that slip one word

Most of the damage was not in the splitting. On the 458 sentences it split exactly as the gold does, its syntax was still far worse. A head is a number, and a model that numbers its own words points one word off: 11% of words, against 3.7% when the numbers are given.

The cost

15.6 points of attachment

Lemmas and features did not change at all. Attachment fell by 15.6 points, and even on the 458 sentences it split as the gold did, from 82.5 to 68.2. Of everything in the protocol, giving the model numbered words is worth the most.

III

A validator, and one second chance

Custos

A model reviewing its own reasoning rarely improves it (Huang et al., 2024). Given a concrete error from outside, it usually does (Kamoi et al., 2024). Interlinea’s validator is that outside voice. It knows nothing about Latin. It knows what a tree is.

What fails

Two roots, a cycle, Case=Fem

Of 833 sentences, GPT-6 luna’s first answer failed the validator on 57. Most of the messages were about trees: 39 answers had no root, two roots or a cycle. In 12 a value sat where none belongs, such as a gender in the case slot, and 9 named an aorist the features did not. Some answers failed more than one check.

The correction turn

Nine in ten repaired

The model sees its answer and the validator’s messages and tries once more. The turn left a valid answer for 89.5% of luna’s failures, 90.6% of Gemini 2.5 Flash’s and 89.2% of Gemini 3.8 Flash’s. A model checking its own reasoning rarely improves it; a model given a concrete error by code usually does.

Validity, not accuracy

The repair fixes form

Scored as they stood, the failed first answers were as accurate word by word as the repaired ones. What the validator catches is form: an answer that would break the diagram or show a learner a case that cannot exist. With the turn, 99.6% of answers are valid; without it, 93.6%.

A wall at 16,000

When the answer is cut off

Every one of Gemini 3.8 Flash’s 130 failures was different: an answer cut off before its JSON closed, because its hidden reasoning counts against the output cap. No answer that closed used more than 15,984 tokens. With a larger cap, 119 of the 130 came back valid in a single call.

Rules first, then a single question

De Metris

Verse needs a different division of labour. The rules of prosody settle most syllables and the metre settles most of the rest. What they cannot know is the length of a vowel the spelling hides. So the rules do all they can, and a model is asked only what they leave open, as a choice among numbered candidates. The scene below is scored on 600 lines from Hypotactic (Chamberlain, 2026).

Syllables

Vivamus, mea Lesbia, atque amemus

Rules split the line into syllables. Where a vowel meets a vowel across words, the first is elided, so Lesbia and atque lose their last syllable to the word after them, as Catullus’ ear did.

Position and nature

What the rules can know

A vowel before two consonants is long by position. A diphthong is long by nature, and in Greek so are η and ω. What the rules cannot see in a text without macrons is a long vowel by nature in Latin, the ī and ā of vīvāmus.

The metre

Four candidates

The hendecasyllable is a pattern the line must fit. Laid over what the rules know, it fixes every syllable but the free opening pair, and leaves four candidate scansions that differ only there.

The question

Pick a number

A model is shown the four numbered candidates and asked for one number, not a scansion. Outside dactylic metres a second model from another provider must pick the same one, or the line stays plain. A plain line costs a reader little; a wrong one teaches the mistake.

Choosing beats scanning

78.6% against 43.1%

On 600 scanned lines from Homer to Seneca, GPT-6 luna picked the right candidate 78.6% of the time but scanned the same lines whole correctly only 43.1% of the time. The rules alone scan 55.8% of the lines; with two agreeing models the system scans 83.8% at 96.6% precision.

The platform around it

Machina

An annotation protocol is only useful inside something people read. Interlinea’s platform is deliberately small: one app, one database, one way to the models, and a store that makes the expensive part happen once per sentence instead of once per reader.

The platform

A small machine around one idea

Interlinea runs on Vercel and Supabase. Readers and software come in through the same edge, and every request either reads what is already stored or asks a model. Only the second costs money, and it happens once per sentence.

Path one

Opening a library passage

The library holds 3,869 Greek works and 1,365 Latin ones, 3.7 million sentences in 321,827 sections, and the whole catalogue fits on a small plan. A passage someone has read before loads from the store.

Path two

The first reader of a sentence

A sentence no one has read goes to a model, once. Every call is budgeted before it is made, for each reader and for everyone together, so no flood of traffic can run up an unbounded bill.

Path three

Software reads it too

Everything a reader sees is also open to software, read-only and without model calls: a JSON API, an MCP server for agents, and a Markdown twin of every page. Passages are cited by CTS URN, as a scholar would cite them.

Path four

A reader reports a mistake

A reader can flag a word. A second model reviews the report and proposes a fix, code checks it, and nothing changes until a person approves it.

Twenty-four days

Fabrica

Interlinea went from an empty repository to a public product in 24 days. In that time it gained a library of 5,234 Latin and Greek works, a reader that annotates every word, verse scansion, a tutor, accounts and payments, an API for software, and an evaluation rigorous enough for a paper. One person built it.

days: 24days
commits: 379commits
pull requests: 74pull requests
lines of TypeScript: 83,598lines of TypeScript
works in the library: 5,234works in the library
person: 1person

AI agents wrote most of the lines. That is the least interesting part. Agents are fast, tireless and literal, and on their own they build the wrong thing quickly. What made these productive was the conducting: deciding what to build and in what order, writing the system down before any code existed, shaping work so several agents could build at once without colliding, judging every result before it was kept, and measuring the product against expert data instead of trusting impressions.

The next two scenes show that work: first the loop at its centre, then the whole history, every commit and pull request in its place.

The conductor

One person, one intent

Every piece of Interlinea began as a decision: what a reader needed next, what it was worth, and what it must never do. The agents were fast and tireless. Deciding what they should build, and what was good enough to keep, stayed with one person.

Briefs go out

Work shaped to run at once

A good brief names the goal, the ground it may touch and what done looks like. Shaped that way, several pieces of work run side by side without colliding, and the person conducting moves on to the next decision while they build.

Work comes back

Judged, not trusted

Every result came back through the same gates as the product itself, and then through a person. Some arrived right the first time. Others went back with a note, sometimes several times, until they met the bar.

Iteration

Drafts are cheap, taste is not

When a draft costs minutes, the scarce thing is knowing which one is right. The launch film, the scansion and the figures in this article each went through several rounds before one was kept, and 34 features reached readers in 24 days.

Measure, decide, ship

A measurement, not a debate

When the evaluation showed the live model missing one Latin head in four, the answer was more measurement: several models run in parallel overnight, and a pull request to switch open within an hour of the result.

Day one

The whole system, written down first

On 27 September, before the first line of code, I wrote Interlinea down: what it stores, how it answers, what it costs and how it is tested. Every piece of work afterwards was built against that plan, so many hands could build one coherent thing.

In parallel

Several builders at once

From the first week, AI agents built different parts of the product at the same time without colliding: the reader, the library, the verse, the accounts. Up to 3 finished pieces stood ready for review at any moment, and the queue never waited on code.

Continuity

Nothing lost between hands

Every finished piece came with a written note of what it did and what the next builder had to know. 67 of them record the project. They are how work handed from one agent to the next without ever starting from zero.

Gates

Nothing merges red

No change entered the product until it passed the type checker, the linter and the tests, 849 of them by the end. When something broke anyway, the lesson became a rule the same day, and the rule held from then on.

24 days

379 commits, 74 pull requests

24 days, 379 commits, 74 pull requests and 83,598 lines of TypeScript, 84 commits on the first day alone. Half of all pull requests merged within four minutes of opening. 34 features shipped. Scroll back and the history plays in reverse.

Measure, decide, ship

The evaluation ran the same way

The evaluation in this paper was built in days, by agents working in parallel. When it found a better model for live analysis, the pull request to switch was open within an hour of the last measurement.

VII

Measure, decide, ship

Quod Invenimus

The evaluation was not an afterthought. It compares four language models with Stanza (Qi et al., 2020) and UDPipe 2 (Straka, 2018), each trained on the treebank it is tested on, against expert annotation in Universal Dependencies (Nivre et al., 2020) and the Ancient Greek and Latin Dependency Treebank (Bamman and Crane, 2011). Two of the gold sets, from Marseille (Pizzorno, 2026) and Artemidorus (Ferreira, 2021), were released after every model’s training data ends (Anthropic, 2026; OpenAI, 2026; Google DeepMind, 2026). What it found changed the product the same week.

The test

833 sentences of expert annotation

I used existing expert treebanks only: the test splits of four Universal Dependencies treebanks of Latin and Greek, and two sets released in August 2026, after every model’s training data ends. The baselines are Stanza and UDPipe 2, each trained on the very treebank it is tested on.

Heads

The best model finds them better than the parsers

Claude Opus 5.5 attaches words to their heads at 86.9 in Latin and 91.4 in Greek, against 81.4 and 82.9 for UDPipe 2. The parsers keep the lead on exact morphology and on relation labels, which follow each treebank’s own conventions.

Unseen text

Medieval inventories no model has read

On Latin inventories from Marseille, released after the models were trained, Opus still gets 82.9% of lemmas right while UDPipe 2 falls to 67.8%. On Greek prose that follows the parser’s own annotation guidelines, the parsers win. The lead depends on how far the text is from the parser’s training.

Price

What changed in production

GPT-6 luna, chosen for speed, costs 0.1 cents a sentence but attaches only 75.5% of Latin heads. GPT-6.1 Sol at low reasoning matches Opus, at 87.0 and 91.3, for about a third of its price. Interlinea’s live analysis is moving to it.

VIII

Limits

Fines

Each gold set comes from one annotator or project, and my conversion of the Ancient Greek and Latin Dependency Treebank favours the models by a few points of attachment against the official trees. The two sets released after the models’ cutoffs are small and differ in genre. Glosses, notes and translations, which readers use most, are not scored, and nothing here measures whether reading with Interlinea teaches anyone Latin or Greek. The models are commercial services that change.

The design holds up under all of that. A language model can be the part of a reading tool that knows Latin and Greek, if code is the part that knows what an answer may look like. The best model attaches 86.9% of Latin heads and 91.4% of Greek ones, and the code around it makes sure a reader never sees a malformed answer. Try it on a page of Virgil.

Cite this paper

Plain

Rumi Elías Calles (2026). Interlinea: Validated Language-Model Annotation and Rules-First Scansion of Latin and Ancient Greek. Aeterna Institute, De Litteris III. https://aeterna-institute.org/research/interlinea/

BibTeX

@article{calles2026interlinea-validated-language-model-annotation-and-rules-first-scansion-of-latin-and-ancient-greek,
  author    = {Rumi Elías Calles},
  title     = {Interlinea: Validated Language-Model Annotation and Rules-First Scansion of Latin and Ancient Greek},
  journal   = {Aeterna Institute, De Litteris III},
  year      = {2026},
  url       = {https://aeterna-institute.org/research/interlinea/},
}

Download the PDF

References

Anthropic (2026). System Card: Claude Opus 5.5. Anthropic. Source
The Alpheios Project (2026). Alpheios: Reading Tools for Classical Languages and Literatures. The Alpheios Project, Ltd.. Source
David Bamman and Gregory Crane (2011). The Ancient Greek and Latin Dependency Treebanks. Language Technology for Cultural Heritage, Springer. doi:10.1007/978-3-642-20227-8_5
David Chamberlain (2026). Hypotactic: Greek and Roman Verse, Audio-Visually Enhanced. hypotactic.com. Source
Gregory Crane (1991). Generating and Parsing Classical Greek. Literary and Linguistic Computing 6(4). doi:10.1093/llc/6.4.243
Anise Ferreira (2021). UNESP Trees: Coleção de treebanks dos Projetos Abertos em Letras Clássicas Digitais. Perseids Publications. Source
Google DeepMind (2026). Gemini 3.8 Flash Model Card. Google DeepMind. Source
Jie Huang, Xinyun Chen, Swaroop Mishra, Huaixiu Steven Zheng, Adams Wei Yu, Xinying Song, and Denny Zhou (2024). Large Language Models Cannot Self-Correct Reasoning Yet. ICLR 2024. Source
Ryo Kamoi, Yusen Zhang, Nan Zhang, Jiawei Han, and Rui Zhang (2024). When Can LLMs Actually Correct Their Own Mistakes? A Critical Survey of Self-Correction of LLMs. Transactions of the Association for Computational Linguistics. doi:10.1162/tacl_a_00713
Joakim Nivre, Marie-Catherine de Marneffe, Filip Ginter, Jan Hajič, Christopher D. Manning, Sampo Pyysalo, Sebastian Schuster, Francis Tyers, and Daniel Zeman (2020). Universal Dependencies v2: An Evergrowing Multilingual Treebank Collection. Proceedings of LREC 2020. Source
OpenAI (2026). GPT-6 Luna Model. OpenAI API documentation. Source
Gabriel H. Pizzorno (2026). Correction as Annotation: Bootstrapping a Dependency Parser for Documentary Medieval Latin. arXiv preprint. doi:10.48550/arXiv.2609.06266
Peng Qi, Yuhao Zhang, Yuhui Zhang, Jason Bolton, and Christopher D. Manning (2020). Stanza: A Python Natural Language Processing Toolkit for Many Human Languages. Proceedings of ACL 2020: System Demonstrations. doi:10.18653/v1/2020.acl-demos.14
Milan Straka (2018). UDPipe 2.0 Prototype at CoNLL 2018 UD Shared Task. Proceedings of the CoNLL 2018 Shared Task. doi:10.18653/v1/K18-2020
Chahan Vidal-Gorène, Bastien Kindt, and Florian Cafiero (2026). Under-resourced Studies of Under-resourced Languages: Lemmatization and POS-tagging with LLM Annotators for Historical Armenian, Georgian, Greek and Syriac. arXiv preprint. doi:10.48550/arXiv.2602.15753
Brandon T. Willard and Rémi Louf (2023). Efficient Guided Generation for Large Language Models. arXiv preprint. doi:10.48550/arXiv.2307.09702

De Litteris

Continue the inquiry

Complete research index

Back to Research

Open Interlinea Download PDF