Splitting Sentences by Thought Group — and 2 Wrong Turns

It all started with one post on our feedback board:

It sounded like a small fix. Make the chunks shorter, ship it. But the deeper we dug, the clearer it became that this was never a question of length. It was a question of where to cut — properly named, how to split English sentences into thought groups. And the moment we solved that, it spawned a much harder one: how do you translate those phrases without teaching people something false?

Here's the full story of both — including the two times we confidently got it wrong.

Part 1 — Cutting short is easy; cutting correctly is not

The fastest approach is mechanical: five words per chunk, or cut at every comma. We tried it. It produces fragments like this:

"the big" / "red car that I" / "bought last week"

Arithmetically perfect. As listening practice, useless. Nobody pauses between "the big" and "red car" — so a chunk cut that way trains you on a rhythm that doesn't exist in real speech. Worse, it asks you to hold a meaningless fragment in your head, and short-term memory is famously bad at meaningless things.

So the question got rewritten: where are we never allowed to cut?

Part 2 — Hand-written splitting rules: why they failed

We sat down and listed the "never break here" rules, based on how English groups words. Never break inside:

• Determiner + noun — the big red car
• Preposition + its object — in the garden, of a week
• Auxiliary/modal + verb — has been, did not answer
• Verb + particle — give up, look after
• "to" + verb — to share
• Proper nouns and titles — New York City, Dr. Jane Smith
• Numbers, dates, money, measurements
• Fixed idioms and compound nouns — on the other hand, climate change
• Possessive + noun — John's car

Plus one that sounds obvious: never end a chunk on a bare function word — don't leave a chunk hanging on "of".

Then we started implementing it. That's where it fell apart.

To know which "of" is stranded, you have to know which phrase it belongs to — which means parsing the sentence. And it kept getting worse:

• The same word "up": in "give up" it's a particle that must stay glued to the verb. In "walked up the hill" it's a preposition belonging to what follows. To code looking at characters, they're identical.
• "New York City" must stay whole — but code can't know it's a proper noun without a named-entity recogniser bolted on.
• The comma in "1,000" is never a break; the comma in "In 2020, we…" is an ideal one. Same character.
• "to" in "to share" is an infinitive; "to" in "went to London" is a preposition.

Every rule we added spawned three exceptions. Halfway through, we looked up and realised what we were building: an English parser. And even finished, it would still lose to one sufficiently awkward real-world sentence.

Then came a fairly obvious realisation that took us embarrassingly long to accept: today's language models are the masters of exactly this. Telling "give up" from "walked up", knowing "New York City" is one block, knowing which comma is a beat and which is a digit separator — that is precisely what they learned from billions of English sentences. We were hand-coding a competence the model already had.

So we flipped the roles:

• The rules stopped being code and became the brief. The entire "never break here" list above now lives in the prompt.
• The model decides 100% of the boundaries. The code invents nothing — no packing to hit a word count, no size caps.
• The code fell back to what code is actually good at — verifying:
  1. Check every token against the original word list. The model may drop punctuation, but one changed or missing word rejects the whole candidate in favour of a fallback.
  2. Time each chunk from per-word timestamps, so pressing play lands exactly on the phrase.
  3. Map every chunk back to its parent sentence — a detail that saves us later.

One small change that mattered a lot: we feed the model the punctuated transcript, not a re-joined string of raw words. Commas and periods are the strongest phrasing cues the recording leaves behind. Stripping them throws away the best evidence we have.

Part 3 — What standard to split by: thought groups and intonation units

Handing the cutting to a model doesn't mean letting it freestyle. We still had to tell it what standard to cut by — and this is where we needed real grounding, not vibes.

People don't speak in separate words. Speech is organised into thought groups (also called sense groups).

A thought group is a cluster of words carrying one idea, spoken in a single breath — and pauses fall on the boundaries between groups, never inside one.

In linguistics this unit is what Wallace Chafe described as the Intonation Unit — an intermediate-level prosodic phrase that carves the continuous speech stream into chunks sized to fit the listener's limited focus of attention. This isn't an aesthetic point: there's dedicated research on the role of Intonation Units in memory for spoken English.

Add a piece from cognitive psychology: working memory holds roughly 7±2 units (Miller, 1956), revised down by later work to about 4±1 (Cowan, 2001). That is exactly why our shortest level is 2–4 words — not "short for the sake of short", but short enough to fit short-term memory while you're still fighting to catch the sounds.

And the part that reassured us most: professional subtitlers settled this long ago. The BBC Subtitle Guidelines require subtitles to be segmented at a natural linguistic break, so each line forms an integrated linguistic unit — and they name the splits to avoid outright: article and noun, preposition and the phrase that follows, conjunction and the clause that follows, pronoun and verb — while preferring segmentation at clause boundaries.

The "never break here" list we'd written in Part 2 lines up almost exactly. That's no coincidence: both come from the same source — how English groups words. We were re-walking a road broadcast subtitlers had already paved, and that's what convinced us we were walking the right way.

Three levels of phrase chunking — and why three

Once you have that grounding, the three levels stop being arbitrary numbers:

Level	Size	Grounded in	Best for
Steady	2–4 words	Working-memory chunk limit (≈4±1)	Starting out, or fast speech
Natural	5–9 words, clause-sized	Intonation units + "prefer clause boundaries"	Everyday practice
Fluent	Whole sentences	The complete semantic unit	Once you can hold a longer line

Each level keeps its own progress, so switching mid-lesson never costs you the work you've done.

A benefit we didn't plan for: speaking practice

Cutting on thought groups opened something else. Because each chunk is exactly one breath, you can shadow it phrase by phrase — say the whole chunk without running out of air, pausing where a native speaker pauses.

Try shadowing a 25-word sentence in one go and you'll stall halfway and break in a place nobody breaks. Shadowing by chunk is a different exercise: you're training the rhythm of spoken English, not just the pronunciation of individual words. One split, serving both listening and speaking.

New to the three stages of listening practice (listen → dictate → shadow)? Read our earlier post — this one assumes you're already at stage 2–3.

Lesson one: segmentation is a linguistic decision, not an act of division. And when a problem is purely linguistic, the best solver isn't the code we write — it's the model that learned the language. Our job is to set the brief precisely and verify ruthlessly.

Part 4 — Phrase-by-phrase translation: the harder problem

With splitting solved, an ugly seam appeared: every four-word chunk was still showing the translation of the entire sentence. You'd be typing "our brain is working" while a full line of translation sat underneath — redundant, and it spoiled the part you hadn't heard yet.

"Just translate each chunk separately" sounds obvious. But it carries three constraints that fight each other:

1. The fragment must mean what it means → which needs whole-sentence context.
2. But it must not reveal the rest of the sentence.
3. And it must not borrow words from the neighbouring chunk.

Wrong turn #1: slice the full translation to fit

The idea felt airtight: the sentence translation is already correct, so just partition it across the chunks. Concatenate the pieces and you get the original back — nothing added, nothing lost.

On well-behaved sentences it was lovely:

"I was appointed six months ago," → "Tôi được bổ nhiệm cách đây sáu tháng," "and the more I've spoken about feminism," → "và càng nói về chủ nghĩa nữ quyền,"

Then we hit this:

"It's an environmental issue which, like the growing amounts of plastic waste, isn't going away."

The chunk "of plastic waste," was assigned: "nhựa ngày càng tăng," — literally "plastic increasingly rising".

Concatenated, the sentence is still perfect. But a learner is staring at four words, "of plastic waste", being told they mean "plastic increasingly rising". The word "growing" belongs to the previous chunk. We had just taught something false.

Note the detail: it's "of" again. It broke us during segmentation, then came back to break us during translation.

Correct-when-reassembled is not the same as correct-in-each-piece. Languages reorder words differently, so a split that's globally faithful can be locally wrong.

Wrong turn #2: translate each chunk — but still hand over the full translation

Chastened, we switched to translating each chunk on its own, passing in both the full English sentence and the existing translation as context, reasoning it would keep word choice consistent.

The result:

"of plastic waste," → "rác thải nhựa ngày càng tăng,"

Still contaminated. Because once a finished translation is in front of it, the model stops translating and goes back to partitioning that translation. We had personally handed it the crutch we were trying to remove.

The wrong kind of context is worse than no context.

What actually worked

Remove the full translation from the prompt entirely. Keep only the full English sentence, and use it for exactly one job: disambiguating word senses. Add one blunt instruction: translate only the words in this chunk; if a word belongs to another chunk, it must not appear here.

"of plastic waste," → "của rác thải nhựa," ✓ "the growing amounts" → keeps growing where it belongs

The three conflicting constraints resolve once the roles are separated: the English sentence supplies context, but only the chunk's own words get translated.

One last trap: correct isn't enough — it has to be reliably correct

Our first version packed 25 sentences into a single call to save money. The model started miscounting how many chunks each sentence had — and a count mismatch invalidates that entire sentence. The fill rate was 58 of 137 chunks. More than half of learners would still be seeing the old whole-sentence translation.

Two very ordinary fixes:

• Smaller batches — fewer items per call and the model stops losing its place.
• Retry each failed sentence on its own — single-item calls are almost never wrong.

Result: 137 of 137. And to avoid paying for that in latency, the calls run concurrently with a sane ceiling.

Part 5 — The UX: phrase translation, plus the full sentence

After all that, what a learner sees is quiet and simple:

• Each chunk shows its own translation. You understand exactly as far as you've typed.
• On the last chunk of a sentence, the full-sentence translation appears too. You get both the piece and the whole — and nothing is spoiled, because it only arrives once you've finished the sentence.
• Missing gracefully falls back. A single-chunk sentence, or one the model couldn't handle, quietly reverts to the whole-sentence translation. Never a blank.

And translations are not just Vietnamese. They follow the native language you pick in Settings — changeable any time, so the same lesson shows up in your own language:

What stayed with us

Look at the distance travelled. In the early days we leaned on YouTube's free subtitles. They had spelling errors, punctuation errors, sometimes outright wrong words — and worst of all for listening practice, they wrapped lines to fit the video frame, not the meaning. Where a subtitle line ended was a decision made by a video player, not by English.

Today the same video goes through: AI transcription → linguistically-grounded chunking at three levels → token-by-token verification → per-word timing → a dedicated translation for every chunk in your own native language → and a retry for anything that didn't come out right. All of it has shipped — see the changelog.

The cost went up several times over. Every video now takes far more model calls, plus retries, multiplied again per language. That's a real bill.

But both problems in this post started from the same place: something that already worked, which we didn't think was good enough. Mechanical splitting — worked. Showing the whole-sentence translation on every chunk — worked. Slicing the translation to fit — honestly looked elegant.

It just wasn't right. And the gap between working and right is the part learners actually feel.

This post began as exactly one piece of feedback — the screenshot at the top. If something in the product still feels "works, but off" to you, send it our way.