Research
How accurate are free braille translators?
Every one of them says it does Grade 2. None of them says how well. This is what happens when you check.
Search for a braille translator and you will find a dozen free ones. Every single one says it handles Grade 2 contractions and Unified English Braille. Not one of them publishes a figure for how often it is right, and none of them shows you a word it gets wrong.
That is not because the tools are bad. It is because publishing the number requires two things most projects do not have: a reference implementation to be measured against, and a willingness to list your own failures on your own website. The first is free. The second is the awkward part.
The method
- Question
- For a given English word, does this site's Grade 2 output match the reference implementation exactly, cell for cell?
- Compared against
- liblouis 3.29.0 (en-ueb-g2) — the translator that braille displays, embossers and screen readers actually run, using its published UEB Grade 2 table.
- Over
- The 5,000 commonest English words, in frequency order.
- Committed as
forge/topics/braille/data/ueb-reference.tsv
The reference output is generated once and committed, so the comparison runs with no network and no dependency — and so anyone can check the ground truth rather than taking this site's word for what the reference said.
The test runs on every build and asserts a floor rather than perfection. The floor is read from the same figure this page prints, so a claim here that the engine cannot meet fails the build. The number on the page and the number the engine scores are the same number by construction; they cannot drift apart.
The result
| Measure | Result | What it means |
|---|---|---|
| Words identical | 4,986 of 5,000 — 99.7% | Every cell the same, including where a contraction was declined. |
| Weighted by frequency | 99.9% | About one wrong word in 1,156 of running prose. |
| Words that differ | 14 | Listed in full below, with what happened in each. |
Two numbers, because they answer different questions. The flat rate treats “the” and “trombone” as equally important, which is the right way to describe a dictionary and the wrong way to describe reading. The weighted rate applies each word's frequency, and is the figure a reader experiences: roughly one word in 1,156 of ordinary text comes out differently from what a transcriber would write.
All 14 disagreements
Rank is position in the frequency list, so the first row is the most common word this engine gets wrong. Nothing is left out.
| Word | Rank | This site | Reference | Why |
|---|---|---|---|---|
| money | #369 | ⠍⠕⠝⠑⠽ 5 | ⠍⠐⠕⠽ 4 | A contraction the reference makes inside a longer word |
| newsletter | #868 | ⠝⠑⠺⠎⠇⠑⠞⠞⠻ 9 | ⠝⠑⠺⠎⠇⠗ 6 | A shortform inside an unrelated word |
| ed | #1,566 | ⠑⠙ 2 | ⠫ 1 | A letter-group sign used as a word in its own right |
| llc | #1,948 | ⠇⠇⠉ 3 | ⠰⠇⠇⠉ 4 | A grade-1 indicator this engine does not emit |
| multimedia | #2,186 | ⠍⠥⠇⠐⠞⠙⠊⠁ 8 | ⠍⠥⠇⠞⠊⠍⠫⠊⠁ 9 | A contraction applied across a boundary inside the word |
| newsletters | #2,399 | ⠝⠑⠺⠎⠇⠑⠞⠞⠻⠎ 10 | ⠝⠑⠺⠎⠇⠗⠎ 7 | A shortform inside an unrelated word |
| experienced | #3,116 | ⠑⠭⠏⠻⠊⠢⠉⠫ 8 | ⠑⠭⠏⠻⠊⠰⠑⠙ 8 | Both spellings are the same length |
| reverse | #3,789 | ⠗⠐⠑⠎⠑ 5 | ⠗⠑⠧⠻⠎⠑ 6 | A contraction applied across a boundary inside the word |
| reaction | #3,934 | ⠗⠂⠉⠰⠝ 5 | ⠗⠑⠁⠉⠰⠝ 6 | A contraction applied across a boundary inside the word |
| severe | #3,938 | ⠎⠐⠑⠑ 4 | ⠎⠑⠧⠻⠑ 5 | A contraction applied across a boundary inside the word |
| er | #4,100 | ⠑⠗ 2 | ⠻ 1 | A letter-group sign used as a word in its own right |
| coupons | #4,405 | ⠉⠕⠘⠥⠎ 5 | ⠉⠳⠏⠕⠝⠎ 6 | A contraction applied across a boundary inside the word |
| whereas | #4,621 | ⠱⠻⠂⠎ 4 | ⠐⠱⠁⠎ 4 | Both spellings are the same length |
| blvd | #4,962 | ⠃⠇⠧⠙ 4 | ⠰⠃⠇⠧⠙ 5 | A grade-1 indicator this engine does not emit |
The small number after each spelling is its length in cells. The braille characters are real Unicode braille, so a refreshable display shows the actual cells rather than a picture of them.
The fourteen are six problems
Grouped by what actually went wrong, the list gets a lot shorter — and a lot more useful, because each group has a different fix and a different cost.
A contraction applied across a boundary inside the word — 5
The letters are there and the position rules allow them, but they belong to two different parts of the word: mul-time-dia, rev-erse, co-upons. Nothing in the spelling marks the seam, so a rule that reads letters cannot see it. These five are the ones that produce genuinely wrong braille, and they are the reason the engine's position rules are measured per word rather than set once for all of them.
multimedia, reverse, reaction, severe, coupons
A grade-1 indicator this engine does not emit — 2
The cells for the letters are identical; the reference puts a marker in front of the initialism and this engine does not. Nothing is misread — the output is one cell shorter than it should be.
llc, blvd
A letter-group sign used as a word in its own right — 2
A few letter-group signs are also whole words when they stand alone. These two turn up in the frequency list as words, which is unusual enough that the engine does not special-case them.
ed, er
Both spellings are the same length — 2
Two legal ways to write the word come to the same number of cells, and the reference prefers the one contraction covering more letters where this engine prefers two covering fewer. Both decode to the same word.
experienced, whereas
A shortform inside an unrelated word — 2
The reference carries a list of roughly six hundred compounds that keep a shortform buried inside them. This engine contracts shortforms only as whole words, so it spells these out: longer, and never a contraction that reads as the wrong word.
newsletter, newsletters
A contraction the reference makes inside a longer word — 1
The letters keep their meaning here and the reference contracts them. This engine's rule for where this contraction may sit is measured, and this word falls on the wrong side of it — see the position survey.
money
Which direction the errors run
This matters more than the count. Writing a contraction the reference does not use produces braille that is wrong, and a reader is misled. Spelling something out where the reference contracts produces braille that is longer and correct — a reader loses nothing but space.
Of the 14, 5 fall on the wrong side and 7 on the harmless one, with 2 that are neither: both spellings are the same length and decode to the same word, and the reference simply breaks the tie the other way.
The engine is tuned for that asymmetry deliberately. Every rule in it that decides whether a contraction applies was set from a measurement, and set conservatively: where agreeing and disagreeing were close, it declines. That costs cells and buys correctness, which is the right trade for a tool people check their work against.
- This measures words in isolation. A document has capitals, numbers, punctuation and line breaks, and the rules for those interact in ways a word list cannot show.
- The corpus is the 5,000 commonest words. Rarer words are not measured, and rarer words are where contraction rules bite hardest.
- Not one word in the corpus contains an accent, a curly quote or a currency sign, so this number says nothing about them. Those are measured separately and at parity — see below.
- It is not a certification. Good enough to read, learn from, and check your work against. Not a substitute for a certified transcription — if the output is going to be embossed for someone to rely on, have a qualified transcriber check it.
The gap this number was hiding
A word-frequency corpus is all words, and words are all letters. For a long time that meant everything outside a to z went unmeasured — and unmeasured turned out to mean broken. The translator was silently deleting every accented letter, curly quote, dash and currency sign it was given: mañana came back as maana, café as caf, and don't typed on a phone came back as dont, because the apostrophe a phone produces is not the one on a keyboard.
Nothing warned anyone. The output was shorter than it should have been, which is invisible unless you count cells.
That is fixed, and fixed the same way as everything else here: 197 characters — the accented letters, the symbols and the punctuation marks — are now checked against the reference on every build, and they match it exactly rather than to a floor. A symbol has one spelling, so anything less than parity would be a defect rather than a difference of opinion.
Two of them were simply wrong rather than missing. The hash and per cent signs carried the wrong prefix cell, while this site's own dot-number pages stated the right one. That is the kind of contradiction a site can hold for months without noticing, and it is why the check now covers every entry in the table rather than a sample.
Anything the engine still cannot write is now named on the page instead of being dropped in silence.
Check it yourself
The reference output is a committed tab-separated file: word, then the cells the reference produces. The comparison is one file that reads it and one command to run. If you disagree with a row, the row is right there to disagree with.
You can also try any of the fourteen in the translator, or put one through the contraction finder, which shows which contractions applied, which were legal and passed over, and which a rule forbade. The rest of the research reports what the measurements turned up.
The finding, as a card
For anyone who wants to share the number without the argument attached.
