beat

If you've checked the grammar of a Microsoft Word document, you may have encountered a baffling number. The readability formula purports to represent the text's appropriate grade level. But it has its roots in research from 60 years ago.

Before computers, reading researchers attempted to quantify the ease of a work of writing using short excerpts and simple formulas. Despite computing advances, Word still follows the same model: It multiplies 0.39 by the average number of words per sentence, adds that to 11.8 times the average number of syllables per word, and subtracts 15.59 from the total. The result is the supposed minimum grade level of readers who can handle the text in question.

Similar formulas are used by textbook publishers and in dozens of states' guidelines for insurance policies.

NUMBERS GUY BLOG

Is it possible to quantify the readability of a given text? Do you ever use these formulas? Share your thoughts in the blog comments.

From the beginning, these formulas were known to be problematic. A 1935 paper laid out more than 200 variables that affect readability. Most formulas incorporate just two, and not because they are the most important but because they are the easiest to measure. Then they're mashed together, with weights set according to how the formulas work on standard texts.

"Everyone is waiting for this magic bullet that's very easy," says Karen Schriver, who runs an Oakmont, Pa., communication-design research company. But her experience with clients who have overly relied on these formulas have suggested that "maybe it's just a stupid idea."

Noting that the same passage's score can differ by three grade levels or more, depending on the formula, readability consultant Mark Hochhauser says, "One of the things the field really needs is an updated formula."

Even neurolinguist G. Harry McLaughlin says of his own, widely used SMOG Readability Formula, "The theoretical basis is c---."

The formulas treat writing as a mere collection of words and spaces. Word meaning and sentence structure don't figure. George Weir, a philosopher and computer scientist at the University of Strathclyde in Glasgow, says Word's readability test thinks grade-schoolers could handle the nonsense passage, "Acuity of eagles whistle truck kidney. Head for the treacle sump catch and but. What figgle faddle scratch dog and whistle?" Similarly, "Had lamb little a Mary" and "Mary had a little lamb" score identically.

I asked Micro Power & Light Co., which sells readability-testing software, to evaluate a memorable 2004 Wall Street Journal front-page article. Four different formulas found it to be comprehensible to 10th-graders, thanks in part to its short sentences. The reason for the frequent periods: The article was about a new book written without verbs, and the article mimicked its subject, making for intentionally tough reading.

Word length is an imperfect measure. "Important" and "elephant" are long words that are easy for most readers, Dr. Schriver notes. Conversely, frustrated crossword solvers encounter plenty of uncommon three-letter words, such as adz, auk and lea. She adds that no formulas account for document layout -- even short sentences with lean words are challenging when printed in an eight-point type.

The formulas have their defenders. Readability consultant William DuBay calls them "good enough," and adds, "They've been extremely beneficial for millions of readers." Among other uses, they were implemented to simplify newspaper writing a half-century ago, he says.

Some researchers are trying to make the formulas better, using new databases and computing power. Prof. Weir aims to create a formula that incorporates the frequency of words and word combinations in typical English writing, meaning "the" and "adz" finally can be distinguished.

Several more-advanced readability formulas already have been developed. None are as convenient, or as criticized, as the Flesch-Kincaid formula Microsoft uses. Developed by readability researcher Rudolf Flesch in 1948, it was modified by psychologist J. Peter Kincaid in a study for the U.S. Navy in 1975, using reference passages. "Do not swing, twirl, or play with the nightstick" is part of a passage deemed appropriate for seventh-graders. Instructions that included, "All the jet streams of the Northern Hemisphere have their southern analogues" required a college degree.

The formula was tweaked once more by Microsoft when the company incorporated it into Word in 1993. Grade-level scores were capped at 12. Reed Shaffner, Microsoft's product manager for Word, told me that the formula was changed in 2003, at least for Windows users. Those users can see results up to grade level 14, while Mac users won't get results above level 12.

Why cap the results at all? "It's a user-experience thing," Mr. Shaffner says. Essentially, Microsoft is concerned about the readability of readability-formula results.

Prof. Kincaid, who today is the head of a modeling and simulation program at the University of Central Florida, tried unsuccessfully to get the formula corrected years before it finally was. Nevertheless, when he wants to use his own formula, he lets Word do the calculation.

That's rare. "I write long sentences and no computer is going to tell me how to write," Prof. Kincaid says. "I'm going to write the way I want to write."

• Email me at numbersguy@wsj.com.