The HyperTextBooks Daniel Kies
Department of English
College of DuPage
Modern English Grammar
English 2126
Contact Form

Evaluating Grammar Checkers:

A Comparative Ten-Year Study


   

  Current work:
  Days remaining this term:

Notes:
Add Note | 

Log in?
 | Privacy | Change Name & Email

Mail this page to a friend


Introduction

How effective are grammar checking programs at finding errors and suggesting corrections? Here, we will examine how well several popular word processors and one grammar checking utility for the Mac find and correct common grammar and usage errors. We will compare Microsoft's Word (both the Windows and the Mac versions of the program), Corel's WordPerfect (Windows only), Grammarian Pro X (Mac only), and Open Office Writer with the LanguageTool extension added (platform independent) for their ability to find and identify the twenty most frequent errors.1

Below, I list the twenty most frequent usage problems found in a corpus of 3000 college essays (Connors and Lunsford 398). We will look at those problems together, examine how well the software could identify and correct those problems, and discuss the strengths and weaknesses of grammar checking programs. (For more information on these errors, see Twenty Common Usage Errors.)

The data

Each error is identified in the heading and an example of the error follows, sometimes with multiple instances of each error in a single example. (Corrections can be found in the Appendix at the end of this document.)

1. No comma after introductory element

Example:

After we watched the movie we went over to the pizza joint for a bite to eat.

2. Vague pronoun reference

Example:

John Smith reported the problem to Bob Adams, and he corrected it immediately.

3. No comma in compound sentence

Example:

John Smith reported the problem to Bob Adams and Adams corrected it immediately.

4. Wrong word

Example:

Building a new fiber optic network will take less dollars then originally suspected.

5. No comma in nonrestrictive element

Example:

Contemporary animal rights groups who are using the same strategies that the civil rights movement developed in the 1960s find that non-violent civil disobedience is effectively drawing attention to their cause.

6. Wrong or missing inflected endings

Example:

Teacher representatives in the United States’ could have wrote the book themselves.

7. Wrong or missing preposition

Example:

The man to whom you were speaking to was the curator on the exposition.

8. Comma splice

Example:

In 1980, Reagan promised to balance the federal budget, however, by 1988, the federal deficit was the largest in U. S. history — until now.

9. Possessive apostrophe error

Example:

The Presidents speech was punctuated by enthusiastic applause from both sides of the aisle.

10. Tense shift

Example:

John was edgy all day. He challenged anything that anyone said to him. Once at a meeting, John looked right at me and he says, "What do you mean by that?"

11. Unnecessary shift in person

Example:

Students should register for classes early if you want to ensure that you get a good schedule.

12. Sentence fragment

Example:

Publishers today are at a loss trying to cope with the new trends in information technology. Which is not surprising actually.

13. Wrong tense or verb form

Example:

The market has responding favorably to the surprisingly strong earnings that the high-tech sector reports yesterday.

14. Subject-verb agreement

Example:

The President as well as his advisors were in meetings all weekend at Camp David. Neither the President nor his advisors is commenting on the status of the negotiations.

15. Lack of comma in a series

Example:

Forget what Wheaties tells you; the real breakfast of champions is pizza, beer and eggs.

16. Pronoun agreement error

Example:

Everyone should register early to make sure that they get the classes they want.

17. Unnecessary comma with restrictive element

Example:

Several of the guys on the team were here at the time, but the guy, who is talking to the police, is the only witness to the accident.

18. Run-on or fused sentence

Example:

Courses in entrepreneurial businesses are increasingly popular however few faculty members are qualified to teach them.

19. Dangling or misplaced modifier

Example:

While smoking a cigarette, the coffee in his cup had grown cold. He says often he smokes cigarettes.

20. Its/it's error

Example:

Its commonly assumed that any bear will defend it's offspring by attacking any human that wanders too near a cub.

The results

All word processors had considerable difficulties identifying and correcting most of the twenty most common and frequently occurring usage errors. Below, in Table 1, is a side-by-side comparison of how all of the programs fared when doing the grammar check of this document, with its twenty common usage errors, culled from the writing of first year composition students at several American colleges and universities. The grammar checker in each program was set to search at the strictest, most formal level allowed by the program.

If we award two points for each "Yes" and one point for each "Some" in the comparison below, we see that, under Windows, Word 2000, Word 2002, Word 2003, and Word 2007 earn 0 of 80 possible points, while Word 97 earns 19 of 80 possible points (24%). WordPerfect 8, 9, 10, 11, and 12 earn 32 of 80 possible (40%). On the Mac, Word 2004 earns 28 of 80 possible (35%), and Grammarian Pro X earns 40 of 80 possible (50%). The open source, platform independent LanguageTool for Open Office earns 6 of 80 possible (8%). Not one of these programs is spectacular at finding and correcting usage errors, though WordPerfect (Windows) and Grammarian Pro X (Mac) do recognize these most common errors more readily.

Table 1:

A comparison of the grammar checking abilities of several popular word processing/grammar checking programs.

  Word 2000
Word 2002
Word 2003
Word 2007

(Windows)
Word 97
(Windows)
WordPerfect 10
WordPerfect 11
WordPerfect 12

(Windows)
WordPerfect 8
WordPerfect 9

(Windows)
Word 2004
(Mac)
Grammarian
Pro X

(Mac)
Open Office
Writer with
LanguageTool

(platform independent)
Error

 Found

 

Corrected 

 Found

 

Corrected 

 Found

 

Corrected 

 Found

 

Corrected 

 Found

 

Corrected 

 Found

 

Corrected 

 Found

 

Corrected 

1.  

No comma after introductory element

No

 

No

Yes

 

Yes

Yes

 

Yes

Yes

 

Yes

Yes

 

Yes

Yes

 

Yes

No

 

No

2.  

Vague pronoun reference

No

 

No

No

 

No

No

 

No

No

 

No

No

 

No

No

 

No

No

 

No

3.  

No comma in compound sentence

No

 

No

No

 

No

No

 

No

No

 

No

No

 

No

No

 

No

No

 

No

4.  

Wrong word

No

 

No

Some

 

No

Some

 

Some

Yes

 

Yes

Yes

 

Yes

Yes

 

Yes

Some

 

Some

5.  

No comma in nonrestrictive element

No

 

No

No

 

No

No

 

No

No

 

No

No

 

No

No

 

No

No

 

No

6.  

Wrong or missing inflected endings

No

 

No

Some

 

Some

Some

 

Some

Some

 

Some

Some

 

Some

Some

 

Some

Some

 

Some

7.  

Wrong or missing preposition

No

 

No

No

 

No

No

 

No

No

 

No

No

 

No

No

 

No

No

 

No

8.  

Comma splice

No

 

No

No

 

No

No

 

No

No

 

No

Some

 

Some

No

 

No

No

 

No

9.  

Possessive apostrophe error

No

 

No

No

 

No

Yes

 

Yes

Yes

 

Yes

No

 

No

Yes

 

Yes

Some

 

Some

10.  

Tense shift

No

 

No

No

 

No

No

 

No

No

 

No

No

 

No

No

 

No

No

 

No

11.  

Unnecessary shift in person

No

 

No

No

 

No

Yes

 

Yes

Yes

 

Yes

No

 

No

No

 

No

No

 

No

12.  

Sentence fragment

No

 

No

Yes

 

Yes

Some

 

Some

Yes

 

Yes

Yes

 

No

Yes

 

No

No

 

No

13.  

Wrong tense or verb form

No

 

No

No

 

No

Some

 

Some

Some

 

Some

No

 

No

Yes

 

Yes

No

 

No

14.  

Subject-verb agreement

No

 

No

Yes

 

Yes

Yes

 

Yes

Yes

 

Yes

Yes

 

Yes

Yes

 

Yes

No

 

No

15.  

Lack of comma in a series

No

 

No

No

 

No

No

 

No

No

 

No

No

 

No

Yes

 

Yes

No

 

No

16.  

Pronoun agreement error

No

 

No

No

 

No

No

 

No

No

 

No

Yes

 

Yes

No

 

No

No

 

No

17.  

Unnecessary comma with restrictive element

No

 

No

No

 

No

No

 

No

No

 

No

No

 

No

No

 

No

No

 

No

18.  

Run-on or fused sentence

No

 

No

No

 

No

Yes

 

Yes

No

 

No

No

 

No

Yes

 

Yes

No

 

No

19.  

Dangling or misplaced modifier

No

 

No

Some

 

Some

No

 

No

No

 

No

Yes

 

No

Yes

 

Yes

No

 

No

20.  

Its/it's error

No

 

No

Some

 

Some

Yes

 

Yes

Yes

 

Yes

Yes

 

Yes

Yes

 

Yes

No

 

No

Score 0% 24% 40% 40% 35% 50% 8%
Word 2000
Word 2002
Word 2003
Word 2007

(Windows)
Word 97
(Windows)
WordPerfect 10
WordPerfect 11
WordPerfect 12

(Windows)
WordPerfect 8
WordPerfect 9

(Windows)
Word 2004
(Mac)
Grammarian
Pro X

(Mac)
Open Office
Writer with
LanguageTool

(platform independent)

Under Windows, Microsoft's Word has been severely broken ever since the Word 97 version. No newer version of the program could find any of the errors in the twenty sentences since Word 97.2 Back in 1998, shortly after I placed an earlier version of this study online, I received an email from Jerry Neufeld-Kaiser, the manager of grammar checker programming for Microsoft Word at that time. Neufeld-Kaiser seemed suspicious of the results reported above and asked how I came to these findings. I replied that the file I used could be freely downloaded and checked (see below) by anyone. I asked that if Microsoft found any differences when it checked the document I would like to know. I never heard back.

I suspect that the more recent versions of Word have a problem initializing the grammar checker. As far back as Word 2000, my students and I noticed that if one were to copy and paste the original document into a second, new Word document, then the grammar checker seemed to start up. However, Word could not find any of the grammar errors if one simply opened the error-filled, original document in the word processor. Galletta (personal correspondence) confirms this, though he adds that in Word 2003, one has to paste the text into the new Word document using the Edit > Paste Special ... > Unformatted Text option. I see this same problem with my copy of Word 2003, but I don't remember having to do this special paste procedure in Word 2000, however.

Kevin Dulzo reports similar initialization problems with Word 2004 for the Mac:

"Microsoft Word 2004 for Mac Version 11.2 (060202) did not instantiate the grammar checker on load of an existing document. In order to perform the evaluation I was required to run the 'Spelling and Grammar...' tool by hand. Also, even with 'Check grammar as I type' enabled, the problems were not discovered as I typed them (either in an existing or a fresh document). Finally, pasting a problem sentence which Word was capable of detecting was flagged upon completion of the paste.

"None of these issues exist with Grammarian per se, as the solution requires me to initialize the engine and use the input manager menu to launch a check.

"I viewed them on equal footing; that is to say both required the user to intervene in order to achieve best results from their evaluation engine."

My experiments with this copy/paste issue in Word 2002, 2003, and 2007 have convinced me that Word's behavior here is more bizarre than I first imagined. For example, if I do a simple copy/paste operation from the original Word document to a new Word document window, nothing happens and running the grammar checker will show nothing. Word's default paste operation keeps the original document's formatting intact, by the way. However, if I copy/paste from the original document into a new Word document window and select the "Keep text only" paste option from the Smart Tag dialog or if I copy/paste the twenty examples from an email document into a new Word document window, then the grammar checker does start up. But here's the puzzler: Word's grammar checker (in both Word 2002 and 2003) actually works differently depending on the circumstances of the copy/paste operation!

For example, see Figures 1 and 2 below, where I illustrated some of the differences with screen captures of two of the examples, numbers 4 and 18:

Figure 1: Screen capture of a simple copy/paste from email document to new Word 2003 document window

Simple copy/paste from email document

In Figure 1, we see the results of a simple copy/paste operation from an email document into a new Word 2003 document. Word 2002 and 2003 recognize less dollars as an error in example 4 but do not see the run-on at however in example 18. The odd line breaks are an artifact of the copy/paste operation from an email client with text set to wrap at 72 characters, and likely the odd line break after however few causes Word not to recognize the continuing sentence, thereby suppressing Word's ability to find the error. Interestingly, however, there seems to be something in the plain text formatting or in the odd line breaks that allows Word to recognize less dollars as an error. (Remember in the original test results above Word 2000, 2002, 2003, and 2007 could not do this.) When I restore the text flow of the document in the example above by deleting the extra, unneeded line breaks, Word then does not recognize less dollars as an error anymore but does recognize however as an error, similar to the way it functions in the example below.

Figure 2: Screen capture of a copy/paste from the original Word 2003 document to a new Word 2003 document window either with "Keep text only" option selected from the Smart Tag or with Paste Special... > Unformatted Text selected from the Edit menu

Copy/paste from Word document to new Word document with "Keep text only" option

In Figure 2, we see the results of a copy/paste operation from the original Word 2003 document to a new Word 2003 document window either with "Keep text only" option selected from the Smart Tag or with Paste Special... > Unformatted Text selected from the Edit menu. Word 2002 and 2003 now fail to see less dollars as an error in example 4 but do see the run-on at however in example 18.

Once Word's grammar checker does begin to respond through the copy and paste operation, we can see that Word 2002 and 2003 can find some of the usage errors: Word 2007, however, responds differently after the copy and paste operation. Compare Tables 2 (Word 2002 and 2003) and Table 3 (Word 2007) below.

Table 2:

Word 2002 and 2003's grammar checking results after the copy/paste operation described above, required to "truly start" the grammar checker

Error

 Found

 

Corrected 

1.  

No comma after introductory element

Yes

 

Yes

2.  

Vague pronoun reference

No

 

No

3.  

No comma in compound sentence

No

 

No

4.  

Wrong word

Some

 

Some

5.  

No comma in nonrestrictive element

No

 

No

6.  

Wrong or missing inflected endings

Some

 

Some

7.  

Wrong or missing preposition

No

 

No

8.  

Comma splice

No

 

No

9.  

Possessive apostrophe error

No

 

No

10.  

Tense shift

No

 

No

11.  

Unnecessary shift in person

No

 

No

12.  

Sentence fragment

Some

 

Some

13.  

Wrong tense or verb form

No

 

No

14.  

Subject-verb agreement

Yes

 

Yes

15.  

Lack of comma in a series

Yes

 

Yes

16.  

Pronoun agreement error

Yes

 

No

17.  

Unnecessary comma with restrictive element

No

 

No

18.  

Run-on or fused sentence

Yes

 

No

19.  

Dangling or misplaced modifier

Some

 

No

20.  

Its/it's error

Some

 

Some

Score after using the copy/paste trick to "awaken" the grammar checker   31%

Table 3:

Word 2007's grammar checking results after the copy/paste operation described above, required to "truly start" the grammar checker

Error

 Found

 

Corrected 

1.  

No comma after introductory element

Yes

 

Yes

2.  

Vague pronoun reference

No

 

No

3.  

No comma in compound sentence

No

 

No

4.  

Wrong word

No

 

No

5.  

No comma in nonrestrictive element

No

 

No

6.  

Wrong or missing inflected endings

Some

 

Some

7.  

Wrong or missing preposition

No

 

No

8.  

Comma splice

No

 

No

9.  

Possessive apostrophe error

Yes

 

Yes

10.  

Tense shift

No

 

No

11.  

Unnecessary shift in person

No

 

No

12.  

Sentence fragment

Yes

 

Some

13.  

Wrong tense or verb form

No

 

No

14.  

Subject-verb agreement

Yes

 

Some

15.  

Lack of comma in a series

Yes

 

Yes

16.  

Pronoun agreement error

Yes

 

No

17.  

Unnecessary comma with restrictive element

No

 

No

18.  

Run-on or fused sentence

No

 

No

19.  

Dangling or misplaced modifier

Some

 

Some

20.  

Its/it's error

Yes

 

Yes

Score after using the copy/paste trick to "awaken" the grammar checker   34%

Using the same scoring procedure outlined above, we see that Word 2002 and 2003 earn 25 of 80 possible points (31%), and Word 2007 earns 27 of 80 possible points (34%). These results are only slightly better than Word 97, and not near what WordPerfect was doing ten years ago.

In example 4, Word 2002 or 2003 will find one of the "Wrong word" errors occasionally, depending it seems on some circumstances of the document's creation. Word 2007, however, fails to find any of the wrong words in example 4. Similarly, I notice that Word 2002 or 2003's ability to find the subject-verb agreement errors in example 14 or commas in a series in example 15 or pronoun agreement errors in example 16 also depends on the manner of the document's creation.

Shockingly, Word 2002, 2003, and 2007 actually give truly bad advice about two of the usage errors — both highly stigmatized mistakes. In example 8, the comma splice, Word completely misses the real comma splice but (worse yet) misidentifies another comma as a comma splice error. See Figure 3, a screen shot of Word 2007 displaying this problem.

Figure 3: Screen capture of Word 2007 misjudging the location of the comma splice.

Screen shot of Word 2007 misidentifying a comma splice error.

In example 18, the run-on sentence, Word 2002, 2003, and 2007 actually do correctly identify the error but recommend a correction that would change the run-on into a comma splice! See Figure 4 below.

Figure 4: Screen capture of Word 2007 suggesting to fix a run-on sentence by making a comma splice

Word 2007 mispunctuating a run-on sentence to create a comma splice.

Although the scores for WordPerfect 8, 9, 10, 11, and 12 have remained the same, the program's functioning has changed slightly, mostly for the better. WordPerfect 8 and 9 (like all the others) could not find one of the major sentence structure errors — the run-on (or fused) sentence. However, WordPerfect 10, 11, and 12 could. That is a real boon. On the other hand, WordPerfect 10, 11, and 12's ability to recognize a fragment has diminished: fragments beginning with which (as in example 12 above) are misidentified as questions (prompting the user to consider putting a question mark at the end of the fragment). Furthermore, WordPerfect 10, 11, and 12 were only able to find some of the commonly misused words: in example 4 above, for instance, WordPerfect 10, 11, and 12 missed the use of then for than. WordPerfect 8 found both of the wrong words in that example.

The fragment-beginning-with-which flaw is not unique to WordPerfect. Word suffers the same problem. See Figure 5:

Figure 5: Screen capture Word 2007 confusing a relative clause fragment for a question

Word 2007 wondering if a fragment is a question.

Also, Word occasionally offers bad options. In Figure 6, below, notice how Word 2007 offers to change the meaning of the sentence rather than its grammar to correct a subject-verb agreement error in example 14. This is a step back from Word 2002 and 2003, I think, both of which could accurately find and correct both agreement issues in example 14. For this reason, I have penalized Word 2007 one point (from "Yes" in Table 1 to "Some" in Table 2). To be fair, I must say that I suspect Word here is trying to compensate for "false positive" subject-verb agreement errors, but I have not been able to verify my suspicion.

Figure 6: Screen capture Word 2007 offering to change the meaning of the sentence rather than its grammar

Screen capture Word 2007 offering to change the meaning of the sentence rather than its grammar

However, Word is not alone in offering bad advice. LanguageTool in Open Office Writer misidentified the adverb too for the infinitive marker to or the preposition to:

Figure 7: Screen capture of LanguageTool 0.9.1 in Open Office 2.3 misidentifying the adverb too

LanguageTool in Open Office misidentifying the adverb 'too.'

LanguageTool's error, though, is not as egregious as the bad options suggested by Word.

LanguageTool's potential strength may be its multilingual nature. LanguageTool's aim is to produce a grammar and style checker for a host of different languages. See the LanguageTool Options dialog in Figure 8 for example.

Figure 8: Screen capture of LanguageTool 0.9.1 Options dialog

LanguageTool's Options dialog.

Discussion

Real humans work in a context, as do our languages. To evaluate the acceptability of any usage or structure, we need to see that construction in a social and linguistic context. The ability to adapt to context is something that most humans do easily, yet that same sensitivity to context requires enormous processing power for a computer. Therefore, in order to make the programs function within a reasonable amount of time given the current technology, most grammar checking functions are done context-free, meaning that the program takes very little of the context into consideration. As the technology improves, the software can evaluate a structure or word choice better in context, in a context-sensitive manner, as humans do.

In 2002, Bruce Wampler, the original author of the Grammatik grammar checker, which is still used with WordPerfect, wrote to me to tell me of a discussion he had with John Markoff of the New York Times:

"I've been in a discussion with John about how the grammar checking available today, 2002, is essentially no different than it was in 1992 when I sold my company to WordPerfect and quit working on the code. Essentially, what has happened is that Microsoft has decided that its version of a grammar checker is 'good enough' and has stopped significant work on improvement. No one else in the world has the resources to build a better grammar checker.

"Who wants to try to compete with Microsoft Word's 95% market share?

"I'm fairly confident that if you ran your test through versions of Grammatik or other grammar checkers of around 1992-1994 that the results would be almost the same as you got with the most recent versions. And I'd bet you'll get the same results 5 years from now. Very sad."

Indeed. What's more, he's right — about all of it. It is five years now since Wampler wrote those words, and these tests justify his pessimism.

After ten years of benchmarking the progress of these grammar checking programs, not one of them has made significant improvements toward creating a system that can reliably find and correct the twenty most common usage errors made by first year composition students at American colleges and universities. In ten years of product development, Microsoft, for example, has only managed to improve Word's grammar checking functionality a mere 10%, judging by these test results. Small improvement.

However, given that Word 2002, 2003, and 2007 offer such horrifically bad advice in examples such as 8 and 18 and given that Word 2000, 2002, 2003, and 2007 do not appear to initialize the grammar checker adequately, "improvement" is a word I really would rather not use here.

Coda

Download the files below, save them to disk, and try this experiment for yourselves. It is quite interesting to see how each grammar checker examines a document, and this discussion does not replace the value of seeing how both programs work side by side.

Download this document with the twenty most common errors
in Word 6/7 file format,
in Word 97/2000/2002/2003/2007 file format, or
in WordPerfect file format
And just for fun, try Abraham Lincoln's "Gettysburg Address"
in Word 6/7 file format,
in Word 97/2000/2002/2003/2007 file format, or
in WordPerfect file format.

Web resources

The links below offer a variety of sites on the internet with more information about grammar checking, parsing, and natural language processing. Several sites for usage study are available too.

Links to Style Sheets     Links to Writer's Guides
and Usage Handbooks
   
       
  1. College of DuPage Library's Footnote and Citation Style Guides

  2. Web Extension to the APA Style Sheet

  3. Montana State University's Collection of Style Guides
   
  1. Fowler, H. W. 1908. The King's English

  2. Purdue On-line Writing Lab

  3. Jack Lynch's Grammar and Style Notes

  4. The University of Illinois Writers' Workshop Online Handbook

  5. Mary McCaskill's Handbook for Technical Writers and Editors
   

i18n and L10n

"i18n" and "L10n" are abbreviations for internationalization and localization, efforts to make software that is free of linguistic and cultural biases. If you are a user of any of the software programs mentioned here, using the program in a language setting other than "US-English," I would love to hear from you.

Notes

1 The software versions in these tests are
Word 2007 (12.0.6015.5000) on Windows Vista
Word 2004 on MacOS X Version 11.2 (060202)
Word 2003 (11.6359.6350) SP1 on Windows XP SP2
Word 2002 (10.4219.4219) SP2 on Windows XP SP1
Word 2000, 97, 7, and 6 version information not available since I no longer have those machines; however all tests were done on Windows operating systems — Windows 98 SE in the case of Word 2000, Word 7, and Windows 3.11 in the case of Word 6.

WordPerfect 12 (12.0.0.238) on Windows XP SP2
WordPerfect 11 (11.0.0.233) on Windows XP SP2
WordPerfect 10 (10.0.0.990) on Windows XP SP2
WordPerfect 9, 8, and 6 version information not available since I no longer have those machines; however all tests were done on Windows operating systems — Windows 98 SE in the case of WordPerfect 9 and 8, and Windows 3.11 in the case of WordPerfect 6.

Grammarian Pro 1.7 from Linguisoft on MacOS X

Open Office 2.3 with the LanguageTool 0.9.1 extension installed on Windows XP SP2

I am gratefully indebted to Kevin Dulzo, Senior Network Security Engineer, U. S. Cellular, for all of the Mac data. His results were obtained using Microsoft Office for the Mac 2004 and Grammarian Pro 1.7 from Linguisoft.

2 Running exactly the same tests through Word XP (Word 2002), Galletta, et al. found that Word XP showed some improvement, with 6 of the 20 errors found; however, they offer no details about which items Word XP could find successfully.

References

Connors, Robert J. and Andrea A. Lunsford. "Frequency of Formal Errors in Current College Writing, or Ma and Pa Kettle Do Research," The St. Martin's Guide to Teaching Writing 2nd ed. Ed. Robert Connors and Cheryl Glenn. New York: St. Martin's, 1992.

Dulzo, Kevin. "Evaluation of Mac Grammar Check Software" Email to the author. 3 May 2006.

Galletta, Dennis F. "Evaluating Grammar Checkers." Email to the author. 7 March 2005.

Galletta, D., Durcikova, A., Everard, A., and Jones, B. (forthcoming) "Does Spell-Checking Software Need a Warning Label?" Communications of the ACM.

An older, pre-publication version of their article is available online as a PDF: "Cognitive Fit and an Intelligent Agent for a Word Processor: Should Users Take All That Advice?" Proceedings of the 36th Hawaii International Conference on System Sciences (HICSS '03), Washington, DC: IEEE, 2002.

Neufeld-Kaiser, Jerry. "Evaluating Grammar Checkers." Email to the author. 30 November 1998.

Wampler, Bruce. "Grammar Checking." Email to the author. 13 April 2002.

Appendix

Below are the examples again with corrections. It's interesting to compare what the grammar checkers suggest to these corrections.

1. No comma after introductory element

Example:

After we watched the movie we went over to the pizza joint for a bite to eat.

Correction:

After we watched the movie, we went over to the pizza joint for a bite to eat.

2. Vague pronoun reference

Example:

John Smith reported the problem to Bob Adams, and he corrected it immediately.

Correction:

John Smith reported the problem to Bob Adams, and Adams corrected it immediately.

OR

John Smith reported the problem to Bob Adams, and Smith corrected it immediately.

3. No comma in compound sentence

Example:

John Smith reported the problem to Bob Adams and Adams corrected it immediately.

Correction:

John Smith reported the problem to Bob Adams, and Adams corrected it immediately.

4. Wrong word

Example:

Building a new fiber optic network will take less dollars then originally suspected.

Correction:

Building a new fiber optic network will take fewer dollars than originally suspected.

5. No comma in nonrestrictive element

Example:

Contemporary animal rights groups who are using the same strategies that the civil rights movement developed in the 1960s find that non-violent civil disobedience is effectively drawing attention to their cause.

Correction:

Contemporary animal rights groups, who are using the same strategies that the civil rights movement developed in the 1960s, find that non-violent civil disobedience is effectively drawing attention to their cause.

6. Wrong or missing inflected endings

Example:

Teacher representatives in the United States’ could have wrote the book themselves.

Correction:

Teachers' representatives in the United States could have written the book themselves.

7. Wrong or missing preposition

Example:

The man to whom you were speaking to was the curator on the exposition.

Correction:

The man to whom you were speaking was the curator on the exposition. (more formal)

OR

The man whom you were speaking to was the curator on the exposition. (less formal)

8. Comma splice

Example:

In 1980, Reagan promised to balance the federal budget, however, by 1988, the federal deficit was the largest in U. S. history — until now.

Correction:

In 1980, Reagan promised to balance the federal budget; however, by 1988, the federal deficit was the largest in U. S. history — until now.

9. Possessive apostrophe error

Example:

The Presidents speech was punctuated by enthusiastic applause from both sides of the aisle.

Correction:

The President's speech was punctuated by enthusiastic applause from both sides of the aisle.

10. Tense shift

Example:

John was edgy all day. He challenged anything that anyone said to him. Once at a meeting, John looked right at me and he says, "What do you mean by that?"

Correction:

John was edgy all day. He challenged anything that anyone said to him. Once at a meeting, John looked right at me and he said, "What do you mean by that?"

11. Unnecessary shift in person

Example:

Students should register for classes early if you want to ensure that you get a good schedule.

Correction:

Students should register for classes early if they want to ensure that they get good schedules.

12. Sentence fragment

Example:

Publishers today are at a loss trying to cope with the new trends in information technology. Which is not surprising actually.

Correction:

Publishers today are at a loss trying to cope with the new trends in information technology, which is not surprising actually.

13. Wrong tense or verb form

Example:

The market has responding favorably to the surprisingly strong earnings that the high-tech sector reports yesterday.

Correction:

The market has responded favorably to the surprisingly strong earnings that the high-tech sector reported yesterday.

14. Subject-verb agreement

Example:

The President as well as his advisors were in meetings all weekend at Camp David. Neither the President nor his advisors is commenting on the status of the negotiations.

Correction:

The President as well as his advisors was in meetings all weekend at Camp David. Neither the President nor his advisors are commenting on the status of the negotiations.

15. Lack of comma in a series

Example:

Forget what Wheaties tells you; the real breakfast of champions is pizza, beer and eggs.

Correction:

Forget what Wheaties tells you; the real breakfast of champions is pizza, beer, and eggs.

16. Pronoun agreement error

Example:

Everyone should register early to make sure that they get the classes they want.

Correction:

Everyone should register early to make sure that she or he gets the classes she or he wants.

OR

All should register early to make sure that they get the classes they want.

17. Unnecessary comma with restrictive element

Example:

Several of the guys on the team were here at the time, but the guy, who is talking to the police, is the only witness to the accident.

Correction:

Several of the guys on the team were here at the time, but the guy who is talking to the police is the only witness to the accident.

18. Run-on or fused sentence

Example: