Re: Corpora: Comparative evaluation of taggers/parsers

Philip A. Bralich, Ph.D. (bralich@hawaii.edu)
Fri, 14 Nov 1997 08:56:00 -1000

At 02:10 AM 11/14/97 -1000, Jean Hudson wrote:
>Has anyone carried out official tests of the various grammatical taggers
>available in terms of accuracy (recall and precision), robustness, and
>speed? Could someone point us to appropriate research comparison papers
>(rather than just sales publicity)? How many of them mark some structural
>information such as subjects and objects?
>
>Please reply to: aharley@cup.cam.ac.uk

There is actually very little. There are the results from the MUC
conferences at http://cs.nyu.edu/cs/faculty/grishman/muc6.html.
And there are the pages posted at the Amalgam page at
http://www.scs.leeds.ac.uk/amalgam/amalgam/amalghome.htm. I have
seen some vague references in some of the materials I have but as
far as I know there is no such comparison in the literature. I will
be watching avidly for your summary. Hopefully, there is more than
I realize.

Recently there was a nice collection of addresses of on-line parsers
posted to the "Linguist" mailing list. The address is:
http://www.ifi.unizh.ch/CL/InteractiveTools.html

However, this list contains no comparative evaluation. If you are
become aware of anyone who is doing such work or who would like to
do such work, I would also like to know.

Phil Bralich

Philip A. Bralich, Ph.D.
President and CEO
Ergo Linguistic Technologies
2800 Woodlawn Drive, Suite 175
Honolulu, HI 96822

Tel: (808)539-3920
Fax: (808)5393924