Re: Corpora: Ergo's Parsing Contest

Anne Sing (annes@htdc.org)
Mon, 22 Feb 1999 09:23:40 -1000 (HST)

The only problem with the contests you mention is that they
are based on searching huge corpii of unrestricted text
which of coures is a valid area of research but which
does nothing for the more immediate problems of improving
navigation and control devices, q&a dialoging and so
forth. The existence of these contests also does not
explain why you are incapable of handling the far simpler
sentences that are in the Ergo Contest. The MUC and
TREC contests require huge dictionaries to work with
language such as found in the New York Times and Wall
Street Journal but does not contribute a bit to the
other areas of NLP that are quite primitive yet need
further research. Education, navigation and control,
Multi-Media and games can all be signficantly improved
with the addition of tools like those discussed in our
contest. There are a huge number of jobs and papers
and projects that are possible in this area that the
greater NLP community is ignoring at the expense of
students, departments, and companies everyhere. Thus
there is something somewhat fishy about contests working
with the New York Times when basic language cannot be
handled and the evidence of this is in the inability of
all these other parsing systems to work with the
sentences in the Ergo contest.

And if you actually had the tools you are talking about you
could and would put me in my place by displaying their
effectiveness contrary to ours. Don't think you can fool
me or anyone else with charges of unprofessionalism. If the
tools existed your upraised hackles would delight at the
prospect of dropping incontrovertable counterevidence in my
lap. The unprofessionalism exists only in a refusal to
relate to evidence--the evidence being that the existent
parsing systems cannot handle the very simple sentences
offered in the Ergo contest. Anyone reading this should
just download the results from the contest posted at our
website at http://www.ergo-ling.com to see for themselves.

And anyone who has a parsing system should REMEMBER the sentences
we are offering in this contest are far simpler than those in
the corpii that are being searched by TREC and MUC. For that
reason if TREC and MUC have any value at all, all entrants from
the past should be able to participate in this contest easily
and demonstrate clearly the superiority of their tools and
show me once and for all the error of my ways.

Anyone with the slightest interest in NLP should go to the
Ergo web site (http://www.ergo-ling.com) download the contest
rules, sentences, and the Ergo results and then ask this one
question: "WHY IN GOD'S NAME DON'T WE SEE THIS FROM THE
NLP COMMUNITY OUTSIDE OF ERGO AND WHY IN GOD'S NAME DON'T
THEY JUST POST A WEB SITE WHERE THEIR RESULTS CAN BE FOUND."

Phil Bralich

At 06:01 AM 2/22/99 -1000, Ken Litkowski wrote:
>I can only speak for myself. I have a parser which is reasonably decent
>and can handle pretty complex stuff (as evidenced in the recent Senseval
>competition). I wouldn't waste my time on your parsing contest because
>it is rather unprofessional, not because my "computational linguistics"
>isn't up to it. The challenge is not to us, but to you. Can your
>system participate in well-structured "competitions" like Senseval,
>TREC, and MUC? Can you put your money where your mouth is? If you do
>well in such competitions, can you be professional and not use your
>accomplishment to ballyhoo your product, but rather to advance the state
>of computational linguistics? Can you let the rest of us alone and
>simply sell your product wherever you can prove its value to your
>customers (like those on the list who already have superior parsers and
>making money with them)?
>
>My apologies to the rest of the list for my raised hackles.
>--
>Ken Litkowski TEL.: 301-482-0237
>CL Research EMAIL: ken@clres.com
>9208 Gue Road
>Damascus, MD 20872-1025 USA Home Page: http://www.clres.com
>
Philip A. Bralich, Ph.D.
President and CEO
Ergo Linguistic Technologies
2800 Woodlawn Drive, Suite 175
Honolulu, HI 96822

Tel: (808)539-3920
Fax: (808)539-3924
bralich@hawaii.edu
http://www.ergo-ling.com

Philip A. Bralich, President
Ergo Linguistic Technologies
2800 Woodlawn Drive, Suite 175
Honolulu, HI 96822
tel:(808)539-3920
fax:(880)539-3924