Re: Corpora: Ergo's Parsing Contest

Philip A. Bralich, Ph.D. (annes@htdc.org)
Tue, 23 Feb 1999 11:18:19 -1000

At 11:02 AM 2/23/99 -1000, you wrote:
>I can only speak for myself. I have a parser which is reasonably decent
>and can handle pretty complex stuff (as evidenced in the recent Senseval
>competition). I wouldn't waste my time on your parsing contest because
>it is rather unprofessional, not because my "computational linguistics"
>isn't up to it. The challenge is not to us, but to you. Can your
>system participate in well-structured "competitions" like Senseval,
>TREC, and MUC? Can you put your money where your mouth is? If you do
>well in such competitions, can you be professional and not use your
>accomplishment to ballyhoo your product, but rather to advance the state
>of computational linguistics? Can you let the rest of us alone and
>simply sell your product wherever you can prove its value to your
>customers (like those on the list who already have superior parsers and
>making money with them)?
>
>My apologies to the rest of the list for my raised hackles.
>--
Raised hackles or not, if you or anyone else who participated
in MUC, TREC or Senseval were able to meet the challenge of
the contest I am proposing you would simply post your results
or the location of the tools and that would be that. No need to
excuse yourself or accuse me of unprofessionalism. Meet the
challenge on those very simple sentences and my point is
finished.

The point being of course is that there is something a little fishy
in those other contests in that they are looking at huge corpora
of unrestricted text but are unwilling to demonstrate their abilities
with the sorts of sentences that are required right now for products
such as speech rec and navigation and control devices. If you
could indeed generalize to the smaller sentences based on your
tools in that contest then you and others would merely post the
results to those 100 sentences in my contest and whatever you
call my posts "mere bombasts" or "a significant academic and industrial
challenge" would be over.

So in sum, however you seek to characterize me or the contest,
the fact remains that you have not demonstrated your ability to meet
the challenge. You have only demonstrated a willingness to excuse
yourself from it. And this in the face of the fact that there is a hidden
assumption that the MUC, TREC and SENSEVAL tests somehow already
cover what we are demonstrating in the Ergo contest. However, if these
other contests do indeed demonstrate everything that the Ergo contest
does then you could post in minutes or at least a few days your results on
the 1st Annual Ergo Parsing contest and immediately lay to rest my claims
to be working in an area that is largely ignored by the NLP community
today. So let's end the conversation once and for all. Post your results,+
demonstrate the errors in my claims, and I can say no more.

Phil Bralich

Philip A. Bralich, Ph.D.
Ergo Linguistic Technologies
2800 Woodlawn Drive, Suite #175
Honolulu, HI 96822
fax: (808)539-3921
tel: (808)539-3924