a workshop jointly organised by the CEC Language
Engineering 1 projects SPARKLE and ECRAN
to be held at the
FIRST INTERNATIONAL CONFERENCE
ON LANGUAGE RESOURCES AND EVALUATION
GRANADA, SPAIN, 26 MAY 1998
This workshop will provide a forum for researchers interested in the
development and evaluation of natural language grammars and parsing
systems, and in the creation of syntactically annotated reference
corpora.
Organisers: John Carroll, Roberto Basili, Nicoletta Calzolari,
Robert Gaizauskas, Gregory Grefenstette
WORKSHOP SCOPE AND AIMS
The aim of this workshop is to provide a forum for discussion of
evaluation methods for parsing systems, and proposals for the
development of syntactically annotated language resources.
With increased attention to evaluation of component technology in
language engineering, evaluation of parsing systems is rapidly becoming
a key issue. Numerous methods have been proposed and while one, the
Parseval/Penn Treebank scheme, has gained wide usage, this has to some
extent been due to the absence of workable alternatives rather than to
whole-hearted support. Parseval/PTB evaluation has several limitations
and drawbacks, including a commitment to a particular style of
grammatical analysis, and oversensitivity to certain innocuous types of
misanalysis while failing to penalise other common types of more serious
mistake. Also, the original published description of the scheme -- and
the evaluation software widely distributed as a follow-up to it -- is
specific to the English language. It may be that there are currently no
alternative more workable schemes or proposals, but this needs to be
more fully discussed: this workshop will provide an opportunity for such
a debate.
This workshop is particularly timely given the large number of CEC
Language Engineering projects that involve parsing in one form or
another and which need to evaluate and share the results of their
efforts. Parsing is an essential part of many larger applications, such
as Information Extraction, which have gained in importance over the last
few years. Often in such systems, the strength of the parser and
grammar has a direct effect on the desired results, and thus achieving
good results rests on being able to determine and improve weaknesses in
the parser/grammar. Without a reliable parser evaluation method this
cannot be done effectively.
A parsing evaluation workshop is also appropriate at this time given the
imminent creation of large-scale syntactically annotated resources for
European languages. Contributions from those involved in such activities
are welcomed, so as to improve communication between the resource
construction and the resource utilisation communities. This should
ensure that the resources constructed are maximally useful to the
general language engineering community.
The organisation of this workshop brings together two European language
engineering projects which are closely related and whose partners share
similar research interests: SPARKLE and ECRAN.
The organisers solicit contributions from the general community on the
following topics:
-- descriptions of generic syntactic annotation schemes
-- methodologies and metrics for parsing system evaluation
-- reports and analyses of the results of utilising particular parser
evaluation schemes
-- description/analysis/experience of language-dependent (especially
for languages other than English) and task-dependent syntactic
annotation schemes
PROGRAMME COMMITTEE
Roberto Basili Gregory Grefenstette
Ted Briscoe Mark Hepple
Nicoletta Calzolari Tony McEnery
John Carroll Maria Teresa Pazienza
Roberta Catizone Paola Velardi
Robert Gaizauskas Yorick Wilks
PAPER SUBMISSION
Papers should not exceed 4000 words or 10 pages. Submission may be in
either hard copy or electronic form. The submission deadline is February
15th, 1998.
Hard Copy Submission:
Three copies of the paper should be sent to:
Dr John Carroll
Cognitive and Computing Sciences
University of Sussex
Brighton BN1 9QH
UK
Electronic Submission:
Electronic submission may be in either self-contained Latex, Postscript,
or RTF formats, to john.carroll@cogs.susx.ac.uk. For each submission --
whether hard copy or electronic -- a separate plain ascii text email
message should be sent to John Carroll, containing the following
information:
# NAME : Name of first author
# TITLE: Title of the paper
# PAGES: Number of pages
# NOTE : Any relevant instructions
# KEYS : Keywords
# EMAIL: Email of the first author
# ABSTR: Abstract of the paper
. . . . . .
IMPORTANT DATES
Paper submission deadline (hard copy/electronic) February 15th
Notification of acceptance March 10th
Camera-ready papers due April 10th
Workshop May 26th
CONFERENCE INFORMATION
General information about the conference is at:
<http://www.icp.inpg.fr/ELRA/conflre.html>
Specific queries about the conference should be directed to:
LREC Secretariat
Facultad de Traduccion e Interpretacion
Dpto. de Traduccion e Interpretacion
C/ Puentezuelas, 55
18002 Granada, SPAIN
Tel: +34 58 24 41 00 - Fax: +34 58 24 41 04
reli98@goliat.ugr.es