EVALUATION IN SPEECH AND LANGUAGE TECHNOLOGY

Robert John Gaizauskas (R.Gaizauskas@dcs.shef.ac.uk)
Fri, 25 Apr 97 10:59:26 BST

*** IF YOU HAVE NOT RECEIVED ACKNOWLEDGEMENT ***
*** OF ELECTRONIC SUBMISSION PLEASE RESUBMIT ***

2nd Call for Abstracts
and
Registration Information

Speech and Language Technology (SALT) Club Workshop
on

EVALUATION IN SPEECH AND LANGUAGE TECHNOLOGY

June 17-18, 1997
Halifax Hall, University of Sheffield, Sheffield, UK

Sponsored by
Department of Trade and Industry
and
Institute for Language Speech and Hearing (ILASH)
and
Department of Computer Science
University of Sheffield

Evaluation is now an issue no speech and language researcher can afford to
ignore. The new empiricism in speech and language research, occasioned by the
massive increase in on-line corpora, together with the need for well
understood system performance characteristics implicit in the very concept of
language engineering, have pushed the topic of evaluation onto the agendas of
most researchers. Current evaluation metrics may be incomplete, simplifying,
distorting, or just plain irritating, but they are here to stay and our best
option is to make them better. This workshop aims to bring together
researchers in speech and language with an interest in evaluation in order to
reflect on their experiences with evaluation to date and to discuss how to
improve metrics and how to introduce metrics into areas where they have not
yet evolved.

Abstracts of up to 500 words describing proposed presentations are invited on
topics including (but not limited to):
* the role, significance, and limitations of evaluation in speech and
language research
* critical analysis of existing evaluation regimes or metrics
(e.g. MUC,TREC,ATIS,CSR)
* discussion or presentation of the EU-sponsored EAGLES proposals on
evaluation and standardisation
* how to foster small-scale, low overhead evaluation (e.g. in the
the upcoming EPSRC-sponsored Speech and Language programme)
* proposals for new evaluation regimes or metrics for language systems
that carry out tasks such as:
- speech recognition
- information extraction
- information retrieval
- machine translation
- natural language generation
- speech synthesis
- spoken language understanding
- topic spotting in spoken language
or for component technologies used in these systems, such as:
- tagging
- parsing
- coreference
- proper name recognition and classification in text
- discourse segmentation
* results of evaluation studies carried out on speech or language
systems or component technologies
* tools for evaluation -- scorers, visualisation tools, markup tools
* data provision for evaluation -- proposals for corpus markup


In the tradition of relatively informal SALT workshops, authors of accepted
abstracts will be invited to give a presentation of roughly 30 minutes.
Abstracts, or complete papers, should authors choose to submit them (length
limit 3500 words), will be published in an informal workshop proceedings.
Abstracts will be reviewed by a local programme committee consisting of:
Dr. R. Gaizauskas (chair)
Prof. Y. Wilks
Dr. S. Renals
Dr. P. Green

Abstracts due: May 1
Notification to authors: May 15
Camera-ready: June 1
Early registration deadline: May 15

For review abstracts should be submitted in ASCII, postscript, or LaTeX
(single, self-contained file only, please). Final abstracts/papers
should be in LaTeX -- authors of accepted presentations will be given
more detailed instructions on formatting later.

Registration and accomodation details will follow shortly.

Further information may be obtained by contacting:

Dr. Robert Gaizauskas
SALT Steering Committee

email: R.Gaizauskas@dcs.shef.ac.uk
phone: +44 (0)114 222 1827
fax: +44 (0)114 222 1810/278 0972
www: http://www.dcs.shef.ac.uk

University of Sheffield
Department of Computer Science
Regent Court, 211 Portobello Street
Sheffield S1 4DP, U.K.


________________________________________________________________________

GENERAL INFORMATION
________________________________________________________________________

VENUE

The venue for registration and the workshop is:

Halifax Hall of Residence,
Endcliffe Vale Road,
Sheffield,
S10 5DF, U.K.

The University of Sheffield's Halls of Residence are situated on the western
side of the city in a leafy residential area described by John Betjeman as
``the prettiest suburb in England''. Halifax Hall is centred on a local
Steel Baron's house, dating back to 1830 and set in extensive grounds. It
was acquired by the University in 1830 and converted into a Hall of
Residence for women with the addition of a new wing.

Halifax Hall is one mile from Sheffield's city centre, and is easily reached
from Sheffield Station by bus or taxi, and from Manchester International
Airport and train to Sheffield. The # 50 bus from the station will drop
you off at the Botanical Gardens but we recommend taking a taxi for around
3.50 pounds if you have heavy cases.

ACCOMMODATION

Single en-suite room accommodation has been reserved for conference delegates
in Halifax Hall.

PARKING

There is ample free parking for approx. 50 cars in the vicinity of Halifax
Hall.


________________________________________________________________________

REGISTRATION NOTES
_______________________________________________________________________

ADDRESS (for registrations)

Gill Wells,
Research Co-ordinator
Department of Computer Science,
Regent Court,
211 Portobello Street,
Sheffield,
S1 4DP

Email: g.wells@dcs.shef.ac.uk
Fax: +44 (0) 114-278-0972
Phone: +44 (0) 114-22 21819

PAYMENT

Payment should be made by cheque or money order payable to `University
Sheffield' , drawn in pounds sterling on a British clearing bank. Payment
should be sent together with the Registration Form to the above address.

CONFIRMATION

Confirmation of booking and a map with travel details will be
sent on receipt of this application form.

DISCLAIMER

The University of Sheffield and the Conference Organisers accept no
responsibility for injury to persons attending the Conference, nor
for loss of or damage to their property. The Conference Organisers
reserve the right to change the details given in this document without notice.


________________________________________________________________________

SALT Workshop on Evaluation in Speech and Language Technology

Registration Form

________________________________________________________________________

Registration Details

Early registration (before Friday 16 May): £50.00
Late registration £ 70.00

Total: _____

Registration includes attendance, meals, proceedings and refreshments

Accommodation

Bed and Breakfast (ensuite bathroom) is available at £33.55 per night (inc.
VAT). Please specify in the table below which night(s) you wish to stay.
Accommodation will be available on Monday 16 June (the night preceding the
Workshop) and Tuesday 17 June.. Please note that there will be NO
accommodation available after the workshop closes.

Bed & Breakfast:
16 June: _____ (33.55)

17 June: _____ (33.55)

Total _____

For special dietary considerations please contact Gill Wells
(g.wells@dcs.shef.ac.uk) giving full details of your requirements.

Personal Details

Name: ________________

Address: _______________

________________

________________

Phone: ________________

Fax: ________________

Email: ________________

WWW: ________________

I enclose a cheque or money order drawn in pounds sterling on a British
clearing bank for ______ pounds, payable to the University of Sheffield.

Signed: ___________________

Date: ___________________