Corpora: Perplexity and corpus size

Adam Kilgarriff (Adam.Kilgarriff@itri.brighton.ac.uk)
Tue, 23 Dec 1997 17:09:32 GMT

Can anyone point me to results/discussions of how perplexity (and
related info-theoretic measures, eg cross-entropy) vary with
size of training and test corpora?

Adam Kilgarriff

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
Adam Kilgarriff
Senior Research Fellow tel: (44) 1273 642919
Information Technology Research Institute (44) 1273 642900
University of Brighton fax: (44) 1273 642908
Lewes Road
Brighton BN2 4GJ email: Adam.Kilgarriff@itri.bton.ac.uk
UK http://www.itri.bton.ac.uk/~Adam.Kilgarriff
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%