[Corpora-List] variant log likelihood calculations

From: Don Hardy (Don.Hardy@Colostate.edu)
Date: Tue Dec 14 2004 - 19:38:54 MET

  • Next message: Don Hardy: "Re: [Corpora-List] variant log likelihood calculations"

    Can anyone tell me whether the log likelihood calculation that follows (from Rayson and  Rayson and Garside) produces the exact same log likelihood figures for a 2X2 table that the formula in Dunning (1993) does?

    Rayson and Garside:

    -2lnλ = 2*(((a*ln(a/E1)) + (b*ln(b/E2)))

    Dunning:

    -2ln
    λ =2[logL(p1,k1,n1) + logL(p2,k2,n2) - logL(p,k1,n1) - logL(p,k2,n2)]

    I've tried both with Dunning's data in his 1993 paper and get very slightly different results with each.  However, as is obvious from this email, I am neither a mathematician nor a statistician.

    Many, many thanks,

    Don


    Dunning, Ted.  "Accurate Methods for the Statistics of Surprise and Coincidence."  Computational Linguistics, (1993): 61-74.

    Rayson, Paul. “Matrix: A Statistical Method and Software Tool for Linguistic Analysis Through Corpus Comparison.” Ph.D. diss., Lancaster University, 2003.

    Rayson, Paul, and Roger Garside.  “Comparing Corpora Using Frequency Profiling.”  Proceedings of the Workshop on Comparing Corpora, (2000): 1-6.

     
    Donald E. Hardy
    Associate Professor
    Department of English
    Colorado State University
    Fort Collins, CO 80523
    970-491-5349
    Don.Hardy@Colostate.edu
    http://textant.colostate.edu



    This archive was generated by hypermail 2b29 : Tue Dec 14 2004 - 19:38:45 MET