A Comparative Study of PMI and its Variants for Meaningful Word Selection

Toprak A., Toprak F. S.

2025 Innovations in Intelligent Systems and Applications Conference, ASYU 2025, Bursa, Türkiye, 10 - 12 Eylül 2025, (Tam Metin Bildiri) identifier

  • Yayın Türü: Bildiri / Tam Metin Bildiri
  • Doi Numarası: 10.1109/asyu67174.2025.11208273
  • Basıldığı Şehir: Bursa
  • Basıldığı Ülke: Türkiye
  • Anahtar Kelimeler: chi-square, keyword extraction, local mutual information, natural language processing, pointwise mutual information, semantic association measures, t-score, word selection
  • İstanbul Ticaret Üniversitesi Adresli: Evet

Özet

Pointwise Mutual Information (PMI) and its statistical variants - such as Positive PMI (PPMI), Normalized PMI (NPMI), Local Mutual Information (LMI), T-score, and Chi-Square - are widely used in natural language processing (NLP) for identifying significant word associations. Despite their widespread application, there is a lack of comprehensive comparative evaluations across different corpora. In this study, we conduct an extensive empirical analysis of six association measures to evaluate their effectiveness in selecting semantically meaningful word pairs. Experiments are performed on the Simple English Wikipedia dump, representing a high-resource language setting. Evaluation encompasses intrinsic semantic similarity tasks using WordSim-353 and SimLex-999 datasets, and extrinsic unsupervised keyword extraction on the Inspec dataset. Results demonstrate that NPMI outperforms other metrics in semantic similarity tasks, with a 7.2% improvement in Spearman correlation over standard PMI. In keyword extraction, PPMI achieves the highest F1-score of 0.684, exceeding TF-IDF and Chi-Square by more than 6%. These findings offer valuable guidance for selecting appropriate association measures tailored to specific NLP tasks.