Distributional Semantics

Based on the idea that words appearing in similar contexts have similar meanings.

1. Skip-gram (in Word2Vec) – predicts context from a word

   • Learns word embeddings that reflect word meaning.

2. CBOW (Continuous Bag of Words) – predicts word from context

   • Similar goal, different direction.

3. GloVe (Global Vectors for Word Representation)

   • Combines matrix factorization with co-occurrence statistics.

4. Latent Semantic Analysis (LSA)

   • Reduces dimensionality of word-document matrices via SVD.

5. Latent Dirichlet Allocation (LDA)

   • A probabilistic model for discovering topics in text.

6. Contextual Embeddings (Neural-based semantics)

   • E.g., ELMo, BERT — generate embeddings based on the entire sentence.