Neural Models

• Connolutional Neural Networks (CNN) -

• RNNs, LSTMs, GRUs

• Pooling

• Attention Mechanism, self-attention and BERT

• Transformer-based Models

  • Self-attention, positional encoding, encoder-decoder

  • Examples: BERT, GPT-3, T5

• Large Language Models (LLMs)

  • Trained on massive corpora with billions of parameter

Exotic

• memory neural networks (neural turing machine)

• tree-recursove NN

• graph Connolutional Neural Networks (GCN) -