Large Language Models (LLMs) are neural networks with billions of parameters, trained on massive text datasets. The Transformer architecture (Vaswani et al., 2017) was revolutionary: the self-attention mechanism lets any part of a text relate to any other part.
How ChatGPT works: RLHF (Reinforcement Learning from Human Feedback) — trainers chose the best responses, and the model was fine-tuned on that feedback. Tokenisation splits text into small fragments. The "hallucination" problem — the model can present incorrect information confidently. For this reason, important decisions always require verification.
Our courses on this topic