Prep 5: Large Language Models

📋 Main Topics¶

• Introduction to Large Language Models
• Definition and Evolution of LLMs
• Notable LLM Examples (GPT Series, PaLM, BERT)
• Text Generation Fundamentals
  • Auto-regressive Generation
  • Sampling Strategies (Greedy, Beam Search, Top-K, Top-P)
  • Temperature and Other Generation Parameters
• LLM Capabilities and Limitations
• Evaluation Methods
• Preview of Advanced Techniques
  • Prompt Engineering
  • Reinforcement Learning from Human Feedback (RLHF)
  • Retrieval-Augmented Generation (RAG)

🧠 Class Activity - Labs¶

• Lab 4: LLMs

📚 Recommended Readings¶

• Foundational Large Language Models by Google (Whitepaper)
• Google Machine Learning Crash Course: LLM
• Transformers in Google Machine Learning Crash Course

🎥 Recommended Videos¶

• Foundational Large Language Models by Google (Podcast) (30 min)
• Introduction to LLM by Andrej Karpathy (59 min)
• Introduction to LLMs by Ryan Rad (24 min)