Architecting Large Language Models from Scratch: From Perceptrons to Post-Training

Course Overview

Demystify Large Language Models (LLMs) by building, training, and optimizing a GPT-2-style transformer from first principles. This course bridges the gap between high-level API usage and low-level implementation, taking you on a journey from a single perceptron to advanced reinforcement learning techniques.

What You'll Learn

• Fundamentals of Neural Networks: Build perceptrons, implement activation functions (ReLU, GELU, ReLU²), and construct Multi-Layer Perceptrons (MLPs) using PyTorch.
• Training Mechanics & Optimization: Understand cross-entropy loss, backpropagation, and state-of-the-art optimizers like AdamW.
• Deep Network Stability: Prevent training collapse using He/Xavier initialization, additive residual streams, and pre-norm RMSNorm/LayerNorm.
• The Transformer Architecture: Implement causal self-attention, Multi-Query Attention (MQA), Grouped-Query Attention (GQA), and assemble a complete generative transformer.
• Data & Pre-Training: Clean raw web crawls, implement Byte Pair Encoding (BPE), build high-throughput sharded dataloaders, and run pre-training.
• Post-Training & Alignment: Fine-tune models using LoRA/QLoRA, and align them with human preferences using direct policy optimization (SimPO).

Prerequisites

• Strong proficiency in Python programming.
• Comfort with basic programming concepts (no prior advanced math or machine learning experience required, as fundamentals are built from the ground up).
• Access to Google Colab or a local GPU environment for training exercises.

Tools & Technologies

• Frameworks: PyTorch, Hugging Face Transformers, Tiktoken
• Hardware Acceleration: CUDA, Triton, GPU-performant coding
• Analysis & Visualization: Microsoft Excel / Google Sheets (for visual math demos), Bertviz, Neuronpedia