CS/AI 684 — Efficient Long-Context Agent Systems

Architecture, KV Memory, Sparse Attention, Agent Training, Graph Retrieval, and Multi-Agent Systems.

Level: Advanced undergraduate / graduate / practicing ML engineer
Duration: 14 weeks · 2 lectures + 1 lab per week · 8–12 h/week
Final outcome: build and evaluate a complete agent system combining long-context reasoning, structured retrieval, verification and tool use.

How to use this mirror

1. Open the week page. Prerequisites are ordinary links into this site (problems, ML math, LLM internals).
2. Read the public papers listed on that week (report PDF, arXiv, official docs).
3. Do the lab. Assignments A1–A4 have the same link style.
4. For official TensorTonic exercises (starter code / hidden tests), subscribe on TensorTonic.

Full bibliography: Required readings.

Modules

I — Long-context architecture

• Week 1 — Why Agent Workloads Change Model Architecture — KV cache · report
• Week 2 — Transformer Memory and Attention Economics — FlashAttention-2 · GQA
• Week 3 — Causal Encoder–Decoder: Reducing Prefill — YOCO
• Week 4 — CSA2: Sparse Attention Across Layers — report / CSA2
• Week 5 — Hierarchical Sparse Indexing — BM25

II — Memory-efficient inference

• Week 6 — FP4 KV Caches and Bounded Replay — LLM.int8() · quantization
• Week 7 — mHC, Engram and DSpark — speculative decoding

III — Training the model

• Week 8 — MoE and Multimodal Architecture — MoE chapter · Switch
• Week 9 — Optimizers and Large-Scale Pretraining — Muon · AdamW
• Week 10 — Distributed Multimodal Training Infrastructure — Megatron-LM

IV — Agent post-training

• Week 11 — Training Agents on Environments Rather Than Text — SWE-bench
• Week 12 — Asynchronous RL, Reasoning Effort and Agent Infrastructure — PPO

V — Graph engineering

• Week 13 — Knowledge Graphs as Structured Agent Memory — GraphRAG · Neo4j
• Week 14 — Grounding, Verification, Scaffolds and Multi-Agent Systems — ECE

Assignments

• A1 — Attention Resource Profile — Week 2
• A2 — CSA2 Mode-Frequency Sweep — Week 4
• A3 — Optimizer Comparison on a Large Embedding — Week 9
• A4 — Verifiable Agent Task — Week 11

Capstone

• Build an evidence-grounded enterprise agent: ingest → extract → entity resolution → graph store → hybrid retrieval → agent → verification → update.
• Implement or simulate ≥2 DeepSeek mechanisms (hierarchical selection, cross-layer KV reuse, KV quantization, bounded replay, speculative decoding, conditional memory, effort control, async rollout).
• Evaluate vector-only RAG vs graph-assisted retrieval on lookup / multi-hop / insufficient-evidence questions across answer correctness, evidence correctness, retrieval recall, tokens, latency and failure type.

How this fits the mirror

This library supplies the foundational drills and LLM-internals vocabulary. The systems half of this course is not in the problem catalog; see mastery/gaps.md for the exact ledger. The companion mastery curriculum is in mastery/.