CS/AI 684 — required and supporting readings
Every link below is a public URL (paper, official docs, or a page on this site). TensorTonic Plus is never required.
Primary text for weeks 1–12 and 14:
Systems and attention
| Resource | Link |
|---|---|
| Attention Is All You Need | arXiv:1706.03762 |
| FlashAttention-2 | arXiv:2307.08691 |
| PagedAttention / vLLM paper | arXiv:2309.06180 |
| vLLM docs | docs.vllm.ai |
| Roofline (Williams et al., 2008) | EECS-2008-134 PDF |
| GQA | arXiv:2305.13245 |
| DeepSeek-V2 (MLA) | arXiv:2405.04434 |
| DeepSeek-V3 | arXiv:2412.19437 |
| YOCO | arXiv:2405.05254 |
| Longformer | arXiv:2004.05150 |
| BigBird | arXiv:2007.14062 |
| RoPE | arXiv:2104.09864 |
Memory, MoE, optimizers
| Resource | Link |
|---|---|
| LLM.int8() | arXiv:2208.07339 |
| Speculative decoding | arXiv:2211.17192 |
| Switch Transformers | arXiv:2101.03961 |
| ViT | arXiv:2010.11929 |
| Muon | kellerjordan.github.io/posts/muon |
| Megatron-LM | arXiv:1909.08053 |
| DeepSpeed | deepspeed.ai |
Agents, graphs, verification
| Resource | Link |
|---|---|
| SWE-bench | arXiv:2310.06770 |
| InstructGPT / RLHF | arXiv:2203.02155 |
| PPO | arXiv:1707.06347 |
| GraphRAG | arXiv:2404.16130 |
| Neo4j graph concepts | neo4j.com docs |
| Cypher manual | Cypher manual |
On this site (foundations)
Start from the week page; each lists clickable prerequisites on this site. The most reused ones:
Back to the syllabus.