Week 93 — Disaggregation

LLM inference and performance engineering · Read / diagram

Hard

Phase 8 — LLM inference and performance engineering

Overview

DistServe, Splitwise, Mooncake, autoscaling; Kubernetes topology (KServe + KEDA + HPA) and edge deployment (ONNX, TensorRT-LLM, WebLLM) as alternative topologies

Mode this week: read_diagram. Match the work to that mode (operating model): implement ships running code; read/diagram ships a doc; deploy/benchmark ships a measured run. Personal dates, checkboxes, and the RPG layer stay in a learner journal.

Core deliverable

Disaggregated serving plan + depth-mode sub-deliverable: deploy the W84 pipeline to K8s with HPA-driven autoscaling on a synthetic spike AND port one vLLM config to ONNX/TensorRT-LLM for edge benchmarking (Modal A10 fallback if no edge hardware)

Gate 8 this week: Deployed vLLM or SGLang, benchmark methodology, observability dashboard, quantization report, speculative decoding report, inference cost model

Optional depth

Add a production-shaped benchmark, cost or safety analysis, and an architecture trade-off note.

Week loop (from the cookiecutter journal)

Mon — Theory: DistServe, Splitwise, Mooncake, autoscaling; Kubernetes topology (KServe + KEDA + HPA) and edge deployment (ONNX, TensorRT-LLM, WebLLM) as alternative topologies
Tue — Build: start the core deliverable above
Wed — Systems / papers: one paper or system note (see below if this week has assigned reading)
Thu — Eval / interview: write the four evidence items
Fri — Algorithmic Forge: Priority scheduling, interval allocation, and binary search on capacity backed by a benchmark. (2-3 hr; Boss fight)

Forge theme: Week 93 - Inference boss

Leetcode Darbar (parallel)

This week's band (82-102): Heaps, priority queues, graphs, trees, range queries. Production connection: Inference schedulers, quotas, queues, tenant fairness.

Standard: one aligned problem. Auror: the five-slot queue on the Darbar page. Tracker stays in the learner journal.

Topics: graphs · heap · queue · range query · trees

All Paths From Source To Target · medium · graphs
The K Weakest Rows In A Matrix · easy · heap
Implement Stack Using Queues · easy · queue
Queue Reconstruction By Height · medium · range query
Binary Tree Paths · easy · trees

Evidence contract

Every core week records all four (cookiecutter journal/weeks/week_093.md):

Code / implementation (or the design artifact on a writing week)
Benchmark / result
Design doc / technical explanation
Retrospective / learning note (Sailboat: destination, wind, anchor, rocks, heading)

On this site

Agent-workload cost model · prerequisite
684 · distributed training

Official Tensor-to-Tenant

Week 93 of Tensor-to-Tenant · Previous week · Next week.