Week 14 — Grounding, Verification, Scaffolds and Multi-Agent Systems

Verification · Scaffolds · Multi-Agent

Advanced

V — Graph engineering

Overview

Graph answers must cite actual paths and distinguish direct evidence from inference; contradictions are retained and flagged, not silently overwritten. The scaffold matters: the same model under different prompts, tool schemas and interaction protocols scores differently. Multi-agent systems use a lead agent, persistent teammates, shared repo state, asynchronous messaging and a shared task board, with an RL objective over task, collaboration and critical-path latency.

What You Will Learn

Distinguish direct evidence from inference and retain contradictions.
Show that scaffold changes score for a fixed model.
Design a multi-agent objective with a critical-path latency term.
Justify parallelism by task structure, not cosmetics.

Core Concepts

The synthesis

\text{Agent Quality}=f(\text{Model},\text{Memory},\text{Retrieval},\text{Scaffold},\text{Environment},\text{Verification})

Multi-agent objective

task performance + collaboration + critical-path latency of the collaboration DAG.

Scaffold sensitivity

Same model, different harness → different score; the scaffold is part of the system.

Prerequisites

Work these pages on this site before the lecture.

Lecture notes

Grounding means: every claim cites a path or span that actually supports it. Direct evidence vs inference must be labeled; contradictions stay in the graph and are flagged, not overwritten.

The scaffold (prompt, tools, protocol) is part of the system. Same model, two harnesses, different score — that is the week-14 experiment.

Multi-agent is a DAG: lead + teammates + shared repo + async messages. The objective includes critical-path latency, not just final accuracy. Parallelism has to match the task graph.

Capstone: vector-only vs graph-assisted on lookup / multi-hop / insufficient-evidence, six metrics, confidence intervals, and a paragraph on where the graph lost.

Required readings

Lab / Implementation

Run the same task under two scaffolds and attribute the differences; add evidence-path citation and a contradiction flag; then run the capstone evaluation.

Not on this site (paper reading required)

grounding/verification discipline
scaffold evaluation
multi-agent design (graph source + DeepSeek)

Mastery Check

DERIVE: direct vs inferred evidence; the multi-agent objective
IMPLEMENT: evidence paths + contradiction flags; two scaffolds
BENCHMARK: vector-only vs graph on lookup/multi-hop/insufficient · 6 metrics + CIs
DEBUG: a cited path that does not support the claim
EXPLAIN: the six-factor quality function with a concrete example each

Part of CS/AI 684 — Efficient Long-Context Agent Systems.