Capstone · Full-Length Mixed Practice Exam

Final Practice Exam

16 fresh scenario-framed questions, interleaved across all 5 domains — the closest simulation of the real format. New items, not repeats. Click to grade + explain.

Target: 14/16 (≈88%) before sitting the real exam. Real pass = 720/1000; practice-exam goal per Anthropic = 900+.

Scenario A — Customer Support Resolution Agent

A1. The agent must never issue a refund before confirming the customer's identity, yet logs show occasional refunds on unverified accounts. The change that guarantees the fix:

Strengthen the system prompt to mandate verification before refunds
Add few-shot examples always verifying identity before any refund
A prerequisite gate blocking refund tools until verification returns an ID
A classifier that enables refund tools only for verified-looking requests

Option 3. Financial guarantee → deterministic gate. Prompt/few-shot are probabilistic; the classifier changes availability, not ordering. (D1)

A2. A customer opens with "Just get me a human." The correct behavior is to:

Escalate immediately without first attempting an investigation
Investigate the issue and escalate only if it can't be resolved
Ask why they want a human before deciding whether to escalate
Attempt resolution and escalate if sentiment stays negative

Option 1. Explicit human request = escalate now. Sentiment is not a trigger. (D5)

A3. Over a long multi-issue chat, the agent starts mixing up which refund amount belongs to which order. Best mitigation:

Summarize the conversation more tightly to conserve context
Persist per-issue structured facts (order ID, amount, status) in a separate layer
Increase max_tokens so the entire history is always retained
Place the running transcript in the middle of each request

Option 2. Structured per-issue fact layer beats lossy summarization; middle position is where content drops. (D5)

A4. get_customer returns three people matching the stated name. The agent should:

Pick the most recently active account as the best-guess match
Proceed with all three and merge their order histories together
Ask the customer for an additional identifier to disambiguate
Escalate to a human because the lookup returned ambiguous data

Option 3. Multiple matches → request more identifiers, don't guess heuristically (and it's not escalation-worthy yet). (D5)

Scenario B — Multi-Agent Research System

B1. Reports on "renewable energy adoption" omit wind and hydro entirely. Every subagent succeeded; the coordinator log shows subtopics "solar panels," "solar subsidies," "rooftop solar." Root cause:

The synthesis agent failed to detect coverage gaps in its inputs
The web search agent used queries that were too narrowly worded
The coordinator decomposed the topic too narrowly when assigning work
The document analysis agent over-filtered non-solar sources out

Option 3. Subagents worked within given scope; upstream decomposition is the defect. (D1)

B2. The synthesis agent needs simple fact-checks on ~80% of tasks; routing each through the coordinator adds heavy latency. Best fix:

Grant the synthesis agent the full web-search toolset directly
Give it a scoped verify_fact tool; route complex checks via the coordinator
Batch all verifications to the end and send them together once
Pre-cache extra context on every source during initial search

Option 2. Least privilege: scoped tool for the common case, coordinator for the hard 20%. (D2)

B3. Two credible papers report different adoption percentages for the same year. The synthesis output should:

Report the figure from the higher-impact-factor journal only
Average the two percentages into one reconciled figure
Present both, annotated with source attribution (and check dates)
Exclude the metric entirely because the sources conflict

Option 3. Annotate conflict with attribution; a date difference may explain it. Don't pick/average/drop. (D5)

B4. The document-analysis subagent's host service times out mid-task. To let the coordinator recover intelligently, the subagent should return:

An empty analysis marked successful so synthesis can proceed
A generic "analysis unavailable" status after silent retries
Structured context: failure type, attempted work, partials, alternatives
A thrown exception that aborts the entire research pipeline

Option 3. Structured error context enables retry/reroute/partial-proceed. (D5)

Scenario C — Code Generation & CI/CD with Claude Code

C1. React components, API handlers, and DB models each need distinct conventions; test files live beside their code everywhere and must share one testing convention. Most maintainable:

.claude/rules/ files with glob paths per code type
One root CLAUDE.md with headed sections Claude infers between
A CLAUDE.md dropped into every directory containing tests
A skill per code type that developers invoke before editing

Option 1. Glob rules apply by file type across scattered locations automatically. (D3)

C2. A nightly job posts review findings as inline PR comments and must be machine-parseable and never hang. Correct CLI usage:

Run interactively and redirect stdin from /dev/null to avoid hangs
-p with --output-format json and --json-schema
Set CLAUDE_HEADLESS=true and parse the plain-text output
Use --batch so it runs unattended and returns JSON by default

Option 2. -p prevents hangs; json + schema gives parseable findings. CLAUDE_HEADLESS/--batch don't exist. (D3)

C3. Re-running the review after new commits floods the PR with duplicate comments. Fix:

Lower the model temperature so it reports fewer findings overall
Switch the review to the Batches API to deduplicate on custom_id
Include prior findings in context; report only new/unaddressed issues
Require developers to resolve all comments before any re-run

Option 3. Feed prior findings and instruct report-only-new. Temperature/batch/process-gates don't address duplication. (D3)

C4. You must migrate a library across 45+ files with two viable integration approaches differing in infra. Approach:

Direct execution, fixing files one by one as issues surface
Plan mode to explore and choose an approach, then direct execution to implement
Direct execution with exhaustive upfront per-file instructions
Batch API run to change all files at once overnight for savings

Option 2. Architectural, multi-approach, many files → plan first, then execute. (D3)

Scenario D — Structured Data Extraction

D1. Invoices pass strict JSON-schema validation, but line items sometimes don't sum to the stated total. Best safeguard:

Mark every numeric field required so totals are always present
Force tool_choice to the extractor so it always runs first
Extract calculated_total alongside stated_total and flag discrepancies
Switch models to one with a larger context window for accuracy

Option 3. Sum mismatch is semantic; schemas don't catch it. Add a calculated-vs-stated validation flow. (D4)

D2. Some source docs omit the "vendor tax ID." The model fabricates one to fill the required field. Fix:

Add a prompt instruction sternly forbidding any fabrication
Make the field optional/nullable so it can return null when absent
Retry the extraction until a plausible tax ID is produced
Force the extraction tool so the field is always populated

Option 2. Nullable lets the model report absence rather than fabricate. (D4)

D3. You must process 5,000 archival PDFs into structured records overnight, no interactive tool calls needed, cost matters. Best API choice:

Synchronous API to guarantee predictable per-document latency
Message Batches API, correlating results with custom_id
Synchronous API with a timeout fallback to batch when slow
Batches API but block the pipeline until every result returns

Option 2. Non-blocking, latency-tolerant, cost-sensitive, no multi-turn tools = textbook batch fit. (D4)

D4. Your extractor reports 96% overall accuracy. Before cutting human review, you should first:

Reduce review immediately since 96% exceeds the target bar
Raise the confidence threshold uniformly across all doc types
Segment accuracy by document type and field to expose weak spots
Trust the model's self-reported per-field confidence to route review

Option 3. Aggregate accuracy can mask a poor segment; segment first. Calibrated (not self-reported) confidence routes review afterward. (D5)

Score interpretation 15–16: exam-ready across the board. 12–14: solid; review the domains behind your misses. ≤11: re-run the domain quizzes for the weak domains before booking the exam — remember you get one attempt.

Exam-day strategy

Read the stem's verb. "Guarantee / must / reliably / never" → deterministic enforcement. "First step / most effective / proportionate" → the cheapest fix that addresses the root cause (usually descriptions/criteria/few-shot).
Find the root cause before the fix. Many questions bait you into fixing a downstream symptom. Ask: which layer is actually broken?
Eliminate the archetype distractors (see cheat-sheet): train-a-classifier, routing layer, sentiment/self-confidence, bigger-window-fixes-attention, blame-downstream, fabricated features (--batch, CLAUDE_HEADLESS).
Two options feel right? One usually over-engineers or is disproportionate to "first step." Pick the lower-effort root-cause fix unless a guarantee is demanded.
No penalty for guessing — never leave a question blank. Eliminate, then choose.
4 scenarios shown of 6. All are covered here; expect several questions per scenario.

Tell your teacher your score and which numbers you missed. I'll log it to a learning record, generate extra targeted drills for weak domains, and (optionally) set a recurring study reminder before your exam date.