20 Context Management & Reliability Practice Questions & Answers

1. 1. You architect a multi-agent research system. Each research subagent forwards its full raw source text — roughly 138,000 tokens in total — to a single synthesis agent. The synthesis agent's reports are accurate about the first and last sources but garble the ones in between. What is the correct fix?

   • A.Split the 138,000 tokens across three synthesis agents and concatenate their three reports
   • B.Have each subagent emit a compact synthesis narrative plus a structured citation index, and forward only those to the synthesis agent✓ Answer
   • C.Raise the synthesis agent's output token limit so it has room to address every source it was given
   • D.Add a reranking layer in front of the synthesis agent so the most relevant of the 138,000 tokens appear first

   The root cause is that raw material was never trimmed before the handoff: the context window is a finite budget shared by instructions, history, tool results and the response itself, and attention degrades over material buried in the middle of a very long context. Compressing each subagent's output to a narrative plus a structured index fixes the design; a reranker only reorders content that should never have been forwarded in full, and a bigger output budget does nothing about an overloaded input.

   Source: Anthropic context-window guidance: the window is a shared finite budget; trim raw content and intermediate reasoning before downstream handoff (Claude Certified Architect – Foundations, Domain 5: Context Management & Reliability)Report a problem with this question

2. 2. A customer-support resolution agent is 60 turns into a single session. The case facts — order number, entitlement tier, and the applicable refund limit — were established at turn 12 and now sit roughly in the middle of a very long transcript. The agent has begun contradicting them. What is the right structural change?

   • A.Resend the entire verbatim transcript on every request so that nothing can be lost
   • B.Insert a restatement of the facts as a note roughly halfway through the transcript, near where they were first established
   • C.Instruct the model to reason harder about the middle of the conversation before answering
   • D.Maintain a pinned fact block at the very top of the context, re-rendered on every turn, and summarize the resolved turns below it✓ Answer

   Material buried in the middle of a long context receives the least reliable attention, so the remedy is positional: load-bearing facts belong at the top of the context where they are re-read on every turn, with resolved history compressed beneath them. Re-inserting the facts mid-transcript places them back in the weakest position, resending everything verbatim makes the window problem worse, and no instruction to 'reason harder' overcomes a placement problem.

   Source: Anthropic long-session context guidance: pin case facts at the top of long sessions and summarize resolved turns ('lost in the middle' mitigation)Report a problem with this question

3. 3. In a support agent session, a shipping-quote tool is called once per turn as the customer changes the destination. Every superseded quote remains in the conversation history; context is growing and the agent has twice cited an outdated quote. Which remedy fits this symptom?

   • A.Clear the superseded tool results out of the conversation, keeping only the current quote✓ Answer
   • B.Summarize the older quotes into a single paragraph so their substance survives in compressed form
   • C.Write every quote to the agent's cross-session memory store so nothing is lost between sessions
   • D.Reduce the reasoning effort so the agent reads less of the history before answering

   Summarizing preserves substance in compressed form; pruning removes material outright; persisting stores knowledge beyond the session. Superseded tool results have no substance worth preserving — they are simply wrong now — so pruning is the matching remedy, and summarizing them would keep a compressed version of information that can still be miscited. Cross-session memory addresses a different problem entirely.

   Source: Anthropic context management: context editing clears stale tool results within a session, distinct from compaction (summarize) and memory (persist across sessions)Report a problem with this question

4. 4. A long-running agent compresses its history every 20 turns by summarizing the previous summary together with the new turns. After several rounds it starts quoting a reimbursement amount and an order number that do not match the source records. What is the correct fix?

   • A.Run summarization on a cheaper model at a shorter interval to keep each compression step smaller
   • B.Summarize more aggressively so that fewer competing details remain in context
   • C.Exempt a structured fact block — identifiers, amounts, dates, entitlements — from summarization and carry it forward verbatim✓ Answer
   • D.Instruct the summarizer to restate the amounts and identifiers in flowing prose so they read naturally

   Summarization is lossy compression, and recursive summarization compounds that loss with every pass — exact-valued facts drift precisely because prose is being rewritten from prose. The mechanism-level fix is to exclude exact facts from the lossy path and carry them forward unchanged; folding them into prose, summarizing harder, or summarizing more often all keep them inside the lossy path.

   Source: Anthropic long-conversation guidance: structured fact blocks (IDs, amounts, dates, entitlements) must remain uncompressed and never be folded into summarized proseReport a problem with this question

5. 5. A long-running coding agent shows two symptoms: within a single run its context fills with old tool output until quality drops, and across runs it rediscovers the same repository conventions from scratch every time. Which architecture addresses both?

   • A.Pair a within-session mechanism — clearing stale tool results and/or summarizing earlier history — with a durable store that lives outside the conversation✓ Answer
   • B.A larger context window, which resolves both symptoms at once
   • C.Summarizing earlier history alone, since the resulting summaries travel with the conversation into later runs
   • D.Persist the full verbatim transcript to disk and replay it at the start of every run

   The two symptoms are different problems: clearing and summarizing operate within a session and do nothing once that session ends, while a durable store outside the conversation is what survives across sessions. This is why long-running agents commonly need more than one remedy. A larger window only delays the within-run bloat, and replaying a full transcript recreates the bloat it was meant to avoid.

   Source: Anthropic agent design: context editing and compaction operate within a session; memory is for cross-session persistence — long-running agents commonly use bothReport a problem with this question

6. 6. An agent is given a durable memory directory and told to record what it learns for future sessions. Reviewing what it wrote, which entry does NOT belong there?

   • A."This package prefers table-driven tests; reviewers reject one-assertion-per-function test files."
   • B."The last migration attempt failed because the index was rebuilt before the backfill completed."
   • C."Staging deploy token: <value> — reuse this instead of asking for it again next session."✓ Answer
   • D."The build script must be run from the repository root, or it fails with a misleading path error."

   Memory files are replayed into the context of every later session that mounts them, so a credential written there is durably persisted and re-exposed in contexts you never audit — secrets belong in a credential store, never in memory or a system prompt. The other three entries are exactly what memory is for: hard-won operational lessons that a future session would otherwise have to rediscover.

   Source: Anthropic memory-tool guidance: never store API keys, passwords, tokens, or secrets in memory files — they persist into every later sessionReport a problem with this question

7. 7. A production service sends the same large system prompt on every request. Usage reporting shows cache-creation tokens on every single call and cache-read tokens that never rise above zero. Which of the following explains it?

   • A.The cache breakpoint sits at the end of the system prompt rather than on the tool definitions
   • B.Requests are issued from more than one worker process rather than a single one
   • C.The system prompt's header line interpolates the current timestamp, so no two requests share a byte-identical prefix✓ Answer
   • D.The user's varying question is placed after the cached portion rather than before it

   The cache is matched as an exact prefix, so a single changing byte anywhere in that prefix invalidates everything after it — an interpolated timestamp at the top of the system prompt makes every request's prefix unique, which is why writes occur but reads never do. Placing the varying question after the cached portion is correct practice, not a fault, and neither breakpoint placement on the system prompt nor multiple processes prevents a shared prefix from matching.

   Source: Anthropic prompt caching: caching is an exact prefix match — any byte change anywhere in the prefix invalidates everything after it; verify with cache-read vs cache-creation token usageReport a problem with this question

8. 8. An analysis service answers 200 different questions against the same large reference corpus. Each request is assembled with the question first, then the corpus, with a cache breakpoint at the end of the request. The cache never produces a read. What should change?

   • A.Place the stable corpus first and the varying question last, with the breakpoint at the end of the shared corpus✓ Answer
   • B.Cache the question and leave the large corpus uncached, since the corpus is too large to cache economically
   • C.Extend the cache entry's lifetime so entries survive long enough to be reused
   • D.Issue all 200 requests concurrently so that they share a single cache entry

   Because matching is by prefix, everything after the first differing byte is uncacheable — putting the question first makes each request diverge immediately, so the corpus behind it can never be reused. Stable content must physically precede volatile content, with the breakpoint at the end of the shared span. A longer lifetime cannot help an entry that is never matched, and firing identical requests concurrently makes them all miss, since an entry only becomes readable after the first response begins.

   Source: Anthropic prompt caching placement: shared prefix first, varying suffix after the last breakpoint; concurrent identical requests all miss until the first response beginsReport a problem with this question

9. 9. An architect is deciding whether to cache a large stable prefix on an endpoint that is invoked once per customer per month. The gap between two calls with the same prefix is far longer than a cache entry survives. What should they conclude?

   • A.Cache it using the longest available entry lifetime, since longer-lived entries are cheaper to write
   • B.Cache it — the write premium is refunded automatically when an entry expires without being read
   • C.Cache it — cache writes and cache reads are billed identically, so there is no downside to enabling it
   • D.Do not cache it — a cache write costs more than processing the same tokens uncached, and with no read before the entry expires that premium is never recovered✓ Answer

   Caching is an economic bet: writing an entry costs a premium over ordinary input processing, while reading one costs a small fraction of it, so the write only pays for itself once enough subsequent reads land before the entry expires. With a single call per prefix and no reuse inside the entry's lifetime, every request pays the premium and never collects the discount. Longer-lived entries carry a higher write premium, not a lower one, so they require even more reuse to break even.

   Source: Anthropic prompt caching economics: cache writes are billed above base input rate and cache reads at a small fraction; a longer entry lifetime raises the write premium and requires more reuse to break evenReport a problem with this question

10. 10. A multi-tenant agent platform builds each request's tool list from that tenant's enabled integrations, serializing it from a hash map with no fixed key order. The system prompt is byte-identical for all tenants, yet cache reads are near zero across the whole fleet. Why?

    • A.Tool definitions render ahead of the system prompt, so a per-tenant or non-deterministically ordered tool list changes the very front of the prefix and invalidates everything after it✓ Answer
    • B.The cache breakpoint should be on the system prompt rather than anywhere in the tool definitions
    • C.Tool definitions are structurally excluded from caching, so only the system prompt can ever be reused
    • D.Cache entries are scoped per tenant by design, so cross-tenant reuse is impossible regardless of content

    Tool definitions occupy the very front of the rendered prompt, ahead of the system prompt and messages, so any per-tenant variation — or merely non-deterministic key ordering that changes the serialized bytes — moves the first differing byte to position zero and makes the entire remainder uncacheable. The fixes follow directly: serialize deterministically and keep the tool list stable rather than assembling it per tenant.

    Source: Anthropic prompt caching: render order is tools → system → messages; per-user tool sets and non-deterministic serialization are silent cache invalidatorsReport a problem with this question

11. 11. A structured-extraction team must process 50,000 archived documents overnight. No user and no downstream step waits on any individual document; the whole set is consumed the next morning. Which execution choice is correct, and why?

    • A.Restructure each document into an interactive multi-turn tool loop so each extraction can self-correct as it runs
    • B.Send them synchronously, because the asynchronous batch path cannot reuse a cached prompt prefix
    • C.Send them synchronously at high concurrency, because batch results are returned in submission order and are therefore easier to reconcile
    • D.Submit them through the asynchronous batch path and key each result by its own request identifier, since nothing is blocked on any individual response✓ Answer

    The batch path trades away any latency guarantee in exchange for a substantially lower rate, so it is the right choice exactly when nothing is blocked on the response — and the synchronous path is right when a user or downstream step is waiting. Two distractors rest on false premises: batch results are not ordered by submission and must be keyed by request identifier, and batch does support prompt caching. Wrapping offline extraction in interactive tool loops adds cost and steps without any latency benefit.

    Source: Anthropic Message Batches: asynchronous processing at a reduced rate with no latency SLA; results return in any order and must be keyed by the per-request custom identifierReport a problem with this question

12. 12. Midway through a long agent session with a large cached prefix, the orchestrator needs a bulk classification subtask that a cheaper model could handle. It switches the model for those calls and switches back. Throughput collapses. What is the correct design?

    • A.Reorder the conversation so that all cheap turns are grouped before the expensive ones
    • B.Keep switching, but move the classification instructions into the system prompt so the subtask runs on the already-cached prefix
    • C.Keep the main loop on a single model and delegate the subtask to a subagent that runs the cheaper model with its own separate context✓ Answer
    • D.Keep switching; the original cached prefix is restored automatically as soon as the main model resumes

    Cache entries are scoped to the model that created them, so switching models mid-session cannot reuse the existing entry and switching back does not restore it — each swap forces a full reprocess of the prefix, which is what collapsed throughput. Delegating to a subagent keeps the main loop's prefix intact on one model while the cheap work runs in its own context. Moving instructions into the system prompt would also edit the front of the prefix and invalidate it.

    Source: Anthropic agent-design caching guidance: caches are model-scoped — a model switch invalidates the entry; spawn a subagent on the cheaper model instead of switching the main loopReport a problem with this question

13. 13. A structured-extraction pipeline reports 97% field-level accuracy on its evaluation set, and the team is ready to remove human review. What should be done before that decision?

    • A.Enlarge the evaluation sample until the overall accuracy figure stabilizes above 98%
    • B.Segment accuracy by document type and by individual field before deciding what to automate, because an aggregate can conceal a subset that fails badly✓ Answer
    • C.Have the model emit a self-reported confidence score per extraction and automate every field above a chosen threshold
    • D.Nothing further — 97% already exceeds the measured human error rate on the same task

    An aggregate accuracy figure is a weighted average, so a minority segment that fails badly — one document type, one field — can be hidden entirely behind a strong overall number, and automating on that number ships a silent production failure. Stratifying by segment is what makes the failure visible. A larger sample moves the same average without exposing the split, and a model's self-reported confidence is not a reliable gate for automation.

    Source: Anthropic evaluation guidance: segment accuracy by document type and by field — aggregate metrics hide catastrophic minority-subset failuresReport a problem with this question

14. 14. A team judges every prompt change by having two engineers skim a dozen outputs and agree it "feels better". Regressions keep reaching production. What is the correct evaluation practice?

    • A.Raise the spot check to fifty outputs per change so the sample is larger
    • B.Build a fixed test set drawn from real production failures, define explicit pass criteria, and measure every change against a recorded baseline on that same set✓ Answer
    • C.Ship each change to a small slice of live traffic and keep whichever version generates fewer complaints
    • D.Have a second model rate each output on a 1-to-10 overall quality scale and track the average

    Anecdotal spot checks cannot detect a regression, because the sample changes with every review and there is no recorded baseline to compare against — so an improvement on one behavior can silently break another. A fixed set built from real observed failures, graded against explicit criteria and compared to a stored baseline, is what makes a change measurable. A larger ad-hoc sample and an unanchored 1-to-10 quality score are still impressions, and live traffic surfaces regressions only after users absorb them.

    Source: Anthropic evaluation guidance: build a representative test set from real failures and measure changes against a recorded baseline rather than by impressionReport a problem with this question

15. 15. An architect must choose graders for three evaluations: (a) does the extraction return the invoice total as the correct number, (b) does the generated CSV contain a numeric price column in every row, (c) is a drafted support reply both factually correct and appropriately empathetic. What is the right assignment?

    • A.Use a model-based judge for all three, since a single grading approach generalizes across tasks
    • B.Use exact string match for all three, since only exact match gives reproducible results
    • C.Use exact match for (a), a programmatic structural check for (b), and a rubric-driven model-based judge for (c)✓ Answer
    • D.Use human review for all three, since automated graders cannot be trusted on production data

    The grader must match the shape of the claim being tested: a single correct value is checked by exact match, a structural property of an artifact is checked programmatically and deterministically, and a judgement with no single right answer needs an explicit rubric applied by a model-based judge. Using one grader everywhere either wastes cost and adds noise on checks a program can settle exactly, or forces an open-ended judgement into a comparison it cannot express.

    Source: Anthropic evaluation guidance: define graders appropriate to the task — exact match, structural validity, rubric, or a model-based judge; evaluate with explicit gradeable criteriaReport a problem with this question

16. 16. A team assembled 60 evaluation examples, iterated on the prompt against those 60 until accuracy reached 95%, and now reports 95% as the expected production accuracy. What is wrong with that number?

    • A.The figure is optimistic because the same examples were used both to tune and to measure; a held-out set that is never used for tuning is required✓ Answer
    • B.Nothing is wrong; 60 examples is a statistically adequate sample for a production estimate
    • C.The sample is too small — expand it to 600 examples and re-tune the prompt against all of them
    • D.The metric is wrong — report per-field accuracy on the same 60 examples instead of a single figure

    Once a prompt has been tuned against a set, performance on that set measures how well the prompt was fitted to those specific examples, not how it will generalize — the number is biased upward by construction. The remedy is a held-out set that is never used for tuning. Enlarging the set and re-tuning on all of it reproduces exactly the same contamination, and changing which statistic is reported does not remove it.

    Source: Anthropic evaluation guidance: measuring on the same examples used for tuning inflates results — hold out an evaluation set that is never used to tuneReport a problem with this question

17. 17. You are defining observability for a production agent so that reliability problems are caught before users report them. What should be instrumented?

    • A.End-to-end latency and a total error count only, since every other signal can be derived from those two
    • B.Per-request token consumption including cached versus uncached input, latency, error and refusal rates, per-tool failure rates, and the number of steps per task✓ Answer
    • C.The full text of every prompt and response, retained indefinitely, so that any incident can be replayed exactly
    • D.A daily human review of a random sample of transcripts, used in place of automated metrics

    Each of those signals detects a distinct failure the others miss: token consumption and the cached-versus-uncached split expose context bloat and cache regressions, per-tool failure rates isolate a single broken dependency, refusal rate separates declined requests from errors, and step counts catch runaway loops. Latency and a bare error count show that something is wrong without localizing it, indefinite full-text retention creates a data-protection liability rather than a metric, and sampled human review is far too sparse to catch a regression early.

    Source: Anthropic production reliability guidance: instrument token consumption (cached vs uncached), latency, error and refusal rates, tool failure rates, and loop/step lengthsReport a problem with this question

18. 18. Within one hour a production agent hits three distinct failures: (a) a response that was cut short by the output limit, (b) a request the model declined, and (c) a temporary service overload. How should each be handled?

    • A.Retry (c) with exponential backoff; for (a) raise or restructure the output budget (or stream) because an identical retry reproduces the truncation; for (b) surface the decline rather than resending the same request✓ Answer
    • B.Treat all three as permanent and surface each of them to the user without retrying
    • C.Retry (a) and (b) with backoff; treat (c) as permanent and surface it to the user
    • D.Retry all three with exponential backoff, since all three are transient conditions

    A retry is only useful when the same request could plausibly succeed later, which is true of a transient overload and false of the other two. Truncation is a deterministic consequence of the request's own output budget, so an identical retry hits the same wall while doubling the cost, and a decline is a decision about the request's content that resending unchanged will reproduce. Handling every terminating condition distinctly — rather than routing all of them into one retry loop — is the reliability rule here.

    Source: Anthropic stop-reason and error handling: overload/transient errors are retryable with backoff; output-limit truncation and refusals require a different response, not a plain retryReport a problem with this question

19. 19. An agent's tool call fails schema validation. The harness catches every exception in one broad handler and retries three times with exponential backoff; each attempt fails identically, and cost per task has tripled. What is the correct fix?

    • A.Suppress the error and return an empty result so the agent can continue without interruption
    • B.Return the specific validation detail to the agent as a structured error carrying an error category and a retryable flag, so it can correct the call, and reserve the retry loop for transient failures✓ Answer
    • C.Hand the failing call to a second agent that reissues it with the same arguments
    • D.Increase the retry count and lengthen the backoff so the validating service has more time to recover

    A validation error is permanent for an unchanged input: nothing about waiting makes a malformed call valid, so retrying burns model cycles on a call that cannot succeed. The root cause is the single broad handler, which erases the distinction between retryable and non-retryable failures; a structured error carrying a category, a retryable flag, and a human-readable message lets the agent self-correct instead. Returning an empty result substitutes silent failure for a loud one, and a second agent reissuing identical arguments repeats the same permanent error.

    Source: Anthropic error-handling guidance: catch typed exceptions most-specific-first and preserve the retryable/non-retryable distinction; return validation errors to the agent with explicit detail instead of retryingReport a problem with this question

20. 20. A long-running extraction pipeline crashed roughly 70% of the way through. Its conversation history contains stale tool results and a half-finished plan. How should the work be resumed?

    • A.Extract a structured checkpoint of completed, partial, and pending items, start a fresh session, and place that checkpoint at the top of the prompt✓ Answer
    • B.Discard everything and restart from zero, since partial state after a crash cannot be trusted
    • C.Resume the crashed session and append a message at the end listing what still needs to be done
    • D.Resume the crashed session exactly as it stands and let the model work out from the history what was already completed

    Resuming a crashed session carries the stale tool results and the abandoned plan forward as if they were current, so the model reasons over state that no longer reflects reality. Extracting an explicit checkpoint that marks each item complete, partial, or pending converts unreliable history into reliable state, and placing it at the top of a fresh prompt applies the same positional rule that governs any load-bearing fact. Appending the remaining work to the end still leaves the stale history in context, and restarting from zero discards 70% of paid-for work that the checkpoint can preserve.

    Source: Anthropic session-recovery guidance: extract a structured checkpoint (complete / partial / pending), start a fresh session, and inject the checkpoint at the top of the prompt rather than resuming with stale tool resultsReport a problem with this question

Practice questions based on the official Claude Certified Architect – Foundations (CCAR-F) exam guide and Anthropic's public documentation. This is an independent study tool, not affiliated with or endorsed by Anthropic, and does not grant certification. The real exam is 60 questions, 120 minutes, passing at a scaled 720/1000, delivered via Pearson VUE ($125). Official certification page →