Chapter 4: State: Feeding the Model the Right Context

The Three Shapes of State

In a System One request, state is the material the model looks at, and questions are the judgments you want made about it. The split is deliberate — like separating a routing table (data) from a route-map (policy applied to it). You do not stuff policy clauses into the prefix list.

Figure 4.1: The Three Shapes of State

String State

The simplest state is a plain string — reserved for genuinely single-piece inputs. The moment you catch yourself concatenating a ticket summary, a hostname, and counters into one newline-separated blob, you have outgrown the string shape: you have built an object with the field names deleted, and no question can point at any one part of it.

state = (
    "<189>Sep 17 14:02:11 dc1-acc-07 %LINEPROTO-5-UPDOWN: "
    "Line protocol on Interface GigabitEthernet1/0/24, changed state to down"
)

response = client.system_one(
    state=state,
    questions={"subsystem": Choice(instructions="Which network subsystem?", criteria={...})},
)

Object State

Object is the documented default: "Use an object for most requests so each part of the state has a descriptive name and its relationships remain clear." The field name is itself context — admin_down_by_change_ticket tells the model more than a bare true. Named fields are addressable, which is what makes path references possible. Grouping matters too: keep an incident and the telemetry that proves or disproves it in one object, because the judgment depends on their relationship.

{
  "incident": {"number": "INC0042771", "short_description": "Uplink flapping on dc1-core-a"},
  "device": {"hostname": "dc1-core-a", "platform": "Arista EOS", "role": "spine"},
  "interfaces": {"Ethernet1": {"lineProtocolStatus": "down", "linkStatusChanges": 214}}
}

Array State

Arrays fit "a sequence of messages or records" where items are the same kind of thing and order carries meaning — a syslog burst is the canonical case, since ordering often is the diagnosis. The constraint hiding in the spec: array state is an array of text values. If your records are themselves structured, put the array inside an object field instead — which also lets you label and pair it with related data.

[
  "13:50:02 dc1-core-a: Interface Ethernet1 changed state to down",
  "13:50:03 dc1-core-a: BGP peer 10.0.2.3 Down - interface flap"
]

Text-Only: No Images, Audio, or Video

"State must be a string, JSON object, or array of text values. Images, audio, and video are not supported (yet)." You cannot hand the model a NetFlow screenshot, a topology diagram, or a raw .pcap. The workaround: render it to text before it becomes state. A packet capture becomes tshark field output; a topology becomes an adjacency list {"dc1-core-a": ["dc1-leaf-01", "dc1-leaf-02"]}, which is more useful than the picture since it is already the data the picture was drawn from.

Key Takeaway: State comes in exactly three shapes — string, JSON object, and array of text values — and objects are the recommended default because descriptive field names preserve both meaning and addressability. Use a string only for a genuinely single passage, an array for ordered same-kind records, and convert any non-text artifact into text before it can become state.

Building State from Network Sources

Arista eAPI and Cisco NX-API JSON Output

Arista's eAPI exposes EOS commands over JSON-RPC 2.0: POST to /command-api, authenticate with HTTP basic auth, send a list of commands in one transaction.

{
    "jsonrpc": "2.0",
    "method": "runCmds",
    "params": {"version": 1, "cmds": ["show interfaces", "show vlan"], "format": "json"},
    "id": 1
}

The response is an array with one structured JSON element per command — interface names as keys, nested properties for status, duplex, and error counters. NX-API on Nexus platforms does the same job with a different wrapper: method is cli, cli_array, or cli_ascii, the command goes in params.cmd, and auth issues an nxapi_auth session cookie valid ten minutes. Responses arrive inside an ins_api envelope whose outputs entries hold a TABLE_interface with ROW_interface records. Arista hands you the data directly; Cisco wraps it. Neither wrapper belongs in your state — both are transport artifacts. Strip to the payload, then strip again to the fields the decision needs.

Junos and Aruba CLI Output: Raw Text or Parse First?

Juniper devices are commonly automated through NETCONF (with YANG models), typically via PyEZ — valuable in multi-vendor shops because it is one model to code against instead of a parser per vendor. But NETCONF/PyEZ return XML-derived structures, and Aruba AOS-CX consoles or show tech captures often give nothing but text.

That last row matters: admin down, notconnect, and errdisabled mean three different things. Collapsing all three to "down" destroys signal before the model sees it. Keep the vendor's own string as the field's value; let the field name supply the structure.

Combining a ServiceNow Incident with Device Telemetry

The running example: an incident lands in ServiceNow, a collector pulls live Arista interface state, and both assemble into one object state. Selection is the step that matters most — out of dozens of eAPI keys per interface, six inform a triage decision:

def summarize_interfaces(eapi_result, names):
    summary = {}
    for name in names:
        raw = eapi_result.get("interfaces", {}).get(name)
        counters = raw.get("interfaceCounters", {})
        summary[name] = {
            "description": raw.get("description"),
            "lineProtocolStatus": raw.get("lineProtocolStatus"),
            "linkStatusChanges": counters.get("linkStatusChanges"),
            "crcErrors": counters.get("inputErrorsDetail", {}).get("crcErrors"),
        }
    return summary

The incident contributes number, short_description, and assignment_group; the assembled object adds device identity, the filtered interfaces, and a recent syslog slice. The same pattern extends to Splunk rows (keep timestamp, host, message; discard search metadata) and NETCONF/PyEZ leaves (pull the two or three the question needs). The collector's job is to hand a small dictionary to the assembly function, which names each part and puts them side by side.

Figure 4.2: Data Source to State Pipeline

Key Takeaway: Arista eAPI and Cisco NX-API both return show output as structured JSON over JSON-RPC 2.0, differing mainly in response wrapping — Arista's direct JSON versus Cisco's ins_api envelope with TABLE/ROW entries. Strip the transport wrapper, select only the fields a decision needs, keep vendor CLI wording raw when its exact phrasing carries meaning, and assemble everything into one named object.

Referencing State in Questions

Dot-and-Bracket Notation

Once state is a JSON object, question instructions can point at specific fields using dot notation for named fields and bracket indexing for array positions and keys. A path such as ticket.messages[0].text or interfaces["Ethernet1"].lineProtocolStatus clarifies exactly which components should inform a judgment. If you have written a JSONPath in an Ansible filter or an xpath in a NETCONF filter, the syntax looks familiar — the difference is that you are not extracting a value in code, you are telling the model which part of the state a judgment hinges on. The whole state is still present; the path is emphasis, not a filter.

questions={
    "owning_team": Choice(
        instructions="Using incident.short_description and the port states in interfaces, decide the owning team",
        criteria={"data_center": "...", "wan": "...", "wireless": "...", "unknown": "..."},
    ),
    "physical_layer_suspected": Noul(
        instructions='interfaces["Ethernet1"].lineProtocolStatus is down with rising crcErrors, '
                     "pointing at a cable or optic fault rather than a config change",
    ),
}

Note the Noul instruction is written as a statement, not a question, because a Noul returns the probability that the statement is true. One caveat: the model reads instructions literally and is not a calculator — compute thresholds in Python, put the result in state as a named field, and let the question judge what that result means.

Policy Alongside the Request It Governs

The same "keep related information together" guidance that pairs a refund request with its policy maps directly onto network operations. A change-freeze calendar, an escalation matrix, or an SLA tier is the network equivalent of the refund policy. If you want the model to judge whether an incident warrants paging on-call, the escalation policy must be in the state, not assumed. Your escalation matrix changed last quarter; the model's training data did not.

state = {
    "incident": {...},
    "interfaces": interfaces,
    "escalation_policy": (
        "Page on-call immediately for: any spine/core device with an active "
        "traffic-affecting fault; any site isolation. Queue for business hours: "
        "single access-port faults, APs affecting fewer than 20 clients."
    ),
    "change_freeze": {"active": True, "window": "2026-09-15 through 2026-09-19"},
}

Now a Noul instructed "the escalation_policy requires paging the on-call engineer for this incident" is answerable from evidence actually in the request. Without the policy field, the model must guess at your organization's rules — and a confident wrong guess is worse than no answer.

Every Question Sees the Same State Independently

Figure 4.3: Every Question Sees the Same State Independently

Three consequences follow. First, questions do not see each other's answers — if a decision depends on a prior decision, that is two requests: run the first, write its answer into state as a new field, then run the second. Second, paths scope attention, not visibility — writing a path to one field does not hide the rest of state from that question; every field is visible to every question, which is why filtering discipline is not optional. Third, state and questions share one budget — around 32,000 tokens total — so asking twelve questions instead of three is nearly free on the state side, since you pay for the large interface dump once, not per question.

Key Takeaway: Dot-and-bracket paths such as incident.short_description and interfaces["Ethernet1"].lineProtocolStatus let a question's instructions name exactly which part of state governs it. Put the policies that govern a decision in state alongside the request rather than trusting model weights, and remember that every question sees the whole state independently and shares one token budget with it.

Filtering and Token Budget

Context Rot: Irrelevant Fields Degrade Answers

Context rot is the phenomenon where model performance degrades progressively as input context grows, even on straightforward retrieval and reasoning tasks. A comprehensive evaluation of 18 leading frontier models found every single one degraded with longer contexts, with severity depending on where relevant information sits and whether topically related distractors are present. The "Lost in the Middle" finding: identical facts placed at different positions scored roughly 70-75% at the beginning, 55-60% in the middle, and 70-75% at the end — a 15-20 point swing from position alone, reproducing across six major model families.

For a network engineer, the mental model is a TCAM: attention is a finite budget diluted across non-essential content. Every token spent on mtu or duplex for a decision hinging on error counters is a token not spent on the error counters. The subtler and more dangerous finding: topically related distractors are worse than obviously irrelevant ones. Counters for 47 healthy ports next to the one flapping port are maximally confusable with the signal, because they are the same kind of data in the same format. TypeSafe's own guidance: "Include only the context relevant to the current questions. This helps the model avoid distractions and context rot."

Selecting Only the Fields a Decision Needs

Decomposing state means asking, field by field, whether the data actually informs the judgment. Applied to the triage decision:

Two rows generalize into rules: "precompute, don't ask" (derived values belong in Python; the result goes into state) and "same-shape noise is worst" (be most aggressive filtering data that looks like the signal). Positioning is the last lever — since accuracy is higher at the start and end of a context window, order decision-critical fields first and push long supporting material like escalation_policy lower.

The Roughly 32,000-Token Budget

State and questions draw from the same ~32,000-token pool (~150,000 characters of English text) — but JSON keys, braces, and quotes tokenize less efficiently than prose, so budget conservatively.

def rough_tokens(state_obj) -> int:
    text = state_obj if isinstance(state_obj, str) else json.dumps(state_obj)
    return len(text) // 4

BUDGET = 32_000
estimate = rough_tokens(state)
if estimate > BUDGET * 0.6:
    raise ValueError(f"State is {estimate} tokens; filter before sending.")

response = client.system_one(state=state, questions=questions)
print(response.usage.input_tokens)  # actual count, from the response

The 60% threshold is a working discipline, not a documented limit — it leaves room for questions and criteria strings. The filtered triage state in the running example lands in the low hundreds of tokens; the unfiltered 48-port dump would run to several thousand. The filtered version is cheaper, faster, and more accurate.

Figure 4.4: Before and After Filtering

Key Takeaway: Context rot is measurable degradation from long or noisy context, affecting every frontier model tested, with a documented 15-20 point accuracy swing based on where a fact sits in the window. Include only the fields your questions actually use, be most ruthless about same-shape distractors like healthy-peer counters, put decision-critical fields near the start, and keep the combined state and questions well under the roughly 32,000-token shared budget.

Key Terms