Chapter 5: The Choice Primitive: Routing and Classification

When to Use Choice

Every NOC runs on classification: which team owns this incident, which vendor wrote this config, what kind of event just fired. The Choice primitive is the typed question for exactly that shape of decision — picking one label out of a fixed list. You supply the option set; the model never invents a new category, never returns two categories, and never returns prose. That constraint is what makes a Choice answer safe to wire directly into automation.

Unordered Options: Teams, Vendors, Categories

A Choice question has three required parts: type (always "choice"), instructions (the question, phrased about the state), and criteria (a map of option names to descriptions). The keys in that map are the labels the model may return; the values describe what each key means, or may be null when the label name alone is unambiguous. Crucially, the map contains no notion of sequence, rank, or distance — swapping two entries changes nothing about the question's meaning. Teams, vendors, and ticket categories are all unordered in exactly this way.

Choice Versus Score: Does the Order of Options Matter?

The clean test is a single question: if you reorder the options, does the meaning change? If reordering is meaningless, use Choice. If the options sit on a line from less to more, use Score.

Figure 5.1: Choosing between Choice and Score

The two primitives even look different in code, which is a useful mnemonic: Choice takes a map — Choice(instructions=..., criteria={...}) — because a map has no meaningful order. Score takes a list — Score(instructions=..., criteria=[...]) — because a list does. If you find yourself writing criteria={"low": ..., "medium": ..., "high": ...}, stop: that is an ordered scale wearing a Choice costume, and you are throwing away the model's ability to land between two rungs. The two primitives also compose well: one Choice can decide the primary intent while a Score asked in the same request assesses complexity or urgency.

Analogy: A VLAN Assignment Versus a QoS Priority Level

A VLAN ID is a label. VLAN 10 might be user data and VLAN 20 might be voice, but 20 is not "twice as much" as 10, there is no VLAN 15 sitting halfway between them in meaning, and renumbering them changes nothing. That is a Choice: discrete, unordered, mutually exclusive. A QoS priority level is the opposite — CoS 5 really is above CoS 3, and a value halfway between two class definitions is a coherent answer rather than a bug. That is a Score.

The mistake engineers make when they first pick up these primitives is treating severity as a Choice — "is this P1, P2, P3, or P4?" — because that is how the ticketing tool renders it. The tool is showing buckets on a scale. Ask for the scale with Score, then bucket it yourself with deterministic thresholds. Ask for the team with Choice, because "the team" is genuinely a set of labels.

Writing Effective Criteria

Plain-String Criteria for Simple Cases

The simplest form maps each label to a one-line description string. Start here — most classification problems in network operations are genuinely easy, and a plain-string criteria map is faster to write, faster to review, and cheaper in tokens.

from typesafe_sdk import TypeSafeClient, Choice

client = TypeSafeClient()  # reads TYPESAFE_API_KEY from the environment

event_kind = Choice(
    instructions="What kind of network event does this syslog message describe?",
    criteria={
        "link_state": "A physical or logical interface changed up/down state",
        "routing": "A routing protocol adjacency, neighbor, or session changed state",
        "auth": "An authentication, authorization, or accounting event",
        "hardware": "A power supply, fan, optic, or line card fault",
        "other": "Anything that does not clearly fit one of the categories above",
    },
)

Because the type definition permits null for undescribed labels, you may also write {"link_state": None, "routing": None} and let the label names carry the whole meaning. That is defensible for unambiguous industry terms — bgp, ospf, isis — and a trap otherwise. A bare label like escalate tells the model nothing about when to escalate. Write the description.

Structured Criteria with what, not_for, and examples

Plain strings stop working the moment two options overlap in the model's mind. When that happens, the value becomes an object instead of a string:

{
  "return_status": {
    "what": "Progress of a return already sent",
    "not_for": "Whether and how an item can be returned",
    "examples": [
      "Has my return arrived yet?",
      "When will my refund be paid?"
    ]
  }
}

what is the inclusion rule. not_for is the exclusion rule — the neighboring option this one keeps getting confused with. examples are concrete inputs that unambiguously belong here. The field names are user-defined and non-reserved, but the model sees both the names and the values, which is why the documented convention works: a key named not_for is itself a signal that what follows is an exclusion.

from typesafe_sdk import Choice

wifi_vs_security = Choice(
    instructions="Which team should own this incident?",
    criteria={
        "wireless": {
            "what": "Client association, roaming, RF, AP, or WLAN/SSID problems",
            "not_for": "A client that associates successfully but is then blocked by policy",
            "examples": [
                "Users on SSID CORP drop every few minutes in Building 4",
                "AP-3F-12 is stuck in a reboot loop",
            ],
        },
        "security": {
            "what": "Firewall policy, NAC/802.1X authorization, VPN, and access-control problems",
            "not_for": "RF coverage or AP hardware, even when the user is on Wi-Fi",
            "examples": [
                "802.1X authentication rejects contractors on the guest VLAN",
                "Site-to-site VPN tunnel to the Dallas branch is down",
            ],
        },
    },
)

Read the two not_for lines together: they form a boundary. Wireless owns getting the client onto the RF medium; security owns what the client is allowed to do afterward. A practical rule for when to upgrade: write plain strings first, look at the probability distributions on a sample of real tickets, and add structure only to the options that keep splitting. Structured criteria cost tokens — spend them where the confusion actually is.

Adding an other or unknown Option to Catch Edge Cases

A Choice answer is always one of your labels — the model does not have the option of declining. If a ticket about a failed Salesforce SSO integration reaches a Choice whose criteria list only WAN, Wireless, Security, and Data Center, it will not return an error; it will return one of those four, with probability mass smeared across whichever two feel least wrong. This is why every production Choice needs a catch-all option:

"unknown": {
    "what": "Not clearly a network incident, or missing the detail needed to route it",
    "not_for": "A clear network incident that merely spans two teams",
    "examples": [
        "Please reset my Salesforce password",
        "Something is broken, call me",
    ],
},

Note the not_for on the catch-all. Without it, unknown becomes a magnet: the model learns that anything even slightly hard goes there, and your automation rate collapses. The catch-all is for inputs that are out of scope or underspecified — not for inputs that are merely close calls. Close calls are handled by confidence gating, covered in the worked example below.

Common Mistakes in Criteria Design

Worked Example: Routing ServiceNow Tickets to Network Teams

Here is the first half of the NOC triage service end to end: a ServiceNow incident comes in, a single Choice decides which network team owns it, and — if the model is confident enough — the assignment_group field gets written back through the Table API.

Figure 5.2: ServiceNow incident routing and write-back sequence

Criteria for WAN, Wireless, Security, Data Center, and Unknown

import os
from typesafe_sdk import TypeSafeClient, Choice

client = TypeSafeClient()  # reads TYPESAFE_API_KEY from the environment

TEAM_CRITERIA = {
    "wan": {
        "what": "Branch/site connectivity, MPLS and internet circuits, SD-WAN overlays, "
                "WAN edge routers, and carrier faults",
        "not_for": "Problems contained inside a single building's LAN or inside a data center fabric",
        "examples": [
            "Denver branch has been offline since 02:14; carrier ticket CX-88213 open",
            "BGP session to ISP on the Chicago WAN edge is flapping every few minutes",
        ],
    },
    "wireless": {
        "what": "Client association, roaming, RF coverage, access points, WLAN controllers, "
                "and SSID configuration",
        "not_for": "A wireless client that associates successfully but is then denied by policy, "
                   "and wired switchport problems",
        "examples": [
            "Users on SSID CORP in Building 4 drop every few minutes",
            "AP-3F-12 will not join the controller after the firmware upgrade",
        ],
    },
    "security": {
        "what": "Firewall policy, NAC and 802.1X authorization, VPN tunnels, and access-control lists",
        "not_for": "RF coverage or AP hardware, and routing problems with no policy component",
        "examples": [
            "802.1X rejects contractors and drops them onto the guest VLAN",
            "Site-to-site VPN to the Dallas branch is down after a firewall change",
        ],
    },
    "data_center": {
        "what": "Top-of-rack and spine switching, EVPN/VXLAN fabric, server-facing ports, "
                "and east-west connectivity inside a data center",
        "not_for": "Circuits that leave the data center, which belong to WAN",
        "examples": [
            "Leaf-07 lost its MLAG peer link and half the rack went dark",
            "VXLAN tunnel endpoints on spine-02 stopped learning MAC addresses",
        ],
    },
    "unknown": {
        "what": "Not clearly a network incident, or missing the detail needed to route it",
        "not_for": "A clear network incident that merely touches two network teams",
        "examples": [
            "Please reset my Salesforce password",
            "Internet is slow",
        ],
    },
}

Five options: four real teams and one catch-all. Every real team carries a not_for that names its nearest neighbor — WAN excludes the LAN, wireless excludes policy denials, security excludes RF, data center excludes anything that leaves the building. Those four exclusions draw the boundaries between the four teams. Now the request — the ticket text goes into state, and the question goes into questions:

state = f"""ServiceNow incident {incident['number']}
Short description: {incident['short_description']}
Description: {incident['description']}
"""

response = client.system_one(
    model="jev-latest",
    state=state,
    questions={
        "team": Choice(
            instructions="Which network team should own this incident?",
            criteria=TEAM_CRITERIA,
        ),
    },
)

In the real triage service, questions can carry the team Choice, a vendor Choice, and a complexity Score together in one call, at roughly the cost of the first question — the layered pattern from the intent-routing guidance.

Reading the Probability Spread When Two Teams Are Plausible

Every Choice answer carries three fields: choice is the highest-probability option, probabilities is the full distribution across all options (summing to 1.0), and confidence is a 0–1 score reflecting how concentrated that distribution is. Here is a representative response body:

{
  "answers": {
    "team": {
      "choice": "wireless",
      "probabilities": {
        "wan": 0.03,
        "wireless": 0.46,
        "security": 0.07,
        "data_center": 0.41,
        "unknown": 0.03
      },
      "confidence": 0.44
    }
  },
  "usage": { }
}

The choice field says "wireless," and if you looked only at choice you would page the wireless team and move on. But 0.46 versus 0.41 is not a decision, it is a tie — and the tie is correct, because the ticket genuinely contains both signals: users are on an SSID (wireless) and an access switch uplink was re-trunked during the window (a data center/campus switching problem). A flat shape, with probability spread across several options, means low confidence; a single peak on one option means high confidence.

This is where the confidence gate goes, using attribute-style access on the response:

CONFIDENCE_FLOOR = 0.5

answer = response.answers["team"]
team = answer.choice
confidence = answer.confidence
spread = sorted(answer.probabilities.items(), key=lambda kv: kv[1], reverse=True)

if team == "unknown" or confidence < CONFIDENCE_FLOOR:
    runner_up = spread[1]
    escalate_to_noc_lead(
        incident,
        reason=f"top={team} ({spread[0][1]:.2f}), runner_up={runner_up[0]} ({runner_up[1]:.2f}), "
               f"confidence={confidence:.2f}",
    )
else:
    assign_incident(incident["sys_id"], team, confidence)

A word about that CONFIDENCE_FLOOR. The 0.5 comes straight from the intent-routing pattern, which is the right place to start when you have no data of your own — it is a documented default, not a measured value. Chapter 8 shows how to replace it by plotting confidence against accuracy on your own closed tickets, and the capstone service in Chapter 12 settles on a floor of 0.60 and an auto-assign line of 0.85 after doing exactly that. Treat 0.5 as the number you ship on day one and expect to move.

Two things about that gate are worth copying into your own code. First, it escalates on the catch-all and on low confidence — those are different failure modes, and both belong in the same branch. Second, when it escalates it hands the human the runner-up and its probability. A NOC lead who sees "wireless 0.46 / data center 0.41" makes the call in four seconds; a NOC lead who sees "the AI wasn't sure" reads the whole ticket from scratch, and you have saved nobody any time.

Figure 5.3: Ticket routing across five teams with a confidence gate

Writing the Assignment Group Back to ServiceNow

The ServiceNow field you want is assignment_group, a reference to the sys_user_group table, accepting either a sys_id (preferred and more reliable) or the group's display name. Build the mapping from your Choice labels to real sys_ids once, and keep it in config rather than in code:

# sys_ids come from the sys_user_group table in YOUR instance — these are placeholders
# except the first, which is the group used in the ServiceNow documentation's example.
SNOW_GROUPS = {
    "wireless":    "287ebd7da9fe198100f92cc8d1d2154e",
    "wan":         "<sys_id of the WAN group>",
    "security":    "<sys_id of the Security group>",
    "data_center": "<sys_id of the Data Center group>",
}

The Table API exposes incidents at https://{instance}.service-now.com/api/now/table/incident. To update an incident that already exists, append its sys_id to the path and PATCH it, using httpx:

import httpx

INSTANCE = os.environ["SNOW_INSTANCE"]
# Basic auth keeps this example short. Chapter 8 explains why production
# integrations should send an OAuth 2.0 bearer token instead.
AUTH = (os.environ["SNOW_USER"], os.environ["SNOW_PASSWORD"])


def assign_incident(sys_id: str, team: str, confidence: float) -> dict:
    url = f"https://{INSTANCE}.service-now.com/api/now/table/incident/{sys_id}"
    resp = httpx.patch(
        url,
        auth=AUTH,
        headers={"Accept": "application/json", "Content-Type": "application/json"},
        json={"assignment_group": SNOW_GROUPS[team]},
        timeout=10.0,
    )
    resp.raise_for_status()
    record = resp.json()["result"]
    log.info(
        "assigned %s to %s (sys_id=%s, confidence=%.2f)",
        record["number"], team, record["assignment_group"], confidence,
    )
    return record

Read assignment_group back off the response rather than assuming your write landed; ACLs can silently decline a field. Four operational cautions that will bite you in a real instance:

• Roles. The calling account needs the snc_platform_rest_api_access role to use the REST Table API at all, and some fields sit behind field-level ACLs that require the ITIL role.
• ACLs are per field. A 200 response is not proof that assignment_group changed.
• Patch, don't recreate. Updating the existing record by sys_id changes only the fields you send; POSTing a replacement blanks everything you left out.
• Mandatory fields vary. assignment_group is optional by default but may be mandatory in specific organizational configurations — confirm against your own instance.

Worked Example: Detecting Vendor from a Config Snippet

The second Choice in the triage service answers a different question: what is this? Snippets arrive pasted into tickets, scraped from backups, and attached to change requests, often with no device metadata at all. Before you can parse a config or match it against a golden template, you need to know whose syntax it is.

Figure 5.4: Vendor-detection flow from config snippet to handler

Distinguishing Cisco IOS, Arista EOS, Junos, and Aruba AOS-CX Syntax

The four platforms fall into two families. Cisco IOS and Arista EOS use a hierarchical CLI syntax structured similarly, while Junos uses a set-based CLI syntax with braces. Aruba AOS-CX joins the first family: its configuration follows a similar hierarchical structure to Cisco IOS. Junos is the outlier — it organizes commands hierarchically within curly braces or as flat set lines, and it stores changes in a candidate configuration before committing to the active configuration, unlike the other three, which apply changes immediately.

Two vocabulary traps are worth flagging. In Cisco and Aruba, "trunk" means an 802.1Q VLAN-tagged interface; AOS-CX calls aggregated interfaces a Link Aggregation Group (LAG), while the older ArubaOS-Switch called aggregation a trunk. And protocol placement differs: Junos puts BFD timers inside the routing protocol configuration rather than on individual interfaces. Those are exactly the kinds of details that belong in not_for and examples.

Using examples in Criteria to Sharpen Boundaries

The examples field earns its keep here, because vendor detection is pattern matching on literal strings and literal strings are what examples carries. "Provide concrete examples that exemplify the option's intent" translates directly into "paste real config lines":

from typesafe_sdk import Choice

vendor_question = Choice(
    instructions="Which network operating system produced this configuration snippet?",
    criteria={
        "cisco_ios": {
            "what": "Cisco IOS/IOS-XE: flat running-config, interfaces named by media type "
                    "and slot/port, VLANs configured with switchport commands",
            "not_for": "Snippets whose interfaces are bare numeric triples like 1/1/1, "
                       "or that use 'vlan trunk allowed' instead of 'switchport trunk allowed vlan'",
            "examples": [
                "interface GigabitEthernet0/0/1",
                "switchport trunk allowed vlan 1,10,20,30",
                "switchport trunk native vlan 1",
            ],
        },
        "arista_eos": {
            "what": "Arista EOS: Cisco-like hierarchical CLI, interfaces named Ethernet<n> "
                    "with no media-type prefix, three-space indentation",
            "not_for": "Interfaces with a media-type prefix such as GigabitEthernet, "
                       "which indicate Cisco IOS",
            "examples": [
                "interface Ethernet24",
                "switchport access vlan 33",
                "spanning-tree bpduguard enable",
            ],
        },
        "junos": {
            "what": "Juniper Junos: either 'set' statements or a brace-delimited hierarchy with "
                    "semicolon-terminated statements, and a candidate config committed explicitly",
            "not_for": "Any snippet that uses switchport or vlan trunk commands",
            "examples": [
                "set interfaces ge-0/0/0 unit 0 family ethernet-switching vlan members vlan-10",
                "description \"Link to Core\";",
                "commit",
            ],
        },
        "aruba_aoscx": {
            "what": "Aruba AOS-CX: Cisco-like hierarchy, interfaces named as a bare numeric "
                    "member/slot/port triple, VLANs configured with 'vlan trunk' commands",
            "not_for": "Snippets using 'switchport', which indicate Cisco IOS or Arista EOS",
            "examples": [
                "interface 1/1/1",
                "vlan trunk native 1",
                "vlan trunk allowed 1,10,20",
                "vsf member 1",
            ],
        },
        "unknown": {
            "what": "Not a switch or router configuration, or too short or generic to attribute "
                    "to a specific network operating system",
            "not_for": "A snippet that clearly belongs to one of the listed platforms",
            "examples": [
                "no shutdown",
                "hostname core-sw-01",
            ],
        },
    },
)

Look at how the not_for fields chain. cisco_ios excludes numeric-triple interfaces and vlan trunk; arista_eos excludes media-type prefixes; aruba_aoscx excludes switchport; junos excludes both VLAN dialects. Each exclusion points at a specific competitor using a specific literal token — contrastive criteria doing the work a regex would otherwise do, except it degrades gracefully on snippets a regex never anticipated. A high-confidence answer looks like this:

{
  "answers": {
    "vendor": {
      "choice": "arista_eos",
      "probabilities": {
        "cisco_ios": 0.04,
        "arista_eos": 0.91,
        "junos": 0.01,
        "aruba_aoscx": 0.02,
        "unknown": 0.02
      },
      "confidence": 0.89
    }
  }
}

One peak, everything else near zero — the single-peak shape that indicates high confidence. Compare that to the ticket-routing answer earlier and the difference is visible at a glance, which is the practical reason to log the full probabilities map and not just choice.

Testing with Ambiguous Snippets

You cannot tune criteria you have not stress-tested. Build a small fixture set of deliberately hard snippets, run them through the Choice, and assert on the shape of the distribution rather than only on the top label. Ambiguous inputs should produce ambiguous distributions — that is correct behavior, not a bug to be tuned away.

The last rows are the ones that will change your criteria. A snippet from an unlisted platform is the argument for the catch-all in concrete form: with no unknown option, that input must come back as Cisco, Arista, Junos, or Aruba, and your downstream template matcher will happily parse it against the wrong grammar. When a correct answer arrives with low confidence, the fix is almost always another entry in examples, not a rewrite of what.

A final note on expectations: the model is fast and reads literally. It is very good at "does this text contain the token switchport" and much weaker at anything requiring arithmetic or multi-step inference. Vendor detection plays to its strengths because the evidence is surface-level tokens. Do not extend the same Choice into "is this config compliant with our standard?" — that is a different question with a different failure mode.