Serving a language model (ChatGPT, Claude, and friends) is not an easy undertaking. The inference infrastructure aside, there's also a matter of security and defense against adversarial interactions that seeks to exploit the service - e.g. jailbreaking for harmful use, sophisticated denial of service, and of course, the much debated distillation attack. Here, I take a stab at the problem of defending, a made up frontier lab - Five Elements Inc. - against such adversary at the inference layer. In order to facilitate this, I've introduced a custom-made Structural Causal Machine(!) to generate a realistically grounded dataset for our fantastical model, and a simple yet sufficient (read: practical) model to detect and classify each user-session into either benign or malicious interactions, and if it's the latter, what kind. For those Fellow Daoist interested to try the problem out for yourself, you can find the full dataset via the following link. Happy reading!

The Need to Defend AI Systems

AI. AI. Greetings Fellow Daoist - we've been getting bombarded by "AI" in almost every aspect of our technological experience as of late, and love it or hate it (of which, I have much to say!), more and more white-collar workflows now have AI as a dependency one way or another - from ~40% of Coinbase's daily code being AI-generated, to Cloudflare letting AI agents register their own cloud accounts, subscriptions, and domains end-to-end, to Anthropic's own Claude in Excel finding its way into the spreadsheets of people who probably still call a VLOOKUP "the lookup thing". However, like all man-made systems, it is fraught with exploit vulnerabilities if wielded inappropriately (read: naively) - and yet, companies are rushing head long into adoption with blatant disregard (or at least, mere theatrical display of regard) for security, despite GenAI usage in organisations leaping from 33% in 2023 to 71% in 2024 without the security stack to match; and this includes services and products where consistency, security and stability are a core, uncompromisable requirement. Coinbase's CEO has openly admitted that "non-technical teams are now shipping production code" at the exchange, days before the trading engine itself went down; Cloudflare itself notes that "85% of IT decision makers report that employees are adopting AI tools faster than their IT teams can assess them" - all sidelined for the sake of not getting left behind by "AI", for the sake of "efficiency", despite METR's randomised controlled trial finding that experienced developers were actually 19% slower with AI tooling while believing they were 20% faster.

Conventional software is already fraught with exploits - known or otherwise. Google's Threat Intelligence Group tracked 90 zero-days exploited in the wild in 2025 alone, a 15% increase year-over-year, and these are only the ones we know about; the population of living zero-days - unknown to the vendor, exploitable for an extended period, sometimes never patched at all - is by definition uncountable. Any sufficiently expressive man-made system is incapable of being simultaneously consistent and complete, and software, being a man-made formal-ish system layered on top of other man-made formal-ish systems, inherits that incompleteness in the form of edge cases nobody thought to check. This is why we get the steady drumbeat of high-profile blowups - the MOVEit/GoAnywhere/Cleo managed-file-transfer cascade, CVE-2026-32202: a zero-click Windows Shell vulnerability actively exploited by APT28 via weaponised LNK files, and so on, ad nauseam. Now, deep generative models further expand this attack surface by introducing new vectors unbeknownst to most of the general public that relies on the service itself to be accurate, secure, and consistent - vectors that are harder than ever to detect and guard against. Here's a curated list of my favourite attack vectors from recent months:

List of exploits and attack vectors

1. Poison 250 documents in a corpus of billions to backdoor any LLM. A joint Anthropic / UK AI Security Institute / Alan Turing Institute study found that as few as 250 adversarial documents - ~0.00016% of a 13B model's training tokens - reliably installs a backdoor, regardless of model scale (Turing Institute writeup).
2. Trivial jailbreak via past-tense framing and persona injection. The OpenAI–Anthropic joint safety evaluation found Claude models were "most vulnerable to the past tense jailbreak", and the Mexican government attacker simply told Claude it was working a bug bounty programme to unlock the full kill chain.
3. Distillation attack at industrial scale. Anthropic disclosed that DeepSeek, Moonshot and MiniMax ran ~24,000 fraudulent accounts and 16M+ exchanges against Claude to extract chain-of-thought training data - MiniMax pivoted to a new Claude model within 24 hours of release.
4. Agentic abuse / weaponisation as a multiplier. Anthropic's Disrupting AI Espionage report describes a Chinese state-sponsored actor (GTG-1002) achieving 80–90% autonomy of a cyber-espionage campaign with only 4–6 human touchpoints; their model was making "thousands of requests, often multiple per second".
5. AI-authored zero-days in the wild. Google's GTIG just published a case where they could fingerprint a 2FA-bypass zero-day exploit as AI-generated from its hallucinated CVSS score and textbook docstrings.

and a non-trivial list of recent high-profile attacks that I can recall:

1. The Mexican government breach (Dec 2025 – Feb 2026). A single operator used Claude Code + GPT-4.1 to compromise 10 government bodies and exfiltrate ~150GB / 195 million identities - Claude executed ~75% of the remote commands.
2. Linux kernel LPE + Chrome zero-day chain (early 2026). A high-severity Linux kernel local-privilege-escalation landed alongside four actively exploited Chrome zero-days in the first quarter of 2026 alone - Google was the second-most-targeted vendor of 2025.
3. North Korean IT-worker fraud, AI-augmented. Anthropic disclosed that DPRK operatives were using Claude to fraudulently secure and maintain remote employment at Fortune 500 tech companies, funnelling salaries back to the regime's weapons programmes.
4. LLMjacking at scale - Operation Bizarre Bazaar. Pillar Security's honeypots recorded ~972 attack sessions per day over 40 days targeting exposed LLM endpoints - a 9-minute credential-to-first-exploitation time sold via the silver.inc marketplace on Telegram/Discord.
5. Supply-chain compromise of LiteLLM and Trivy. In March 2026, the TeamPCP / UNC6780 group embedded the SANDCLOCK credential stealer into LiteLLM and Trivy builds, exfiltrating AWS and GitHub secrets - and, by extension, the AI environments those keys unlocked.

From the above listicles, it's obvious that the types of attack are very varied and have been demonstrated to great effect. So, how can we go about defending or preventing such attacks in our own workflow or infrastructure? Unfortunately the solution is non-singular, and requires a complete reimagination of not only how we build our technologies, but also of how these models are served. Here in this essay, I would like to focus on the latter, where we try to secure the serve-side of the equation.

Why intervene at this point?

1. Model weakness & vulnerabilities are a function of many things - e.g. training dataset, training procedure, infrastructure, model architecture (e.g. energy-based models are less likely to hallucinate whilst LLMs are great at making things up; a weaker language model doesn't present such a level of control because it's too dumb to be coherent / consistent). All of these are baked in at training time - by the time the model is on the serving fleet, there is very little we can change about its priors without a multi-million-dollar retraining run.
2. The "serve" side is much easier to control, since the serving platform has visibility and deterministic levers that are tried and tested - e.g. rate-limiting APIs, harmful-content filtering, IP/device blacklisting, velocity throttling, etc. The same toolbox payment networks have used for decades to keep card fraud at ~7 bps of GMV is, mostly, transferable.
3. The interventions introduced here can better generalise across different model types and attack types. A rate limit doesn't care whether the model behind it is GPT-5.5 or a fine-tuned Qwen 3.6 - and an attacker's hydra-cluster account graph has the same "hyper-shape" regardless.

So in this exercise, we'll imagine that we're a frontier LLM service provider, Five Elements Inc., and we have a popular frontier LLM service. Now, how can we serve this responsibly and defend ourselves against abuse and attack from Demonic Path practitioners?

Defending Five Elements Inc

The following diagram summarizes our task setup @ Five Elements Inc. Platform Trust & Safety - take a look and understand how they connect first, visually:

Now, we have the above setup for Five Elements Inc. that I've just made up - pretty standard stuff - for context, so we can sketch out the exact problem statements next. Our goal is simple: allow good users to have minimal friction and a good experience, and identify, then block out, all adversarial actors.

In order to do that, we need to ask ourselves a few questions

1. What should our solution and interventions be, and how can we manifest them?
2. How can we identify these adversarial actors reliably, when each type could present a different profile and behaviour (e.g. bot farm vs. jailbreak)? Is a simple binary good/bad classification sufficient?
3. How do we definitively define what's good or bad? (e.g. a genuine researcher could look like a distillation attacker)
4. At what level should we intervene? (e.g. user level? session level? transaction level?)
5. So on, and so on.

Each practitioner goes above the above in their own way per their experience, and I'd always like to start with a nifty mental construct I devised - the Ladder of Abstraction - and sketch it out from here. The following is a simple sketch of the Ladder, it's simply a solution "roadmap" or taxonomy of sort; starting from the most fundamental atomic unit of raw data from event streams at L0, all the way to the most complex solutions I can think of in the forms of noisy, partially observable, multi-agent games at L7. Feel free to construct your own ladder and share it with me, I would love to see how things look like from your perspective!

Since we're just starting out at Five Elements Inc. [image of newbie Sir Duckalot with his backpack of myriad things and pots and pans, eager to set out on a journey with a map in hand], we'll start with the simplest lower level and go from there - point-predictor inference models, a'la good ol' XGBoost. Once we're able to make meaningful inferential point predictions, we can then layer up and build up our intervention stack & security framework from there as a pre-requisite! (e.g. to make meaningful bandit decisions, having a good action-state estimator is immensely helpful.) Each rung consumes the one below it. A bandit (L4) needs a reward signal, and that reward is only as trustworthy as the point-estimate (L2) and the uncertainty around it (L3) feeding it. A Stackelberg solver (L6) reasoning about an adversary's best response needs a belief state (L3) to reason over. Skip the lower rungs and you get a strategic-game layer optimising against garbage estimates - confidently wrong, which is the worst kind of wrong in an adversarial setting. So we earn each rung before climbing it.

Concretely, our task here is as follows:

• The Task: Given a user-session input, we have to determine if they are benign ($0$) or adversarial ($1$), and if they're malicious, what kind of bad actor they are.
• The Input: A user-session record and metadata. Roughly, each session row carries:
• The Output: A binary score of goodness, plus a multi-class label for the type of actor. The full label space, as per our SCM:

Why aggregate to the user-session? A single transaction is almost information-free in isolation: one prompt that trips the safety filter could be a curious researcher, a typo, or the first probe of a jailbreak. The discriminating signal lives in the shape of a session - the trajectory of perplexity, the accumulation of refusals, the diurnal rhythm, the inter-request timing variance (low for bots, high for humans). Aggregating to the user-session is the smallest unit that preserves that behavioural shape while still being granular enough to act on. Too coarse (whole-account, all-time) and we lose the onset of an attack; too fine (per-transaction) and we're classifying noise.

$$\begin{aligned} y_{\text{adversarial}} &\in \{0,\;1\} \\[6pt] y_{\text{type}} &\in \mathcal{C} \end{aligned}$$ $$\mathcal{C} \;=\; \left\{\;\begin{array}{rl} 0: & \texttt{benign} \\[3pt] 1: & \texttt{distillation} \\[3pt] 2: & \texttt{jailbreak} \\[3pt] 3: & \texttt{bot\_farm} \\[3pt] 4: & \texttt{harmful\_use} \\[3pt] 5: & \texttt{dos} \\[3pt] 6: & \texttt{credential\_abuse} \\[3pt] 7: & \texttt{data\_extraction} \\[3pt] 8: & \texttt{agentic\_misuse} \end{array}\;\right\}$$

where each non-benign class maps to a documented real-world surface:

Class	What it is
distillation	Capability extraction from competitors
jailbreak	Safety bypass (past-tense framing, persona injection)
bot_farm	Coordinated fake-account infrastructure
harmful_use	Extortion, malware, ransomware-as-a-service
dos	Unbounded-consumption / token flooding
credential_abuse	Stolen-key / reverse-proxy LLMjacking
data_extraction	Training-data / PII / system-prompt exfiltration
agentic_misuse	Tool-chain abuse as an attack platform

For our fun project here, we'll content ourselves with the above types only for brevity; at the campaign level there's a tenth type that I've also included in the dataset for extra fun, multi_surface - a coordinated operation spanning several of the above types, but that's a group property requiring cross-account aggregation, so it's out of scope for our per-session point predictor here, and we shall defer it to the higher rungs.

To recap, given a feature vector $\mathbf{x} \in \mathbb{R}^d$ for a session, we want model and estimate the following:

$$\hat{y}_{\text{adversarial}} = f_{\text{adversarial}}(\mathbf{x}) \in [0, 1], \qquad \hat{\mathbf{y}}_{\text{type}} = f_{\text{mc}}(\mathbf{x}) \in \Delta^{8}$$

where $\Delta^{8}$ is the $8$-simplex over the $9$ classes (probabilities summing to $1$), and $f_{\text{bin}}$ collapses to the complement of the benign-class probability in the simplest single-model framing:

$$\hat{y}_{\text{bin}} = 1 - \big[f_{\text{mc}}(\mathbf{x})\big]_{\texttt{benign}}$$

Now for the metrics, which given the I/O above, they are pretty straightforward! Of all the user-sessions, how many did we correctly identify as benign / adversarial? Specifically, we'll compute precision and recall curves and observe their interplay, and for convenience, we'll also compute their AUC for a threshold-agnostic comparison. Also, given each user-session that's malicious, what type of threat actor are they? This is essentially just multi-class classification, so we'll compute the per-class precision and recall, and then for convenience, their class-aggregate values (macro or micro) for easy comparison.

Precision asks: of everything I flagged as adversarial, how much actually was? (Low precision = crying wolf, punishing good users.)

Recall asks: of all the adversarial sessions out there, how many did I catch? (Low recall = letting attackers through.)

They trade off against each other as you slide your decision threshold $\tau$: clamp down hard ($\tau \to 1$) and precision rises while recall falls; loosen up ($\tau \to 0$) and the reverse. The PR curve plots that whole trade-off, and the area under it (AUPRC) gives one threshold-agnostic number - especially honest on skewed data, where a high-AUROC model can still be useless because the rare positive class is drowned out.

A quick refresher for those that still needs to Google the formulae like me - for the binary case, with $\mathrm{TP}$ / $\mathrm{FP}$ / $\mathrm{FN}$ as true positives / false positives / false negatives:

$$\text{precision} = \frac{\mathrm{TP}}{\mathrm{TP} + \mathrm{FP}}, \qquad \text{recall} = \frac{\mathrm{TP}}{\mathrm{TP} + \mathrm{FN}}$$ $$\text{AUPRC} = \int_{0}^{1} \text{precision}\big(\text{recall}\big)\; d(\text{recall})$$

For multi-class, we compute these per class $k$ in a one-vs-rest fashion, then aggregate. Let $\mathrm{TP}_k, \mathrm{FP}_k, \mathrm{FN}_k$ be the counts for class $k$, over $K$ classes:

$$\text{precision}_k = \frac{\mathrm{TP}_k}{\mathrm{TP}_k + \mathrm{FP}_k}, \qquad \text{recall}_k = \frac{\mathrm{TP}_k}{\mathrm{TP}_k + \mathrm{FN}_k}$$ $$\underbrace{\text{macro-P} = \frac{1}{K}\sum_{k=1}^{K} \text{precision}_k}_{\text{every class counts equally}}, \qquad \underbrace{\text{micro-P} = \frac{\sum_{k=1}^{K} \mathrm{TP}_k}{\sum_{k=1}^{K} \big(\mathrm{TP}_k + \mathrm{FP}_k\big)}}_{\text{every sample counts equally}}$$

Macro treats agentic_misuse (rare) as importantly as benign (the majority), whereas micro lets the majority class dominate. On our skew, the gap between the two is itself diagnostic.

With that out of the way, now you may ask - where do we get the dataset from? Whilst there are some LLM-serving datasets out there, such as BurstGPT: A Real-world Workload Dataset to Optimize LLM Serving Systems and ServeGen: Workload Characterization and Generation of Large Language Model Serving in Production, they are insufficient and do not meet the requirements of our task. We not only need the serving pattern and transactions of generic interactions, we also require

1. The labels of whether each transaction or user-session is benign or adversarial, and if it's the latter, what type they are.

To the best of my knowledge, there is no public labelled dataset of this kind, and I'd reckon companies are strongly disincentivized to release one. Publishing confirmed-adversarial traffic means admitting how much got through (reputation damage), exposing yourself to legal and regulatory liability, and handing adversaries a labelled map of exactly which behavioural signatures your detectors key on - a free evasion guide. The labels that do exist live behind T&S walls and stay there. This is also why our ground-truth labels arrive delayed and incomplete in the sim - a confirmed distillation label might land months later, when a competitor's suspiciously-similar model ships, and $30\text{–}70\%$ of adversarial accounts are simply never confirmed at all.

1. To be realistic, a lot of transactions - on the order of at least millions of data points, because adversarial interactions are very rare compared with the benign interactions and population, and therefore we need many transactions to yield a meaningful amount of labels for training.

Take a deliberately conservative transaction-level floor of $p = 0.01\%$ adversarial (1 in $10{,}000$). Then to collect just one malicious example you need $\sim\!10{,}000$ benign ones alongside it, so to get a usable $1{,}000$ malicious training rows you're already scanning

$$N_{\text{total}} = \frac{1{,}000}{p} = \frac{1{,}000}{10^{-4}} = 10^{7} \;\; (\text{10 million transactions}).$$

Where does $0.01\%$ sit? The open-web figure is wildly higher - Arkose Labs found 73% of web/app traffic in 2023 was bots-and-fraud-farm traffic, and Statista's 2024 read puts bad-bot traffic around 37% - but those are traffic-level on the open internet. At the account level on a friction-gated platform (payment, KYC, regional restrictions), the right anchor is much lower: the CERT insider-threat benchmark runs at $\sim\!1\text{–}3\%$ adversarial users. The truth for any given platform likely sits somewhere on the $0.01\%\text{–}5\%$ band at the transaction level (e.g. active users have higher activity), so to be conservative, we will go with $0.01\%$.

All that said, because of the above challenges and lack of existing open dataset that satisfies our need, we'll settle for the next best thing and come up with our own synthetic dataset - which is what we're going to touch on next!

Data-Synthesizer-3000

There are many ways to make up a dataset - e.g. the most trivial first-order idea would be to first figure out the set of features or signals you want and then defining the parameters of said feature set, and randomly sample each column as needed; the problem with this approach is immediately obvious upon some mental poking-around, the most obvious being how the features correlate and covariate, are not accounted for. We can do better and generalize this to account for realistic interdependent relationships by conditioning certain features with another, adding some way to approximate a consistent session-to-session behaviour for a single user across time, and even ways to simulate cross-account coordination for more sophisticated multi-account campaigns.

For example, if we have a user-session associated with a bot farm, it's likely that it has an outlying velocity footprint, correlated intra-cluster activity cadence, potentially high-degree of device or IP sharing with other bot-instances from the same source, and so on. Each one of these correlations only makes sense in the context of the other sessions in the same cluster - sampling them independently is a category error.

Concretely, the value of feature $j$ on session $s$ for user $u$ at time $t$ is not a free-standing random variable - it's conditioned on three different axes at once:

$$x^{(j)}_{u,s,t} \;=\; f_j\!\Big(\;\underbrace{\mathrm{pa}_j(x_{u,s,t})}_{\text{intra-session parents}},\;\;\underbrace{\mathbf{h}_{u,\,<t}}_{\text{user trajectory}},\;\;\underbrace{\mathbf{c}_{C(u),\,t}}_{\text{campaign context}}\;\Big)\;+\;\varepsilon^{(j)}_{u,s,t}$$

where $\mathrm{pa}_j(\cdot)$ is the set of causal parents of feature $j$ within the same session (e.g. total_tokens depends on n_requests), $\mathbf{h}_{u,<t}$ is a sufficient statistic over user $u$'s prior sessions (escalation, coverage saturation, baseline geo, etc.), $\mathbf{c}_{C(u),t}$ is the shared state of the campaign $C(u)$ that $u$ belongs to (e.g. jailbreak success, coordinated burst tick, domain distillation progress), and $\varepsilon^{(j)}$ is irreducible per-session noise. Drop any one of these axes and the dataset stops being interesting - drop the first and features are uncorrelated; drop the second and accounts have no memory; drop the third and a "bot farm" is just N independent accounts with no shared signature.

We can achieve the above through a Structural Causal Model (for a more formal yet approachable treatment, try this writeup from Pearl); though I personally prefer to call them Structural Causal Machine instead, because machine is much cooler than model(!), and I shall take the liberty to use the term within this article, because I can - ha!

The modern SCM formalism is largely due to Judea Pearl, crystallised in Causality: Models, Reasoning, and Inference (2000) and the do-calculus that earned him the 2011 Turing Award. The underlying idea - distinguishing structural equations from regression equations - traces back further to Sewall Wright's path analysis (1921) and the Cowles Commission econometricians of the 1940s.

So what is a Structural Causal Machine? Intuitively, instead of treating our dataset as a flat spreadsheet where each column is sampled in blissful ignorance of its neighbours, an SCM treats the world as a little assembly line of cause-and-effect: you lay out your features as nodes on a graph, draw an arrow from each cause to its effect, and then generate each feature as a function of its parents plus a dash of its own randomness. The arrows are the whole point, they encode "this depends on that". Sample the parents first, then their children, walking the graph in order, and out pops a synthetic instance whose features hang together the way real ones do - a bot farm's session isn't a bag of independently-rolled dice, but the downstream consequence of being a bot farm (low cadence variance, shared infra, the works). And because the structure is causal rather than merely correlational, we get a third freebie: we can reach in and intervene - clamp a node to a value, sever its incoming arrows, and ask "what would this session look like if it weren't rate-limited?" - which is exactly the kind of counterfactual surgery you can't do with a pile of np.random calls and will come in handy down the line when we perform counterfactual analysis.

Formally, an SCM is a tuple $\mathcal{M} = \langle \mathbf{U}, \mathbf{V}, \mathbf{F}, P(\mathbf{U}) \rangle$ where $\mathbf{V}$ are the observed/endogenous variables (our session features), $\mathbf{U}$ are the exogenous noise terms (one per node), $\mathbf{F} = \{f_j\}$ is the set of structural equations $v_j := f_j(\mathrm{pa}_j, u_j)$, and $P(\mathbf{U})$ is the joint over the noise. The induced graph $\mathcal{G}$ is acyclic by construction, and observational sampling is just ancestral sampling: walk $\mathcal{G}$ in topological order, evaluate $f_j$ at each node. Interventions $\mathrm{do}(v_k := c)$ replace $f_k$ with the constant $c$ - surgically severing $v_k$'s incoming arrows while leaving every other equation untouched. This last property is what separates an SCM from "a Bayes net with extra steps": the same object answers observational, interventional, and counterfactual queries.

Our SCM, is organised in three concentric layers, mirroring the three conditioning axes in the equation above.

I shall not dwell too much on the topic of SCM in this essay - that can be a topic for another essay in of itself - so we don't get distracted; but to completely gloss over it may also rob certain readers without prior exposure a sufficiently vivid apprehension to appreciate the subsequent parts. Therefore, I will now provide a necessary and somewhat sufficient overview of our SCM in action, and then we'll proceed to examine a slice of the final dataset and its semantics before diving into the modeling and results (ah, exciting!).

Coordinating a Distillation Attack

Now I will walk you through the simplified end-to-end journey on how we generate a realistic synthetic dataset for our use case here; note that in my description, I have stripped down and abstracted away the exact mechanistic components of my SCM since it is merely one of the possible instantiation of the workflow described below. For simplicity, we're also going to focus only on a single distillation attack campaign where there's a coordinated group of actors (or it's just a single person with a bunch of Agentic Mac Minis!) looking to extract high-value data from our model for their own adversarial purposes. Further note that we're narrowing the aperture here to epistemic distillation - the kind where the attacker is systematically tiling the model's knowledge across a domain space, accumulating coverage slot by slot. There is a structurally distinct cousin: orchestration distillation, where the attacker feeds the model elaborate multi-tool agentic tasks and captures the resulting action chains for imitation learning. That variant has a very different signature - e.g. deep tool chains, near-zero query entropy, potentially higher tool-use rate, preference for strong-agentic models.

On the highest level, there are really only 2 major components to our generative process:

1. The definition & instantiation step where we define the parameters of our universe and cast of characters
2. The roll-out process where we manifest the tick-by-tick user-sessions for each account of the campaign

Defining & Instantiating the Campaign Universe

Before any user ever touches the model, we first have to set up the world they'll act in. Recall the three axes our feature equation leaned on - the intra-session parents $\mathrm{pa}_j$, the account's own trajectory $\mathbf{h}_{u,<t}$, and the shared campaign context $\mathbf{c}_{C(u),t}$. This definition-and-instantiation step is where we lay the groundwork for all three, and it's cleanest to picture it as four layers cascading into one another, left to right.

At the far left sit two universe draws - the campaign type $C$ and the actor type $A$ - which fix the strategic context: what kind of attack we're simulating, and who is running it. From those we draw a sophistication score $s$, a single scalar that turns out to be the master dial of the whole operation, feeding downstream into nearly every behavioural parameter to come. $C$, $A$, and $s$ together are then realized into the campaign details - the once-per-campaign quantities like fleet size, phase schedule, and infrastructure pool - which fan out into the per-account realizations: the leaf draws unique to each account on the roster. Once instantiated, an account doesn't just sit there - it acts on Five Elements Inc.'s serving stack, and the platform acts back: every rate limit, CAPTCHA, or monitoring flag it applies conditions that account's future behaviour and, for sophisticated campaigns, the coordinating campaign layer itself. That act/condition loop is the one piece of the picture that isn't a one-shot sample - we'll see it spelled out tick-by-tick in the rollout section below.

We'll only walk a representative slice of this here - the full SCM carries far more campaign types, account archetypes, and platform dynamics than any one essay can bear - but it's enough to watch the three axes take shape. The diagram below makes the dependencies explicit; we'll walk each layer in turn, then close the section by mapping every block back onto its argument in the equation above.

Campaign type $C \sim \operatorname{Cat}(\boldsymbol{\theta})$ draws which attack surface this campaign pursues. Distillation is the modal case at 30%, reflecting a world where systematic knowledge extraction is a more common commercial motive than outright harmful use or disruption:

distillation	harmful_use	jailbreak	credential_abuse	data_extraction	dos	multi_surface	bot_farm
0.30	0.20	0.15	0.10	0.10	0.07	0.05	0.03

Actor type $A \sim \operatorname{Cat}(\boldsymbol{\phi})$ identifies who is running the campaign - $\boldsymbol{\phi} = (0.30,\;0.25,\;0.25,\;0.10,\;0.10)$ for (criminal_org, competitor, script_kiddie, nation_state, insider). Criminal organizations are the modal adversary; nation-states are rare but disproportionately capable.

Sophistication $s \mid A, C$ is sampled last and is the most consequential of the three - it encodes how capable and disciplined this particular operator is:

$$s \mid A,\,C \;\sim\; \operatorname{Beta}\!\bigl(\alpha_{A,C},\,\beta_{A,C}\bigr)\Big|_{[s_{\min},\,s_{\max}]}$$

The Beta prior varies by actor type, reflecting our beliefs about each adversary class's capability floor and ceiling:

$$s \mid A, \text{distillation} \;\sim\; \begin{cases} \operatorname{Beta}(8,2)\;\;[\,\mathbb{E}[s] = 0.80\,] & A = \texttt{nation\_state} \\[2pt] \operatorname{Beta}(5,3)\;\;[\,\mathbb{E}[s] = 0.63\,] & A = \texttt{competitor} \\[2pt] \operatorname{Beta}(4,4)\;\;[\,\mathbb{E}[s] = 0.50\,] & A = \texttt{criminal\_org} \\[2pt] \operatorname{Beta}(3,3)\;\;[\,\mathbb{E}[s] = 0.50\,] & A = \texttt{insider} \\[2pt] \operatorname{Beta}(1,5)\;\;[\,\mathbb{E}[s] = 0.17\,] & A = \texttt{script\_kiddie} \end{cases}$$

For distillation campaigns specifically, $s$ is then clamped to $[0.40,\,0.95]$ - even a script kiddie running a distillation operation needs to be minimally organized, and nobody operates with perfect omniscience.

I have simply picked these categories and their values from thin air according to my own priors for this exercise - reparameterize freely! Amongst the three root parameters, sophistication is the one that rewards the most attention: it propagates into infra reuse, account-creation spread, TLS spoofing enablement, evasion response complexity, load redistribution capability, and burnt-account replacement speed, amongst others.

Now that we have the campaign's meta-statistics defined, we sample and realize an instantiation of a singular distillation campaign. Once again, we'll only look at a subset of the campaign's key states, and the following values and configurations reflect my own belief of the states, you are welcomed to tweak them as you need.

Number of accounts, $n$:

$$n \;=\; \max\!\bigl(1,\;\operatorname{NegBin}(r{=}3,\,p{=}0.3)+5\bigr), \quad \mathbb{E}[n] \approx 12$$

NegBinomial (not Poisson) is chosen for its over-dispersion: most operations deploy a modest cohort, but a heavy right tail allows occasional large fleets. Here $r$ is the stopping parameter and $p$ the per-trial success probability; the $+5$ offset enforces a floor of genuine coordination. This is an intrinsic property of the distillation type itself - distillation must tile a large domain space, so it is inherently multi-account - and is independent of $A$ and $s$.

Phase schedule - a sampled, accumulated timeline from $t_{\text{start}}$:

$$\begin{aligned} \tau_{\texttt{setup}} &\sim U(1,5) \\ \tau_{\texttt{warmup}} &\sim \operatorname{Exp}(\lambda^{-1}=7) \\ \tau_{\texttt{probing}} &\sim U(3,10) \\ \tau_{\texttt{active}} &\sim \operatorname{LogNormal}(3.5,\,0.5), \quad \text{median} \approx 33\,\text{d} \\ \tau_{\texttt{cooldown}} &\sim U(1,5) \end{aligned}$$

Durations are sampled per phase and accumulated into absolute end-days. Crucially, the setup/warmup phases deliberately emit benign cover traffic - a distillation account looks like an ordinary casual user for its entire early life, and only reveals its template-driven, systematic behavior from probing onward.

Termination. The campaign runs until one of two things ends it: it declares victory the moment aggregate domain coverage crosses $0.80$ - it has tiled the 80% of the space it set out to cover - or it aborts the moment the suspended fraction crosses $0.70$, folding once attrition has burned through 70% of its fleet.

Account creation spread, $t_{\text{create}}$:

$$t_{\text{create}} \;\sim\; \mathcal{N}\!\bigl(t_{\text{start}},\;\sigma = (1-s)\cdot 14\bigr), \quad t_{\text{start}} \sim U(0,\;0.85\,H)$$

Accounts are spawned clustered around the campaign's start day $t_{\text{start}}$ (drawn over the first 85% of the horizon $H$ so it has room to run). The spread narrows with sophistication: $s \to 1$ provisions the whole fleet within a tight window - deliberately, often well ahead of activation during the benign setup/warmup cover phase - while $s \to 0$ scatters registrations chaotically across ~2 weeks. Note the genuine tension here: a tight burst actually raises registration-velocity (many sibling registrations in one window), so the sophisticated operator's edge comes not from this axis but from pairing the burst with low infra reuse (near-unique IPs per account) and hiding it inside the cover-traffic window before any adversarial behavior invites scrutiny.

Infra pool size & reuse:

$$n_{\text{IP}}\!\sim\!\operatorname{Poi}(5),\quad n_{\text{dev}}\!\sim\!\operatorname{Poi}(8),\quad n_{\text{email}}\!\sim\!\operatorname{Poi}(10);\qquad \text{reuse} = 1 - s$$

The campaign provisions a shared pool of IPs, device fingerprints, and email domains. The reuse factor $1-s$ governs how aggressively accounts share that pool: high $s$ → low reuse → near-unique infra per account → faint shared-IP / shared-device footprint (hard to catch); low $s$ → heavy reuse → dense, easily-clustered footprint. Sophistication ($s>0.6$) additionally unlocks TLS-fingerprint spoofing.

Load redistribution:

$$\text{mult} = \max\!\Bigl(3.0,\;\tfrac{1}{1-f_{\text{burn}}}\Bigr), \qquad \text{applied only if } s > 0.5$$

When accounts are suspended (burn fraction, $f_{\text{burn}}$), only sophisticated campaigns ($s>0.5$) compensate by redistributing the lost throughput across surviving siblings, scaling each survivor's activity by up to $3\times$. Unsophisticated campaigns simply absorb the loss and slow down.

From the campaign definition, the individual accounts of the campaigns are spawned to populate its roster.The multi_surface campaign is the apex predator of the taxonomy: its roster is a deliberate mixture of roles - e.g. jailbreaker (35%), distillation (30%), agentic_misuse (20%), and cover_traffic (15%) - and it is restricted to the nation-state sophistication tier ($s \in [0.70, 1.0]$). The variety is intentional: it simulates the realistic operational security of a well-funded adversary who cycles attack surfaces to frustrate pattern-matching defenses. For this walk-through we hold to a single campaign type and a single role; the multi-surface case is its own essay. Each one then draws its own realized values from the latent distributions we've been laying out - its personal sophistication, how many sessions it runs per day and how many requests within each session, the average inter-request time, its prompt perplexity and query-structure entropy, the epistemic domain coverage it targets both per-session and in coordination with its siblings, how it carries itself in each phase, its evasion-response complexity, its diurnal rhythm, and so on down the roster of latents.

Last but not least, the label arrivals for each user-session is also a controlled parameter drawn from a Lognormal distribution to mimic realistic label discovery / annotation efforts at Five Elements Inc. Specifically,

$$\Pr(\text{ever labeled}) = 0.10, \qquad \text{delay} \sim \text{Lognormal}(\mu_{distillation}{=}4.5,\,\sigma_{distillation}{=}0.5) \;\Rightarrow\; \text{median} \approx e^{4.5} \approx \textbf{90 days}$$

That is to say, it's non-trivial to identify and flag adversarial distillation user-sessions (e.g. it has non trivial overlap with other benign user type such as a diligent autodidact) - there's no "sure-fire" smoking gun indicator that we can rely on, hence it takes many days to accumulate and then analyze historical user-sessions over time for reliable identification, and even after all that, we are only able to identify 10% of all sessions with up to 90 days delay, of such type in my fantastical setup. (The Trust & Safety team at Five Elements Inc. has their work cut out for them indeed!)

To see just how fuzzy that overlap gets, here is the behavioural DNA of an active distillation session laid side by side with a benign power-user's - two profiles that, on more than half these axes, are genuinely hard to tell apart:

Feature	Distillation (active)	Benign power-user	Separation
sessions_per_day	~4/day	~5/day	overlap
n_requests_per_session	~33/session, heavy	~25/session, heavy	overlap
is_api	95% API	25% API	separable
avg_inter_request_time_sec	~3s, regular	~12s, varied	partial
std_inter_request_time_sec	~8s jitter, robotic	~45s jitter, bursty	key signal
avg_prompt_perplexity	~0.0, tight	~0.2, looser	overlap
std_prompt_perplexity	~0.15, template-driven	~0.8, varied phrasing	separable
query_structure_entropy	~0.25, narrow/repetitive	~0.43, varied	separable
avg_safety_score	~0.03, benign-looking	~0.04, benign-looking	overlap
domain_coverage_delta	~0.50, aggressive new coverage	~0.20, narrow focus	separable
total_tokens	~1.8k/session, high	~3.0k/session, high	overlap
model_endpoint	OPUS .70 / SONNET .25 / HAIKU .05	OPUS .55 / …	partial

Read straight down that last column and the fuzziness speaks for itself in our synthetic dataset. On cadence, request volume, perplexity, safety score, and raw token count, the two archetypes are effectively indistinguishable - a distillation account working the active phase looks, feature-for-feature, an awful lot like a heavy but legitimate power user. The separation the model actually gets to lean on lives in a handful of regularity tells - the std_ jitter columns and the query-structure entropy - propped up by API usage and coverage aggression. Which means in this fantastical setup, by design, a distillation account gives itself away not by what it asks but by how mechanically it asks it.

We've now built every block in the left half of that DAG, so let me make good on the promise from the top of the section and hang each one back onto the feature equation. Recall it had four pieces - the intra-session parents $\mathrm{pa}_j$, the trajectory $\mathbf{h}_{u,<t}$, the campaign context $\mathbf{c}_{C(u),t}$, and the noise $\varepsilon^{(j)}$ - and every block we just walked feeds exactly one of them:

• Intra-session parents, $\mathrm{pa}_j$. These are the base parameters $\theta^{(j)}_{\text{base}}(C, \phi_t)$: the within-session wiring that lets total_tokens ride on n_requests, and with it the whole cast of per-session distributions. Campaign type $C$ settles which family of those distributions is in play, and the phase $\phi_t$ settles which row of the DNA table above goes live on a given tick, with the fleet size $n$ and the schedule $\boldsymbol{\tau}$ between them deciding which phase an account is sitting in to begin with. Actor type $A$ never reaches this axis on its own; whatever pull it has arrives entirely through $s$.
• Campaign context, $\mathbf{c}_{C(u),t}$. This axis is made concrete by the shared infra pool, the reuse factor $1-s$, and the redistribution gate $\text{mult}$ - the cross-account coordination signals, recomputed once per tick and held in common across every sibling in the fleet.
• Trajectory, $\mathbf{h}_{u,<t}$. This is the one axis no static configuration can set. It begins empty and only takes shape as the rollout runs, each day's sessions accreting into a history that the next day reads straight back. The instantiation step can do no more than seed it through the per-account leaf draws; it is the rollout that actually fills it in.
• Noise, $\varepsilon^{(j)}$. The irreducible exogenous draw sitting at each node, folded into every feature so that no two sessions ever come out quite the same.

And sophistication $s$ is the thread stitched through nearly all of it: it fixes the reuse factor and redistribution gate on the campaign-context axis, and it bends the trajectory axis, through evasion dynamics, saturation rate, and how fast burnt accounts get replaced. That reach is exactly why $s$ was worth sampling so carefully back at the start - nudge that one dial and the entire downstream picture shifts.

The table is a photograph of one specific moment: $C = \texttt{distillation}$, $\phi_t = \texttt{active}$, an account mid-campaign. In reality those distributions are moving targets - the same account on day one of warmup looks nothing like it does forty days into active extraction, and a rate-limited one looks nothing like either. Written down properly, for each account $u$ on day $t$:

$$P(\mathbf{x}_{u,t} \mid \mathcal{Z}) \;=\; \prod_{j \in \text{topo}(\mathcal{G})} P\!\Big(x^{(j)}_{u,t}\;\Big|\;\mathrm{pa}_j,\;\theta^{(j)}(\mathcal{Z})\Big), \qquad \mathcal{Z} = \big(C,\;\phi_t,\;\mathbf{h}_{u,<t},\;\mathbf{c}_{C(u),t},\;\text{Interv}_{u,t}\big)$$

$\mathcal{Z}$ is a context bundle assembled fresh each day from the campaign config, the account's own accumulated history, the campaign's shared coordination state, and whatever the platform has decided to do to this account today. The per-node parameters $\theta^{(j)}$ encode all of it by starting from a base that depends on $(C,\phi_t)$ alone - the DNA-table snapshot - and then pushing that base through a three-pass modifier stack, each pass folding in one more slice of $\mathcal{Z}$ (written after the semicolon, the context that conditions that pass):

$$\theta^{(j)}(\mathcal{Z}) \;=\; \mathcal{T}_{\text{traj}}\!\Big(\mathcal{T}_{\text{camp}}\!\big(\mathcal{T}_{\text{interv}}\!\left(\theta^{(j)}_{\text{base}}(C,\,\phi_t)\;;\;\text{Interv}_{u,t}\right)\;;\;\mathbf{c}_{C(u),t}\big)\;;\;\mathbf{h}_{u,<t}\Big)$$

So the base carries $(C,\phi_t)$, and the three passes fold in the remaining components of $\mathcal{Z}$ one at a time - platform pressure ($\text{Interv}_{u,t}$) first, campaign coordination ($\mathbf{c}_{C(u),t}$) second, the account's own trajectory ($\mathbf{h}_{u,<t}$) last. Between the base and the stack, all of $\mathcal{Z}$ is accounted for, which is exactly why $\theta^{(j)}$ is a function of the whole bundle and not just $(C,\phi_t)$. The rollout is what happens when $t$ starts ticking.

The Rollout

We're almost there! After all that hard work setting up the stage, the next order of things is to simulate the tick-by-tick rollout of the campaign across time in the form of user-session records, and the backbone for such rollout is the aforementioned Phase Schedule.

Time, in our simulation, advances in discrete daily ticks, and on each tick the campaign wakes up and runs its housekeeping before a single session is ever drawn. Think of it as the morning standup of the operation - the campaign surveys yesterday's damage, decides who is still in the fight, advances its own clock, tallies how much ground it has taken, and only then hands each surviving account its marching orders for the day. The ordering is deliberate: each step reads the state the previous step just wrote, so the whole thing is a little cascade that runs top-to-bottom every single day.

First, decay. Any interventions the platform imposed earlier - a rate limit, a CAPTCHA gate, a stint of enhanced monitoring - carry a countdown, and the first thing each day is to tick those timers down by one and lift any that have expired. This is what lets a punished account gradually claw its way back to full throughput once the heat is off.

Second, fallout. Next the campaign reacts to whatever policy threw at it overnight. Every account suspended since the last tick is marked burned and removed from the active roster, and the campaign checks its own survival condition - if the cumulative burn fraction crosses $0.70$, the whole operation aborts then and there. Short of that, the sophisticated campaigns ($s > 0.5$) don't just absorb the loss; they redistribute the load of the fallen across the survivors, scaling each remaining account's activity by

$$\text{mult} = \max\!\Big(3.0,\; \frac{1}{1 - f_{\text{burn}}}\Big),$$

so the campaign tries to hold its total throughput roughly constant even as its fleet thins - capped at $3\times$ so survivors don't betray themselves with an absurd spike. Unsophisticated campaigns skip this entirely and simply bleed out.

Third, the phase transition we just described - the campaign consults its schedule (and its coverage progress) and decides whether today is the day it graduates from, say, probing into active extraction.

Fourth, the shared-state update - the one genuinely cross-account moment in the day. Here the campaign sums up how much of the domain space its accounts have collectively chewed through and converts it into a single progress number:

$$\text{covered\_slots} = \sum_{\text{active distill. accounts}} (\text{slots that account has covered}), \qquad \text{aggregate\_coverage} = \frac{\text{covered\_slots}}{\text{total\_domain\_sectors}}.$$

This is the campaign's scoreboard. It's what the success check reads tomorrow ($\geq 0.80 \Rightarrow$ victory), and - just as importantly - it's the signal that tells each account how much territory is still on the table, which we'll see shapes how hard they push.

Fifth and last, activity allocation - the marching orders. The campaign walks its surviving accounts one by one and writes each a small daily directive. For a distillation account in the active phase, the key field is how many fresh domain slots to attack today,

$$\text{slots\_today} = \min\!\big(\text{slots remaining in that account's block},\; \operatorname{Poi}(2) + 1\big),$$

Outside the active phase, or for any non-distillation role, this is simply zero (the account is told to lie low). The directive also folds in the load-redistribution multiplier and a rate-limit penalty ($\times 0.3$ if currently throttled), and during warmup it flips a "generate cover" flag that forces the account to behave like an ordinary benign user regardless of its true intent.

Only now, with the campaign-level bookkeeping settled, do the accounts actually sit down and use the model. For each active account the rollout assembles a sampling context - a compact bundle of everything the generative model needs to know about this account, on this day - and then draws sessions one at a time. That context is stitched together from three sources, each carrying a different flavor of influence:

Influence	Captures	What it carries for distillation
Intervention state	what the platform is currently doing to the account	whether it's rate-limited; its sophistication (which governs how it evades)
Campaign context	the cross-account coordination signals	today's slot allocation; remaining coverage, $1 - \text{aggregate\_coverage}$
Trajectory	the account's own accumulated history	its personal saturation, $\text{covered}/\text{assigned} \in [0,1]$

The crucial subtlety is that sessions within a single day are not drawn independently in one shot - they're generated sequentially, and the account's cumulative statistics are updated between each draw. After every session, the new ground it covered is booked straight back into the ledger,

$$\text{covered\_slots} \mathrel{+}= \max\!\bigl(1,\; \lfloor \delta \cdot \text{assigned\_slots} \rfloor\bigr),$$

so the very next session that same day already sees a slightly more saturated trajectory. It's a small thing, but it's what keeps the within-day dynamics honest: an account doesn't get to cover the same territory twice, and its push naturally tapers as its own block fills in. And because that ledger is exactly what the campaign re-aggregates at the top of the next tick, we've closed the loop - today's sessions become tomorrow's scoreboard, the scoreboard decides tomorrow's marching orders, and the campaign grinds forward, one quiet day at a time, until it either tiles 80% of the space and declares victory or burns down trying.

Behold, Our Dao-Achieving Dataset

The following is a snippet of our Five Elements Inc's raw service usage dataset for its frontier models. Here, I've taken the liberty to provide a stratified sample of 100 samples each, per attack type (note the difference between campaign types and individual attackers of specific type per campaign) for your perusal. Two things to note, first of which, is that for those with no attack type (-), these are benign accounts, and secondly, for certain columns two variants provided (e.g. noisy_avg_prompt_perplexity and avg_prompt_perplexity) where the former is the realistic noise-injected signal and latter is the oracle value of said signal. We'll talk a bit more about how we noised up the data in a bit.

There are a few quick points, amongst others, to note about our synthetic dataset before we go further:

1. Campaigns don't shapeshift. An adversarial account is locked to a single campaigns and attack type for its entire life - no harmful_use account quietly graduating into distillation, or vice versa. Real operators absolutely pivot surfaces mid-campaign; I've simply kept the roster monogamous here for tractability.
2. Every account starts cold. The fleet spawns straight into setup with no burn-in, so the first days of any run sit in a transient rather than a steady state. A more faithful sim would run a longer horizon and clip the head off once things settle - I didn't bother, so treat the earliest ticks as warm-up noise rather than signal.
3. Label delay has a prior, not a mechanism. Each surface draws its arrival lag from its own Lognormal delay distribution - dos clears in under a day, distillation sits at a ~90-day median - so at the type level, delay does track how hard a surface is to catch, and that's deliberate: I hand-tuned those medians to encode exactly the "loud and dumb resolves fast, quiet and patient resolves slow" intuition a real T&S team would recognise. What's missing is any account-level causality underneath that prior: two distillation accounts draw from the identical Lognormal regardless of sophistication or how convincingly either one impersonated a power user.
4. Infra pools are too clean. Benign users draw from one shared pool of IPs/devices/emails, and every campaign gets its own private pool - a tidy partition that doesn't hold up in the real world, where attackers and honest users routinely share the same datacenter ranges, VPN exits, and cloud infrastructure. That cleanliness is quietly doing some of the detector's work for it.

The takeaway from the above is this - as it stands, the synthetic dataset has some artificial boundaries and simplifications, which renders it an easier challenge compared to a real counterpart, but nonetheless, sufficient for our task here.

Finally, we'll now get to the fun part - modelling!

---

Training the Detection Model

In this section, we'll first do some data preparation work - e.g. quick exploratory analysis, feature engineering and the like - and then defining the training setup before finally looking at the results of our custom detection models.

Recall from the Defending Five Elements Inc. section that our task is the following - given a session feature vector $\mathbf{x} \in \mathbb{R}^d$, we want to predict two targets:

$$\underbrace{\;y_{\text{adversarial}} \in \{0,\;1\}\;}_{\substack{\text{binary} \\[2pt] \text{benign vs. adversarial}}} \qquad\qquad\qquad \underbrace{\;y_{\text{type}} \in \mathcal{C}\;}_{\substack{\text{multi-class} \\[2pt] \text{which kind of adversarial account}}}$$

Specifically, $\mathcal{C}$ here corresponds to attack_type (not campaign_type) in the previous dataset snippet; means we're focusing on detecting the type of adversariality of a given account and not the campaign that the account belongs to. This is a deliberate choice as the former is simpler than the latter and constitutes a decent starting point for our task; thus, our dataset is focused on single-attack campaigns, with the exception of multi_surface campaigns where it's an ensemble of various attack types. Additionally, $y_{\text{type}}$ is strictly more descriptive than $y_{\text{adversarial}}$ (e.g. we can simply define benign class), hence we'll be focusing on the multi-class case.

Dataset Preparation

Some of the more astute Fellow Daoist, after perusing the raw dataset in the prior section, would've realized that we have a few different labelling columns such as the campaign_type and campaign_label, noisy_campaign_label triplets, amongst others. The question is, "why do we have these and what are their utilities"?

Labels & Ocularity

Our labels live on two independent axes, Fellow Daoist. The first axis is what a label tells you - its purpose. The second is how cleanly you were ever allowed to see it - its ocularity, how close to omniscient the channel that produced it is.

Axis 1, Purpose: what a label is about. There are three label families, nested from coarse to fine:

1. is_adversarial - the binary verdict. 1 for an adversarial account of any stripe, 0 for an honest one. This is the headline number our binary task ultimately cares about, but note it carries no information the type label doesn't - it's a pure collapse of attack_type, where anything outside the benign.* archetypes folds down to a 1.
2. attack_type - the per-account type. The surface a given account is actually working: benign.{casual | power_user | developer | enterprise | researcher}, or one of distillation | harmful_use | dos | jailbreak | credential_abuse | data_extraction | agentic_misuse | bot_farm | cover_traffic. Each adversarial account is enlisted for exactly one surface (recall our monogamy assumption), and this is the label our multiclass head chases. Two wrinkles worth remembering from the sim: during WARMUP/SETUP an adversarial account falls back to a benign archetype (it hasn't started misbehaving yet), and inside a multi_surface ring an account can play a cover_traffic role that diverges from what the ring is really doing.
3. campaign_type - the per-ring type. The surface of the whole coordinated operation the account was recruited into: same value space as above plus multi_surface, and simply absent for organic benign accounts with no campaign affiliation. Several accounts map up to one campaign, so this is a group property - out of scope for our per-session predictor, but we carry it so that a higher-rung model can one day aggregate across accounts and recover the ring.

Axis 2, Ocularity: how cleanly you get to s ee it. Here is where synthetic data is awesome to work with! In the real world you never observe a label directly - you observe it through some evidentiary channel that is late, lossy, or lying. We emit three, ordered by how cruel the world has been to them. I call them M3 → M2 → M1, in decreasing order of oracularity (and increasing order of realism).

1. M3 - Oracle. The ground truth the simulator knows by construction: attack_type, campaign_type, is_adversarial, the exact attack_phase, and the latent sophistication knob. Available the instant a session is sampled - no noise, no delay, no ambiguity. In a real Trust & Safety setting M3 simply does not exist; you cannot run a frontier lab on counterfactual omniscience (unless you happen to be a Soul Transformation Dao Lord holding the right heaven-reading rank-5 artifact). So M3 is evaluation-only - it's the answer key, never a training input.

2. M2 - Investigation-confirmed ("golden"). Strip away the latent fields (sophistication, exact phase boundaries) and keep only the coarse label - is_adversarial ∈ {0,1}, attack_type ∈ {distillation, jailbreak, …, benign} - but pay for it with a realistic investigation delay $\tau \sim \text{Lognormal}(\mu_d, \sigma_d)$, typically 14–90 days between a session firing and a human analyst confirming the verdict. M2 lives in the label / campaign_label columns. This is the golden dataset a mature T&S team accretes over years: clean, trustworthy - and late. Evaluation-only for us as well.

3. M1 - Operational signal. Now strip the investigation channel too. All you are left with are the policy outputs of the system itself: an account got suspended or it didn't - plus a label-noise model bolted on top. False positives from trigger-happy policies, false negatives from attackers who exfiltrate and vanish before any rule fires, misattribution when the wrong surface gets blamed, and the same ever-present delay. M1 lives in noisy_label / noisy_campaign_label. This is what a T&S team actually holds most of the time, and the only channel that scales without setting fire to analyst-hours (or, these days, token quota). M1 is what we train and evaluate on.

Laid out as a grid, the whole labelling scheme is as follows:

Channel	Binary verdict	Attack type (per account)	Campaign type (per ring)
M3 - Oracle (no delay, no noise)	is_adversarial	attack_type	campaign_type
M2 - Golden (+ investigation delay)	(derived)	label	campaign_label
M1 - Operational (+ delay + noise)	(derived)	noisy_label	noisy_campaign_label

The information-loss hierarchy

Because each row is a stochastic function of the row above it - M2 coarsens and delays M3, M1 corrupts and delays M2 - the chain forms a Markov chain $\text{M3} \to \text{M2} \to \text{M1}$. And the moment you say "Markov chain", the data-processing inequality hands you a hard, first-principles ceiling on how much any model could ever recover about the true adversariality $Y$, no matter how clever:

$$\underbrace{I(Y;\,\text{M3})}_{\text{full ground truth}} \;\;\ge\;\; \underbrace{I(Y;\,\text{M2})}_{\text{coarsened, delayed}} \;\;\ge\;\; \underbrace{I(Y;\,\text{M1})}_{\text{noised, operational}}$$ Why three? A real T&S team is rarely handed a clean (features, label) pair. Labels arrive late, partially, and through wildly different evidentiary channels - a suspended account today might be confirmed-adversarial next month by a fraud team, or reinstated as a false positive. So we emit three label variants per run, ordered by how cruel the world is to the modeller. Alas, the Dao is cruel and ruthless!

The practical payoff of carrying all three is that the gap between them is itself the most honest number in the whole study. Modelling against M3 tells you the ceiling - the irreducible Bayes error once the world is fully observed. Modelling against M2 tells you what a good T&S team could plausibly achieve given enough analyst-hours. Modelling against M1 tells you what you can ship today, with the labels you actually have - and, by subtraction, exactly how much performance you're leaving on the table by not investing in a labelling pipeline. We'll report numbers against all three in the results.

A note on how the noise and delay are actually applied

It would be tempting to generate M1 by flipping labels i.i.d. per session - and it would be wrong. The moment a single session of an account is annotated, that verdict recontextualises the entire account. A real analyst doesn't rule "Tuesday's session was a jailbreak but Wednesday's was fine" - they rule that the account is bad, and that judgement colours everything the account has ever done. Per-session dice-rolling destroys exactly the cross-session consistency that a real labelling process imposes.

So our corruption is an account-level confusion roll: one draw per account - not per session - from a confusion distribution (hand-tuned per surface, in the same spirit as the Lognormal delay from earlier), deciding whether and how that account's labels are corrupted: benign flagged as adversarial (FP), adversarial missed entirely (FN), or one attack type misread as another (misattribution). Whatever the roll decides is then applied consistently across every session that account owns.

The mirror image of consistent noise is consistent backfilling. When an account is finally confirmed bad, a real T&S team doesn't just tag the one session that tripped the alarm - they walk back through its history and relabel. But how far back is not uniform, and getting this right is what keeps the dataset both realistic and free of temporal leakage. The pivot is whether a surface is always-adversarial - malicious from its very first session, $\texttt{ALWAYS\_ADVERSARIAL} = \{\texttt{bot\_farm},\,\texttt{credential\_abuse},\,\texttt{data\_extraction},\,\texttt{distillation},\,\texttt{dos}\}$ - versus a surface that can plausibly look innocent early (a jailbreak prober who warms up looking like a curious researcher). The rule the generator follows:

Account fate	Surface	What gets labelled
Suspended / revoked	always-adversarial	All sessions backfilled - it was intentional from day one
Suspended / revoked	not always-adversarial	Only the session that triggered suspension - the earlier ones were genuinely ambiguous
Active, label lands late	always-adversarial	All sessions backfilled once the investigation retroactively confirms them
Active, label lands late	not always-adversarial	Only sessions from label_arrival_day onward - preserve temporal honesty
Never confirmed	any	Unlabelled - the adversarial accounts that simply slip through

The whole point of that split is temporal integrity. An always-adversarial account was bad by construction from day one, so backfilling its entire history leaks nothing - those early sessions really were malicious. But a not-always-adversarial account genuinely was ambiguous early on, so we only label it from the confirmation day forward; pretending we "knew" on day one would be leakage from the future, and would flatter every offline metric into uselessness.

Which lands us cleanly at the split protocol we'll use throughout:

• Train / Val - M1 only. noisy_label, with the account-level consistent noise and honest backfill above. You train on the labels you'd genuinely have had at the time.
• Test - ground truth. attack_type (and the M2 label where delay is the thing under study). You grade against the truth, even though you never got to train on it.

Feature Engineering

The raw dump you perused earlier is a session-level firehose - dozens of columns, oracle and noisy signals sitting side by side, benign and adversarial traffic all interleaved. Our job now is to distil that into a clean numeric matrix $\mathbf{X}$ that a gradient-boosted tree can chew on, and the single principle governing every transformation below is this: stand at day $t$, wearing only the defender's eyes - no oracle, no crystal ball - and build features out of what you could genuinely have seen from that vantage. Everything that follows is that one constraint, applied different ways.

First, we clip the horizon at both ends. Recall the caveat from the raw dataset - every account spawns straight into setup with no burn-in, so the opening ticks of the run sit in a cold-start transient rather than a steady state. The tail of the horizon has the opposite defect: a session near the final day hasn't had time for its 14-to-90-day investigation lag to elapse, so its M1/M2 labels still read benign purely because the verdict hasn't arrived yet - a right-censoring artefact of the delay. So we discard a burn-in head and a burn-out tail and keep the steady-state middle band,

$$n_{\text{burn\_in}} \;\le\; \text{day} \;\le\; n_{\text{burn\_out}}$$

which we set to $40$ and $320$ respectively. The head clip drops the warm-up transient; the tail clip is a label-delay backstop, keeping us from training on a session whose benign label is just an un-arrived verdict in disguise.

Second, we materialise the label grid into concrete columns. The M3 → M2 → M1 hierarchy has so far lived as a conceptual table; here it becomes nine actual columns - one triplet (is_adversarial, attack_type, campaign_type) per channel, read straight off the mapping we already laid out. The construction rule is uniform across all three: a null in the source label means "no confirmed adversariality on this channel," so is_adversarial is just an is_not_null cast to an integer, while the attack-type and campaign-type strings get their nulls filled with the literal benign.

Third, we compute per-account rolling history features. A single session is nearly information-free in isolation (the argument we made when we chose the user-session as our unit); the discriminating signal lives in the trajectory. A bot farm's refusal count creeps up day over day; a distillation ring's domain coverage marches upward; a jailbreak prober's safety scores drift as it warms up. To hand the tree that temporal shape without reaching for a recurrent model, we summarise each account's recent past into a trailing seven-day window. Let

$$W^{(7)}_{u,t} \;=\; \bigl\{\, s : u_s = u,\;\; t - 7 < d_s \le t \,\bigr\}$$

be the set of sessions belonging to account $u$ that fall in the seven days ending on - and inclusive of - day $t$. Each rolling feature is a simple aggregation over that window,

$$\operatorname{mean}_{u,t}(x) \;=\; \frac{1}{\bigl|W^{(7)}_{u,t}\bigr|}\sum_{s \in W^{(7)}_{u,t}} x_s, \qquad \operatorname{sum}_{u,t}(x) \;=\; \sum_{s \in W^{(7)}_{u,t}} x_s,$$

and likewise for running maxima and counts, applied to a handful of base signals - e.g. session volume, average request and token counts, the noisy safety / timing / coverage measurements, accumulated refusals, intervention histories, and so on.

The constraint that does all the work here is $d_s \le t$: the window is strictly trailing. It reaches back over the last seven days and stops at today, never summing in a session from tomorrow. This is the same temporal-integrity discipline we imposed on label backfilling, now applied to features - a prediction for day $t$ is assembled only from evidence a defender would have held by the close of day $t$.

Fourth, we settle on the allowed feature set - and the exclusions here are as important as the inclusions. It helps to think of the observable columns in tiers by how much machinery stands between the raw event and the number:

• Tier 1 - fully observable. Counts and hard facts the platform logs directly, no inference required: n_requests, session_duration_min, total_tokens, output_input_ratio, the inter-request timing mean/std, the is_api / is_datacenter_ip / is_vpn flags, n_refusals.
• Tier 2 - model-derived. Signals that are themselves outputs of an upstream model - prompt perplexity, safety scores, query-structure entropy, tool-use rate, domain-coverage delta. Here we deliberately take the noisy variants (noisy_avg_prompt_perplexity and kin), because in production the defender only ever sees the noised measurement, never the oracle value.
• Tier 3 - derived / aggregated. Cross-account graph signals like noisy_shared_ip_count, noisy_shared_device_count, registration_velocity, concurrent_sessions - again taken noisy.

These make up the base feature set, and the final training matrix is that set plus the seven-day rolling block from above.

Left out entirely is the family of surface flags - template_query_pattern, divergence_probe_pattern, token_flood_pattern, and the rest. These are highly oracular from our synthetic generator and unlikely to be available in meaningful ways in the real world without a lot of preparation work and pre-processing, so we'll omit them to be fair; though they are useful for downstream diagnostic and analysis. A real Trust & Safety team holds the raw perplexity and entropy signals and has to learn the boundary a surface flag hard-codes, so we hold our model to the same bar and let it find those boundaries on its own.

Rounding it all up, here's the full pipeline from raw dump to training matrix at a glance:

Step	Transformation	What it does	Why
1	Clip horizon	Keep the burn-in/burn-out band ($n_{\text{burn\_in}}{=}40$ to $n_{\text{burn\_out}}{=}320$)	Drop the cold-start transient (head) and the censored tail whose labels haven't arrived
2	Materialise label grid	Fold the noisy_* / label / attack_type channels into m1/m2/m3 × (is_adversarial, attack_type, campaign_type)	Turn the M3→M2→M1 hierarchy into concrete train/eval columns
3	Rolling features	Trailing 7-day per-account aggregates ($W^{(7)}_{u,t}$, strictly $d_s \le t$)	Hand the tree behavioural trajectory without leaking the future
4	Select allowed features	Tier 1–3 (noisy) + geo + cumulative, plus the rolling block	Restrict to production-observable signals; exclude oracular surface flags

And that's the lot. Run all five passes over the raw firehose and out comes the dataset we actually train on - the same stratified sample you explored earlier, now almost ready to be trained on! Here's a preview of a snippet, and the full dataset can be found here.

Training Setup

With a clean numeric matrix $\mathbf{X}$ finally in hand, we can turn to the mechanics of actually fitting a model on it. Two properties of this dataset shape almost every decision that follows, Fellow Daoist: its sheer size - millions of sessions, far too many to hold in memory at once - and the sparsity of its labels, the same brutal skew we've been circling since the very beginning. Once the raw firehose is clipped, materialised, and rolled up, the adversarial prevalence across our three splits lands roughly here:

Split	M1 (operational)	M2 (golden)	M3 (oracle)
Train	1.53%	0.74%	2.77%
Val	1.35%	0.57%	2.61%
Test	1.47%	0.70%	3.22%

The columns fall in exactly the order the information-loss hierarchy predicts - the oracle channel M3 sees the most adversariality, the operational M1 rather less, since a good chunk of it is quietly missed or backfilled late - but the headline is that no matter which channel you read, barely one session in seventy is adversarial.

On the split itself. We don't split rows at random - that would let the same account (and worse, the same campaign) bleed across train, val and test, leaking the answer. Instead we use a time-population split: partition by account so no user appears in two splits, and cut along the time axis so training lives strictly in the past relative to validation and test. No overlapping users means no overlapping campaigns; the temporal cut means the evaluation is causal - we only ever grade the model on a future it was never shown.

This is not an unfamiliar bind - anyone who's worked the highly asymmetric domains of fraud, payments, or intrusion detection has stared down the same 1-in-many skew. In a plain binary setting the standard toolbox is well established: reach for prevalence-sensitive metrics that don't let the majority class flatter you (AUPRC over raw accuracy), apply a label-compensation knob like XGBoost's scale_pos_weight to rebalance the two classes, or swap in a loss that deliberately down-weights the easy majority, à la Focal Loss. All of these are variations on a single theme - stop the abundant negative class from drowning out the rare positive one.

Our task, though, is multi-class, so we have to generalise that theme from two classes up to nine. Concretely, we fit an XGBoost model under the multi:softprob objective - which emits a full probability simplex $\hat{\mathbf{y}} \in \Delta^{8}$ over the nine classes rather than a hard label - and track mlogloss (multi-class cross-entropy) as the metric that governs early stopping. The scalar scale_pos_weight knob no longer applies in more than two dimensions, so in its place we bring our own per-sample weight vector, a small augmentation that reshapes the loss so the model stops under-valuing the rare surfaces.

Weights are a training-time device, and only a training-time device. This is the single most important principle to keep straight, and it's worth stating from first principles: a sample weight changes the loss - and therefore the gradient - so that during fitting the optimiser can no longer minimise its objective by simply ignoring the rare classes. That is its entire job, and its job ends the moment training does. At evaluation time you want the opposite: an honest picture of how the model behaves on data as it actually arrives in production, where the skew is real and the rare classes really are rare. Re-weighting the validation set would distort early-stopping and turn every val metric into a description of a fictional, balanced world that will never exist at serving time.

• Train streams through a weighted iterator - the weights are live, shaping every gradient step.
• Validation & Test streams through a plain, unweighted iterator - so mlogloss stays honest and early stopping triggers on the real distribution, not an imagined one.

So the weights live entirely inside the training stream and are invisible everywhere else. The only question left is how to set them. Our scheme is three moves stacked on top of each other.

Move one - inverse-frequency weighting. Give each class $k$ a weight inversely proportional to how often it appears, so that a class the model sees a thousand times less often is pushed a thousand times harder per example. Wrapped with the smoothing exponent below, the weight for class $k$ is

$$w_k \;=\; \left( \frac{N}{K \cdot n_k} \right)^{\!\gamma}, \qquad N = \textstyle\sum_k n_k, \quad K = 9,$$

where $n_k$ is the count of class $k$, $N$ the total sample count, and $K$ the number of classes. The weight is then attached to each row by indexing this length-$K$ vector with that row's label, $w = \texttt{class\_weights}[y]$ - a single gather, cheap enough to do on the fly inside the streaming iterator.

Move two - the square-root smoothing $\gamma = \tfrac{1}{2}$. Setting $\gamma = 1$ recovers raw inverse-frequency, and on our skew that is a disaster: a surface that is $10{,}000\times$ rarer than benign would receive a $10{,}000\times$ weight, and a single ultra-rare class's handful of examples would hijack every gradient update, sending the loss into wild, exploding-gradient oscillation. Wrapping the ratio in a square root ($\gamma = \tfrac{1}{2}$) compresses that dynamic range - the same $10{,}000\times$ rarity now earns roughly a $100\times$ weight instead of $10{,}000\times$. It trades a sliver of minority emphasis for a great deal of training stability, which on severely imbalanced data is a bargain worth taking every time.

Move three - the benign clamp. Plain inverse-frequency has an ugly side effect: because benign is the majority, it is handed the smallest weight of all, and after every rare surface has been aggressively up-weighted, the dominant benign class can be effectively drowned out of the gradient entirely. A model trained that way stops learning where the benign boundary actually is and starts firing false positives at everything that twitches - catastrophic when benign is 98%+ of your traffic and every false positive is a paying customer you just slapped with friction. So we floor benign's weight to at least that of the heaviest non-benign class,

$$w_{\texttt{benign}} \;\leftarrow\; \max\!\Big(\, w_{\texttt{benign}},\;\; \max_{k \,\neq\, \texttt{benign}} w_k \,\Big),$$

ensuring the correction toward rare types never swings so far that the model forgets what normal looks like.

Step back, Fellow Daoist, and you might reasonably ask why we stop here at all - if up-weighting the minorities is good, why not push $\gamma$ all the way to $1$ and balance the classes perfectly, handing every surface exactly equal say in the gradient? Because perfect balance is a luxury our labels can't pay for. Full inverse-frequency trusts each session of our rarest surface as heavily as a thousand-odd benign ones - and recall which labels we're actually training on: the noisy, operational M1 channel, riddled with the false positives, missed attacks, and misattributions we so carefully baked in earlier. Bet the model that hard on so few rows and it no longer learns the rare surface, it memorises it - quirks, sampling accidents, mislabels and all - amplifying every unit of label noise by the very same factor to amplify the signal. That is the deeper reason the $\sqrt{\cdot}$ smoothing and the clamp aren't afterthoughts bolted onto raw inverse-frequency; the goal was never to equalise the classes, only to make the rare ones learnable without letting a handful of noisy examples, or one over-eager class, seize the wheel and dictate the entire fit. We temper toward balance; we do not chase it, and thus walking the Great Dao.

The cleanest way to see what these three moves do is to lay them side by side. The figure below takes one fixed, imbalanced class distribution and shows the resulting per-class weights at each stage of the recipe, one column per stage - plain inverse-frequency, then with the benign clamp added, then with the $\sqrt{\cdot}$ smoothing on top. Each bar is a class's weight multiplier, printed at its end and scaled to the heaviest weight in that column.

Plain inverse-frequency (row 1) starves benign - its bar is essentially flat on the floor while the rarest surfaces tower, and the spread runs to a thousandfold. Adding the clamp (col 2) rescues benign in a single stroke: its bar snaps up to match the heaviest surface; this is of course, a matter of choice and design on my own whim, but the intuition is obvious and empirical results promising; I'll leave the theoretical grounding aside for the theoreticians in the crowd! Finally, adding $\sqrt{\cdot}$ smoothing (col 3) does the final tempering - the whole distribution flattens toward even, the tallest and shortest bars pulled closer still, so no single surface can bully the gradient, ensuring a stable training process.

Results & Evaluation

We have a trained model, now we have to find out whether it's any good and is it actually useful to a T&S team of Five Elements Inc.

Question, is this malicious?

Here's how a straight binary is_malicious evaluation fares on the test split with respect to the levels of ocularity:

Channel	Prevalence	AUROC	AUPRC	Precision @1% FPR	Recall @1% FPR
M1 - operational	1.47%	0.842	0.313	0.360	0.379
M2 - golden	0.70%	0.994	0.609	0.360	0.801
M3 - oracle	3.22%	0.997	0.993	0.767	0.993

Against the oracle M3, the problem is very nearly solved - an AUPRC of $0.99$ and a recall of $0.99$ at an impressive $1\%$ false-positive budget, meaning the model catches essentially every adversary while barely grazing a good user. Against M1 - the most abundant type of labels a real Trust & Safety team have by default - the same model looks decidedly mediocre, an AUPRC of $0.31$, which makes the developer in charge sweat bullets and start doubting their competency. M1 is simply M3 seen through a fog of delayed, noisy, half-missing verdicts, so grading against M1 doesn't measure how good your model is - it measures how lossy your labels are. The model is, quite literally, better than your operational labels - Fellow Daoist, beware of bad labels!

Thankfully in our hypothetical Five Elements Inc., we have the privilege of M2 & M3 to guide us; the qualitative gap between the M1-M2-M3 evaluation results are worth chewing on as it reveals the importance of a Golden Dataset, costly as it may be, and a reminder why XGBoost is still the undisputed King amongst practitioners.

The results on M3 are fantastic, but note that our synthetic world is tidier than the real one, so read these as an optimistic ceiling instead.

Now let's take it up a notch, and look at how well it performed in terms of pinning down the exact attack type - benign is just one of many faces, and knowing what kind of bad actors we're dealing with, is key to applying the correct intervention and routing the cases to the right team downstream. For example, here's a breakdown of how things work at Five Elements Inc.

Attack Type	Routed to	What a miss actually costs
distillation	Legal & Policy	a competitor ships your exfiltrated capability
data_extraction	Legal & Policy	training-data, PII, or system-prompt leakage
jailbreak	Trust & Safety	a safety bypass loose in the wild
harmful_use	Trust & Safety	real-world harm - extortion, malware, RaaS
agentic_misuse	Trust & Safety	your own tools turned into an attack platform
bot_farm	Platform Integrity	a fake-account fleet scaling unchecked
dos	Platform Integrity	unbounded consumption, token flooding
credential_abuse	Security	stolen-key / reverse-proxy LLMjacking
multi_surface	Threat Intelligence	a coordinated multi-vector campaign
benign	No action	(the flip side: a good user slapped with needless friction)

Attack Type Prediction with Naive Policy

Now the table below, provides us the result on the multi-class evaluation and note the significant discrepancy between the sparkling Top-N metrics and the abysmal Macro-acc number. Also, for context, here we're simply taking the $\text{argmax}(\cdot)$ of the class probability vector as a naive determination policy.

Channel	Top-1	Top-3	Macro-acc	Log-loss	Routing
M1 - operational	0.988	0.997	0.194	0.067	0.989
M2 - golden	0.997	1.000	0.244	0.021	0.997
M3 - oracle	0.983	0.998	0.247	0.058	0.983

Top-1 accuracy of $0.98$, routing accuracy of $0.98$ - ship it, surely? Under a nine-way softmax we take the $\text{argmax}$ - which has no threshold to tune and therefore unbiased, but as a downside, it's prior-sensitive. When benign is $98\%$ of everything, the majority class wins every close call by default, so a model can post a gorgeous top-1 just by betting on benign uniformly. As such, we need to lean on the prevalence-agnostic Macro-acc in this case, which gives a more meaningful read on the model's performance which looks pretty bad and downright trash at a glance; though there's still a noticeable M1-M2-M3 ocularity gap.

To see what's really happening we have to break the aggregate open and look class by class:

The figure above sets a class's ranking beside its decision. PR-AUC here is calculated in a piece-wise one-vs-rest manner so it measures whether the model can surface a class's true members above everyone else across the spectrum, and the F1 bars next to it is measured on the naive argmax policy's output on the final predicted attack type class. The results can be roughly grouped into 3 performance-bins:

Good Ranker + Good Policy Compatibility. benign`` (F1 $0.99$),dos($0.62$),bot_farm($0.48$),credential_abuse` ($0.45$) falls into this bin - here ranking and decision agree. The model separates these surfaces well and argmax picks them well enough. These are the classes with loud, distinctive signatures and they're the ones a T&S team could act on immediately with swift thunderous might!

Good Ranker + Bad Policy Compatibility. These are the ambiguous lot. Look at distillation (PR-AUC $0.79$, argmax-F1 $0.00$) and jailbreak (PR-AUC $0.88$, $0.01$): the model ranks these beautifully - it plainly knows what they look like - yet the realized performance from the policy's determination is absolutely horrible! There's something more to the model here, a PR-AUC of $0.88$ is proof it separates them superbly, but these rare classes are unable to out-score benign's massive prior to win in the $\text{argmax}$ filter.

Feckless All Around. harmful_use (PR-AUC $0.25$), data_extraction ($0.14$), agentic_misuse ($0.01$) are classes that the model truly struggles with. For these classes, we either do not have sufficient samples of them in our dataset or engineered feature set is insufficient on capturing the existing signal in the samples. These are the classes that need more data or methods from a higher rung of the ladder, not a cleverer cut.

For completeness sake, here are the Precision-Recall curves of each class as well for your reference - lest I be slandered with negligence of diagnostic-essentials!

A Smarter Approach with Smarter Policies

So we retire the naive argmax and come up with something smarter by introducing finer controls in the form of per-class thresholds for determination since each attack type has a different prevalence and hence, a different sensitivity in the final predictions. Of course, this introduces a new problem - how can we decide, amongst those that manages to clear the threshold, which class is the correct choice? To deal with this, we introduce a simple decision engine, with two main levers that can be used to generate a set of varying policies.

The first: how do we crown the winner from a field of contenders? Take the largest by plain probability via argmax again, only this time it's over a shortlist rather than all nine classes or, among the classes that actually cleared their own per-class threshold, and then take the one that cleared it by the widest margin, in raw terms (the absolute lift, $p_k - t_k$) or relative to the bar itself (the relative lift, $p_k / t_k$, fairer when the bars sit at such different heights that a raw margin means little). The second lever: do we treat benign differently from the rest? Specifically, either it competes for the win like any other class, or as the overwhelming default with higher prevalence, it's excluded from consideration unless it clears a markedly higher threshold, so a genuinely suspect session is given a fair looked at with a scrutinizing gaze.

Since I'm not sure what's the best configuration is, I have thus assembled a suite of 6 other policies (in addition to the naive one), and then applied them in evaluation to decide heuristically which is the best. Note that all the tuning of thresholds below are done on the validation set using the noisy M1 labels to keep things fair.

Let $p = (p_0, \dots, p_8)$ be the softmax vector over $\mathcal{C}$, let $t_k$ be the tuned threshold for class $k$, and write $b = 0$ for benign. Everything hinges on the contender set - the classes that cleared their own bar:

$$\mathcal{K}(p) \;=\; \{\, k \in \mathcal{C} \;:\; p_k \ge t_k \,\}$$

Whenever a policy is left with an empty contender set it abstains to benign. With that in hand:

1. Absolute Lift. Every class competes, and the winner is the one that overshot its own threshold by the widest raw margin.

$$\hat{y} \;=\; \arg\max_{k \in \mathcal{K}(p)} \;\bigl(p_k - t_k\bigr)$$

Because the margin is measured in probability mass, a class with a high bar needs a correspondingly high probability to win - which quietly favours the classes that were already confident.

2. Relative Lift. The same contest, but the margin is scaled by the bar itself, so clearing a low threshold twice over beats clearing a high one by a hair.

$$\hat{y} \;=\; \arg\max_{k \in \mathcal{K}(p)} \; \frac{p_k}{t_k}$$

This is the fairer comparison when thresholds sit at wildly different heights, and it's what gives the rare classes - whose bars are necessarily low - a real chance at the win.

3. Benign Floor → Argmax. Here we pull the second lever: benign is no longer an ordinary contender but must clear a deliberately high floor $\tau_b = 0.65$ to be believed at all. If it fails, it's struck from the ballot and the plain argmax runs over the remaining classes.

$$\hat{y} \;=\; \begin{cases} b, & p_b \ge \tau_b \\[6pt] \displaystyle\arg\max_{k \in \mathcal{C} \setminus \{b\}} p_k, & \text{otherwise} \end{cases}$$

This essentially bakes in an explicit prior or forcing function on establishing a hard decision boundary to tilt the scale away from the majority class.

4. Benign Floor → Absolute Lift. Identical in spirit, except that once benign is dismissed the survivors are ranked by raw margin rather than raw probability, with the floor tuned to $\tau_b = 0.60$.

$$\hat{y} \;=\; \begin{cases} b, & p_b \ge \tau_b \\[6pt] \displaystyle\arg\max_{k \in \mathcal{K}(p) \setminus \{b\}} \bigl(p_k - t_k\bigr), & \text{otherwise} \end{cases}$$

5. Benign Floor → Relative Lift. The most aggressive configuration in the suite: benign must be near-certain ($\tau_b = 0.95$) to carry the session, and the set it leaves behind is settled on scaled margin.

$$\hat{y} \;=\; \begin{cases} b, & p_b \ge \tau_b \\[6pt] \displaystyle\arg\max_{k \in \mathcal{K}(p) \setminus \{b\}} \frac{p_k}{t_k}, & \text{otherwise} \end{cases}$$

Both levers are pushed to their limits at once - the majority class held to a punishing standard, the minority classes compared on terms that don't punish their rarity.

6. Meta-Stacker. The one policy that isn't hand-written, and instead learned via a smaller meta-model - here I use another XGBoost instance - to enable a higher-order decision boundary.

$$\hat{y} \;=\; \arg\max_{k \in \mathcal{C}} \; g_k\bigl(p,\; p - t\bigr)$$

Elegant in principle, and it does recover most of what the tuned rules find - but it's a second model to train, calibrate, and defend, which matters a great deal when the rarest classes have only a few dozen validation examples in our case.

The final results are quite promising! Take the strongest of them, #5: Benign Floor → Relative Lift where the macro-F1 climbs from $0.295$ to $0.489$. Rescuing the rare tail is supposed to cost you the head - you buy minority recall with majority precision - yet here no bill came due, because argmax had been quietly misfiling enough rare attacks as benign, and we can see evidence of that by the positive $\delta$macro-F1 across the board!

Decision rule	Accuracy	Macro-F1	Macro-recall	Routing	Δ Macro-F1
argmax (baseline)	0.983	0.295	0.247	0.983	-
absolute lift	0.984	0.324	0.267	0.984	+0.029
relative lift	0.987	0.422	0.407	0.989	+0.127
benign floor → argmax	0.984	0.358	0.316	0.985	+0.063
benign floor → absolute lift	0.984	0.321	0.271	0.984	+0.026
benign floor → relative lift	0.992	0.489	0.520	0.996	+0.194
meta-stacker	0.988	0.435	0.445	0.991	+0.140

Digging into the per class performance, we can see that key attack vectors such as distillation improved significantly, with argmax-F1 jumping from $0.00$ to $0.65$; notably, jailbreak also posed a significant improvent going from $0.006$ to $0.83$. Here's a graphical breakdown of the per class results, for those more visually inclined like yours truly!

Toggling through the above though, we can see that there are indeed finer trade-off at the class level - alas, there is no free lunch afterall!

1. That same aggressive champion, #5: Benign Floor → Relative Lift saw dos slips from $0.616$ to $0.513$, and credential_abuse detection deteriorated as well alongside bot_farm detection.
2. On the flip side, looking at #3: Benign Floor → Argmax and the trade inverts. It lifts the middle (bot_farm $0.475 \to 0.695$, credential_abuse $0.451 \to 0.608$) while barely touching the buried rankables it hasn't the nerve to fire such as distillation.
3. There is also a wall none of them climb. Watch agentic_misuse and data_extraction as you flip between policies - they barely twitch, topping out near $0.042$ and $0.244$ across the policies set.
4. The one happy exception makes the point from the other side - harmful_use, whose ranking looked merely middling ($0.25$), still yields a respectable $0.47$ at its best.
5. For the #6: Meta-Stacker, it did quite well! With a macro-F1 of $0.44$, and the steadiest hand across the nine classes of any option here. Perhaps this will be the preferable approach in the case where labels are more abundant, and the complexity of having a meta-model is not a concern.

In short, no single rule is best everywhere; each is a different point on one curve, trading the comfortable classes against the desperate ones.

Quick Look at Feature Importance

Before we close, a quick look under the hood via a SHAP reading to see which features the model leaned on, to understand if it had learned something sensible after all.

At a glance, it seems pretty reasonble to me! The heaviest hitters are request volume (the seven-day mean request count towering over all else), cumulative domain coverage - an obvious signal for distillation rings, where they methodically farming the model's knowledge, precisely as per the mechanic we baked into our dataset via the SCM - and a cluster of infrastructure-graph signals such as registration velocity, shared devices and IPs, datacenter and API flags, which are the tell-tale indicators for coordinated fake-account fleets. Timing cadence and safety-score peaks round out the top of the list. In other words, the model did manage to learn something sensible! It recovered certain heuristics and priors we've programmed into the SCM; I'll set aside the question of circularity here for the exercise.

---

Closing

Well that's been a pretty lengthy journey, I'd hope it's a been worthwhile read (or at least, perusal) for you, Fellow Daoist See you on the next one :)

References