> ## Documentation Index > Fetch the complete documentation index at: https://docs.xysq.ai/llms.txt > Use this file to discover all available pages before exploring further. # PII scrubbing > Strip identifying details out of what you push, at ingest, before anything is written to the log. Say you're pushing support tickets into a vault, and those tickets have names, emails, order numbers, all the stuff a real conversation carries. You want the vault to remember the *problem* (a firmware bug, a refund, a login issue) without remembering *who* reported it. That's what PII scrubbing does: it rewrites identifying details out of the content before the first byte is stored. ## Why ingest-time Every layer downstream of a push inherits whatever the push contained: the wiki pages distilled from it, the facts pulled out of it, the embeddings computed over it. And the raw log itself is immutable (it's the audit trail) so nothing written to it can be edited out later. That leaves one place to catch PII: the door. Scrub once, before the first durable write, and every derived layer is clean by construction. ## The vault toggle `pii_scrub` is a vault setting, off by default. Turn it on at create: ```python theme={"dark"} from xysq import Xysq with Xysq() as client: vault = client.vaults.create("customers", project_id=project_id, pii_scrub=True) ``` or later in the app, under the vault's Settings. Flipping it on only affects pushes from that point forward: existing content isn't rewritten, because (as above) the log is immutable. The Settings panel says this too, so nobody expects a flip to launder history. ## Pushing with `pii` and `known_pii` A push can override the vault default with `pii=True`, and it can hand over `known_pii`: the identifiers you already know because they came from your own system of record (a helpdesk, a CRM). This is close to free for most integrations and it makes the scrub deterministic instead of best-effort. ```python theme={"dark"} result = client.vaults.push( vault.vault_id, "user: Alice Chen (alice@example.com) reported that ticket #A-1042, " "her Widget Pro keeps flickering on startup.\n" "agent: escalated to firmware, will follow up.", known_pii=["Alice Chen", "alice@example.com", "#A-1042"], ) ``` What gets stored is neutral: ```text theme={"dark"} user: The customer reported that ticket , their Widget Pro keeps flickering on startup. agent: escalated to firmware, will follow up. ``` Sent, never stored: `"Alice Chen"`, `"alice@example.com"`, `"#A-1042"`. `known_pii` is transient input to the scrub, not a field on the source; it never lands in the log. Every sensitive match, deterministic or `known_pii`, collapses to the same literal token: ``. There's no per-kind phrasing (no `[email]` vs `[phone]`) because a typed label is itself a hint about what was there. Prose identity is different: the model pass rewrites a name into a gender-neutral noun like "the customer" or "they", not into ``, because the sentence still needs to read. If the scrub can't produce a clean result, the push fails closed: a `422` with the message `"content could not be scrubbed of personal data; nothing was stored"`, and nothing is stored (not even a pending row). A partially-scrubbed push is worse than a rejected one, so xysq doesn't do partial. ## How it works Three stages run inside the push, before the first durable write: * **Deterministic scrubbers.** Pattern-based detectors catch the list below and every `known_pii` string (case, whitespace, and punctuation don't matter). Replaced with ``. * **Model neutralization.** One model call rewrites remaining prose identity into neutral voice: "the customer", "they", never a gendered pronoun (gender is an inference, not a fact, and we don't ink inferences into the log). * **Deterministic gate.** The same detectors from step one re-run against the model's output. If anything survives, the model pass retries once; if it still fails, the push is rejected. The model rewrites, the gate guarantees. Titles go through the same deterministic pass and gate (no model call - a title is a handful of words, not worth the round trip). A title the deterministic pass can't fully clean rejects the push, same as content. ### The deterministic detector list This is the actual guarantee. It catches exactly this, no more: * **Emails** * **Phone numbers** with 10 or more digits * **Card numbers** that pass a Luhn check * **IPv4 and IPv6 addresses** * **IBANs**, mod-97 validated (not just IBAN-shaped) * **Every `known_pii` string you pass**, matched with case, whitespace, and punctuation normalized so `"Alice Chen"` also catches `"alice chen"` Anything on this list, or anything you named in `known_pii`, is deterministically absent from what's stored - the gate checks every push, so a survivor fails the push rather than shipping quietly. Everything else (a name you never passed in, a physical address, a nickname) goes through the model pass only: best-effort, not gate-verified. ## What's actually guaranteed Be precise about what "scrubbed" means here, because not every kind of identity is removable the same way: | Category | Guarantee | | ----------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------- | | The deterministic list above, and everything in `known_pii` | Deterministically absent. The gate verifies this on every push; a survivor fails the push, not the guarantee. | | Unknown prose identity (a name you never passed in `known_pii`) | Best-effort. The model pass catches most of it, but it's probabilistic, not gate-verified, since the gate only checks for what it can detect. | | Quasi-identifiers ("the only customer in Iceland with two units") | Not removable. No scrub can strip context that's identifying by elimination rather than by naming. We're not claiming otherwise. | That middle row is why `known_pii` matters: anything you can name, the gate can guarantee. Anything you don't name is left to the model's best judgment. ## Things worth knowing * **Thread turns are raw until flush.** `client.threads.append` writes to a working buffer verbatim; scrubbing runs when a flush promotes those turns to long-term memory, not when each turn lands in the buffer. See [Thread-level memory](/sdk/threads). * **A writer key can opt out per push.** `pii=False` on a single push bypasses scrubbing even on a `pii_scrub` vault. Read the promise as "masked unless a writer explicitly opts out", not "impossible to store raw". * **IPv4-mapped IPv6 is a known edge.** Forms like `::ffff:1.2.3.4` may be only partially masked. Don't rely on this shape being fully caught. * **Re-pushing identical content doesn't dedupe on a `pii_scrub` vault.** Dedup keys off the stored (scrubbed) hash, and the model rewrite isn't deterministic, so two pushes of the same raw text can scrub to two different strings and land as two sources. * **Binary uploads are rejected on a `pii_scrub` vault.** There's no text to scrub before storage, so a file push comes back `422`. Push text content instead. ## Best practices * **Pass `known_pii` from your system of record.** If your helpdesk knows the customer's name and email, hand them over. It's the difference between "guaranteed absent" and "best-effort". * **Use opaque customer ids in metadata, not emails.** A `customer_id` metadata value that's an internal id, not `alice@example.com`, keeps the metadata layer as identity-free as the content layer. * **Keep identity in metadata, semantics in content.** The content should read like it's about anyone; anything that has to tie sources back to a specific customer belongs in a declared metadata key, not in prose. ## Works with filters A `pii_scrub` vault still supports [metadata filters](/sdk/metadata-filters) and [tags](/sdk/tags) as usual. Scrubbing changes what the content *says*, not how you search it.