> ## Documentation Index
> Fetch the complete documentation index at: https://docs.datafog.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Tokenization and restoration

> Issue opaque reversible tokens through an application-owned provider and restore them atomically.

Tokenization is the reversible privacy operation. DataFog Core defines request
validation, provider batching, a canonical token envelope, and atomic text
mutation. Your provider owns storage or reversible cryptography, authorization,
lifecycle, retries, and audit logging.

## Configure tokenization

```json theme={null}
{
  "default": {
    "strategy": "tokenize",
    "token_ref": "customers/default"
  }
}
```

Selected tokenization and every restoration request require an exact,
case-sensitive request scope:

```json theme={null}
{ "scope": "tenant-a" }
```

## Provider contract

A token provider implements two asynchronous batch methods:

* `tokenize_batch(scope, items)` / `tokenizeBatch(scope, items)`
* `restore_batch(scope, items)` / `restoreBatch(scope, items)`

Each tokenization item contains an opaque request `id`, the `exact_value`, and
the configured `token_ref`. Return the same `id`, opaque payload bytes, and a
concrete resolved profile version.

Each restoration item contains an `id`, `token_ref`, resolved version, and
opaque payload. Return the same `id` and restored value.

## Python round trip

```python theme={null}
import asyncio

from datafog_core import PrivacyManager


async def round_trip(token_provider):
    manager = PrivacyManager(None, token_provider=token_provider)
    context = {"scope": "tenant-a"}

    tokenized = await manager.scan_and_transform(
        "Email jane@example.com",
        {
            "transform": {
                "default": {
                    "strategy": "tokenize",
                    "token_ref": "customers/default",
                }
            }
        },
        context,
    )

    return await manager.restore(tokenized.text, context)


restored = asyncio.run(round_trip(token_provider))
assert restored.text == "Email jane@example.com"
```

## Node.js round trip

```javascript theme={null}
import { PrivacyManager } from "@datafog/node";

const manager = new PrivacyManager({ tokenProvider });
const context = { scope: "tenant-a" };

const tokenized = await manager.scanAndTransform(
  "Email jane@example.com",
  {
    transform: {
      default: {
        strategy: "tokenize",
        token_ref: "customers/default",
      },
    },
  },
  context,
);

const restored = await manager.restore(tokenized.text, context);
console.assert(restored.text === "Email jane@example.com");
```

## Atomic and non-recursive behavior

* Repeated source values are separate tokenization items and may receive
  different tokens.
* Identical envelopes are deduplicated before restoration provider calls.
* Every canonical token in the supplied text is restored, or no result is
  returned.
* Nested tokenization and recursive restoration are rejected.
* Partial restoration and ignore-failure modes are not available.

Token envelopes use the canonical
`DFTOKENv1(<body-length>):<ref>.<version>.<payload>` form with unpadded
Base64URL components. Treat the envelope as opaque application data; do not
parse or construct it yourself.

Browser/WASM rejects selected tokenization and every restoration call with
`unsupported_strategy`.