Detection quality
The model detects first names, last names, email addresses, phone numbers, street addresses, customer IDs, and account numbers. On 90,018 held-out synthetic Nemotron-PII documents, containing 248,739 annotated entities:
A prediction counts as correct when both its category and boundaries match the
annotation. Separate partitions were used for checkpoint selection, confidence
calibration, and threshold selection. The inference artifacts were frozen before
test scoring.
Results across all seven categories:
For email and phone numbers, the model was compared with the legacy DataFog
Python 4.8.1 regex engine. These were the two categories supported by that
baseline within the seven-category evaluation:
These results describe performance on this synthetic dataset. Structural
validation removed malformed annotations, but semantic annotation errors remain.
Generalization to real customer-support data has not yet been established.
A separate set of seven hand-authored support fixtures recovered 14 of 18
entities. Those examples exposed failures that the aggregate score alone
would not convey.
Local deployment
The model runs through a native Rust runtime and returns UTF-8 byte offsets into the original text. Long inputs are processed in overlapping windows. The deployment bundle is approximately 300 MB, including the inference runtime. On the development Mac, individual warm requests produced these measurements:
These are preliminary observations at four input lengths. Repeated latency
measurements and throughput testing are still needed.
Confidence calibration and the acceptance threshold are experimental.
Detection is limited to the seven listed categories.