Source-Traced Compliance: Lex as Part of the Regulation

A Number Without a Source

A data satellite holds the statutory values of one parameter domain for one cycle: allowances, rates, ceilings, thresholds. Wage types read them through lookups, and integration tests prove that the calculation applies them correctly. What the tests cannot prove is that the values themselves are right. A ceiling typed in with one wrong digit is applied consistently, passes every test and produces a precisely wrong payslip — the gap described in Data Regulation Satellites.

After a few annual cycles, four questions usually have no reliable answer: Where exactly does this value come from, and which legal norm defines it? Was it checked against the source — by whom, and when? Has the source changed since the value was taken over? Is every statutory value of the satellite documented at all?

Lex answers these questions with one rule: every compliance-relevant value has a traceable, frozen source. It is part of the open-source Payroll Engine: a manifest format, lex.json, and a group of Payroll Console commands that cover the life of a satellite from the first source document to the next year’s rollover. The full command set is available from Payroll Engine 1.2.0.

The .lex Folder

A satellite keeps its lex in a .lex folder next to its Regulation folder. The example below is a fictitious tax satellite; the structure is the same for every jurisdiction:

Data.Tax.2026/
├── .lex/
│   ├── lex.json              # manifest: sources, values, files, exclusions
│   ├── snapshot.json         # generated by LexSnapshot, never edited
│   ├── tax-guide-2026.pdf    # evidence files
│   └── tax-rates-2026.html
└── Regulation/
    └── *.json                # regulation with lookups

Because the evidence is stored in the folder, a satellite can be verified without network access — years later, on an auditor’s laptop, or in a build agent without internet.

The Manifest: Sources, Values, Files, Exclusions

The manifest names the jurisdiction (authority), the satellite (regulation), its effective date (validFrom, which must match the regulation) and the languages its sources are written in (languages). Four lists carry the content:

{
  "$schema": "https://payrollengine.org/schemas/lex/v1.json",
  "authority": "XX",
  "regulation": "Data.Tax",
  "validFrom": "2026-01-01",
  "provider": "Tax authority",
  "languages": ["en"],
  "sources": [
    {
      "url": "https://example.org/tax-guide-2026.pdf",
      "description": "Employer tax guide 2026",
      "legalBasis": "Income Tax Act s. 12",
      "review": "FourEyes"
    }
  ],
  "values": [
    {
      "key": "TaxParameter.BasicAllowance",
      "content": "12000",
      "unit": "EUR",
      "sourceUrls": ["https://example.org/tax-guide-2026.pdf"],
      "sourceLocation": "Employer tax guide 2026, page 4, section 2.1",
      "legalReference": "Income Tax Act s. 12(1)",
      "documentRef": "tax-guide-2026.pdf",
      "description": "Basic tax-free allowance per year"
    }
  ],
  "files": [
    {
      "fileName": "tax-guide-2026.pdf",
      "sourceUrls": ["https://example.org/tax-guide-2026.pdf"],
      "description": "Employer tax guide 2026"
    }
  ],
  "exclusions": [
    {
      "kind": "coverage",
      "pattern": "TaxParameter.RoundingMode",
      "reason": "Technical setting, not a statutory value"
    }
  ]
}

How an entry maps to the regulation

The key has the form Lookup.Key. The entry above covers the value with key BasicAllowance in the lookup TaxParameter of the satellite. A further key part addresses a property of a value object (Lookup.Key.Property), so a lookup that stores a JSON object per year can be documented field by field or as one structured entry.

Small values are stored inline in content — a scalar or a compact JSON object. Large tables, such as a full tariff, are referenced with contentFile; such an entry covers its whole lookup and the file is copied into the .lex folder like any other evidence.

Where the authority publishes the tariff itself as a text file — fixed columns or tab-delimited, often one file per region inside a ZIP archive — nothing is retyped. The entry points to the file in the archive (tariffs-2026.zip!tariff-a26.txt) and a small map in contentMap names the columns and the record types to take over. The authority file stays the evidence: it is verified by file identity against its declared download URL and frozen by the snapshot hash, and the lookup with tens of thousands of rows is generated from it.

Where in the source

A URL alone rarely answers an auditor. sourceLocation is free text that pins the value down: a page and section of a PDF guide, a table and row of a gazette, a margin number of a circular, a named table on a government web page. The more precise the location, the faster a second person can confirm the value. Where a page holds several similar figures, sourceQuote adds the exact text passage that contains the value — for example one and one-half times the regular rate — and anchors the automatic verification to it.

Documented exceptions

Exclusions record deliberate gaps instead of hiding them. Besides coverage (a technical lookup value that is not statutory), orphan (a rule implemented in a script) and sourceLocation, three kinds address the verification: autoVerification keeps the tool away from an entry whose number is in the source but means something else; derived marks a property of a bundle that is computed from other values, with the formula and legal reference as reason; and notApplicable justifies a property with the value 0 because the rule does not apply in the jurisdiction. A zero without such a reason is reported, so a rate set to 0 by mistake does not stay unnoticed.

The Lifecycle at a Glance

Each step of a satellite’s life has its own command. All of them are regular Payroll Console commands, so they run the same way on a developer machine and in a pipeline:

Except LexRollover, which reads the payroll calendar of a tenant, the commands work on files only. The processing commands accept a path or file mask and walk a whole repository tree with /recursive.

Collecting Evidence and Recording Values

Work starts with the source document, not with the number. LexFetch downloads a document unchanged into the .lex folder and declares it in files. If the file already exists with the same content, it is kept; a different version is only replaced with /overwrite, so evidence is never swapped silently.

LexFetch https://example.org/tax-guide-2026.pdf
LexFetch https://example.org/rates tax-rates-2026.html "Description:Tax rates 2026"

LexAdd records a value. The value is read from the source by a person; the command checks the form: the key must be new and have the form Lookup.Key, structured content must be valid JSON, the source URL must be declared in sources and the document in files. Provenance fields that are not given are written empty — they show up later as findings instead of being forgotten.

LexAdd TaxParameter.BasicAllowance Content:12000 Unit:EUR SourceUrls:https://example.org/tax-guide-2026.pdf "SourceLocation:page 4, section 2.1" "LegalReference:Income Tax Act s. 12(1)" DocumentRef:tax-guide-2026.pdf

Verification Records: Auto, Manual, Four Eyes

An entry with a source is documented. It is not yet verified. Lex distinguishes the two explicitly: each entry can carry a verification record that states how the value was confirmed.

Automatic: LexVerifySource

LexVerifySource looks up the numbers of every entry in its source — by default in the local evidence file (/local, offline and reproducible), or with /live at the source URL. The result per entry is found, ambiguous (the numbers are there, but the evidence is weak), partial / not found (review the value), quote not found or unreadable / no local evidence. With /writelocations, the command fills in a missing sourceLocation and writes an auto record for every found entry; ambiguous and excluded entries are never written:

"verification": {
  "mode": "auto",
  "verifiedOn": "2026-11-03",
  "method": "LexVerifySource 1.2.0; languages en 3f2a9c1b"
}

Weak matches and the year of a value

A number found somewhere in a long document does not prove the value. The tool therefore rates a single number as ambiguous when the source writes it with very few digits (22%, 1.5), when a short number occurs in many sections, or when it is only found in words (one and one-half). Sources also often list several years side by side, so each hit is rated by the closest year in its text: a number found only next to another year is ambiguous with multi-year section. Validity phrases (effective January 1, 2026), citations of legal acts (Finance Act, 2025) and tax years that do not start on 1 January (FY 2025-26) are taken into account, in seven languages: English, German, French, Spanish, Italian, Dutch and Portuguese. A value that legally belongs to another year — a surcharge for 2025 in the 2026 satellite — declares it with referenceYear.

An ambiguous entry is resolved with a sourceQuote — then only the quoted passage is rated — or confirmed by a person. It is never resolved by rewriting the value or the source.

Automatic reading works for HTML pages, plain text, CSV, spreadsheets (xls, xlsx, xlsb, ods), PDF documents with a text layer and text documents (docx, odt); the format is detected from the content. A value whose numbers are spread over several documents lists all of them in sourceUrls, and a number counts as found when any of them holds it. Scanned PDFs without a text layer, script-only pages and hosts that block automated access are reported as unreadable — those values need a readable local copy of the official text, or a person.

Manual: LexAttest

LexAttest records that a person confirmed the values against an evidence document. The evidence must be declared in files and exist in the .lex folder. The key accepts a pattern, so a whole lookup can be attested in one step:

LexAttest TaxParameter "VerifiedBy:Jane Doe" Evidence:tax-guide-2026.pdf
LexAttest TaxParameter.* "ReviewedBy:John Doe"

A manual record replaces an automatic one; a manual record itself is only replaced with /overwrite, and LexVerifySource never overwrites it.

Four eyes per source

The review rule is set per source: SingleAuthor (default) or FourEyes. For values from a FourEyes source, a manual verification is only complete with a second person and date (reviewedBy, reviewedOn) — recorded with the second LexAttest call above. LexCheck reports manual records of such sources that lack the review.

Freezing the Evidence: LexSnapshot and LexVerify

LexSnapshot turns the manifest into an immutable record. It copies every referenced file into the .lex folder (downloading files that only have sourceUrls), records SHA-256 hash, size and content type per file, computes a sourceHash over all values and file hashes, and stores a hash of the normalized text of every readable source URL. After that, no external access is needed.

LexSnapshot .lex/lex.json /dryrun
LexSnapshot .lex/lex.json /overwrite
LexSnapshot lex.json /recursive

An existing snapshot is skipped with an info message rather than overwritten, so recursive runs across a repository are safe. Re-creating a snapshot is a deliberate act (/overwrite) after the lex has changed. /dryrun validates and reports without writing.

LexVerify is the counterpart: it recomputes the hashes from lex.json and the files in the folder and compares them with the snapshot. It detects changed or missing evidence files, values edited after the snapshot was created, and a stale snapshot — and returns exit code 6 when anything differs.

Lex Writes the Regulation: LexApply

Documenting values next to the regulation is half the job. The other half is making sure the regulation actually contains those values. LexApply writes the lex values into the lookup values of the satellite, using the same key mapping as above:

• Only differing values are replaced; missing values of an existing lookup are added.
• Descriptions and file formatting are kept — a second run changes nothing.
• Open entries (see rollover) are skipped until they are verified.
• A text table replaces the whole lookup when the table differs; the lookup file is written as a compact, deterministic ZIP archive — the same table gives the same bytes, so version control shows no change.
• /dryrun shows the differences without writing.

LexApply .lex/lex.json /dryrun
LexApply lex.json /recursive

This turns the working rule “no lex entry, no value” into the normal path: values are entered once, with their source, and the regulation JSON follows. Editing a lookup value by hand is still possible — and the next check reports it.

Lex Checks the Regulation: LexCheck

LexCheck compares each manifest with the lookup values of its satellite and reports what does not fit:

The release gate

Before a satellite is released, LexCheck /release applies the strictest rules. It requires a precise location and a verification record on every entry, and verifies every entry again against its local evidence. The following are then findings:

• a main source declared as secondary (official: false), such as a commercial summary or a tax calculator,
• an entry without manual record whose automatic match is ambiguous or whose quote is not found,
• an entry without manual record that can be verified neither locally nor live (a blocked host, a script-only page or a scanned PDF without readable copy),
• a zero value without justification,
• an evidence file that backs no entry and is therefore never read.

LexCheck lex.json /recursive /release Report:Reports/LexRelease.md

A satellite that passes the gate can be verified again by anyone, from its own folder, at any later time.

Documented exclusions are listed in the report with their reason instead of appearing as findings — a technical setting in a lookup, or a rule that is implemented in a script rather than a lookup, is visible and justified rather than silently ignored. The report also counts the verification records per satellite: automatic, manual, without record, open.

Gates in the Pipeline

The commands return exit codes designed for automation:

A check gate in a Payroll Console command file needs two lines:

# verify the frozen evidence, then the regulation against the lex
LexVerify lex.json /recursive
LexCheck lex.json /recursive Report:Reports/LexCheck.md

The Report parameter writes a Markdown report and a CSV file with the same name. With Root and RepoMask, one run covers every repository below a folder — useful for a provider maintaining many satellites across several country regulations.

The Next Cycle: LexRollover

Every year the same work starts again, and the most common error is a value carried forward without being checked. LexRollover prepares the satellite of the next cycle so that this cannot happen silently:

• The cycle is read from the tenant’s payroll calendar — a calendar year, or for example a fiscal year with Calendar:FiscalYear. Its start becomes the new validFrom, its year the new year tag.
• A new satellite folder is created; the previous satellite is not changed, and an existing target is never overwritten.
• Names and references with the year tag are renamed; validFrom and the version are set.
• Values are not changed — but every entry becomes open: its verification record is removed until it is verified against the new source. The snapshot is not carried over.
• Lines that still mention the previous year are listed for review.

LexRollover Data.Tax.2026 Date:2027-01-01 Tenant:MyTenant /dryrun
LexRollover Data.Tax.2026 Date:2027-07-01 Tenant:MyTenant Calendar:FiscalYear

From there the cycle repeats: new evidence, updated values, verification, LexSnapshot, LexApply, LexCheck. Open entries are reported by LexCheck and skipped by LexApply, so an unverified value cannot reach the regulation through the lex.

Committed, Reviewed, Not Hidden

The .lex folder is committed together with the satellite. That makes the compliance record part of the normal review: a pull request that changes a statutory value shows the new value, its source location, its legal reference and its verification record in the same diff. LexCheck enforces the last step — evidence that is missing, untracked or ignored in git is a finding.

The result is a chain that a regulation developer can maintain and a reviewer can follow: statutory source → evidence file → lex entry → lookup value → wage type → payslip. How compliance and audit teams use that chain day to day — audits, annual updates, changed sources — is the subject of the companion article, “Show Me Where This Number Comes From”.

The complete reference — every command, parameter and schema property — is part of the open-source documentation: the Lex page on payrollengine.org and the Payroll Console command reference.