Vendor mirrors¶

Audience: internalStatus: reference

How a vendor app stores what a vendor said and what we sent back. The shared implementation is esz_shared.vendor_sync; this page is the reasoning behind its shape, so the code can state rules without re-arguing them.

core owns the business model (Domain modelling); a mirror is a sync artefact. Nothing outside the vendor app reads a mirror as a source of truth, and the admin's CSV export is canonical-only for that reason.

One change, end to end¶

A customer moves and sales enter the new billing street in Salesforce:

sequenceDiagram
  participant SF as Salesforce
  participant SFM as Salesforce mirror
  participant Core as core
  participant BPM as Büro+ mirror
  participant BP as Büro+
  SFM->>SF: poll
  SF-->>SFM: changed account
  SFM->>SFM: normalise into observed, compare with base
  SFM->>Core: merge change (origin salesforce)
  Core->>Core: update canonical row, record history with source
  BPM->>BP: next Büro+ poll
  BP-->>BPM: old street
  BPM->>BPM: vendor still at base, core differs
  BPM->>BP: push canonical field slice
  Note over BP,BPM: Following poll matches core<br/>No further change

Fetch. The poll stores the vendor document as observed, parsed and normalised, so a formatting difference is not mistaken for a change.
Check. Only a real difference reaches core; an unchanged value is not written again (see Convergence).
Record. Core updates the canonical row, and the history entry names the vendor the change came from.
Deliver. Every other vendor finds the difference on its own next poll: the row is merged first, then whatever core still holds differently is pushed inline, after the merge commits. There is no fan-out and no delivery job per change, so a write path that bypasses the facade is still delivered. The vendor the change came from already holds the value and is owed nothing. A row withheld from core, or whose merge failed, is not pushed. A manual sync_inbound run is the same poll, so it may write to vendors too. A transient push failure fails the row in the poll's result and is retried by the next poll; a permanent error pauses the row (see Pausing).

Which fields travel in which direction, and which side wins, is set per domain on the data pages, for example Customers.

Envelope columns, JSON snapshots¶

A mirror row is one vendor's view of one core row, split in two:

Part	Storage	Why
Envelope — `external_id`, `observed_at`, `merge_pending`, `sync_error`, `pause_reason`, `last_synced_at`, `last_pushed_at`	Real columns	Indexable, constrainable, typed. The sync itself queries these on every poll.
Vendor document	JSONB snapshot	A vendor adding a field costs no migration, and vendor field sets drift constantly.

The split is the whole design: sync state is ours and stays typed, vendor data is theirs and stays shapeless. A vendor's schema change should not reach our migration history.

Promoting a snapshot key to a column is allowed for exactly two reasons: a database constraint, or a query the profiler says needs it. Admin display is not a reason — a snapshot renders without a column. Promoted keys are declared in promoted_fields and read back through the mirror field, because a vendor omits a key far more readily than a typed, usually NOT NULL column tolerates one.

Admin searches on snapshot keys use JSON text extraction followed by a case-insensitive match. PostgreSQL cannot use a normal JSONB index for that expression, so these searches scan the mirror table unless a matching expression index is introduced. Keep the key in JSON until measurements show that the search needs promotion or a dedicated index.

Two snapshots, never conflated¶

observed is the only believed vendor state. Merge and push-diff read it and nothing else. It holds the parsed, normalised payload, so it and the canonical row agree by construction rather than by luck.
base is, per shared field, the last vendor value core has accounted for. Every poll moves it to the vendor's current value, except for a field core still owes, which keeps its base until a poll reads the vendor agreeing with core. It never moves on a push acknowledgement, so a vendor that accepts a write and stores something else never looks delivered.

base decides who wins when both sides changed a field that both may write:

Vendor vs `base`	Core vs `base`	Outcome
same	same	nothing to do
same	differs	core's edit is pending: push it
differs	same	vendor edit: import it
differs	differs	core's pending edit wins and is pushed

A field with no base yet - a first link - takes the vendor's value, and so does one where core holds an empty value it would never deliver. That merge then records the vendor's value as the base, so an admin edit made after the link is pending, not overwritten by the vendor's next change to another field. A change undone before delivery (A to B and back to A) owes nothing. Two vendors editing the same field between polls resolve in poll order: the first import wins, and the second vendor receives it.

What a push delivered is not a third snapshot. last_pushed_at says when it landed, the vendor's own record says what it holds, and the payload goes to the log. A column would keep only the most recent push and hold a second copy of the row's personal data forever; the log keeps every push and ages out on its own retention.

Convergence¶

On the inbound side, core compares incoming values with the canonical row and saves only when something changed. On the outbound side, the poll pushes only the fields where core differs from observed.

PushField(core=…, api=…, mirror=…) names a pushed field's three representations — canonical attribute, vendor API key, vendor-side snapshot key — so the fingerprint, the outbound payload and the mirror write all derive from one tuple and adding a field is one entry.

Two rules keep a converged row quiet:

Both sides are read back as the core field's type before comparing. JSON renders a Decimal as a string and a datetime as ISO text; comparing raw would re-push every row carrying one, on every reconcile, forever.
An empty canonical value is an absence, not an instruction to blank the vendor's. We never deliver it, so it must not be reported as drift either.

max_len on a PushField validates against the vendor's column width and raises before delivery, so the mirror records a push failure instead of a silently truncated vendor value.

Measuring divergence¶

manage.py survey_field_divergence --entity customer [--fields city,phone] [--samples 20]

--entity names the entity as the inbound registry spells it - customer, supplier - rather than the service, so the command serves every service without knowing one. An adapter that carries no canonical payload is link-only (Salesforce products is one: every fact it holds is a value core pushed there), and an entity left with fewer than two field-carrying vendors is refused rather than reported on: a lone vendor can only be compared against the row it already wrote.

Two more entities are refused rather than half-answered. A vendor that keeps several mirror rows per core row - a Hivebuy procurement_item carries one per variant - states several values for one field, and the report has one slot per vendor, so procurement_item is rejected until the survey can carry the variants. An adapter whose payload annotation cannot be resolved is an error too, not a skip: it does carry fields, and leaving it out would report agreement nobody looked for.

A read-only report, written as CSV to stdout, over the mirror snapshots the polls have already stored. No vendor is contacted and nothing is written. The field list comes from the adapters' payload types, so a run covers the same fields whichever rows happen to carry a mirror.

One row per (field, left, right). left and right are two vendors, or a vendor and core — the canonical row is the last witness, so every vendor is also compared against what it would be merged into.

Column	Meaning
`compared`	Rows where both sides state a value. A field one side leaves empty states nothing and is not compared.
`one_sided`	Rows where exactly one side states a value. `rows - compared - one_sided` is the rows neither side speaks for.
`raw_diff`	Rows where the two stated values differ at all.
`folded_diff`	Rows still differing after `names.fold` removes case, umlaut spelling and punctuation.
`normalized_diff`	Rows still differing after the field's own comparison form: country alias, postal spacing, dialling prefix, street abbreviation, legal form on a company line.
`uncertain`	Rows the comparison form cannot rule on: a number `phonenumbers` will not parse, a country name no register knows. Neither an agreement nor a conflict, and a subset of `folded_diff`: only a difference that survives folding is put to the comparison form at all.
`would_fill`	Rows where the vendor's next poll writes a value into a field core holds nothing in. Only on a `core` comparison.
`would_replace`	Rows where it writes over a value core already holds.
`would_clear`	Rows where it blanks a value core holds (only reachable under `AUTHORITATIVE` survivorship).
`observation_gap`	Age gap between the two snapshots behind each difference, bucketed.

Coverage is reported even where it is zero: a field no vendor pair ever both states still gets a row, so "the vendors agree" and "nobody ever looked" are not the same reading of a quiet table.

The three strengths separate work from noise: only normalized_diff costs a person a decision. The three write columns then split what happens if nobody decides, and they are not equally serious — a would_fill is the sync doing its job, and only would_replace and would_clear put one party's answer over another's. The replay runs the shipped resolver and then the shipped write filter, so a field the vendor states nothing about - parent_customer_number or a ProductUpsert column carrying None - is no write, a field the payload lists in not_columns is not a write either (it resolves a related row rather than a column), and a row whose adapter precondition refuses proposes nothing at all. They stay proposals: withhold_core is not replayed, because an override may write on the canonical side and this command does not, and no model validation runs, because nothing is persisted. A vendor that withholds rows reports an upper bound. A difference the payload already resolves — CustomerUpsert normalizes country on construction — never reaches the report at all.

observation_gap guards the conclusion. Two snapshots taken weeks apart can differ because one vendor was polled late rather than because the vendors disagree, so a report whose conflicts sit in the 30d+ bucket measures poll lag. It reads on vendor pairs only: the canonical row records no observation time, so every (vendor, core) difference lands in unknown.

The comparison columns and the write columns read an empty value differently, because the questions differ. A side that states nothing has nothing to disagree with, so an empty value is never compared. A vendor payload carrying a blank is a statement - the ERP cleared that column - so the write columns replay it as the clear the poll would perform. Only a field the payload leaves unset is silence on both sides.

--samples prints example rows to stderr, so stdout stays a CSV to redirect. Only differences that survive folding, and the rows that came back uncertain, are sampled; a spelling variant is not something to look at.

Every polled entity's changelist carries a Measure field divergence button for the same report, superusers only. It runs the command inside the request and answers with the CSV as a download rather than deferring it: a queued run would leave the report in the worker's log instead of in the hands of the person who asked for it, and the survey reads stored snapshots only, so it finishes well inside gunicorn's 60-second worker timeout - six seconds for the 10,166 customers across three vendors in production on 2026-09-22.

The button is on every entity, including the ones the survey refuses today, so the page states the reason rather than hiding the question. customer and supplier answer; product, shipping_address, supply_option and employee are short of a second field-carrying vendor, and procurement_item keeps several Hivebuy rows per core row. Each refusal arrives as an error message on the changelist, not as an empty file. --fields and --samples stay command-line only.

A vendor the command skips - one whose payload carries no fields for the entity - is named in a # comment row at the end of the downloaded CSV. The download leaves the changelist on screen unrendered, so an admin message would wait for the next page load, by which time the file reads as though every vendor agreed.

Pausing and the stale-observation guard¶

A permanent push error pauses the row: pause_reason records why, and the row is not pushed again until an operator runs Resume paused pushes on the mirror changelist. Later core edits do not resume it, so a bad row cannot generate a retry storm. Imports keep running while a row is paused, and core no longer holds its own value against the vendor's: the vendor refused it, so a correction made on the vendor side is imported. Resuming clears the pause and the last error; the next poll pushes whatever core still holds differently.

A failed merge is not a pause. It records sync_error and is retried by the next poll from the stored document, and the row's push waits until the merge succeeds, so a push never sends a value core has not yet reconciled.

Polls and pushes run in different worker processes, so ordering them by wall clock would order them by container drift. Both fetch_started_at and last_pushed_at are read from the database clock (SELECT statement_timestamp()), and a poll whose fetch started before the latest completed push is rejected as stale rather than overwriting what we just sent.

Provisioning¶

An unmirrored core row is provisioned by looking the vendor record up by its business key first and minting one only if that finds nothing, so a re-run adopts rather than duplicates. When provisioning adopts a row a poll has already observed, the polled document wins every key it carries: the seed names only what provisioning knows, and a thin slice must not replace a full document.

Provisioning has no mirror row on which to park a permanent error until lookup or creation succeeds. A permanent vendor error from either step therefore leaves the delivery job failed and visible to the unmirrored sweep; it is not a paused mirror push.

Migration deployment constraint¶

The mirror migration histories were squashed before launch by rewriting their 0001_initial migrations. They are safe only for an empty database: Django identifies applied migrations by app label and migration name, so it will not rerun a rewritten 0001_initial against an existing schema. Any environment that applied the earlier histories must be dropped and recreated before this version is deployed.