Domain modelling for integrations

Audience: internalStatus: reference

Internal rulebook for step 1 of every integration (INTEGRATION-OVERVIEW.md §7). Auth, endpoints and retry policy can be changed later; the domain model cannot — once data has landed and other systems read it, every change is a migration and a coordination window. So the model is settled on paper before anyone writes a client.

core models the business, not the vendors — in master-data terms, the core row is the golden record. An Employee has a supervisor because ESZ has supervisors, not because a vendor API has a supervisor_id field. Vendor names, ids, enum spellings and meaningful nulls stay in the vendor app.

(For readers coming from DDD literature: each service is a bounded context, the vendor app is a conformist downstream plus anti-corruption layer, and an export endpoint is an open host service speaking a published language.)

Question zero: which bounded context?

Before any entity is modelled, decide which service owns it. This is the question everything downstream inherits from — a mis-homed entity means every later decision (identity, authority, published fields) is negotiated with the wrong owner, and moving it later is a cross-service migration, the most expensive change this platform knows.

A bounded context is the reach of one consistent vocabulary: inside it, every word means exactly one thing. Our services are drawn on those lines — employees, customers, products, procurements each own the words of one domain. The boundary is what lets core stay small: "employee" inside the employees service never needs qualifying.

How to find the boundary, in practice:

• Same word, different meaning → two contexts. When sales and payroll both say "customer" or "rate" but cannot agree on one definition, that is not a modelling failure to resolve in a meeting — it is two contexts, each keeping its own definition. Forcing one shared model produces a field salad nobody owns.
• Different meaning, same word across a boundary is fine. Personio's "status" and ESZ's EmployeeStatus share a name and differ in vocabulary; the translation map at the boundary is the correct place for that difference to live (see Q4 in the example below).
• Invariants draw the inner line. Rules that must hold in one database transaction ("a supervisor must be an employee of the same company") must live inside one context. Rules that tolerate eventual consistency ("the ERP knows about the new customer within minutes") may cross contexts — and crossing is always async (scheduled sync, queued jobs), never a shared table.
• Different source of truth → lean toward different contexts. If two clusters of fields answer to different systems of record and different business owners, splitting them costs less than co-owning them.
• Follow the pivotal events. Walk the business process and note where state changes hands ("hired" → HR context ends, directory provisioning begins). Hand-offs are natural context borders; if a workshop is needed, event storming is the tool for surfacing them.

For an integration the question is usually not "do we need a new context?" but "which existing service owns this entity?" — employee-shaped data goes to employees regardless of which vendor carries it. A new service is warranted only when the data speaks a genuinely new vocabulary that no existing service owns — never merely because a new vendor arrived; that is a vendor app (INTEGRATION-OVERVIEW.md §2).

How to run it

One session per entity, business owner present. Engineering prepares, the business owner decides, the contract table records it.

1. Pull real payloads first. Fetch sample records from the vendor sandbox (or their docs if the sandbox is late) for every entity that crosses the boundary. Model from actual payloads — vendor docs lie by omission about nulls, enum values and undocumented fields.
2. Map vendor objects to ESZ entities. For each vendor object: which ESZ concept is it (Employee, Article, …), or is it mirror-only noise? Name it in ESZ's words before looking at its fields.
3. Walk the six questions (below) per entity, in order, and write the answers down. Don't skip ahead to fields while identity (Q2) or overlap (Q3) is open — field decisions inherit from them.
4. Do the field pass. Every field that crosses, in either direction, gets a row in the contract table. Each row must pass the vendor-removal test and hub-not-bus (below), and have an authority answer. A row you can't fill is a field that isn't ready.
5. Sketch the core delta. Which fields/relations does core gain, which convergence keys (if Q3 said yes), which stay in the mirror. Paper or the contract table's ESZ-name column — not code, not a migration.
6. Gate. Done when: every table row is complete, the business owner has signed off on authority answers, deletion and match-nothing behaviour are written down, and — where the vendor reads from us — the published field list is final. Only then does step 2 (mirror) start.

The six questions

1. What is the entity, in ESZ's words? An Employee, not an Acme WorkerRecord.
2. What identifies it permanently? A stable id for the life of the record. Names and e-mail addresses do not qualify. Choosing wrong means a migration.
3. Do two systems describe the same real thing? Only then add a convergence key — a normalized, partial-unique column. Never speculatively: it constrains the table for as long as the table exists.
4. For each field held on both sides, which side wins? "Sync both ways" is a race condition, not an answer. Most common cause of schedule slip, and not one engineering can settle alone.
5. What happens when a vendor record matches nothing? Create a core row, or error.
6. What does deletion mean? Hard, soft, archive, or silently absent from the payload.

Answering Q4 — the three authority rule types

Master-data practice calls these survivorship rules. In order of preference:

• Source authority (default): one named system of record per field. HR fields → the HR system, payment terms → the ERP. Deterministic, auditable, and the only rule a business owner can sign off field by field.
• Recency: latest modification wins. Only for genuinely bidirectional fields, and only when both sides expose a trustworthy modified-at — clock skew and bulk-import timestamps silently corrupt this rule.
• Completeness: non-empty beats empty. A tiebreak for enrichment fields, never a primary rule — it cannot distinguish "cleared on purpose" from "never filled".

If a field seems to need recency or completeness as its primary rule, that usually means Q4 was asked at the wrong granularity — split the field or name a source.

The three field tests

Vendor-removal test. Would the field still need to exist if the vendor system were removed tomorrow? Yes → business concept, core. No → vendor concept, mirror. Ignoring this is how core becomes the union of every vendor's schema, one field at a time.

Start minimal. Adding is a PR; removing is a negotiation once a single use case depends on it. So: no speculative entities, no convergence keys without a live second system, no published field without a named consumer and a stated purpose. When in doubt, leave it out — adding it next month costs nothing, carrying it forever does.

Hub, not bus. esz-backend is the hub; connected systems are spokes, not relays. Push each system only what it needs for its own function, never data it would forward to a third system — if B needs what A holds, that flows A → core → B, both hops ours. A relay hop is a hidden contract: we can't rotate the shape, audit who sees it, or know the third system exists, and field authority breaks when our data comes back wearing another vendor's shape. In discovery: every requested field gets "what does your system do with it?" — "we hand it to X" means integrating X, not widening the feed.

Published fields are contracts

For any endpoint a vendor reads (the export shape, INTEGRATION-OVERVIEW.md §4.5). Publishing turns a freely refactorable model into one a third party's production system depends on.

• The serializer is the contract, the model is not. Explicit, hand-declared field list — never fields = "__all__". Type conformance is fixed in the serializer, never by changing a model field type.
• The endpoint lives in the vendor app: GET /v1/acme/export/employees/, reading core via domain/queries.py. (Bare /v1/<vendor>/<entity>/ is the passthrough shape — the prefix keeps them apart.)
• One serializer per consumer. Sharing one couples two partners' contracts to each other and both to us.
• Name the exact fields. Each one passes all three tests above; each one is a field we can no longer rename freely.

Prefer pushing over letting the vendor read where they can accept it — push is retryable and decouples availability. Reads suit unpredictable on-demand lookups; "tell me when something changed" is a push or webhook, not polling.

The data contract template

Step 1's deliverable: one table per entity, one row per crossing field.

Field (ESZ name)	Vendor name	Type / unit / tz	Direction	Authority	Needed by, for what	PII	Notes

Plus per entity: stable identifier, deletion semantics, match-nothing behaviour (Q2, Q5, Q6).

"Needed by, for what" is where hub-not-bus gets enforced — "to pass on to X" fails the row. The PII column feeds the data-processing-agreement check before cutover. A filled example follows.

Worked example: Personio → employees

The live Personio integration, read back through the six questions. Code: services/employees/core/ and services/employees/personio/.

Q1 — entity. Employee. Personio's shape (nested attributes.<name>.value JSON, German labels, custom fields like dynamic_14478730) never leaves the vendor app: personio/models.py mirrors it, core/models.py knows nothing of it.

Q2 — identity. external_id stores the integer personio_id that Personio guarantees stable. core.Employee keeps its own surrogate key and carries no vendor id at all: the mirror's core_employee OneToOne is the crosswalk. That is the general pattern — the vendor id lives on the mirror, so unplugging the vendor deletes the crosswalk with the vendor app.

Q3 — same real thing in two systems? Yes: Personio and Microsoft 365 both describe the same person. Yet there is no convergence key, because only one direction matches inbound — Ms365Employee rows are provisioned from core and polled back only to detect drift, never matched against incoming vendor data. A convergence key is only needed when two systems are matched inbound; provisioned mirrors link by construction.

Q4 — authority. Source authority: Personio (the HR system) owns first_name, last_name, email, position, status. MS365 receives pushes and owns nothing in core; its operational fields (account_enabled, user_principal_name) stay in its mirror. Authority does not mean copying: Personio's status vocabulary is translated, not adopted — PERSONIO_STATUS_MAP folds Personio's "leave" into ESZ's active and maps anything unknown to inactive, because ESZ's EmployeeStatus enum is the business's vocabulary, not Personio's.

Q5 — match nothing. Create: an unknown Personio external_id creates the core row. (A new hire appears in HR first, so this is the normal path, not an edge case.)

Q6 — deletion. Personio never says "deleted"; records go silently absent from the payload. Decision: soft-delete the mirror (deleted_at) and set the core employee to terminated — core keeps the row because other systems still reference the person. One guard from the session where this was decided: an empty payload is treated as an API failure, not a company-wide offboarding, and skips deletion entirely.

The flow that falls out — also the anatomy of every ACL (adapter → translator → facade):

Personio API                      (adapter: esz_shared client)
  → PersonioEmployeeUpsert        (translator: pydantic payload model —
                                   validates identity, flattens vendor JSON)
  → PersonioEmployee mirror       (vendor document under `observed`)
  → .core → EmployeeUpsert        (translator: vendor-agnostic dataclass,
                                   status map applied here)
  → apply_employee_change(        (facade: the one canonical-write funnel)
        origin="personio")
      → upsert core.Employee

Two details worth stealing for the next integration:

• Echo prevention. Other vendors pick the change up on their own next poll by comparing core with what they hold; the source already holds the value, so nothing is pushed back to it and nothing loops.
• Second-pass references. supervisor arrives as a raw supervisor_personio_id and is resolved to a core FK only after all rows of the run have landed — the supervisor may appear later in the same payload. Vendor-side references generally cannot be resolved row by row.

The tests in action. company_employee_id ("Mitarbeiternummer") sounds like a business concept, but it sits in the mirror: no ESZ use case consumes it yet, so start-minimal keeps it out of core — the day a consumer shows up, promoting it is a PR. sync_error, personio_last_modified_at, last_synced_at fail the vendor-removal test outright and stay mirror-only.

The contract table for this integration:

Field (ESZ name)	Vendor name	Type / unit / tz	Direction	Authority	Needed by, for what	PII	Notes
first_name	first_name	string	vendor → core	Personio	core model	yes
last_name	last_name	string	vendor → core	Personio	core model	yes
email	email	string	vendor → core	Personio	core model, matching	yes	required by identity validator
position	position	string	vendor → core	Personio	core model	no
status	status	enum	vendor → core	Personio	core model	no	translated via status map, never copied
supervisor	supervisor.…id	employee ref	vendor → core	Personio	org chart	yes	second-pass FK resolution
— (mirror only)	dynamic_14478730	string	vendor → mirror	Personio	none yet	yes	start-minimal: not promoted to core

Stable id: external_id containing the Personio id. Deletion: absent from payload → soft-delete mirror, terminate core (empty payload = fail safe). Match nothing: create.