Second half of the mass import: the transactional loader, the /import page
and a template generated from the same schema the validation uses.
Everything happens in one transaction. A half-loaded organisation — areas
without departments, positions without people — is worse than none, because
it looks like data. The dry run is the same code path with a rollback at the
end, so the report is built against the real current state rather than a
copy, and nothing is cached between checking and committing: the file is
sent twice. That costs one upload and avoids server-side state that can
expire, fill up, or be confused between two people.
Personnel numbers are taken from the file, not reassigned. personnel_number
is GENERATED ALWAYS AS IDENTITY, so this needs OVERRIDING SYSTEM VALUE and a
hand-written insert — worth it, because the number is on payslips, in files
and on badges. An import that reissues it is not a migration. The identity
counter is advanced afterwards; without that the next hire draws a number
the import already used, and the unique index refuses it weeks later, far
from the cause.
Three defects the first real run against the database exposed, none of which
typecheck, lint or 231 tests could have found:
- weekly_hours is bound to employment type by a CHECK constraint: full time
is exactly 38.5. The import reached the insert and was rolled back. Now
it is a finding with a row number.
- Titles are restricted to a fixed list by another CHECK. Same treatment.
- setval() needs UPDATE on the sequence, which `usage, select` does not
grant. Migration 20260803120000 adds it; until it is applied, an import
containing people will fail at the last step and take itself back.
I also had exit_date > entry_date where the database has >=. Someone who
never starts enters and leaves the same day; the stricter rule would have
rejected a real case.
Verified against the live database through the actual route and session: a
file with four deliberate faults produced exactly four findings, each with
sheet, row and column, and the rollback left nothing behind.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
First half of the mass import: a file becomes named sheets with typed rows,
and every rule that could reject a row is stated in one place.
Nothing here touches a database. The parser turns bytes into sheets, the
schema says which columns exist, and validation reports findings — the
existing state is passed in as a parameter. That is what makes 36 tests
possible without a connection, and the rules are the part worth testing.
Three decisions where the easy choice would have been silent corruption:
- A two-digit year is refused. "15.08.68" is 1968 as a birth date and 2068
as a contract end, and any rule invented here creates people not yet
born.
- "31.02.2026" is refused. Date turns it into March 3rd without complaint.
- An unrecognised value in a yes/no column is an error, not "no". Read the
other way, a typo in "Betriebsrat" quietly removes someone's dismissal
protection.
CSV is parsed rather than split. German Excel writes semicolons because the
comma is the decimal separator, so the delimiter is sniffed from the header;
a semicolon inside a quoted address would otherwise shift every following
column and import the row plausibly wrong. Quoted newlines, doubled quotes
and the byte-order mark Excel prepends are all handled — the last one makes
the first column read as "?Personalnummer", which is invisible in an editor.
Validation collects every finding instead of stopping at the first. With 800
rows that is the difference between correcting once and uploading eight
hundred times.
One rule earns its place from experience: a history event dated before the
entry it belongs to is refused here, with a row number, because the database
refuses it too — mid-insert, without one.
My own slip, caught by the type checker: `a ?? b ? c : d` does not mean what
it looks like; ?? binds tighter than the conditional.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
