BOM Voyage

The challenge

A text export pipeline produced files that look identical in an editor but behave differently in scripts. Find the encoding landmines before they break the investigation.

One question per file, the same field every time:

Determine text.utf8_bom_present for <file>.

Answer format: an enum, true or false, trimmed.

Reading the prompt: what it said we'd need

What a BOM is

A byte order mark is an optional marker at the very start of a text file that says which Unicode encoding it uses:

UTF-8 doesn't need one, but some Windows tools add it (Notepad in older versions, PowerShell 5.1's Out-File -Encoding utf8). Editors hide it, so two files can look identical while one starts with three extra bytes. Scripts trip over it: a CSV's first header becomes name instead of name, so row["name"] fails, and a line-1 comparison like == "Case ..." comes out false.

What was in the files

Every file was tiny, plain ASCII text. Only the first bytes (and, once, the line endings) differed:

The contents, as bytes (the case IDs are random per variant):

evidence-note.txt        Case 3DF9200ADB1A evidence note.\n
roster.csv               host,case\nWS-01,0DFBA6451415\n
evidence-note-copy.txt   Case E189CF04FA4B evidence note.\n
export-001-with-bom.txt  \xEF\xBB\xBF Export 1\r\nCase 68240FF2A75D\r\n

In an editor, the last file would look like any other two-line text file. As bytes, it has two landmines: the BOM, and Windows CRLF line endings, the only CRLF file seen in the event so far. A script splitting on \n would leave a stray \r at the end of Case 68240FF2A75D, and an exact match on the case ID would fail.

The zips: the Single Solve and Time Trial were one zip each. Mastery was one outer zip with the two set zips nested inside (729 B in total). All local and central name records agreed. The Hash to Ride trap wasn't used this time, but the solver still checked it.

The name …-with-bom.txt told the truth here. That's still no reason to trust it: the next variant could carry the same name without a BOM, and the solver would get it right either way.

Single Solve: CORRECT in 28 seconds

Started at 14:47:58 UTC. The zip (205 B) was downloaded with its SHA-256 matching. Its one member started 43 61 73 65, with no EF BB BF. false was CORRECT at 14:48:26, flag be80373f…a4d0.

The solver

Our Day 3 solver loads Day 2's Looks Can Be Deceiving solver by path, which in turn loads Day 1's Hash to Ride. That gives it all the zip handling for free: - names from the zip's own headers; - the local/central name check; - nested and stored zips; - per-set scoping for Mastery.

It adds one field: - …-text-utf8-bom-present: true only if the file's first three bytes are exactly EF BB BF. - false for a UTF-16 or UTF-32 BOM, and for a BOM that isn't at the very start. - The file name is never consulted. - Any other text.* field stops the run before submitting, rather than guessing.

It also prints every file's hidden traits (BOM type, CRLF/LF/CR counts, trailing newline, NUL, and non-ASCII bytes), ready for whichever text.* field a later day asks about.

6 tests, all passing before any timed run: - the real Single Solve zip; - true and false cases; - a file named …-with-bom.txt without one, a UTF-16 BOM, and a BOM after a space, all false; - the Mastery shape: two zips, an answer ID that doesn't match the file name, one zip stored; - an unknown text.* field, which stops; - the trait counts.

Time Trial: 1.94 s of a 6-second clock

started_at 2026-10-07T14:52:16.656Z  deadline 2026-10-07T14:52:22.656Z
  artifact metadata-day-03-evidence.zip 187 bytes sha256 941acab50c421c8f… (checked)
  roster-text-utf8-bom-present: false
  start         726 ms
  artifact      390 ms
  solve           1 ms
  submit        819 ms
  TOTAL        1936 ms
result CORRECT  flag 40db8a6f…f2a6

Mastery: 3.05 s of a 5-second clock, first attempt

started_at 2026-10-07T14:53:30.784Z  deadline 2026-10-07T14:53:35.784Z
  artifact metadata-forensics-day-03-bom-voyage-mastery.zip 729 bytes sha256 ed058314fa5b627b… (checked)
  set1.evidence-note-copy-text-utf8-bom-present: false
  set2.d03-text-001-text-utf8-bom-present: true
  start        1883 ms
  artifact      336 ms
  solve           1 ms
  submit        834 ms
  TOTAL        3053 ms
result CORRECT  flag 62d5bae1…2d95

This was the tightest margin of the day, under 2 seconds spare. Mastery's clock (5 s) was shorter than the Time Trial's (6 s), and the start call alone took 1.9 s, like every Mastery start on 2026-10-07 (1.75 s, 1.80 s, 1.88 s). The solve itself took 1 ms; nearly all the time is network and server.

Time to solve

All three modes were solved on 2026-10-07 by 14:53:33 UTC, 5 min 35 s after the first start. The Single Solve took 28 s, and the Time Trial (1.94 s of 6 s) and the Mastery (3.05 s of 5 s) each went through on the first submit.

Takeaways

• Read text as bytes when it matters. An editor's view hides BOMs, CRLF, and UTF-16. xxd, od -c, or Python's repr(bytes) show the truth.
• A name that makes a claim is a claim, not evidence. …-with-bom.txt happened to be right, but the solver never relied on it.
• Answer only what was asked, but record the rest. CRLF turned up unasked in Mastery. It's noted, and the solver already counts it, ready for a future text.line_endings-style field. Meanwhile, an unknown field stops the run instead of guessing.
• Compounding lanes reward reuse. Three days of Metadata solvers stack, so this one only needed the new field. The Single Solve took 28 s.
• Mastery start calls are the bottleneck. All three Mastery starts on 2026-10-07 took 1.75–1.9 s each, against 0.5–0.7 s for other starts. With a 5-second clock that leaves about 2 seconds, so if Mastery clocks shrink further, speeding up our own network calls stops being optional.
• Landmines are everywhere. While writing up this challenge, a real BOM character slipped into the middle of a line in our notes. It was found by searching for the bytes EF BB BF and replaced with the visible text .