The upload feature could never work: four bugs only a browser could see

[adamjohnwright]

Spec 012's upload flow went to beta on 2026-09-22 (#292) and nobody had run it. Driving it with a headless browser — a private copy of the same image, Turnstile off, against the real ReactomeGSA — found four bugs. Every one had passed 67 tests, CI, a 15/15 answer sweep and the routing probe.

The four

1. Submission always failed. Both POST endpoints answer text/plain — a bare, unquoted identifier (swagger: produces: text/plain, example Analysis00371643). submit and load_public_dataset called response.json() on it. The docstring above the parser stated the opposite — "the bare ID as a quoted JSON string" — and had never been measured.

Every test stubbed submit() at the method level and returned a Python string, so the one line that was wrong was the one line no test ran. Even the parser's own test handed it a dict, because it was written from the same belief.

2. The failure message was untrue. "I could not reach the analysis service. Nothing was run." The service had been reached and had accepted the job with a 200 — the failure was in reading the reply, so the analysis was running, orphaned.

3. The whole chat UI died at the moment of success. For an element given by path, Chainlit infers the MIME type with filetype.guess(), which reads magic bytes. A TSV has none, so it came out null — Chainlit only falls back to the extension for URLs. The browser then called mime.startsWith() on null and replaced the entire chat with "Cannot read properties of null (reading 'startsWith')". The server logged the result delivered 11 ms before the page died.

4. Progress updated where nobody could see it. The progress message was sent first, so it was the first message in the thread, and Chainlit scrolls the latest user message to the top. It updated faithfully, above the fold, while the user saw "Started." and nothing else for minutes.

Found while fixing those

• Pathway names contain markdown. Real names from the run: NOTCH1:M1580_K2555, H139Hfs13* PPM1K …. Paired _ or * become emphasis and a variant identifier silently loses characters; a | shifts Direction and FDR a column right. Escaped.
• The progress percentage contradicted itself against the real service — "60% · Permutation 1000 / 1000". ReactomeGSA holds completed at 0.6 for the whole permutation phase while its description counts through it. The percentage is dropped; the service's own description is shown.

What makes it testable now

• GsaClient takes an injectable transport. Without it the only way to test the client was to stub its methods — which is how Got the embeding generation running! #1 shipped. tests/gsa/test_gsa_http.py goes through httpx.MockTransport with the service's real response shapes, content type included.
• chainlit_flow logs either side of delivering the result. The first real run went silent after "result written", and websocket frames are not logged, so "sent and not shown" and "never sent" looked identical.

Verified in a browser

First against a fake service replaying the real recorded responses — an iteration took 11 s instead of 10 min, and showed the delay was never the cause — then against gsa.reactome.org:

step	result
attach .tsv	chip shows, send enabled
read back	16 samples, in column order
labels	accepted, analysis starts in ~2 s
progress	one message, below "Started.", six stages, updated in place
result	275 of 2,679 pathways significant, top-10 table, Pathway Browser link
download	HTTP 200, 350 KB, all 2,679 rows and nine columns, text/tab-separated-values
afterwards	progress line gone, nothing falsely says "Done"

Sabotage: response.json() back in submit fails 7 tests (0 before this change); dropping the MIME type fails its test; dropping the escaping fails 5.

Known, and your call

The user's own uploaded file is deleted as soon as it is submitted (FR-007, for disk). Chainlit still links it from the attachment chip in their message, so clicking it gives a 500 — and the unhandled FileNotFoundError in Chainlit's file route also drops that connection. Harmless unless they click their own upload.

The alternative is to leave deletion to Chainlit, which removes a session's files when the session expires (session_timeout = 3600 after disconnect). That costs up to ~20 MB per session for up to an hour, on a host that runs at ~5 GB free. I have left the spec's choice in place.

🤖 Generated with Claude Code