Benchmarks And Proof

Current Status

Dev smoke has validated the new large-file surfaces on https://ultra-dev.egomonk.com, including the 10 GiB Turbo resume class, desktop Chromium browser rows, and iPhone Simulator Safari browser rows. Physical-device and broader Safari benchmarking remains pending.

Validated Dev Run

Latest validation event for this docs pass: the Browser Baseline Runner posted desktop Chromium and iPhone Simulator Safari results through the dev-only result handoff after server-side chunk-wait diagnostics were added.

ULTRA_SMOKE_BASE_URL=https://ultra-dev.egomonk.com npm run bench:delivery-baseline -- --json --sizes 300MB,1GB --profiles desktop-chromium,iphone-safari-wifi

For real browser rows, open Browser Baseline Runner on the target browser/device and run the same size classes.

Phase 1.1 validation matrix planning is available through npm run bench:pipe-live -- --dry-run --json. The matrix names desktop-to-desktop-live, desktop-to-iPhone-live, iPhone-to-desktop-live, and large-file-sanity, and requires jitter/failure evidence fields including p95GapMs, physical receiver-side p95ReceiverGapMs, chunkWaitP95Ms, streamFailedCount, and failureReasons. Scoped artifacts can be written with npm run bench:pipe-live -- --dry-run --json --matrix-id desktop-to-desktop-live --output ../docs/phase-1.1-pipe-live-desktop.json. This is the checklist for collecting proof; physical-device rows still need measured dev-channel results.

Measured Phase 1.1 desktop API-level proof is stored in docs/phase-1.1-pipe-live-desktop.json. The dev-channel node-fetch-api run moved 96 MiB with 24/24 live hot-buffer hits, 0 stream failures, 1 ms live p95 receiver gap, and 0 ms live server p95 chunk wait; replay had 8 hot-buffer misses, 835 ms of R2 get time, and 230 ms server p95 chunk wait. This is PipeDO API proof; browser UI proof and physical iPhone rows remain separate validation work.

Measured Phase 1.1 large-file sanity proof is stored in docs/phase-1.1-pipe-live-large-file.json. The dev-channel node-fetch-api run moved 320 MiB in 40 streamed 8 MiB chunks with 40/40 live hot-buffer hits, 0 stream failures, 1 ms live p95 receiver gap, and 0 ms live server p95 chunk wait; replay had 32 hot-buffer misses, 4819 ms of R2 get time, and 474 ms server p95 chunk wait. This is generated PipeDO API-level large-file sanity, not the full Hybrid/Direct-R2 throughput matrix.

1 Gbps Throughput Proof

Dev-channel proof now shows the raw Direct-R2 upload ceiling and browser-native encrypted progressive receiver path approaching practical 1 Gbps utilization on desktop Chrome. Physical mobile-device proof remains pending before production promotion.

The progressive batch target rows recorded progressiveBatchCommitCount values of 1, 2, and 3 for 1 GiB, 5 GiB, and 10 GiB. They prove that chunk 1 can still publish immediately for a live receiver while later chunks are committed through POST /api/upload/progressive/commit-chunks.

The browser throughput runner now includes a progressive-r2 path for browser-native encrypted progressive chunks. Tiny Chrome CDP canary codex_progressive_smoke_20260617e proved mechanics; target-size runs codex_progressive_1gib_chrome_chunkrecordfix_202606170405, codex_progressive_5gib_chrome_chunkrecordfix_202606170406, and codex_progressive_10gib_chrome_batcheddl_202606170439 then proved encrypted desktop throughput with local fragment-key retention, receiver start before upload completion, download-complete ack ok, and cleanup ok. In-upload batch row codex_progressive_10gib_chrome_inupload_batch_20260617a raised 10 GiB browser-native encrypted upload to 867.01 Mbps with progressiveBatchCommitCount: 25, zero retries, early receiver start, and cleanup ok. Current local reporters label throughput rows as sequential_one_sided, overlapped_bidirectional, or upload_only, and keep one-sided peak, overlapped per-leg, bidirectional aggregate, and effective delivery Mbps separate.

The target-size retry also found and fixed a progressive completion race: parallel chunk records were committed under independent keys, but completion could still rely on stale KV list/manifest state and see only the first contiguous chunk. Dev Worker version cd9067ce-68a8-4f97-8a67-c265c843fc63 exact-hydrates committed chunk records with short retry, accepts client-observed chunk numbers for verification, and avoids whole-manifest rewrites from chunk commit endpoints.

The authenticated Chrome CDP runner is now reusable through npm run bench:browser-progressive. Smoke run codex_cdp_runner_smoke_202606162159 confirmed it injects auth only into same-origin API requests, keeps R2 URLs unauthenticated, preserves the fragment key locally, starts the receiver before upload completion, and cleans up normally.

Earlier target-size attempts hit Ultra's app-level dev daily upload quota before bytes moved. The dev authenticated quota is now raised under [env.dev.vars] for validation only; the top-level production quota was not changed.

Target-size /api/dl separation proof is now recorded. Ordinary full Worker-streamed /api/dl completed 1 GiB at 254.15 Mbps, but 5 GiB and 10 GiB full rows truncated before the expected byte count. Session-backed parallel ranges and dev-gated direct R2 GET completed all target sizes after Worker policy/session resolution.

Current tuning rule: 32 MiB parts/chunks for the 1 GiB class, 64 MiB parts/chunks for 5-10 GiB, and concurrency 8 for the dev-certified desktop Chrome fast path. Desktop Chrome encryption is not the current bottleneck: browser AES-GCM measured 4908.53-8654.85 Mbps while upload measured 627.95-867.01 Mbps. Safari and physical iPhone rows remain compatibility-pending.

Mobile Safari Validation documents the receiver result handoff. A physical receiver URL can include mobile_validation_run_id before the fragment key so the receive page stores Service Worker, fallback, completion/failure, visibility, pagehide/pageshow, online/offline, connection-change, and sawUnsafePageAssemblyStart evidence in the dev-only result endpoint. Expected rows use mobileSafetyStatus values bounded_service_worker_stream_started or unsafe_fallback_blocked, with sawUnsafePageAssemblyStart: false. The repeatable dev command is npm run validate:mobile-safari -- --run-id=iphone_sw_YYYYMMDDa; it creates a 1088 MiB encrypted progressive share, prints the physical iPhone receiverUrl, uploads direct-R2 chunks while the receiver is open, and polls the stored row. Stored rows can be rechecked with npm run validate:mobile-safari -- --audit-run-id=iphone_sw_YYYYMMDDa, which requires Safari/iPhone identity, lifecycle evidence, sawTerminalSignal: true, and no unsafe page assembly.

Phase 1.1 live-pipe physical validation now has a repeatable harness: npm run validate:pipe-physical -- --dry-run --json --direction desktop-to-iphone --run-id=iphone_pipe_YYYYMMDDa emits the phase-1.1-physical-pipe-validation plan with no network access. The dry-run artifact includes a collectionChecklist with expected phase-1.1-pipe-physical-*.json artifact paths, collection/audit commands, row-level manualSteps, copy/paste-ready resolvedCommands, preflightCommands/resolvedPreflightCommands, localRowFiles, resultFetchCommands/resolvedResultFetchCommands, resultFetchPolling, the physical iPhone senderUrl, resultUrls with senderResultUrl and receiverResultUrl for the dev browser-result rows, shareUrlInputMethods for safe clipboard/stdin receiver URL preparation, paired iPhone sender/desktop receiver run ids, and required evidence fields for both missing physical rows. The result fetch commands use --save-result-row to poll for and save browser-result rows under /tmp/ultra-phase11-rows so offline preflight/audit commands can use concrete local files; the structured resultFetchPolling field exposes --poll-timeout-ms and --poll-interval-ms for tuning. Timeout failures name the run id and result URL so the missing browser-result row can be retried directly. Generated result-fetch commands include the resolved polling flags explicitly, so copy/paste handoffs do not depend on remembering CLI defaults. Use --output ../docs/phase-1.1-physical-plan.json --runbook-output ../docs/phase-1.1-physical-runbook.md to save that dry-run collection plan and generated operator runbook without touching the network; append --print-output-path when automation needs the resolved written path or the compact JSON path bundle for both saved artifacts. docs/phase-1.1-physical-plan.json is the checked-in operator handoff snapshot for the current missing physical proof rows, and docs/phase-1.1-physical-runbook.md is the generated operator-readable checklist for the same commands, preflight checks, result URLs, required evidence, URL-fragment handling caution, and final gate. The handoff plan also exposes a structured finalGate command that reruns the Phase 1.1 matrix audit with --print-output-path. Live mode creates an encrypted UPCH pipe, adds validation_run_id before the fragment key, prints a physical iPhone Safari receiver URL, and audits p95ReceiverGapMs, chunkWaitP95Ms, streamFailedCount, and failureReasons from the dev browser-result row. For iphone-to-desktop, dry-run prints a sender URL with sender_validation_run_id; the iPhone sender posts live-pipe upload evidence and the desktop receiver row can use receiver_validation_run_id. The helper also supports --prepare-receiver-url --share-url-stdin so the copied share URL is rewritten locally with the validation id before the fragment key. The dry-run plan exposes shareUrlOriginPolicy, and wrong-origin share URLs are rejected before any receiver URL is printed so dev validation ids cannot be paired with a non-dev copied share. It also exposes requiredAuditInputs; incomplete local-file or stored-run-id audits fail before writing artifacts, so the iPhone-to-desktop row requires both sender and receiver evidence. After rows exist, --audit-output can write phase-1.1-pipe-physical-*.json artifacts from stored run ids or local row files passed with --audit-sender-file and --audit-receiver-file. Add --print-output-path to saved physical audit commands when automation needs the written artifact path instead of the full audit artifact on stdout; generated handoff artifact-writing commands include that flag by default, and failed-audit handoff regeneration commands include it so saved plan/runbook refreshes can echo only the written paths. Add --preflight to those audit commands to run the same pass/fail checks without writing the final artifact; the handoff also exposes localRowFiles, resultFetchCommands, resolvedResultFetchCommands, localAuditCommands, resolvedLocalAuditCommands, preflightCommands, and resolvedPreflightCommands so saved local row files can be rechecked without fetching the dev result endpoint again. Then npm run validate:phase11 -- --json --output ../docs/phase-1.1-validation-matrix.json creates the phase-1.1-validation-matrix-audit bundle. The current docs/phase-1.1-validation-matrix.json snapshot intentionally records readyToMarkRoadmapValidationItem: false; that must become true before the roadmap checkbox moves. It also exposes blockingSummary with blockingRows and missingArtifactPaths so automation can distinguish missing physical collection from other gate failures. Failed audits emit nextActions with the exact collection, preparation, artifact audit, final gate commands, and requiredEvidence fields. Default failed audits also include an operatorHandoffPlan summary and enrich matching physical nextActions with the handoff finalGate, handoffPlanPath, runbookPath, senderUrl, manualSteps, resolvedCommands, preflightCommands, resolvedPreflightCommands, localRowFiles, localRowFileStatus, resultFetchCommands, resolvedResultFetchCommands, resultFetchPolling, localAuditCommands, resolvedLocalAuditCommands, recommendedCommandGroup, recommendedCommands, recommendedCommandReason, requiredAuditInputs, shareUrlOriginPolicy, and shareUrlInputMethods, and resultUrls from the checked-in handoff plan. Add --check-local-row-files to have the audit inspect default saved-row paths and recommend either result-fetch or local-audit command groups. Add --print-handoff-readiness when automation needs a readiness-only payload that exits zero for ready_for_physical_collection even while the final matrix gate is still waiting on physical artifacts; use --handoff-readiness-output <path> plus --print-output-path to save that payload and echo only its path. Add --print-recommended-commands with optional --action-id <nextActionId> when automation needs compact next shell commands instead of the full failed matrix JSON; --recommended-output <path> plus --print-output-path saves that compact payload and echoes only its path. Use --silent-npm-commands or invoke through npm --silent run when automation needs stdout-safe shell lines without npm lifecycle banners. Compact command payloads include blockingSummary, operatorHandoffReadiness, availableActionIds, retryCommandBase, retryCommandContext, rawCommandCount, duplicateCommandCount, and retryWithActionIds so automation can recover from mistyped action ids while preserving supplied local-row, silent-output, artifact-path flags, and the current gate context. Shared setup commands are deduplicated in the top-level command list while remaining visible per action. The handoff summary also exposes operatorHandoffPlan.runbook freshness status and missingMarkers if the generated runbook is absent or stale; use --physical-runbook <path> to audit a candidate runbook before replacing the checked-in one. It also exposes operatorHandoffPlan.readiness; ready_for_physical_collection means the checked-in handoff plan, generated runbook, command groups, operator metadata, and final gate are current. The physical artifacts still need to be collected before that gate can pass. With --output, the audit writes quietly by default; add --print-output-path when automation needs a compact stdout path.

Desktop Safari compatibility now has a local WebDriver runner: npm run bench:safari-progressive. It uses anonymous dev transfers only and does not read or inject Ultra auth tokens. The first tiny canary, codex_progressive_safari_smoke_20260617b, was blocked before navigation because Safari's local Allow remote automation setting is disabled; enabling it is a persistent local security change requiring explicit operator approval.

P2P/WebRTC remains a deferred experiment. ULTRA_ENABLE_P2P_EXPERIMENT now enables tokenized dev signaling rooms and P2P Experiment synthetic throughput probes with selected ICE direct/relay classification and mobile lifecycle/network event capture. The dev row codex_p2p_chrome_16m_hostonly_20260617 proved one same-machine headless desktop Chrome direct host/host path at 16 MiB, 64 KiB chunks, no relay, and 30.5 Mbps, but this does not move Ultra file bytes or prove direct peer success rate, TURN relay cost, mobile backgrounding, or 1 GiB/5 GiB throughput against progressive Direct-R2.

The remaining bottlenecks are physical/mobile validation, Safari compatibility rows, and repeatability of the new 10 GiB in-upload batch row across normal network variance. Desktop Chrome 10 GiB progressive upload now reaches the target band.

Each row records path, browser/device, part size, chunk size where relevant, file size, Mbps, percentage of 1 Gbps, time to first contiguous downloadable prefix, time to first receiver byte, retries/failures, and caveats in the dated throughput report.

Rollback And Rollout Guardrails

• ULTRA_ENABLE_DIRECT_R2_MULTIPART can disable Direct-R2 multipart; required Turbo uploads then receive lane_unavailable rather than falling back to a Worker payload route or widening to Live Pipe.
• ULTRA_ENABLE_DIRECT_R2_DOWNLOAD can disable direct R2 GET download URL issuance while preserving the existing /api/dl Worker download path and session-backed range behavior.
• ULTRA_ENABLE_PROGRESSIVE_R2 can disable progressive chunk manifests while preserving classic direct multipart and existing download semantics.
• Completed progressive transfers stay on chunk-manifest serving by default, so upload completion does not add a Worker-side object-stitching step.
• ULTRA_ENABLE_PROGRESSIVE_FINALIZED_COPY is reserved for any future background finalized-object copy experiment. The current path does not create finalized progressive objects during upload completion, receiver session creation, receiver download, or as a mobile-triggered action; any future copy must preserve policy, grant, max-use, delete-token, webhook, audit, encryption, and cleanup semantics for completed >20 GiB or repeated completed-object downloads.
• ULTRA_ENABLE_P2P_EXPERIMENT is dev/local-only and controls the optional WebRTC signaling and measurement prototype. Production promotion remains blocked until direct peer success rate, TURN relay fallback rate, mobile behavior, and comparative throughput are measured.
• The bounded-memory progressive Service Worker receiver is automatic when browser prerequisites and a matching server session are available. Dev/local diagnostics may disable it.
• Sender-side progressive batch manifest publication is automatic under the Hybrid plan. Dev/local diagnostics may narrow it, and chunk 1 still uses the immediate single-commit path so live receivers can start from the first contiguous prefix.
• Large mobile progressive receives prefer the bounded-memory Service Worker stream path. If that route is unavailable, the unsafe whole-file path is blocked before showing the start-download button, creating a receiver session, or allocating the final merged download buffer.
• A signed server TransferPlan selects Live Pipe, Hybrid, or Turbo and bounds geometry. Hostname, query, and local storage cannot widen that decision.
• Unavailable Hybrid or Turbo policy returns lane_unavailable; jumbo files do not silently fall back to Live Pipe.
• Dev/local diagnostics may narrow geometry or disable an experiment, but cannot create server authority.
• Production rollout remains manual until dev benchmarks, browser-native rows, mobile-safety checks, and rollback anchors are reviewed together.

The superseded Phase 1.3 soak, browser-controller, import, audit, and deployment-readiness commands are retired and cannot execute. Their checked artifacts remain immutable historical evidence. All new qualification status, verification, and execution flows through the fixed UQP-v2 command surface documented in the repository authority files; no live row is authorized by this page.

Local No-Transfer Verification

cd workers && npm run ci:protocol:local
cd workers && npm exec tsc -- --noEmit
node --test cli/ultra.test.mjs
cd sdk && npm run test:delete-transfer
cd sdk && npm run typecheck

Pending Matrix

• Physical iPhone Safari receive matrix.
• Desktop Safari and representative larger desktop file classes.
• Physical mobile validation for progressive encrypted send/receive and the new Service Worker progressive download stream using Mobile Safari Validation.
• P2P/WebRTC promotion measurements: direct peer success rate, TURN relay fallback rate, mobile backgrounding, and 1 GiB/5 GiB comparison against progressive Direct-R2.
• Progressive upload optimization for closing the 10 GiB gap beyond the current 779.18 Mbps browser encrypted row.
• Common third-party TUS clients.
• p50/p95 completion time and retry density beyond the current 300 MB / 1 GB desktop Chromium and iPhone Simulator Safari rows.
• Physical-device confirmation of PipeDO jitter and hot-buffer behavior.