# ZID optimized timing paths: source replay

This source-only bundle reruns the three implementations used in the timing
comparison on deterministic synthetic inputs (200 examples per group,
2048 features). It does not include the original feature bank or reproduce
the published 10k/50k timing numbers. Linux x86-64, Python 3.12 and a C++11
compiler are required. The full-matrix allocator step is Linux/glibc-specific.

From the extracted directory:

```sh
sha256sum -c SHA256SUMS
python3.12 -m venv .venv
.venv/bin/python -m pip install -r requirements.txt
.venv/bin/python -s replay_zid_timing.py /tmp/zid_replay_new
```

Use a new output directory. No historical binaries are used: the six C++
sources are rebuilt with the local compiler. The script runs each route
in a separate process, binds all machine paths inside the selected output
directory and uses one thread. It compares 500-by-6 signed arm values and
all 500 aggregated scores to the public reference implementation, verifies
that the feature hashes agree, independently recomputes Simes scores, and
requires exactly equal permutation p-values. Both samples and the 499
random relabelings are deterministic. The expected observed score is about
4.220096834 and the expected permutation p-value is .016 for all routes.

The arm and score tolerances are 1e-8 absolute and relative. The 200-point
full-matrix check selects the historical 10k block configuration. Runtime
input routing and this small-size dispatch are adaptations, not changes to
the arm calculations. SOURCE_MANIFEST.json identifies original-source and
portable-template hashes. VALIDATION.json in the output records fresh results.
This check does not establish equivalence for arbitrary settings, non-Euclidean
metrics, or every platform.


## Large-input timing (separate from the numerical smoke test)

Use two precomputed feature arrays in .npy format, each C-contiguous native
float64 with 2,048 columns. No features are extracted or shipped here.
The rebuttal labels correspond to these command-line routes:

| Rebuttal label | Command-line option | Distance handling |
|---|---|---|
| ZID (full-matrix storage) | `--route full` | Stores the full pairwise distance matrix. |
| ZID (upper-triangle storage) | `--route packed` | Stores only the upper triangle of the pairwise distance matrix. |
| ZID (low-memory version) | `--route fused` | Computes pairwise distances in blocks. The label describes the measured memory advantage at 50k per group, not a guarantee at smaller sizes. |

Run one route at a time on Linux with GNU /usr/bin/time:

    .venv/bin/python -s benchmark_zid.py /tmp/zid_full_10k --route full --x x10k.npy --y y10k.npy
    .venv/bin/python -s benchmark_zid.py /tmp/zid_packed_50k --route packed --x x50k.npy --y y50k.npy
    .venv/bin/python -s benchmark_zid.py /tmp/zid_fused_50k --route fused --x x50k.npy --y y50k.npy

Use a fresh output directory for each invocation. By default, score-only and
499-permutation testing run sequentially as separate one-thread processes.
Use --mode score or --mode test to run only one. BENCHMARK.json reports the
implementation's wall time (excludes feature loading and extraction) and
GNU time's whole-worker peak RSS in KiB (includes resident inputs and process
startup; excludes compilation). Arrays, score vectors and input hashes
are checked before reporting PASS. This is a ZID timing entry, not a runner
for FID or KID. The 50k packed path can require over 40 GiB of worker memory;
check available memory and allow additional headroom before starting.

The reported timing experiment used specific ordered Inception-2048 arrays.
Supplying different arrays, including synthetic arrays, benchmarks that new
input only. Optionally use --expected-hashes X_SHA256 Y_SHA256 to require an
exact input match. These hashes refer to raw contiguous float64 array bytes
(not the .npy header). Machine, BLAS and compiler differences can affect
timings. The original arrays and sampling bank are not distributed here.

Original input content SHA256 (first group, then second group):

* Full, 10k: a64b8739228687369de498ad2fbd3fb00e9ec42b66f6a11da04e8f8a2d322fdb
  and d3dd43fe6f2d254f99ee019526c7dc3bbaf7ddc91e2cdacb33af1f0bd244ecaf.
* Packed/fused, 50k: dcd5f58889e18ab5694cd6872e16fd0b7bdbf3c8cd69c1ecc27ff13dd537d530
  and 80805c409548445a20e3824346486927c391e34cf091cb42a7b26200f848b092.

For a small timing-path check with your own 200-row arrays, add --smoke.
This does not validate throughput or memory consumption at 10k/50k.
