# Validation and its limits

This release was checked by independent computational implementations and by scripted user tasks. It has **not** been evaluated in a human usability study, an external chemical prediction benchmark, or a representative sample of papers. A successful run establishes the listed numerical operations on the bundled inputs, not experimental truth or general error-detection accuracy.

## Checks performed before release packaging

The test plan was recorded before the new CLI was exercised. Both real cases were then run from separate disposable package copies in directories with spaces, outside the original workspace. No sibling source cache, Git metadata, API credentials or browser session was needed.

| Environment | Result |
|---|---|
| Linux x86-64, Python 3.12.3 | All 16 unit and CLI tests passed; both real cases reproduced |
| Linux x86-64, Python 3.11.15 | All 16 unit and CLI tests passed; both real cases reproduced |
| Interpreter isolation | CLI subprocesses used `-I -S`: Python environment path injection and site packages disabled |
| Network boundary | A CPython audit hook rejected every `socket.*` event; its intentional socket self-test was rejected, then the actual CLI attempted zero socket API events |

The network check covers CPython socket API calls in these runs. It is not an operating-system firewall or a claim about every possible network mechanism. The inspected core uses only standard-library local computation and file access.

Each `reproduce` run took about 0.08–0.09 seconds on the available machine. This is a measured smoke-run duration, not a performance guarantee or a measure of researcher time saved. Python 3.10, Windows and macOS were not available for runtime testing; the declared Python 3.10 minimum follows the standard-library APIs used.

## Independent numerical comparison

A separate implementation read the publisher JATS XML directly. It did not import release code or consume the release CSV extraction. It used exact `Fraction` arithmetic for centered least squares, R², average-tie ranks and leave-one-out predictions, with 50-digit `Decimal` square roots.

All ten primary Lauberte analysis variants and the separately labelled post-hoc ABTS source sensitivity were compared with that independent calculation: included compound IDs, sample counts, correlation, slope/intercept, rank-pair counts, MAE/RMSE and individual held-out predictions. All 13 thermal table rows, the full-table and from-250 °C counts, and the four 800 °C table/prose assignments were compared independently as well. For the post-hoc ABTS analysis, the original ten variants stay unchanged: only responses for compounds 5 and 6 change from Table 1 values 0.67/0.96 to prose values 1.45/1.44, with all 14 compound IDs and BDE values held fixed. The independent R² is 0.74716763568591. This substitution does not resolve which source representation is experimentally correct, and table confidence intervals are not assigned to the prose means.

There were **940 numerical and identity checks per interpreter**, all passing. The largest observed absolute numerical difference from the independent calculation was **8.88 × 10⁻¹⁵**. The comparison uses relative and absolute tolerances of `1e-10` to accommodate floating-point implementations; this is a software comparison tolerance, not measurement uncertainty.

The two Python versions produced identical displayed HTML in this check. JSON floating-point representations differed slightly, so cross-version byte identity of generated JSON is not promised. The release verifier compares frozen numerical results within the stated tolerance and requires exact identity for nonnumeric structure and source/file hashes.

These tests verify analysis of published rounded aggregate values. They do not recover raw replicates, validate the authors' DFT calculations, supply experimental confidence intervals for the new reanalysis, or establish external generalization.

## Scripted user tasks

`tests/test_cli.py` exercises the command-line interface in disposable copies without importing its analysis functions:

- Reproduce both bundled real cases with no network or site packages.
- Export the real Lauberte values into a differently named three-column CSV, reverse row order, then analyze it as user data. The same 18 complete DPPH pairs and numerical result are recovered; compound 17 remains visibly missing.
- Run the thermal comparison with `--min 250`: 11 of 12 tabulated temperatures agree with the requested greater-than relation; the counterexample is 250 °C, with a difference of −1.5 percentage points.
- Reject missing/duplicate column names, malformed rows, duplicate IDs, undefined constant-variable correlation, too few complete pairs, malformed measurements, NaN and infinity before creating a result directory.
- Preserve numerical zero as a measurement and explicitly list missing pairs rather than filling them with zero.
- Reject blank unit labels, nonfinite interval limits and duplicate numeric indices such as `250` and `250.0`.
- Preserve an existing nonempty output directory unless the user explicitly requests replacement.

The statistical tests also cover hand-calculated straight-line predictions and training-mean baselines, average ranks for ties, expected-direction reversal, and leave-one-out folds that cannot fit a slope. Both models' aggregate errors use the same evaluable folds.

Unit flags are explicit labels. The program cannot determine from numbers alone whether a CSV mixes solvents, charge states, assays, units or normalizations. The user must establish comparability before requesting a calculation. A numerical consistency flag identifies a mismatch in the given source representations, not misconduct or corrected raw experimental truth.

## Repeat the checks

From the unpacked release directory:

```sh
python3 -m unittest discover -s tests -v
python3 -m ligninqc verify
python3 tools/check_independent.py
python3 -m ligninqc reproduce --out my-reproduced-results
```

Use a new output directory, or deliberately add `--force` when replacing generated outputs. `verify` checks the packaged manifest, source transcription and frozen results; it is a separate check from rerunning tests. `tools/check_independent.py` runs both implementations using the current interpreter, compares all 940 numerical/identity checks plus array/source identity constraints, prints a compact JSON summary, and exits nonzero on a mismatch. It uses temporary outputs and needs no private workspace files. Add `--out comparison.json` to keep its summary. Final published-archive hash and download verification are recorded in the release receipt, outside this self-contained package, to avoid a self-referential checksum.
