Findings and Decisions Log
The numbered record of what has been tried in ELFQuake, what it found, and what was decided as a result. It is a log, not a queue — for what to do next see Next Actions.
Item numbers are stable and must not be renumbered. They are cited by
number throughout the documentation, the skills under .codex/skills/, and
MISTAKES.md. An item that turns out to be wrong is marked superseded or
retired in place, with a pointer to whatever replaced it, so a reference to it
still lands on something explaining why it should not be used. Items 1-88 are
the early standing-instruction list, kept for the references into it; items 89
onward are the detailed findings record.
Numbered record
- Run
./scripts/run-real-transfer-trial.shafter each INGV refresh. Correction (2026-08-09): the identical0.693435/0.315353reported on 2026-07-29, 2026-08-04, and again today was a stale-input artifact, not stability. The trial readsevents_italy_all_available.combined.normalized.csv, which onlybackfill-ingv-history.shused to write, so it stayed frozen at 2026-07-07 through three "refreshed" runs. After rebuilding that catalog (5,007 events through 2026-08-08), the trial scores held-out balanced accuracy0.667730and precision0.279167against a historical spatial-rate baseline of0.687589(precision0.322751). The transfer model is now below its own historical-rate control on both metrics. VLF and astronomy remain missing-mask features, so this is a seismic-history baseline, not evidence of multimodal prediction. Re-run 2026-08-22 on a live catalog (5,088 events, 2,071 train / 532 test samples): balanced accuracy0.671167, precision0.28629, against a historical spatial-rate control of0.69222/0.331633. Still below its own control on both metrics. This is the first run of this trial since the collector was fixed (item 104) and the catalog began refreshing every 30 minutes, so the conclusion no longer rests on a manually refreshed input. - Use
./scripts/evaluate-piezo-group-holdout.shas the primary synthetic stability check. The latest 27-run evaluation averages0.578712balanced accuracy, with positive recall0.166667--0.916667and negative recall0.148148--0.914286; it fails the stability gate. - Keep random-init piezo/VLF-only as the leading controlled Transformer architecture. It averages
0.619033within episodes, but has not passed unseen-episode stability; direct and summary branches remain disabled by default. - Keep label-free real VLF pretraining as the default real-data path while supervised VLF-aligned labels remain one-class or sparse; require reconstruction to beat both zero and last-patch baselines.
- Continue periodic INGV refresh and prospective relabeling; the current catalog-coverage guard prevents false maturity. Region-level tables remain one-class, so use
./scripts/prepare-italy-spatial-model-inputs.shfor the fixed-cell baseline until more temporal coverage arrives. -
Repeat
./scripts/evaluate-italy-spatial-baseline.shafter each refresh; its--group-by-timesplit keeps all cells from one VLF window in the same partition. The 2026-08-04 score0.666539was computed on stale Aug-4 inputs (see item 1 and 89). On genuinely refreshed data (11,020 rows, 7,258 labeled, 5,795 train / 1,463 test) the all-feature ablation scores calibrated balanced accuracy0.415351, precision0.191011, recall0.420330— below the 0.5 majority/always-negative baseline. The apparent skill in earlier runs did not survive fresh data. Keep this separate from the fixed-cell transfer trial, which uses a different contract. Diagnosed 2026-08-10 (item 93): the sub-chance score is a cross-era artifact of a split boundary that coincides with a collector outage. Within one era the score is0.655320/0.575000, but a coordinate control shows all of it is per-cell base rate.Re-run 2026-08-22 after the first refresh that actually propagated. The aligned dataset had been five days behind the labeled table — the existence-only rebuild guard in this script, now fixed under item 106(d) — so every run since 2026-08-17 scored the old record. Rebuilt: 15,979 rows, 13,775 labeled (was 7,258), 2,643 positive, 11,020 train / 2,755 test. Cell-stratified,
stratum_base_ratecontrol at exactly0.500000as required.Held-out label transitions: 5. That number governs everything below it. Stratified scores are
all_features0.625008,seismic_vlf0.623460,full_multimodal0.592414,vlf_only0.548896,seismic_only0.514868,seismic_astronomy0.404074, across 5 of 19 scoreable strata. The apparent0.51->0.62lift from adding VLF rests on five label changes and is not a result. Two further reasons not to read it: this table poolsera_0andera_3raw pixel features, which item 96 forbids; and no shift control has been run on this pooled table. Cite the per-era item-104 numbers instead. 7. Do not promote the spatial model. Label coverage improved materially on the 2026-08-09 refresh (10 of 19 cells contain positives, up from 6; 1,290 positive / 5,968 negative / 3,762 pending), and the re-run leave-one-cell-out evaluation now has 10 two-class folds instead of 5. Their mean calibrated balanced accuracy is0.5031(range0.1558--0.8373) — indistinguishable from chance, with the top fold (0.8373) resting on a single positive row out of 382. The 19-fold mean is0.3712. Better label coverage did not produce spatial transfer. 8. Retired 2026-08-21 by item 105(d); do not cite this item. The timestamp-shuffle control it rests on is not a null — it carried 94-342 held-out label transitions against the real runs' 1 and 19, so it was scored on up to a hundred times the evidence. Under the matched circular-shift control the "all five beat real order" pattern disappears. See Within-Cell Null Control. Original text follows. Treat the permutation result as a hard stop for interpretation. On the 2026-08-09 refreshed table the five timestamp-shuffled controls average0.6413(range0.6008--0.7158) against a real-order score of0.4154: all five controls beat the real chronological ordering, by a wide margin. Destroying temporal structure makes the task easier, which is the opposite of a predictive signal. The most likely mechanism is that shuffling leaks era information across the split — the real chronological test period is the recent dense-capture era with a different VLF feature distribution, and the shuffle mixes eras into training. Diagnose that distribution shift before any further modeling on this table; the permutation control is not a clean null while the shift exists. Diagnosed 2026-08-10 (item 93): the era-leak mechanism is confirmed, and the controls were also run at 100 epochs against a 600-epoch real run. Rerun cleanly within a single era at matched epochs, three of five controls still beat real order in each era, so the no-temporal-signal conclusion survives. 9. Follow Synthetic Event Alignment Strategies: first compare real and synthetic event-process statistics, then calibrate time/rate, magnitude, and spatial density before another transfer-model sweep. 10. Reviewdata/derived/reports/italy_event_catalog_alignment.json; the first comparison shows synthetic magnitudes are far too high and synthetic episodes occupy fewer cells, so do not use raw synthetic magnitudes for transfer without calibration. 11. Compare calibrated catalogs usingdata/derived/reports/italy_event_catalog_alignment_calibrated.json; rate and magnitude alignment improve, but spatial support remains incomplete. Use the new per-event spatial weights only in a weighted-training diagnostic until a source-observation model is validated. 12. Use the matching central-Italy catalog for central-Italy simulation profiles. The current raw seed-40 rate is about 16 times too high; do not select an extractor from a five-event sample. Generate and combine several independent episodes before judging temporal or spatial alignment. 13. Use the three-episode combined calibrated profile as the current alignment candidate:data/derived/reports/central_italy_event_catalog_alignment_combined_spatial.json. Its sample-matched nearest-neighbour distance is now3.469km; validate this improvement with more episodes and held-out spatial cells before promoting it. 14. Do not promote the 40,000-step seed-4500profile. Its tuned ten-event catalog is too small and has poor magnitude and clustering alignment; generate several independent long episodes and require a minimum event count before recalibrating. 15. Use explicit simulation coverage duration in future catalog comparisons. The corrected five-episode report isdata/derived/reports/central_italy_event_catalog_alignment_5episode_duration_final.json; its rate ratio is0.924, but held-out episode and cell validation is still required. 16. Treat./scripts/evaluate-italy-synthetic-episode-alignment.shas a regime-stability diagnostic only after all inputs share one simulator profile. The current0.278--14.990spread is configuration drift: regenerate seeds under the current source-localized refill profile before judging seed sensitivity. 17. Use the matched current-profile baseline atdata/derived/reports/central_italy_matched_3episode_q09996_w240_final.jsonfor the next alignment/model smoke tests. Keep the extractor settings (q=0.9996, window240, no event cap) explicit. 18. Use the per-episode reports underdata/derived/reports/central_italy_matched_q09996_episode_alignment/as a seed-stability gate. Rate is stable, but raw sample-matched clustering spans23.5--42.9 km; do not tune spatial transport against the aggregate alone. 19. Treatdata/derived/models/central_italy_matched_3episode_transfer_suite.jsonas a matched-model smoke baseline only. Its historical-rate control beats synthetic transfer and all thresholds are recall-driven; do not interpret the high balanced accuracy as skill. 20. Usedata/derived/models/central_italy_matched_3episode_target_calibration.jsonfor precision-aware comparisons. Report rate-matched precision and recall alongside balanced accuracy; do not select a threshold from the final holdout. 21. Usedata/derived/models/central_italy_matched_3episode_target_calibration_rolling_controls.jsonas the current precision gate: transfer rate-matched precision0.214is below the historical-rate control at0.228across four rolling folds. Improve features or training before changing threshold policy again. 22. Follow Feature And Training Options: multiscale seismic/neighbour features are now implemented behindFEATURE_MODE=multiscale; the first MLP transfer check is negative, so retain compact features as the baseline and test the multiscale path only with more data and stronger calibration. 23. Usedata/derived/models/central_italy_transformer_sweep_long/summary.jsonas the current five-seed fixed-split Transformer reference. The longer run still favours piezo/VLF-only, but do not select it until the episode-held-out range improves. 24. Usedata/derived/models/central_italy_transformer_episode_holdout/as the current nine-fold multi-task diagnostic. Mean calibrated balanced accuracy is0.5508, with a0.3788--0.7009range; improve regime robustness before adding more capacity. 25. The identical occurrence-only control scores0.5533, slightly above multi-task0.5508; keep occurrence-only as default. Only run count/energy loss-weight sweeps if a representation diagnostic justifies the added objectives, with normalization and threshold selection training-only. 26. Test domain-robust features withFEATURE_MODE=relative: use causal local/neighbour activity and magnitude/energy relative to the preceding Italy-wide baseline. Require improvement over the compact model on rolling folds and at least five of nine episode folds before retaining it. 27. Expand synthetic domain randomization across source schedules, deposition rates, thresholds, erosion, and sensor corruption. Select settings using training episodes only; reserve complete episodes for final evaluation. 28. Require cross-regime consensus and calibrated uncertainty for any future event list. Abstain when ensemble disagreement or domain distance is high, and compare predicted rates with the historical INGV rate. 29. Keep synthetic pretraining and self-supervised representation learning separate from supervised evidence. Once real labels contain both classes, use chronological real validation with seismic-only, VLF-only, astronomy-only, full, and shuffled-modality controls. 30. Treatdata/derived/models/domain_randomized_transformer_episode_holdout/as a failed but reusable stress-test baseline: mean calibrated balanced accuracy0.5096. Before adding more regimes, test target alignment and regime-relative normalization on these same 12 folds. 31. Treatdata/derived/models/domain_randomized_transformer_episode_holdout_per_window/as the normalization control: mean0.4909, below global normalization. Do not add model capacity yet; inspect event extraction thresholds, horizon labels, and causal regime-relative targets on the same folds. 32. Add a train-only target audit for each held-out episode: event count, positive rate, event timing, and source/profile metadata. Require comparable target support before interpreting model scores across regimes. 33. Obtain one exact ISEE CDF file URL from the archive, record its station/date/units/use policy, and store the unchanged file underdata/raw/vlf/japan/. Do not mark the source usable until the pull is reproducibly nonempty. 34. After installingcdflib, runINPUT=data/raw/vlf/japan/<file>.cdf ./scripts/normalize-japan-vlf-cdf.sh; inspect epoch variables and channel units before building features. 35. For the first ISEE sample, retain the CDF metadata JSON as the source contract. Confirm whether the archive variable is a scalar trace or a time-frequency array before adding a feature adapter; do not flatten a spectrum without preserving its time and frequency axes. 36. The native CDF spectrum adapter is implemented inextract-japan-vlf-cdf-features.sh. Validate its band definitions against additional Moshiri months, preserveresearch_use_only, and compare features with Cumiana only in explicitly declared scientific experiments. 37. Usebuild-japan-vlf-cdf-window-features.shagainst Japan seismic windows after the Japan USGS history is populated; require nonempty overlap and retain Japan-only research reports before any cross-region representation experiment. 38. Completed the initial Japan temporal extension through 2025-07-15: 319 normalized events, 26 mature windows, and one overlapping window for each Moshiri sample. Keep this as an ingestion gate only; it is not sufficient model coverage. 39. Use./scripts/process-japan-vlf-manifest.shas the standard Japan preprocessing entry point. Add more manifest rows only after station/date/permission metadata are recorded, then rerun the workflow and audit overlap before model training. 40. Install and monitorelfquake-japan-vlf.timeras a separate Japan research-data collector. Confirm the archive's publication delay and adjustLOOKBACK_MONTHSorMAX_FILESonly after checking storage and overlap growth. 41. Run./scripts/build-japan-vlf-cdf-dataset.shafter each refresh to produce one combined Japan VLF row per seismic window; use this artifact as the input to the Japan design-matrix join. 42. The Japan refresh now rebuilds the combined CDF windows and model-input table automatically whenWINDOWSis set. The current table has 78 target windows but 75 missing VLF rows; acquire matching CDF dates before training or cross-region comparison. 43. Usedata/derived/models/japan_vlf_model_input.m5.csvas the current Japan smoke-training table only. Its comparable M5.0 target has49/29positive/negative windows; eight now contain observed VLF, including three negative and five later/test-era windows. The refreshed 80/20 tabular run scored0.375calibrated balanced accuracy and failed negative recall; do not tune on this sample. 44. Run./scripts/probe-japan-target-thresholds.shafter each Japan refresh. Select thresholds using class balance and VLF overlap; do not optimize the threshold against model scores while only three VLF windows are observed. 45. Run the Japan VLF sequence smoke test with the capture-specific manifests listed indata/derived/models/japan_moshiri_sequences/manifests.txt; treat it as an interface check only until the number of VLF-observed target windows grows substantially. 46. Require more than one observed Japan VLF capture in both chronological train and test periods before evaluating the Japan Transformer. The absolute coverage blocker is cleared, but seven windows remain insufficient for a reliable score and most target rows are still masked. 47. Usedata/derived/models/mixed_source_transformer_fixture.alignment.jsonand Transformer Fixture as the current cross-source inventory. Implement the common window builder only after preserving domain-specific time scales and missing-modality masks. 48. Use./scripts/build-common-transformer-fixture.shto refresh the 5,546-row mixed window fixture. Treat itsready_for_smoke_trainingstatus as a tabular/tensor interface gate only; build continuous per-dataset sequence inputs before patch-Transformer training. 49. Use./scripts/materialize-common-transformer-sequences.shfollowed by a two-epochsequence_common_multimodalCPU smoke run. The current run completes across 48 manifests but scores0.489320calibrated balanced accuracy, below the majority baseline; retain it as an interface/masking test only and do not tune against it. 50. Run./scripts/audit-common-transformer-alignment.shbefore any mixed-source model comparison. The current audit finds 5,301 Italy seismic/VLF/astronomy rows, 8 Japan seismic/observed-VLF rows, and 167 fully co-observed synthetic sensor rows; acquire more temporally matched Japan VLF before evaluating Japan transfer. 51. Use./scripts/compare-japan-synthetic-shapes.shand Japan And Synthetic Shape Comparison as the current signal-shape gate. The first run finds Japan VLF low-band power0.771versus synthetic piezo0.361, and Japan seismic event energy is much sparser and heavier-tailed than the synthetic catalog. Tune rate/clustering and the causal piezo envelope before another transfer sweep. 52. Run./scripts/evaluate-piezo-japan-shape-variants.sh. Retain a slow-envelope setting only if it moves the piezo PSD and low-band ratio toward the Japan feature trace across multiple seeds without creating an artificial trend. 53. Run./scripts/tune-japan-avalanche-events.shover multiple seeds and time-held-out episodes. Treat the current 25-event candidate as a rate-calibration control only; improve clustering without relying on a global event cap. 54. Usedata/derived/reports/japan-avalanche-policy-seeds/summary.csvas the current multi-seed gate. The fixed policy is rate-stable enough for a control but fails clustering, so do not promote it as the simulator default. 55. Add a configurable clustered-loading/relaxation regime to the sandpile simulation, preserving localized source locations and deterministic seeds. The first per-source persistence smoke run was stable but did not improve clustering; do not promote it. 56. RetainSOURCE_REGIME_DECAY=0.2,SOURCE_REGIME_BOOST=0.8, andTARGET_FILL_REGIME_FLOOR=0.25only as a negative control. The 5,000-step probe remained stable but failed event-shape alignment. 57. Refactor the synthetic event aggregation boundary or add a longer-lived stress-release state. Compare event inter-arrival, burst-run, and PSD metrics without changing source coordinates or injecting independent events. 58. Usedata/derived/reports/avalanche-burst-extractor/summary.csvas the current burst-extraction diagnostic. Thedecay99_gap120candidate improves rate, PSD slope, and tails without a global event cap, but remains a single-seed control. 59. Usedata/derived/reports/avalanche-burst-seeds/summary.csvas the multi-seed diagnostic. Rate and tail behavior are promising, but burst clustering and PSD sign are not stable; do not use it for model fixtures yet. 60. Add train-only burst-threshold calibration: estimate the baseline-score threshold from training episodes, apply it unchanged to held-out seeds, and report event rate, burst runs, tails, and PSD without per-test-episode retuning. 61. Usedata/derived/reports/avalanche-burst-train-test/summary.csvas the leakage-safe gate. The fixed threshold fails held-out rate transfer (0.101–0.170versus0.084), so the extractor is not ready for fixtures. 62. Retain the relative-baseline normalization as a negative control. It does not improve held-out rate transfer, so do not use it for fixtures. 63. Use the bounded source stress reservoir as the next simulator candidate. The 500-step probe is stable and produces localized releases, but it has not yet passed event-shape evaluation. 64. Retain the tested stress-reservoir parameters as a stable negative control. They produce releases without safety failure but only one extracted event over 5,000 steps. 65. Do not increase stress release mass blindly. The per-source cooldown control (SOURCE_STRESS_RELEASE_COOLDOWN_STEPS=120) is stable but still yields one extracted direct event over 5,000 steps. Redesign the stress-release coupling or event representation so localized releases create distinct, short-lived global activity episodes, then repeat the causal burst and safety gates. 66. Use the optional*.source_stress.csvoutput to compare source-local stress-release pulses with nearby avalanche activity using causal, per-regime normalization. The output is implemented and preserves source coordinates; evaluate it against the existing direct signal rather than replacing it silently. 67. Build the release-aware diagnostic and report lead/lag, local activity, and event-shape metrics. Promote only if it improves held-out multi-seed alignment without safety failures or independent event injection. 68. Runscripts/analyze-source-stress-alignment.shover tagged episodes. The first 1,000-step cooldown result has 938 release rows, only5.9%with positive local excess activity, local/global excess-AUC ratio0.037, and median local peak lag96steps. This does not support a source-local precursor effect yet. 69. Repeat the release-aware diagnostic over multiple seeds and radii. Completed on seeds40–43: positive local excess was2.2%at radius 16,6.8%at radius 32, and23.5%at radius 64; local/global excess-AUC ratios were0.010,0.043, and0.188, with median lags near 90–102 steps. This is weak and spatially broad, so the stress reservoir remains a negative control. 70. Improve the spatial avalanche activity representation before further simulator tuning. Compact*.avalanche_regions.csvoutput is now available as a configurable regional grid, preserving locality without per-cell output. Use it to rerun the source-stress diagnostic and compare against the existing global signal. 71. Add a region-aware source-stress diagnostic using the regional table. Report activity in the release region versus matched non-release regions, with time-held-out seeds. 72. Track the Japan VLF archive until 28 July data are available. The first pre-event check covers 13 daily Moshiri CDFs from 15–27 July; July 26 is the highest robust-deviation day, but earlier elevated days prevent a precursor claim. Re-run with hourly event-day/post-event coverage and matched controls. 73. Keep the Italy transfer-trial artifact atdata/derived/models/real_transfer_trial/report.jsonas the current chronological baseline. Compare future multimodal runs against its confusion matrix and historical-rate control, with missing-modality masks reported explicitly. 74. Compare the two Italy baselines using identical time ranges and target contracts before interpreting any apparent improvement. The mismatch is documented in Italy Baseline Comparison; do not attribute the transfer-trial difference to VLF or astronomy until the spatial feature and label definitions are matched. 75. Build a real-VLF holdout with observed variation before making a modality claim. A new Cumiana image was captured at2026-07-29T08:45Zand integrated; the descriptive association table now has four VLF-observed weeks, but the new target is pending. The separate M2.5 central-Italy table has 229/50 positive/negative rows, but its chronological test has only one negative and its0.990909score is majority-class driven. Keep it as a data-shape artifact, not model evidence. 76. Extend Cumiana capture coverage until at least three positive and three negative VLF-observed target weeks exist in the same threshold and horizon contract. Then run matched seismic-only, VLF-image-only, and full multimodal time-held-out baselines with training-only threshold calibration. 77. Completed the self-supervised VLF refresh: 280 rows, 257 causal windows, and 41 exploratory alerts at score>=0.8. Re-label the new2026-07-29T08:51Zwindow after its horizon matures, keeping the anomaly threshold and model checkpoint fixed for the audit. 78. Added the dated current-window auditdata/derived/reports/italy_data_coverage_20260729.json: 280 INGV events, 284 VLF metadata records, 257 anomaly windows, and 4 VLF/seismic overlap weeks. Use this alongside the longer historical coverage report to avoid mixing time ranges. 79. Added capture continuity monitoring with./scripts/report-vlf-capture-gaps.sh. The current report finds 280 captures and 8 gaps over one hour, including a 308-hour gap before the July 29 capture; use it to verify the systemd collector is producing sustained VLF coverage before interpreting future anomaly/earthquake overlap. 80. Leave the collector running and rerun./scripts/report-vlf-capture-gaps.shafter several 30-minute intervals. Confirm that new captures are being added and that no multi-hour gaps recur before using future VLF anomaly windows in analysis. 81. The collector has since added a2026-07-29T11:15ZCumiana capture. Current prospective summaries have 280 rows and one pending target each; continue monitoring the 30-minute cadence and rerun the gap report before the next matured-label refresh. 82. Ran the refreshed real-VLF versus synthetic-piezo embedding probe. The closest 25% synthetic windows improved centroid distance3.045->2.531and nearest distance2.186->1.404, but synthetic reconstruction MSE remained11.604versus real0.562. Treat inlier filtering as diagnostic only; next improve simulator signal-shape statistics before model training. 83. Ran the 20,000-step piezo shape sweep.gain_burstis the leading single-run candidate (1.512centroid,1.028nearest distance), withfast_burstclose behind; validate both across multiple seeds and compare against the current profile before changing defaults. 84. Completed the multi-seed variant check with./scripts/evaluate-piezo-vlf-variant-seeds.sh.gain_burstled narrowly on the fixed alignment seed; the independent model-seed rerun gives means of1.7488centroid /1.0537nearest forgain_burst,1.7558/1.0636forfast_burst, and1.7490/1.2575for current. The preference is weak because per-run ranges are wide; do not promote a transform yet. 85. Repeat the cross-region smoke run with at least three seeds. Measure coordinate error, in-Italy rate, spatial dispersion, and duplicate-location rate; compare event count against the historical spatial-rate baseline before treating the output as more than an interface artifact. 86. Use./scripts/trial-weekly-event-forecast.shas the current end-to-end event-list contract smoke test, not as a validated predictor. 87. Use./scripts/balance-italy-synthetic-episode-rates.shonly as an auditable training/observation-model diagnostic. It can thin overactive episodes, but it must not synthesize events for underactive episodes; the matched rerun is preferred. 88. Four pipeline defects were fixed and covered by regression tests (tests/test_common_window_fixture.py,tests/test_cross_region_smoke_map.py,tests/test_japan_model_input.py): the common-window fixture split now counts only labeled rows, so a growing pending-future tail can no longer empty the test partition; the cross-region smoke map now renders the most recently matured held-out week instead of the oldest; Japandataset_idis hashed when a window is covered by many hourly CDFs, keeping sequence directory names within path limits while the full source list stays in its own column; and the burst extractor treats a zero quantile as "no quantile filtering" instead of silently dropping the weakest step. The regenerateddocs/images/cross-region-generative-smoke.pngnow shows 4 generated coordinates against 0 actual events in the newest matured week, replacing the previous oldest-week view; the map remains an engineering visualization, not a forecast. -
Two stale-input defects were found and fixed on 2026-08-09, both of which had been silently inflating apparent reproducibility:
refresh-prospective-labels.shrewrote only theCOMBINE_START_DATE-scoped combined catalogs and never touchedevents_italy_all_available.combined.normalized.csv, which is what the transfer trial, coverage report, VLF/event association, weekly forecast, catalog calibration, and cross-region smoke all read. That catalog was frozen at 2026-07-07 for a month. The refresh now rebuilds bothall_availablecatalogs; verified purely additive against a fullbackfill-ingv-history.shrun (5,007 -> 5,013 rows, 0 removed).prepare-italy-spatial-model-inputs.shrebuilt its two upstream inputs only when missing, so a "refresh" reused Aug-4 spatial targets and reported an unchanged 478 anchors / 5,301 labeled rows. It now rebuilds by default (REBUILD_INPUTS=0restores the old behaviour). After the fix the table is 11,020 rows / 7,258 labeled across 580 anchors.- Treat any score that reproduces to six decimal places across a refresh as a staleness signal, not a stability result. Audit the remaining scripts in the
all_availablelist for the same pattern.
-
Checked the Cumiana VLF record for precursors to the 2026-08-04 M4.3 near Pisa (event
46769822,2026-08-04T08:15:18Z, 43.6925N 10.3263E, depth 8.2 km, 275 km from the Cumiana receiver). No supportable precursor. Details in Pisa M4.3 VLF Check. Two blocking facts: the collector has no captures at all between 2026-07-16 and 2026-07-29, so the first 8 of the 14 pre-event days have zero data; and the two highest-scoring days in the whole record (2026-07-29 max0.9877, 2026-07-30 max0.9765) are a capture-gap artifact, not a signal. Withlookback-steps=24, every window straddling the 13-day gap mixes pre- and post-gap frames: those 21 windows average0.9113and the next 24 clean windows drop as a step function to0.4074. The clean pre-event stretch averages0.4127, below the pre-gap baseline0.5429, while the post-event period is the highest in the record at0.7214. The gap-aware guard this called for is now implemented; see item 91. -
Implemented the gap-aware guard in
score-sequence-anomalies(item 90 follow-up). Each window now carriesmax_step_secondsandspans_capture_gap; a window is flagged when any internal inter-frame interval exceeds--max-capture-gap-seconds, defaulting tolookback_steps x median_step(12 h for the current 24-step / 30-minute Cumiana record). Flagged windows can never raise an alert, and a flagged latest window sets the forecast status toinvalid_capture_gap. Regression cover is intests/test_sequence_anomaly_capture_gap.py, which pins the contiguous-block and step-recovery behaviour and confirms ordinary capture jitter is tolerated. The rescored record shows 66 of 558 windows span gaps, and 43 of the 101 raw>=0.8alerts were gap artifacts — 43% of the historical alert record. Gap-spanning windows average0.7908versus0.5143for clean windows. After filtering, 2026-07-29 and 2026-07-30 leave the top clean days entirely. Any earlier count of "exploratory alerts" from this scorer, including the 41 recorded in item 77, is inflated and should be recomputed before use. -
Resolved 2026-08-10; see item 93 and Capture-Era Shift. One confound plausibly explains three separate negative results from 2026-08-09 at once — the chronological spatial baseline scoring below chance (
0.4154, item 6), all five shuffled controls beating real time order (item 8), and the self-supervised scorer treating the collector restart as the record's top anomaly (item 91). The Cumiana record splits into a sparse early era (2026-06-30 to 2026-07-16, with multi-day gaps and single-capture days) and a dense era from 2026-07-29 onward at a roughly 30-minute cadence. Chronological splits put the eras on opposite sides of the boundary; shuffling mixes them, which is why shuffling helps.Steps:
- Compare VLF image-feature distributions between the two eras directly: per-feature mean, spread, and a two-sample distance for each of the 14
vlf_image_*and 7vlf_metadata_*features. Report which features move most. - Check whether the shift is physical or instrumental.
vlf_capture_count,vlf_latest_age_seconds, andvlf_total_bytesare aggregation artifacts of cadence, not signal, and will shift mechanically with capture density. Separate those from the intensity/band/streak features before drawing conclusions. - Re-run the grouped temporal baseline restricted to the dense era only. If the sub-chance score is a cross-era artifact it should move toward the majority baseline; if it persists within one era, the problem is the features or targets, not the split.
- Re-run the permutation controls within the dense era only. A clean null requires controls that cannot leak era membership.
- Do not "fix" this by dropping cadence-derived features or reweighting until the diagnostic says which mechanism is operating. Record the outcome here either way, including a negative one.
Until this resolves, treat every Italy score in items 1, 6, 7, and 8 as uninterpretable rather than as evidence against multimodal value. They are currently confounded, not refuted.
- Compare VLF image-feature distributions between the two eras directly: per-feature mean, spread, and a two-sample distance for each of the 14
-
Completed the capture-era diagnostic (item 92). The confound is real, it explains the sub-chance score, and removing it does not rescue the model. Full write-up in Capture-Era Shift; reproduce with
./scripts/diagnose-vlf-capture-era-shift.sh, then./scripts/evaluate-italy-spatial-baseline.shand./scripts/evaluate-italy-spatial-coordinate-control.shagainst the per-era CSVs it writes.- Item 92's premise was wrong on one point. The record is not a sparse era followed by a dense one. It is two dense eras at the same 30-minute cadence —
era_02026-06-28 to 2026-07-05 (277 anchors, 5,263 labeled) andera_32026-07-28 to 2026-08-07 (301 anchors, 1,957 labeled) — separated by a collector outage with gaps of 104 h, 127 h, and 308 h and two isolated captures. The largest within-era gap is 15 h, an overnight stop. - The chronological split boundary is the outage boundary. Training runs to 2026-07-29T08:53Z and testing from 2026-07-29T09:23Z, so training is almost all
era_0and the test partition is entirely insideera_3. That is why shuffling helped: it removed a shift the real split imposes. - The shift lives in image content, not cadence. Median absolute standardized mean difference by family: VLF signal
0.842(max1.325), VLF cadence-derived0.321(max0.474), seismic0.508, astronomy0.261, quality0.000. All nine largest shifts are content features — bands 0-3, intensity, high-intensity ratio, and hot-colour ratio all fall, vertical streaks rise, bands 4-5 do not move (d+0.03), image size is unchanged. That pattern suggests a gain or colour-scale change across the outage, but derived features cannot establish it; compare raw spectrograms next. - Targets shift too, and this is genuine, not an artifact: positive rate
0.1543->0.2427,seismic_max_magnitude2.65->3.29,seismic_event_count4.94->6.85. Separately,astro_capture_countgoes0.823->0.000, so astronomy is absent fromera_3entirely and every chronological astronomy ablation tests on a constant-zero feature. - Restricted to one era, the sub-chance score disappears:
0.655320inera_0and0.575000inera_3, against0.412604on the full table. The0.4154in item 6 was a cross-era artifact.era_0's0.655320is the same figure as the original grouped-time smoke baseline, because that 5,301-row table was essentiallyera_0alone. - The within-era permutation null is clean, and the answer is unchanged. Five controls per era at
EPOCHS=600to match the real run — the earlier item 8 comparison ran controls at the script default of 100 epochs against a 600-epoch real run, which was not like-for-like.era_0controls mean0.650519(range0.603404--0.667172),era_3mean0.592576(range0.572778--0.630354); three of five beat real order in each era. - The decisive control: dropping the cell coordinate columns collapses both eras to exactly
0.500000.target_cell_latitudeandtarget_cell_longitudecarry the largest weights in every fitted model. The fixed-cell spatial model is a static per-cell base-rate lookup with no temporal component, which is consistent withseismic_onlyandvlf_onlyalready collapsing to0.5. - Do not reweight eras or drop cadence features on the strength of this. The blocking finding is the absence of temporal signal in either era, which no reweighting addresses.
- Item 92's premise was wrong on one point. The record is not a sparse era followed by a dense one. It is two dense eras at the same 30-minute cadence —
-
Fixed a latent standardization defect found while reading item 93's fitted coefficients. All five copies of
scales.append(scale if scale else 1.0)treated a near-constant column as varying: 4,199 copies of125.69accumulate a mean of125.68999999999998, giving variance2.02e-28and scale1.42e-14, which is nonzero and therefore used. The constant column then standardized to1.0instead of0.0andastro_noaa_solar_cycle_f107_valuecarried the largest weight (-0.894) in theera_0model. Here it was benign — same constant in train and test, so it acted as a second intercept, and the full-table score moved only0.415351->0.412604— but a held-out partition carrying a different constant (next month's F10.7, a quality flag that flips after a collector change) would standardize to order1e13and saturate every prediction. Now guarded by a relative tolerance insrc/elfquake/models/scaling.py, applied acrosstemporal_holdout,ablation_smoke,logistic_smoke,torch_tabular, andtorch_sequence, with regression cover intests/test_capture_era_shift.py. -
Next up, in order — superseded by items 100–103. (a) Completed 2026-08-14; see item 96. (b) Now the binding constraint on every modality; promoted to item 103. Re-examine whether the fixed-cell target design can express anything beyond per-cell base rate, given that removing coordinates zeroes the model — either the temporal features carry no information at this horizon, or the target contract needs per-cell rate residuals rather than raw occurrence. (c) Completed 2026-08-17; see item 102.
era_3now runs2026-07-28–2026-08-16, 483 anchors and 6,802 labeled rows, and the within-era conclusions reproduce on 3.5x the test rows. (d) Completed 2026-08-22; see item 106. Audit the remaining scripts for the item-89 staleness pattern. -
Completed item 95(a). The Cumiana receiver's colour scale was changed during the outage, so
era_0andera_3image features are not on a common scale and must not be pooled. Full write-up in Cumiana Colour-Scale Change; reproduce with./scripts/diagnose-vlf-palette-shift.sh. Everylast_E_VLFcapture embeds its own 96-step colourbar above a fixed-100 dB … 0 dBtick ruler. Across all 699 captures the ramp takes exactly two values with no intermediates: solid-red onset at step59for all 277 captures up to2026-07-06T21:45Z, and step48for all 422 captures from2026-07-11T06:00Zonward — a shift of 11 steps =11.58 dB, downward. The ruler, the right-hand frequency tick rows, and the842x573geometry are pixel-identical across both, so this is a colour-scale setting and not a re-plot. Three consequences. First, the pooling question is settled on its own terms:vlf_intensity_*,vlf_hot_color_ratio,vlf_high_intensity_ratio, andvlf_band_*_meanare all functions of pixel colour, and the same colour denotes a level11.58 dBlower after the change. Second, the change point sits inside the outage (2026-07-06→2026-07-11), so it is invisible to any diagnostic that compares only denseera_0against denseera_3. Third, the item-93 "bands 0-3 fell, bands 4-5 unchanged" split is not frequency-selective physics: bands 4-5 read the separate sub-1500 Hz zoom panel and sit deep in the palette's saturated region in both eras. Decoding pixels back to absolute dB through each image's own colourbar — hour-matched to11:00–13:00UTC against the diurnal cycle, over the rightmost 45 columns of the upper panel — leaves the late era15.8–23.2 dBlower across the four bands resolvable under both palettes (-80.0 … -49.5 dB, the overlap of the two displayed windows; the other four are censored by saturation and reported as such). Not settled: whether that underlying level change is instrumental or atmospheric. The evidence leans instrumental — broadband, roughly uniform over two decades of frequency, step-like, and coincident with an operator changing a display setting — but images alone cannot separate a front-end gain reduction from a quieter period, and this needs station metadata or operator contact. Regression cover is intests/test_vlf_palette_shift.py. -
(a) completed 2026-08-21; see item 105(b). The features exist and a
17.9 dBlevel step survives them, so (b) still applies and the eras stay separate. (d) is now the only open route. (a) Add palette-inverted absolute-dB features tosrc/elfquake/features/vlf_image.py. Each image carries the colourbar it was drawn with, so dB features are era-invariant by construction and would remove the need to keep the eras apart — but they must record the censored fraction per capture, because the two palettes resolve different dB windows and a band below the shared floor is missing, not quiet. (b) Until then, do not poolera_0andera_3raw pixel features, and do not read any cross-era comparison built from them as physical. (c) Re-run./scripts/diagnose-vlf-palette-shift.shafter each capture refresh; a third palette variant would invalidate pooling again. (d) Ask the Cumiana operator whether receiver gain changed in the2026-07-06–2026-07-11window; that is the only route to settling the instrumental-versus-atmospheric question. -
Raw capture coverage now runs to
2026-08-14, well past the four-dayera_3labeled window recorded in item 93. Dense days run2026-07-29to2026-08-12at the nominal 30-minute cadence (with2026-08-06and2026-08-13missing), against2026-06-29–2026-07-06forera_0. Rebuild the spatial model inputs so item 95(c) is evaluated against the coverage that exists rather than the coverage recorded a week ago — but note this is raw-capture coverage, and labeled rows still lag it by the seven-day target horizon. -
Superseded 2026-08-22 by item 107; the state described below no longer holds and should not be cited as current. Status of astronomical data in transformer training, audited 2026-08-14: it was not incorporated in any meaningful sense, and the missing-modality mask that is supposed to say so was broken. Evidence, from
data/derived/models/common_transformer_fixture_sequences/italy_all_astronomy_sequence/anddata/derived/multimodal/all_italy.spatial_vlf_image_windows.labeled.csv:- Acquisition is not running.
data/raw/astronomy/captures/holds 11 files across two days,2026-06-29and2026-07-06, against 699 Cumiana captures across 24 days. Nothing has been pulled in five weeks. - Nothing writes to the astronomy root.
deploy/systemd/elfquake-prospective.servicepassesELFQUAKE_ASTRONOMY_METADATA_ROOTinto bothupdate-prospective-vlf-tablecalls on every 30-minute run, but noExecStartline fetches astronomy. The timer has re-read a frozen directory for five weeks. This is the item-89 staleness class in the service unit rather than in a script, and it should be counted against item 95(d). - Exactly two channels reach the transformer, per the fixture manifest
channel_fields:astro_capture_countandastro_noaa_solar_cycle_f107_value. Neither is usable. astro_noaa_solar_cycle_f107_valueis constant at125.69across all 11,020 table rows and all 9,082 sequence rows — zero variance, no information. It is the monthly solar-cycle series andastro_noaa_solar_cycle_f107_monthis stuck at2026-05while the table runs into August, so it is also stale. This is the exact column that produced the item-94 standardization defect.astro_capture_countis collector bookkeeping, not a measurement: 10,051 of 11,020 rows are0, and the nonzero values (1x19,2x589,7x57,9x304) fall on 3 of the 17 fixture days — the days someone ran the fetch by hand. Feeding it to a model feeds it an indicator for collector activity, the same artifact class as the capture-gap and capture-era defects.- The missing-modality mask never fires.
quality_missing_astrois0for all 11,020 rows and the sequence masks arepresent=1for all 9,082 rows, including the 10,051 rows with no astronomy capture at all. Cause:src/elfquake/features/astronomy.py:60sets the flag present ifcaptures or moon_phase or f107_value, andf107_valueis always non-empty because it comes from a once-pulled historical monthly series.src/elfquake/models/real_transfer_trial.py:133documents astronomy as "represented by an explicit missing-modality mask"; the mask exists but is unconditionally true, so the model is told astronomy is observed everywhere when it is observed almost nowhere. - The connectors already exist and are unused.
fetch-gfz-kp-ap,fetch-kyoto-dst,fetch-ncei-goes-xrs,fetch-f107-dailyand their four normalizers are wired into the CLI, andsrc/elfquake/normalize/space_weather.pydefines the output schemas. There is nodata/derived/astronomy/directory: they have never been run into the derived layer. - Some history is already unrecoverable from the live endpoints.
data/raw/astronomy/manifest.csvmarksnoaa_swpc_kp_3h,noaa_swpc_dst_1h,noaa_goes_xray_7d, andnoaa_swpc_f107_30d"confirmed not archival" — rolling windows. The June–August geomagnetic history must come from the archival mirrors (GFZ Kp/ap since 1932, Kyoto Dst, Spaceweather Canada daily F10.7, NCEI GOES XRS), not from these.
Consequence for the modeling record: no astronomy ablation to date is a negative result.
seismic_astronomyandfull_multimodalcannot separate an astronomy contribution from a constant plus a collector counter, so astronomy's value is untested, not disproven. Do not describe it as tested. - Acquisition is not running.
-
Priority queue for astronomy, ahead of items 95(b)–(d) and 97. (a)–(f) and (h) completed 2026-08-17; see item 101. (g) remains.
(a) Fix the mask contract in
src/elfquake/features/astronomy.py.quality_missing_astromust reflect whether the window was actually observed, not whether a once-pulled monthly constant is on disk. A slowly varying historical series is background context, not an observation of that window. Add regression cover pinning that a window with no captures marks the modality missing.(b) Drop both current astronomy channels from the fixture.
astro_noaa_solar_cycle_f107_valueis zero-variance and stale;astro_capture_countis collector bookkeeping and must never be a model input. Keep them in the table as provenance if useful, but exclude them fromtensor_specchannel selection.(c) Add a channel gate to the sequence/fixture builder that fails loudly on a channel that is constant across the whole axis or whose mask is
present=1everywhere while its values are imputed. This generalises the item-94 fix from the standardizer to the builder, where it would have caught this five weeks earlier.(d) Backfill real geomagnetic history for the Cumiana window (
2026-06-28onward) with the existing connectors: GFZ Kp/ap (3-hourly, CC BY 4.0), Kyoto Dst (hourly, non-commercial), Spaceweather Canada daily F10.7 (daily, from 2004-10-28). Normalize to UTC, preserve source identifiers, URIs, and provenance per the data rules, and write todata/derived/astronomy/.(e) Define the alignment from those cadences (3-hourly, hourly, daily) onto the 30-minute VLF anchors before building features. State the interpolation or hold rule explicitly and carry a per-anchor staleness field; do not let a daily value silently present as a 30-minute observation. This is the multimodal time-window alignment concern in the project priorities, and astronomy is the modality where the cadence mismatch is largest.
(f) Replace
astro_usno_next_phasewith a continuous quantity. A next-event category is a sawtooth countdown, not a physical state; a lunar phase angle or tidal potential at the anchor time is both continuous and cheap to compute.(g) Only after (d)–(f) land, re-run the
seismic_astronomyandfull_multimodalablations. Until then those runs measure a constant.(h) Add an astronomy fetch to
elfquake-prospective.service, or a separate timer, so the collector stops reading a directory nothing writes to. -
Completed item 100(a)–(f) and (h) on 2026-08-17. Astronomy now reaches the model as observations with an explicit alignment rule; it is still untested. Full write-up in Astronomy Alignment. What changed, against the item-99 audit:
(a)
quality_missing_astronow reports observation rather than computability. It is set only when Kp, Dst and F10.7 are all missing at the anchor, and ephemeris channels are excluded from the test by design — they are always computable, so letting them satisfy it would pin the flag to0forever, which is exactly what the old implementation did. Per-channelquality_missing_kp,quality_missing_dst,quality_missing_f107flags carry the detail. Regression cover intests/test_astro_features.py.(b)
astro_capture_count,astro_sources,astro_latest_capture_utc,astro_usno_next_phase*, andastro_noaa_solar_cycle_f107_*are gone from the prospective table.src/elfquake/features/astronomy.pyis marked superseded and retained only for the olderbuild-multimodal-smokepath.(c)
src/elfquake/models/channel_gate.pyruns insidebuild_common_window_fixtureand raises onconstant_channel,unmasked_missing_channel, andempty_channel. This is the item-94 standardizer fix generalized to the builder. A channel that is constant by design must be named with--allow-constant-channel, so the decision is recorded rather than inferred;--no-strict-channelsdowngrades to a report entry.(d) Real archives are normalized to
data/derived/astronomy/: GFZ Kp/ap (276,496 rows, 1932 onward), Kyoto Dst (three months, hourly), Spaceweather Canada F10.7 (23,853 rows, 2004 onward). Kyoto's monthly pages are fixed-width, not whitespace-delimited — a missing hour is the sentinel9999and consecutive missing hours run together with no separating space, so a whitespace split silently merges hours. The parser slices by column.dst_tieris carried through normalization because recent months are realtime-only and realtime values get revised; the previous month is refetched on every run.(e) The alignment rule is zero-order hold, never interpolation: each channel takes the most recent reading whose observation interval has closed at or before the anchor, so a 09:00–12:00 Kp bin is not usable at 09:57.
astro_*_age_hourscarries the staleness, and the hold expires at 6 h (Kp, Dst) or 72 h (F10.7) rather than manufacturing a constant.(f)
astro_usno_next_phaseis replaced by continuous quantities from a new low-precision ephemeris (src/elfquake/features/ephemeris.py, Meeus truncated series, checked against the worked examples and 2026 lunation times to ~2 arcmin): lunar phase angle with a sin/cos encoding, illuminated fraction, lunar distance, and the degree-two lunar + solar tidal potential at the Italy box centre, with min/max/range over the lookback window.(h)
refresh-space-weather.shpluselfquake-space-weather.{service,timer}run daily. Deliberately not on the prospective job's 30-minute cadence: the Kp/ap and F10.7 sources are whole-history archives of roughly 16 MB and 2 MB and publish at most once a day. The prospective job now reads--space-weather-root data/derived/astronomyinstead of a raw-capture directory nothing wrote to.--astronomy-metadata-rootis retained as a deprecated no-op so the installed unit does not break before it is reinstalled; the units indeploy/systemd/need copying to/etc/systemd/system/and asystemctl daemon-reload.(g) Done. Astronomy adds nothing measurable on held-out data. See item 102 for the numbers. The features do reach the model —
build-tensor-specon the rebuilt fixture assigns 19 channels to the astronomy modality against the 2 the item-99 audit found, withmissing_cell_count5890, so the mask fires for the first time — and the ablation still comes back null.Result over the 761 rebuilt anchors:
astro_kp0–7.333(20 distinct),astro_ap0–154,astro_dst_nt-150–+53nT (101 distinct),astro_f10794.8–258.1(65 distinct),astro_tidal_potential-0.804–+1.278(761 distinct). The window contains a real geomagnetic storm, so there is something to test against. This is not evidence that astronomy helps. It makes item 100(g) askable for the first time; until those ablations run on held-out data, astronomy remains untested rather than disproven.The channel gate earned its place on its first real run, rejecting 24 defects in the existing fixture: 19 constant channels across the synthetic and Japan tables (
*_sample_count,relaxation_converged_*,unstable_cell_count_*,safety_released_mass_*,japan_ch*_valid_fraction_*), now named inscripts/build-common-transformer-fixture.shso the decision is recorded; and 5 unmasked-missing astronomy channels caused by a real defect in the fixture builder — a dataset that never carried a modality has no flag column for it, so the unioned row read blank, and blank is not "present". Blankquality_missing_*cells are now filled as1. -
Completed item 100(g) on 2026-08-17. Real astronomy and geomagnetic features do not improve held-out prediction. This is a negative result, and this time it is a real one.
An earlier draft of this item claimed a held-out ablation runner had to be written first. That was wrong:
evaluate_temporal_holdoutalready evaluates every group inABLATIONSon held-out rows with thresholds calibrated on training rows only. Only the in-sampletrain-ablation-smokepath lacks it.Run per era (item 96 requires it — the chronological split of the full table lands on the collector outage), grouped by time, 600 epochs, thresholds calibrated on training rows only. Calibrated held-out balanced accuracy, against a majority baseline of
0.5:Group era_0era_0no coordsera_3era_3no coordsseismic_only0.5000000.5000000.5140240.514024seismic_astronomy0.5000000.5000000.5181230.518123seismic_vlf0.5000000.5000000.5077670.507767full_multimodal0.5000000.5000000.4946060.494606vlf_only0.5000000.5000000.4976270.497627all_features0.6754090.5000000.5573560.494606era_0is 4,199 train / 1,064 test rows;era_3is 5,434 / 1,368.Three readings.
(a) Astronomy contributes nothing.
seismic_only→seismic_astronomymoves0.500000→0.500000inera_0and0.514024→0.518123inera_3. That+0.004on 1,368 test rows is noise. Adding 19 real, varying, correctly aligned channels — Kp to7.333, Dst to-150 nT, a continuous tidal potential — changes nothing. The item-99 finding was that astronomy was untested; it is now tested, and it does not help at this horizon in this design.(b)
full_multimodalis worse thanseismic_onlyinera_3(0.494606vs0.514024), i.e. below chance. Adding modalities to this target design degrades it.(c) The item-96 coordinate finding reproduces exactly, on 3.5x more
era_3rows.all_featuresis the only group above chance and the only one containingtarget_cell_latitude/longitude/degrees; removing them collapses it to0.500000and0.494606. Every other group is bit-identical with and without, because none of them ever contained the coordinates. The model remains a static per-cell base-rate lookup.Do not read (a) as "astronomy is irrelevant to earthquakes." It says these features, at a 7-day horizon, on 1.5-degree fixed cells, in a design whose only demonstrated signal is a per-cell base rate, add nothing. Item 95(b) — whether this target design can express anything beyond per-cell base rate — is now the binding constraint on every modality, not just astronomy. Testing more feature families against a target that only encodes a base rate will keep returning
0.5. -
Completed on 2026-08-20. The fixed-cell target design carries seven independent label changes in its held-out partition. The row count was never the sample size. Full write-up in Target Design. Two new reproducible artifacts:
./scripts/evaluate-italy-spatial-cell-stratified.shscores balanced accuracy inside each cell and averages over cells, so any cell-constant predictor scores exactly0.5. This is the third standing control, alongside the coordinate control and the permutation control, and it should be run on every fixed-cell result from now on. It also reports an explicitstratum_base_ratecontrol — the per-cell training positive rate used directly as the score — so the quantity being neutralized is named and measured rather than inferred../scripts/diagnose-spatial-target-design.shcounts what a candidate design offers before any model is fitted: per cell, how many times the label changes between consecutive anchors, over the whole record and inside the same grouped-time held-out partition the evaluator forms.
Anchors are 30 minutes apart and the horizon is 7 days, so consecutive target windows overlap by
99.7%. A cell's label can change only when an event enters or leaves the horizon. Rows are not observations; label transitions are.era_0: zero of 19 cells have a held-out label that varies at all. Its held-out window is 27.5 h against a 7-day horizon. The stratified metric is undefined, and the0.675409pooledall_featuresscore from item 102 is entirely the per-cell rate.era_3: 3 of 19 cells vary, each by exactly one transition across 72 anchors.all_featuresscores0.604489stratified against0.557356pooled andseismic_astronomyscores0.625000, but those come from three label changes and mean nothing. Thestratum_base_ratecontrol behaves exactly as designed:0.677416pooled,0.500000stratified.
The binding constraint is event supply. The 802-anchor span holds 252 events at M≥2.0 and 68 at M≥2.5. The current 1.5° / M≥2.5 / 7-day design yields 27 full-record and 7 held-out transitions across a 11,989-row table. The sweep over cell size, magnitude threshold, and horizon gives full-record / held-out transitions:
Cells M min 1 day 2 days 3 days 7 days 0.75° 2.0 111 / 32 88 / 23 83 / 18 61 / 10 1.5° 2.0 94 / 27 70 / 19 65 / 14 40 / 6 1.5° 2.5 31 / 10 31 / 8 29 / 6 27 / 7 1.5° 3.0 17 / 6 16 / 5 15 / 4 16 / 4 3.0° 2.0 51 / 14 20 / 7 9 / 2 2 / 0 Italic is the design in use; the full nine-row table is in the write-up. Shortening the horizon helps at every cell size and threshold, lowering the magnitude threshold helps more than anything else (M≥2.0 is the catalog floor), and coarser cells saturate — 3.0° / M≥2.0 / 7 days reaches a positive rate of
0.879556with two transitions in the whole record, which is the target saturation the fixed-cell decomposition was introduced to fix.Recommended: 1.5° cells, M≥2.0, 1-day horizon — held-out transitions
7→27, two-class cells10→15of 19, positive rate0.185170→0.100865. The horizon is fixed when the prospective window table is built, not when targets are labeled, so this needs a separately scoped table; the exact commands are in the write-up. Do not repoint the live 30-minute job.Every fixed-cell score reported so far —
0.415351,0.655320,0.575000,0.675409,0.557356— was computed against single-digit held-out label transitions. They are not refuted; they are uninformative, and the interval on any of them spans chance. This also reframes item 102: astronomy was not shown to add nothing, it was shown to add nothing measurable against seven label changes. That is a weaker claim than item 102 makes, and item 102 should be read with this correction. -
Completed on 2026-08-21. Two findings: the event catalog was four days stale and the live collector never refreshes it; and the shortened target design works but the era split still leaves twenty held-out label changes. Full write-up in Target Design. Reproduce with the
elfquake-fixed-cell-evaluationskill; this run is pinned atAS_OF=2026-08-21T11:22:36Z,CATALOG_END=2026-08-21T11:10:37Z.(i) The live 30-minute service does not fetch INGV events.
elfquake-prospective.serviceupdates VLF image features and the prospective window table against whatever catalog is already on disk. Onlyrefresh-prospective-labels.sh, run by hand, advances the catalog, and it had last run on 2026-08-17. Meanwhilebuild-italy-spatial-vlf-targets.shdefaultedCATALOG_ENDto wall-clock now, asserting coverage the catalog did not have, so 22 real events between 2026-08-17 and 2026-08-21 were labeled as non-events, all inside the held-out partition. On the refreshed catalogera_3held-out transitions go 10 → 19, scoreable cells 6 → 8, andseismic_onlystratified falls0.652961→0.552836. The largest apparent VLF-era result in the project was an artifact of missing events.CATALOG_ENDnow defaults to the catalog's ownmax(ingested_at_utc). The collector defect itself is not fixed — see item 105(a).(ii) The design delivers and it is still not enough. The scoped 1-day / M≥2.0 table gives 15,504 rows against 11,020, 98% labeled against 66%, positive rate
0.100718against0.185170, full-record transitions 27 → 104, whole-record held-out transitions 7 → 31, cells ever two-class 10 → 15 of 19.The 31 does not survive the item-96 era split.
era_0holds 1 held-out transition across 1,064 rows and one scoreable cell;era_3holds 19 across 1,976 rows and eight scoreable cells. Twenty in total. The split that makes VLF features comparable is the same split that removes the label variation needed to test them, and that conflict applies to every design in the item-103 sweep.era_3scores, all on the same 19 transitions:seismic_only0.525464pooled /0.552836stratified;seismic_vlf0.515191/0.539445;stratum_base_ratecontrol0.534581/0.500000;vlf_only0.506800/0.532717;full_multimodal0.489631/0.431988;all_features0.455515/0.404832. Per cell the range is0.300344–0.854430on one to five transitions each. Do not read any of these numbers.(c) is now enforced in code:
_stratified_metricsemitslabel_transitionsper stratum and in the summary, and the CLI prints it beside every score.New finding: the permutation control is not a null at all. Shuffling target timestamps destroys the autocorrelation that makes consecutive rows near-copies, so it manufactures label variation. The shuffled controls carry 94–119 held-out transitions in
era_0against the real run's 1, and 315–342 inera_3against the real run's 19 — 17 to 100 times the evidence. They are a different, better-resourced task, not the same task made harder. This retires the item-8 reading ("all five controls beat real order, so destroying temporal structure makes the task easier") as uninterpretable; the item-93 era-leak mechanism and this evidence-count asymmetry are both present and cannot be separated at this record length. It does validate the metric: given 315–342 transitions the shuffled controls land on0.500000–0.533923stratified, a proper null converging on chance. A within-cell permutation preserving each cell's sequence length, positive count, and block structure would be a valid null; see item 105(d).(d) The blocker is calendar time. 419 Italian events at M≥2.0 across 81 days, through 19 cells, split by an unavoidable era boundary, yield twenty independent held-out label changes. Stop adding feature families. This is a data-volume statement and not evidence that VLF, astronomy, or seismic history lacks predictive value.
-
(a) and (b) completed 2026-08-21. The collector is fixed; the era boundary is not removable. The plan was to add evidence without waiting for calendar time, by making the catalog current and by putting both eras on one ruler. The first worked. The second did not: the ruler was fixable, the
17.9 dBlevel step behind it was not, so the eras stay separate and the item-104(d) blocker stands. What remains is (c)-(f), none of which adds evidence to the current record.(a) Completed 2026-08-21. The live collector now fetches INGV. It previously ran every 30 minutes and never fetched events, so the target catalog was stale by however long it had been since someone ran
refresh-prospective-labels.shby hand. The fetch/normalize/combine steps are now inscripts/refresh-ingv-events.sh, called both byrefresh-prospective-labels.shand as the firstExecStartofelfquake-prospective.service. It fetches a rollingLOOKBACK_DAYS=7window —combine-normalized-eventsdeduplicates byevent_id, so chunks merge cleanly without re-pulling the whole record every half hour — tolerates a failed fetch so a network blip cannot stop VLF capture, and writesdata/derived/ingv/catalog_freshness.json.build-italy-spatial-vlf-targets.shand thelabel-multimodal-targetscalls now takeCATALOG_ENDfrom that report'scoverage_end_utcrather than wall-clock now. Installed and verified on the 17:23 run:fetch_status: ok, 420 events,coverage_end_utccurrent. Checkcatalog_freshness.jsonbefore any evaluation, and confirm the deployed unit still matches the repo withdiff /etc/systemd/system/elfquake-prospective.service deploy/systemd/elfquake-prospective.service.(b) Completed 2026-08-21, and the premise was wrong. Palette-inverted absolute-dB features do not make the eras poolable. Full write-up in Cumiana Colour-Scale Change. Item 97(a) is implemented:
src/elfquake/features/vlf_image_db.py, run by./scripts/extract-vlf-image-db-features.sh. All 824 captures decode through their own embedded colourbar, mean censored fraction0.229,7.65of 8 bands scoreable. Censoring is carried per band as item 97(a) required, and a band's level is withheld rather than filled once censoring passes 50%, because a band below the shared floor is missing, not quiet.But a
17.9 dBlevel step survives the inversion (./scripts/diagnose-vlf-db-era-step.sh): median-17.9, range-16.6to-21.6across all eight bands from200 Hzto15 kHz, at2.8times the within-era day-to-day spread. Inversion removes the display change and cannot remove a gain change — decoding recovers dB as the receiver reported it, not dB at the antenna, so a front-end gain change alters the quantity plotted rather than the plot and passes through untouched. Item 96 stands, the eras stay separate, and the transition budget does not improve. The item-104(d) conclusion is unchanged: the blocker is calendar time.The evidence has moved further toward instrumental — the step is now on a common ruler, is broadband and roughly uniform rather than frequency-selective, and is about
1.5xthe size of the11.58 dBdisplay change that accompanied it, which is what an operator cutting gain and then re-scaling the display to match would produce. That is circumstantial. Item 105(c) is still the only route to settling it.(c) Ask the Cumiana operator about receiver gain in the 2026-07-06 to 2026-07-11 window. A recorded gain change would let the correction be applied as a known constant rather than inferred.
(d) Completed 2026-08-21. A matched null exists, and the item-8 result does not survive it. Full write-up in Within-Cell Null Control.
shift-spatial-targetscircularly shifts the whole labeled matrix in time: per cell the sequence length and positive count are preserved exactly and the run structure survives apart from the wrap seam, while the feature-label alignment is destroyed. Run it with./scripts/evaluate-italy-spatial-shift-controls.sh; the printedtransitions: N before, M afteris the check that it held (74 → 76onera_3, against the shuffle's74 → 322).Held-out transitions, real against control:
era_0real 1, shift 3-6, shuffle 94-119.era_3real 19, shift 10-22, shuffle 315-342. The shift controls bracket the real run; the shuffle exceeded it by two orders of magnitude and was never a null.The item-8 pattern is retired, not reversed. Under the shuffle, 5 of 5 controls beat real order for every ablation in both eras. Under the shift the count is 0, 1, 2 or 3 of 5 depending on the ablation, with no systematic direction. The comparison items 8 and 93 drew cannot be made with that control and should not be cited again.
era_3stratified, real vs control mean:seismic_only0.552836vs0.461116(0 of 5 controls beat it, but the margin over the best control is0.0003— real order ties its best control);seismic_vlf0.539445vs0.483559(1 of 5);vlf_only0.532717vs0.532322(2 of 5, level with its own null to four decimals);all_features0.404832vs0.454386(3 of 5). Every family added to seismic history lowers the score. With a null finally matched on evidence, no modality separates from chance and nothing improves on seismic history alone — consistent with items 102 and 104, and still not evidence of absence at 19 transitions.(e) Keep collecting. Re-run the item-104 evaluation via the
elfquake-fixed-cell-evaluationskill when a single era reaches a held-out transition count in the low hundreds, not before.(f) Leave the per-cell rate residual and single-regional-target designs from item 103 untested until there is enough label variation to distinguish them.
Item 95(d) is closed by item 106.
-
Completed item 95(d). Audited every
all_availableconsumer and the spatial evaluators for the item-89 staleness pattern; found six more instances, two of them live. Write-up and the audit table in Input Freshness. The shared guard isscripts/lib/staleness.sh:require_fresh_inputs REFERENCE INPUT...compares modification times, warns on stderr by default so a diagnostic run still produces its report, and exits3underSTALE_INPUTS=fail. It never rebuilds anything — the artifacts it guards cost more than the reports reading them. Cover intests/test_staleness_guard.py.(a) The transformer was training on dead channels.
common_transformer_fixture_sequences/was 13 days behindcommon_transformer_fixture.csv, so the astronomy sequence still carried the two item-99 channels while the fixture had held item 100's 19 aligned channels since 2026-08-17. Rematerializing took the astronomy modality fromchannel_count2 to 19,row_count9,082 to 14,478, and gave it masks that actually fire (Kp/ap blank on 38 rows at the end of the record).run-cross-region-generative-smoke.shnow guards both hops. See item 107.(b) Both weekly forecast scripts defaulted
AS_OF_UTCto the literal2026-07-08T00:00:00Z. A fresh catalog answering a frozen question — the same defect in the argument list rather than in a path, invisible to any timestamp check. Both now derive it fromcatalog_coverage_end, which readscoverage_end_utcfrom the freshness report; setAS_OF_UTCexplicitly to reproduce an old run.(c)
analyze-italy-vlf-event-association.shandreport-italy-data-coverage.shreadreal_vlf_anomaly_scores.csv, 13 days behind the catalog; both weekly forecast scripts read*.real_vlf_aligned_windows.csvat 15 days and the aligned synthetic windows at 46. Nothing on the refresh path writes any of them — they come fromscore-real-vlf-anomaly-forecast.shandprepare-real-model-inputs.sh. All four now warn before doing any work.(d) The five
evaluate-italy-spatial-*.shscripts each kept the existence-only rebuild guard that item 89 removed fromprepare-italy-spatial-model-inputs.shitself, so a refreshed labeled table stopped at the caller: the aligned dataset exists, the branch is skipped, the evaluation reports an old record under today's date. They now rebuild when the input is missing and warn when it is merely stale.(e)
catalog_coverage_endis now shared rather than duplicated;refresh-prospective-labels.shuses it in place of its inline copy.(f) The guard compares modification times and nothing else. It cannot see a file rewritten with identical content, or the frozen-argument case in (b). Those still need the item-89 tell: a score that reproduces to six decimal places across a data refresh is a staleness signal, not a stability result.
-
Astronomy status as of 2026-08-22, superseding the item-99 audit: the modality is live, reaches both model paths, and does not help. Item 99 described a frozen directory, two dead channels and a mask that never fired. All three are fixed — item 100 did the alignment, item 106(a) carried it into the transformer.
- Acquisition runs.
elfquake-space-weather.timerfetches GFZ Kp/ap, Spaceweather Canada F10.7 and Kyoto Dst daily;data/derived/astronomy/was rewritten2026-08-22T07:53Z. The 30-minute prospective timer reads those tables through--space-weather-root, so every anchor gets current values. - The channels vary. Over the 817 prospective anchors:
astro_kp0–7,astro_ap0–132,astro_dst_nt-146–+55nT,astro_f10794.8–258.1, plus the ephemeris moon and tidal channels. The record covers a real geomagnetic storm, so the geomagnetic channels have something to be tested against. - The masks fire.
quality_missing_kptakes both values; the per-channel sequence masks readpresent=0on the 38 rows where the GFZ 3-hour bin has not closed.quality_missing_astrois still constant0and correctly so — it is set only when Kp, Dst and F10.7 are all missing, which has not happened. - The transformer gets 19 channels, and the run was verified end to end.
italy_all_astronomy_sequenceafter item 106(a): 19 channels, 15,960 rows, 840 time steps, Kp/ap masks readingpresent=0on 152 rows. The whole chain was then rebuilt from the live catalog on 2026-08-22 — labeled spatial table (15,960 rows, coverage end2026-08-22T09:48:48Z), fixture (16,260 rows, 12 datasets, channel gate clean), sequences, thenrun-cross-region-generative-smoke.sh. It completed on 11,004 train / 2,752 test rows: all three modalities0.513960, astronomy masked0.511239, seismic masked0.511085, Italy VLF masked0.493117, linear probe0.498439. Indistinguishable from0.5on one seed, and the masking test measures inference-time reliance rather than modality value. The run establishes that the interface works with 19 real channels and nothing more. - The held-out answer is negative, and worse than negative. On the item-104 design (
era_3, 1-day horizon,M>=2.0, cell-stratified, 19 label transitions),seismic_onlyscores0.552836andseismic_astronomyscores exactly0.500000— the 21-feature model predicts the negative class on all 1,976 test rows after reaching0.613508balanced accuracy in training.full_multimodalis0.431988,all_features0.404832. Twenty-one features against nineteen transitions is the whole result; this measures the target design's evidence, not astronomy. - Do not read a future positive result without a date-proxy check. Over the 799 anchors of the labeled spatial table,
astro_f107correlates-0.880with the anchor index,astro_moon_phase_cos+0.838,astro_moon_illuminated_fraction-0.838,astro_moon_distance_km-0.730,astro_tidal_potential_min-0.604. No future information enters any row, but under a time-based split those five approximate an indicator for which side of the split a row is on, and a model can use them to recover the era base rate rather than any physical state. The lunar ones will decorrelate as the record passes two lunations;astro_f107will not, while the record is short against the solar cycle. Gate any astronomy result against the item-105(d) shift control before treating it as physical. - Next, and only when the evidence exists: re-run the ablation through the
elfquake-fixed-cell-evaluationskill when a single era reaches held-out transitions in the low hundreds (item 105(e)). Until then astronomy is in the same position as VLF — correctly wired, honestly measured, and untestable at this sample size.
- Acquisition runs.
-
Routine checks, 2026-08-22. (a)
./scripts/diagnose-vlf-palette-shift.shre-run over all 849 captures: still exactly two palette variants and one change point, so the item-96 era split is unchanged and the analyses built on it stand.palette_0holds 277 captures from2026-06-29T09:45Zto2026-07-06T21:45Zatred_start_px59;palette_1now holds 572 from2026-07-11T06:00Zto2026-08-22T14:00Zat 48 — 150 more captures than at the item-96 check, all in the same variant. The single change point sits in the same outage with the same-11.579 dBdisplay shift, and the common resolvable window is unchanged at-80.0to-49.474 dB. A "no change" result here is the point: a third variant would invalidate every cross-era comparison again.(b) Measured the accrual rate of the blocking constraint.
era_3has produced 78 label transitions over 22.1 labeled days across 19 cells (3.54per day), of which the held-out final fifth sees about0.71per day — consistent with the 19 transitions currently observed. At that rate the held-out count reaches 100 around 2026-12-14 and 200 around 2027-05-04, assumingera_3stays unbroken. This turns "wait for calendar time" into a date and sets the expectation for item 105(e): the next fixed-cell evaluation worth running is months away, not weeks. -
All-source Transformer run with the most recent matured week held out, 2026-08-22. The classifier collapsed to exactly
0.500000and the coordinate head is beaten by climatology. Rebuilt the whole chain first — labeled spatial table, fixture (16,279 rows, 12 datasets, channel gate clean), sequences — then ran./scripts/run-cross-region-generative-smoke.sh: synthetic masked pretraining, Japan self-supervised continuation, chronological Italy fine-tuning on seismic + VLF + astronomy, CPU, one seed.The holdout is
2026-08-07T09:57Zto2026-08-14T09:57Z. It has to be the most recent week whose outcomes are known: labeled anchors end at2026-08-14T09:57Zand the 2,204 anchors after it are pending because the 7-day target horizon has not closed. Computing the split fraction that puts the boundary exactly seven days before the last labeled anchor returns0.800000— at the current labeled density the standard 80/20 split already is a one-week holdout. 11,020 train / 2,755 test rows, 677 test positives, 5 held-out label transitions.Classification.
italy_multimodalfine-tune balanced accuracy0.500000; linear probe0.499034; masking astronomy0.500000, masking Italy VLF0.500000, masking seismic0.496765. Nothing moves. The same configuration on 16 fewer training rows scored0.513960earlier the same day (item 107), so that0.514was single-seed noise and should not be quoted as a result.Coordinates, and the control that matters. The coordinate head emitted its budgeted 5 continuous epicentres against 13 actual
M>=2.5Italy events in the week. All five fall in the Alps and Po valley between44.4Nand46.7N; eight of the thirteen actual events lie south of the southernmost prediction, and the week's largest event — theM3.7of the three-event Pisa cluster on 2026-08-12 — is missed entirely. One prediction did land13.0 kmfrom theM3.4at44.529N 7.247E, which is what one close pair out of five looks like by chance.Median distance from a prediction to the nearest actual event is
152.3 km. Two controls, both drawn without looking at the test week: uniform random points in the Italy bounding box give a median-of-medians of198.3 km(5th-95th94-335), so the model is inside that spread; drawing five epicentres at random from the 1,686 historicalM>=2.5events that occurred before the holdout opened gives109.5 km(5th-95th32.7-178.7), and86.5%of those draws are at least as close to the week's events as the model is. The generative head is worse than climatology. Map atdocs/images/cross-region-generative-smoke.png, inputs underdata/derived/models/cross_region_generative_smoke/map_inputs/.This is an interface check, not a forecast. It confirms the all-source chain runs end to end on current data and emits continuous coordinates rather than grid cells. At 5 held-out label transitions it cannot do more, which is the item-103 constraint again.
-
Tier-A evidence work, 2026-08-27. Held-out label transitions in
era_3went from 2 to 31 with no additional calendar time, and a defect voided every VLF ablation in item 104. With 15.5 times the evidence the classifier still scores0.502434.(a) The item-104 tables were scored without VLF image features. The 1-day / M≥2.0 design was built by writing
build-prospective-vlf-windowsoutput straight to a path namedall_italy.prospective_vlf_image_windows.h1m20.csv, skipping thejoin-vlf-image-featuresstep the live pipeline runs to reach that filename. The table had 47 columns against the live table's 63, missing all 16vlf_image_*features. Becausevlf_onlyresolves tovlf_metadata + vlf_image + ...(src/elfquake/models/feature_groups.py:46), item 104'svlf_only0.506800,seismic_vlf0.515191,full_multimodal0.489631andall_features0.455515were computed over capture counts, byte totals, JPEG dimensions and file entropy. No spectrogram content entered any of them. Those four numbers are withdrawn;seismic_onlyandstratum_base_ratein the same table are unaffected. Rebuilt through the join. SeeMISTAKES.md.(b)
M≥2.0was never the catalog floor. Item 103 called it one and concluded the threshold could go no lower. It was themin_magnitudedefault ofsrc/elfquake/connectors/ingv.py:27. The INGV FDSN service acceptsminmag=0; what fails islimit=50000, which returnsHTTP 400and looks like a rejected magnitude. The service capslimitat10000, so a long span must be chunked. Over2025-08-01to2026-07-27in 12 monthly chunks it returned 17,196 events down toM0.0. Maximum-curvature completeness isMc = 1.1, Aki MLE b-value0.787over the 9,390 events at or above it.M≥1.5yields2.64times the events ofM≥2.0, matching the10^(0.787x0.5) = 2.48the b-value predicts. Working request shape and the frequency-magnitude table are in Target Design. TheM≥1.5catalog is stored underdata/raw/ingv/m15/anddata/derived/ingv/m15/; the subdirectory is deliberate, sincerefresh-ingv-events.shglobs at-maxdepth 1and would otherwise sweep a lower-threshold catalog into theM≥2.0combined tables.(c) What the two changes bought. Same anchors, same 1.5° cells, transitions counted as label changes between consecutive anchors, held out as the last 20% of anchor times.
era_3only, since item 96 forbids pooling across the palette shift:Design Held-out transitions, era_3Two-class cells Held-out positive rate 7-day / M≥2.5(previous default)2 2 of 19 0.23571-day / M≥2.021 9 of 19 0.12011-day / M≥1.531 14 of 19 0.2414A factor of 15.5 from target design and catalog threshold alone. The
M≥1.5design also holds the positive rate at0.2414, close to the old0.2357, where theM≥2.0design degrades it to0.1201.scripts/build-common-transformer-fixture.shnow takesITALY_LABELED, so the design a run trains on is one environment variable.(d) The extra evidence did not produce a signal. Full chain re-run on the 1-day /
M≥1.5design — fixture 17,438 rows and 12 datasets, channel gate clean, 96 sequence manifests, thenrun-cross-region-generative-smoke.sh. Holdout2026-08-16T07:30Zto2026-08-23T07:30Z, which is nine days more recent than item 109's because a 1-day horizon matures a week faster. 13,376 train / 3,344 test rows.italy_multimodalfine-tune balanced accuracy0.502434against a training figure of0.573554; linear probe0.499472; masking astronomy0.500000, masking seismic0.500736, masking Italy VLF0.489208. Every ablation is within0.011of chance.This is the first
0.5in the project that carries weight. At 2 transitions a chance score was uninformative; at 31 it is a considerably better-supported null. It is still not a demonstration of absence — 31 is well short of the low hundreds — but it is no longer only a statement about sample size.(e) The coordinate head is again worse than climatology. 6 generated epicentres against 8 actual
M≥1.5events; median nearest-actual255.4 km. Drawing 6 epicentres from the 5,055 pre-holdout historical events gives a median-of-medians of149.2 km(5th-95th69.6-269.4), and93.1%of those draws land at least as close to the week's events as the model does — worse than item 109's86.5%. All six predictions cluster in the central Apennines and one Alpine point; five of the eight actual events are nowhere near them. Map atdocs/images/cross-region-generative-smoke-h1m15.png.Nothing here is evidence that VLF, astronomy or seismic history predicts earthquakes.
-
A5 archival backfill, 2026-08-27. The VLF route is closed at both ends; the astronomy route was already open and the doc saying otherwise was stale. Two collector services were found in
failedstate.(a)
abelian.orgno longer exists. Every Abelian endpoint in the repo —live-stream.php(thevlf15raw Ogg stream) andretrieve.php(the archive form) — resolves to nothing.getentreturns "Name or service not known" locally anddig @1.1.1.1 abelian.orgreturnsSERVFAILwith no NS records, whilevlf.itreturnsNOERRORfrom the same resolver. This is the domain's nameservers being gone, not a local block. Nine archive probes spanning 1 to 800 days back all failed at DNS before reaching HTTP.src/elfquake/connectors/vlf_abelian_*.py, thefetch-vlf-abelian-cumiana-archive,probe-vlf-abelian-cumiana-archiveandrecord-vlf-abelian-cumianacommands, and the archiveddocs/archive/vlf-abelian.mdnotes are all dead code against a dead host; the row in Source Inventory is marked accordingly.This closes two options at once: A5's archive route and B3's "move from rendered JPGs to raw samples", which depended on the same
vlf15stream. Raw VLF samples are not obtainable from any endpoint this project knows about.(b) The live captures were never Abelian. They come from
http://www.vlf.it/cumiana/last_E-VLF.jpg(data/raw/vlf/cumiana/manifest.csv), which is alive and serving on schedule;elfquake.serviceis running normally. The two facts are unrelated and it was only luck that the dead host was not the capture host.http://www.vlf.it/cumiana/returns403(directory listing disabled) and no archive index is discoverable, so vlf.it serves current images only. The sole remaining route to historical VLF is asking the operator, which merges A5 into the item 105(c) email — that email is now the only path to both historical data and raw samples, not just to the receiver-gain question.(c) The astronomy archives were already deep, and the "unconfirmed" note was wrong.
data/derived/astronomy/gfz_kp_ap.csvholds 276,576 rows from 1932-01-01, andf107_daily.csvholds 23,883 rows from 2004-10-28. Kyoto Dst is fetched per month. Astronomy needs no backfill work. It also cannot extend the modeling record on its own, because every anchor requires a VLF capture to exist.(d)
elfquake-space-weather.servicehad been failing since at least 2026-08-26, and the derived astronomy tables were four days stale. Kyoto serves each month from exactly one tier and promotes months as they are revised.refresh-space-weather.shpinnedDST_TIER=realtimefor both the current and previous month. Probed 2026-08-27:202608→realtime200,202607and202606→realtime403,provisional200. So the previous month 403s every month, permanently, from the first rollover onward. Underset -euo pipefailthat 403 aborted the script before the normalize steps, so raw captures kept arriving whiledata/derived/astronomy/*.csvstopped advancing on 2026-08-23 — with no error visible to anything downstream. Every astronomy channel in the item-110 runs was four days stale. Fixed: tiers are now triedfinal provisional realtimeand a month no tier serves is a counted warning rather than an abort. After the fix all three months normalized and August gained 4 days of rows.(e)
elfquake-japan-vlf.serviceis failing on a systemd start timeout, killed mid-transfer after ~29 minutes of CPU having processed several files. Japan remains parked with 9 VLF-observed windows against 124 missing and nothing to train on, so this is recorded rather than fixed. Japan ISEE data remain research-use-only.Lesson. Two of three timer-driven collectors were in
failedstate and neither had surfaced anywhere in the documentation, the freshness guard, or any evaluation.require_fresh_inputscompares derived artifacts against the event catalog; it cannot see a collector that stopped, because the file it would check is the one that stopped being written. Checksystemctl list-units --all | grep elfquakeas part of the post-refresh routine. -
Eight days of accrued data, and the first seed ensemble, 2026-09-04. The ensemble shows the classifier is not distinguishable from chance and that three previously reported single-seed scores were all draws from one noise distribution.
(a) What accrued. 1,035 captures (was 849 at item 108), a 465-event
M≥2.0prospective catalog and a 1,235-eventM≥1.5catalog, and astronomy current to 2026-09-04.era_3held-out label transitions went 31 → 41 in eight days, an accrual of1.25/day against the1.09/day projected in item 110. The palette diagnostic still reports exactly two variants and one change point, soera_3remains unbroken across 758 captures from2026-07-11to2026-09-04and no cross-era comparison is invalidated.(b) The Dst tier fix survived the case it was written for. The space-weather collector failed for months on end whenever a month rolled over and Kyoto promoted the previous month off
realtime(item 111(d)). September rolled over on 2026-09-01 with the fallback in place:kyoto_dst_202609.csvexists, the unit reportsDeactivated successfully, and the derived astronomy tables are current. This is the first month boundary the collector has crossed without breaking.(c) Five seeds, and the numbers stop supporting a point estimate. The same configuration, same data, seeds
42-46, on the 1-day /M≥1.5design with 15,701 train / 3,926 test rows and 806 test positives:Ablation Mean SD Min Max italy_multimodal0.5074150.0243320.4751600.541439without_astronomy0.5288250.0296360.4999950.572007without_seismic0.5253120.0260320.5000000.554203without_italy_vlf0.4820550.0207240.4482480.502616linear_probe0.5130980.0185460.4871590.538689The 95% confidence interval on
italy_multimodalis0.4772to0.5376, which contains0.5;t = 0.681on 4 df against a2.776threshold. The classifier is not distinguishable from chance. Two ablations that remove a modality score higher than the full input, which is the signature of noise rather than of a modality being harmful.(d) Three earlier "results" were the same distribution sampled once each. The single-seed values reported in items 107, 109 and 110 —
0.513960,0.500000,0.502434— all fall between the 40th and 60th percentile of this five-seed spread. None of them was informative about anything but which seed was drawn. Item 110's reading that0.502434was "the first0.5in this project that carries weight" was overstated: the weight came from the transition count, but the point estimate itself had a spread of0.066that went unmeasured. Report ensembles with an interval; a single-seed transformer score on this data is not a result.(e) The coordinate head still loses to climatology, by less. Holdout
2026-08-25T07:45Zto2026-09-01T07:45Z, 7 generated epicentres against 10 actualM≥1.5events, model median nearest-actual128.3 km. Against a pool matched to the target (M≥1.5since 2026-06-01, 721 events) climatology gives114.8 kmwith57.8%of draws at least as close; against the deeperM≥2.0full-history pool (5,098 events, comparable to item 111) it gives101.5 kmwith73.7%. Both still favour drawing past epicentres at random over the model, though by much less than item 109's86.5%and item 110's93.1%. The two pools are not interchangeable and the shallower one is the matched control; quote the pool with the number.Nothing here is evidence that VLF, astronomy or seismic history predicts earthquakes.
-
The spatial labeled table was never on the refresh path, 2026-09-04.
refresh-prospective-labels.shbuilt the prospective andlabel-multimodal-targetstables but neverall_italy.spatial_vlf_image_windows.labeled.csv, which is the input every fixed-cell evaluation reads. On 2026-09-04 that table was 13 days old against a catalog current to the minute.This is the item-89 pattern with the guard pointed the wrong way. All five
evaluate-italy-spatial-*.shscripts callrequire_fresh_inputsonSPATIAL_LABELS, so the staleness would have been reported — but nothing anywhere wroteSPATIAL_LABELS, so the guard was watching a file that could only ever get older, and the fix for item 89 gave the appearance of coverage without it.Fixed:
refresh-prospective-labels.shnow builds both designs — the 7-day /M≥2.5table for continuity with existing reports, and the 1-day /M≥1.5table used by current runs, including thejoin-vlf-image-featuresstep whose omission voided item 104 (item 110(a)). TheM≥1.5catalog is refreshed only when it already exists, so creating it stays a deliberate act. The refresh now ends by printing held-out transition counts for both designs, which is the "count before the score" rule of Next Actions made automatic rather than remembered. First run after the fix,era_3: 7-day /M≥2.5gives 4 held-out transitions, 1-day /M≥1.5gives 41.New helper
scripts/count-label-transitions.shprints anchors, labeled rows, held-out rows and positive rate, and full-record and held-out transition counts, withERA_STARTto restrict to one capture era.
Standing practice by area
Longer-running rules rather than immediate actions. These accumulated alongside the numbered record and are kept here for the same reason: they are the reasoning behind current defaults, and several are cited from elsewhere.
Modeling
- Run
./scripts/run-transfer-experiments.shafter each real-data refresh. It compares historical rate, real-only random initialization, synthetic transfer, rolling-origin folds, and a train-only grid selection before one final holdout evaluation. The default synthetic corpus now includes four long episodes; add more 20,000-step episodes before treating transfer changes as stable. - Generate more independent warmed episodes and rerun leave-one-episode-out evaluation; nine episodes are not enough to estimate regime robustness tightly.
- Combine at least five scope-matched long episodes, apply calibration using training dates only, and report rate, magnitude, inter-event, sample-matched clustering, and occupancy metrics together.
- Keep the five-episode candidate as a diagnostic benchmark, not a training default, until its corrected rate and spatial metrics survive held-out episode and cell checks.
- Add regime-conditioned reporting or a mixture-of-regimes simulation before another global calibration pass; the current episode rate spread is too large to treat one global thinning factor as a physical correction.
- Investigate the remaining matched rate and clustering differences using the
4600--4800source/loading trajectories; change simulation dynamics only if a stable cause is found. - Do not add the default piezo potential channel to model training yet. Its spatial average failed a nine-episode causal lead-time check; event-nearest diagnostics are positive but use future event locations and are not valid inputs.
- Calibrate weekly event counts against historical INGV
>M2rates before trusting any neural score scale. - Compare every weekly forecast run with
./scripts/compare-weekly-forecasts.shand track Stage 1/Stage 2 pass/fail status. - Keep direct avalanche-derived seismic features separate from piezo/VLF-like features; use ablations to test their contribution independently.
Data
- Keep accumulating Cumiana VLF image captures and refreshing image features.
- Refresh prospective INGV labels as target windows mature; train supervised real models only after one table has both positive and negative labels.
- Validate Abelian Cumiana live/archive audio only if a reproducible nonempty pull is found; current probes returned zero usable bytes.
- Extend historical INGV backfill earlier than 2024 only if weekly baseline calibration needs longer seasonal coverage.
- Repeat mixed real/synthetic VLF alignment after new Cumiana captures; require improvements over centroid and random controls before relying on inlier selection.
- Keep event-count, energy, and spatial-occupancy targets alongside binary occurrence; do not make one thresholded event label carry all timing, magnitude, and location information.
- Fit magnitude calibration on the real training period only, then compare calibrated and uncalibrated synthetic catalogs before adding temporal-rate or spatial-density transforms.
- Treat rate thinning as an observation model, not a simulation fix; retain the raw event catalog and test whether spatial reweighting improves cell occupancy without moving localized source events.
- Add a joint alignment score with minimum sample gates: rate ratio, magnitude distance, inter-event distance, nearest-neighbour distance, and spatial occupancy must be reported together.
- Preserve the combined-episode time offsets and calibration metadata in every synthetic training artifact; do not collapse episodes back onto their shared demonstration clock.
- Use sample-size-matched nearest-neighbour statistics for catalog clustering. Do not compare a 32-event synthetic catalog directly against all 594 real events.
Simulation
- Run
./scripts/run-longer-synthetic-transformer-batch.shwhen CPU time is available, validate drift, then rerun./scripts/evaluate-piezo-group-holdout.shagainst the larger episode set. - Keep
damage_totalas a validated synthetic precursor diagnostic, not a default Transformer feature. A matched nine-fold screen regressed from0.599648without damage channels to0.586848with them. - Keep the duration-aligned
SOURCE_COUNT=64, refill470, removal interval20, andq=0.998/window=120profile as a valid synthetic target baseline (47.0%positives, temporal drift0.182). It has no confirmed piezo lead and is not a precursor-training profile. - The first two-stage mature-weakness profile failed its nine-episode causal confirmation despite stable target drift. Do not tune its scalar parameters immediately or train a model. Document a stronger physical mechanism proposal, such as a spatially propagating rupture/nucleation state, before another synthetic dynamics run.
- Compare future episode-batch h6 drift against the current scaled
WARMUP_STEPS=3000delta0.187025. - Revisit structured initial fill only with delayed bottom-layer removal; the first fill probe drifted at
0.307937. - Tune the piezo/VLF mapping only from
*.piezo.csvand compare against Cumiana VLF shape reports.
Maintenance
- Keep docs concise: one current source doc, one simulation doc, one modeling doc, one operations/steps doc, and one report.
- Split
tests/test_acquisition_scaffold.pyby subsystem if test maintenance starts slowing changes. - Add chunked sandpile snapshot storage only if larger pretraining runs outgrow current
.npysanity snapshots. - Keep optional dependencies CPU-compatible on this system; do not add GPU-only paths.
Japan parallel data path
- Run
./scripts/backfill-japan-history.shand verify nonempty USGS raw and normalized outputs. - Identify one reproducible current passive broadband ELF/VLF Japan sample and add it to
data/raw/vlf/japan/manifest.csv; prioritize ISEE Moshiri or Kagoshima over WALDO. - Compare Japan and Italy source coverage before any cross-region model training.
- Use the confirmed ISEE permission to obtain one recent Moshiri or Kagoshima digital sample, then build the Japan VLF adapter for its native CDF format.
- Keep WALDO out of the main acquisition schedule; revisit it only for a defined historical case study or optional self-supervised pretraining corpus.
Italy data refresh (2026-07-16)
- Refreshed Italy data through 2026-07-16: 67 new INGV events were pulled, one new Cumiana
last_E_VLFimage was captured, and the prospective tables now contain 279 rows with 277 mature rows. Both all-Italy (277/0) and central-Italy (0/277) remain class-blocked. - Rebuilt the real VLF sequence and model inputs: 279 image rows and 256 anomaly windows now extend through 2026-07-16. The label-free smoke forecast remains a novelty artifact, not a seismic prediction.
- Audited the mirrored all-Italy/central-Italy label counts. Equal row counts are correct because both scopes use the same VLF anchors; the one-class labels are target saturation, not a region-filter bug. All-Italy is
277/0at M3+ and central Italy is0/277; at M2.5+ central Italy is228/49but all-Italy remains277/0. - Added
./scripts/report-italy-data-coverage.sh. The latest report contains 4,836 INGV events, 283 Cumiana capture metadata records, 256 VLF anomaly windows, and only two weeks with both VLF and seismic observations. This is descriptive coverage evidence, not an association result. - Added
./scripts/analyze-italy-vlf-event-association.sh. The first refreshed permutation-controlled association remainsinsufficient_controls: three VLF-observed weeks provide only one M2.5+ event week and two controls.
Italy coverage diagnostics
- Run
./scripts/report-italy-data-coverage.shafter each refresh. It reports INGV event coverage, Cumiana capture coverage, label-free anomaly coverage, and descriptive weekly overlap. - Treat anomaly/event overlap as exploratory only until enough mature windows contain both positive and negative targets.
- Replace binary all-Italy targets with fixed spatial-cell targets or count regression; use central-Italy M2.5+ only as a temporary exploratory control.
The fixed-cell implementation is available in data/derived/multimodal/all_italy.spatial_vlf_image_windows.labeled.csv and is prepared by ./scripts/prepare-italy-spatial-model-inputs.sh. The current smoke artifact has 5,301 rows across 19 cells, with 812 positive, 4,451 negative, and 38 pending labels. This fixes target saturation but does not fix the short time coverage or establish predictive skill.
The first grouped-time logistic smoke baseline reached calibrated balanced accuracy 0.655320 for the all-feature ablation. The seismic-only and VLF-only ablations collapsed to balanced accuracy 0.5 under their calibrated thresholds. These figures are a single short-window diagnostic and are not evidence that either modality predicts earthquakes.
The first 19-cell leave-one-cell-out probe is stored under data/derived/models/all_italy_spatial_cell_holdouts_v2. Only 5 cells have positive test labels; the other 14 folds are one-class. The valid folds range from 0.146597 to 0.855263 calibrated balanced accuracy, with mean 0.333370 across all folds. This instability and class sparsity block meaningful spatial transfer evaluation.
The timestamp-permutation null control is stored under data/derived/models/all_italy_spatial_permutation_controls. It preserves each timestamp's complete spatial label pattern but shuffles those patterns across time. Five controls scored 0.643309--0.709108, mean 0.679362; all five matched or exceeded the real-order 0.655320. The current multimodal score therefore has no demonstrated temporal signal.
Japan event-window follow-up
- Checked the official ISEE/ERGSC archive on 2026-07-31. It lists 24 hourly Moshiri CDF files for 2026-07-28 and further files for 2026-07-29.
- Decoded and scored 2026-07-28 00:00--06:00 UTC. The extended floor-aware robust anomaly score ranged from
2.891to5.538; 2026-07-27 18:00 scored8.772, so the available event-day elevation is not event-specific evidence. - The next acquisition retry should process the remaining 2026-07-28 07:00--23:00 and 2026-07-29 files from
data/raw/vlf/japan/manifest.csv. The previous transfer degraded to roughly 15 KB/s and was stopped; interrupted partial files are cleaned by the downloader. -
After acquisition, rebuild the Japan windows and rerun the anomaly report with a longer pre-event baseline, matched local-time controls, and separate pre-/post-event summaries. Japan data remains restricted to scientific research use.
-
Re-ran the regular CPU controls on 2026-07-31: the real transfer trial remains
0.693435balanced accuracy, the 27-run piezo holdout remains unstable at mean0.578712(0.275641--0.758730), and the grouped Italy spatial baseline completed on 5,301 rows with 1,064 held out. These remain control results, not evidence of multimodal predictive value. - Re-ran
./scripts/evaluate-italy-synthetic-episode-alignment.shacross the five configured profiles. Seeds40--42produce14.324--14.990times the real event rate, while seeds4300and4500produce0.555and0.278; sample-matched spatial distance ranges from9.316to225.626km. This confirms configuration drift, so these profiles remain diagnostics rather than a training corpus. - Re-ran
./scripts/evaluate-avalanche-burst-seeds.shacross seven seeds. The causal burst extractor produces16--30events per episode with broadly stable nonzero rates, but burst-run counts, lag-1 autocorrelation, and PSD slopes remain unlike the real reference. Keepdata/derived/reports/avalanche-burst-seeds/summary.csvas a diagnostic and do not promote the extractor to the training default. - Ran
./scripts/evaluate-avalanche-burst-train-test.shwith a threshold learned only from seeds40--4300. The fixed threshold0.221864346produced25--50training events and48--58held-out events; held-out nonzero rates0.145--0.176exceed the real0.082, and autocorrelation/PSD remain mismatched. Threshold calibration alone does not resolve regime drift; retaindata/derived/reports/avalanche-burst-train-test/summary.csvas a negative control. - Re-ran the source-stress alignment on the 1,000-step stress episode: 938 release rows,
5.9%with positive local excess activity, local/global excess-AUC ratio0.037, and median local peak lag96steps. The result does not support a localized causal precursor signal; retaindata/derived/reports/mountain_256x256_seed40_1000.stressdiag.source_stress_alignment.csvas a negative control. - Re-ran
./scripts/evaluate-piezo-japan-shape-variants.shwith the available Japan CDF features. The slow-envelope variants leave synthetic PSD slope near0.50versus Japan-0.23, and low-band ratio near0.17versus Japan0.76;slow_strongraises kurtosis but does not improve spectral alignment. Retaindata/derived/reports/piezo-japan-shape-variants/summary.csvas a negative shape-control result. - Ran
./scripts/tune-japan-avalanche-events.shon the current 20,000-step profile. The nominal best isq=0.975, window120, capped at 25 events, with shape score0.291279and normalized distance2.083974; it still has 7 burst runs versus 595 in the real reference. Because the apparent improvement depends on a global event cap, retaindata/derived/reports/japan-avalanche-event-tuning-reduced.csvas a calibration diagnostic only. - Completed the Japan archive retry: all 24 hourly Moshiri files for 2026-07-28 and all 24 for 2026-07-29 are now decoded, with one 2026-07-30 file also available. The expanded anomaly report shows 28 July scores of
1.529--5.418, with the maximum after the main event; 27 July reached6.305before it. This does not support an event-specific precursor claim. Keep Japan data restricted to scientific research use. - Fixed the self-supervised Transformer evaluator so its default synthetic-only run skips the unavailable
synthetic_then_japan_then_italyregime and records it as skipped; explicit requests still require both Japan and Italy manifests. The full CPU run completed with full-input balanced accuracy0.413--0.443and piezo-only0.500, so no representation-transfer utility is demonstrated. The targeted Transformer test passes. - Rebuilt the Japan M5 CDF window dataset after the archive refresh. It now scans 129 feature files and produces 78 seismic windows, but only 8 windows contain VLF rows and the catalog currently ends on 2026-07-01; the new 28--29 July event-day files therefore do not yet enter the model table. Extend Japan seismic windows through the event before retraining or evaluating transfer.
- Extended the combined Japan seismic catalog through 2026-08-01: 1,353 normalized events and 133 complete weekly M5 windows. The rebuilt model input has 9 VLF-observed windows and 124 missing. The 2026-07-27 event week is not yet a complete seven-day target window because its end falls on 2026-08-03; keep it pending until the target horizon matures rather than treating the absence as a join bug.