This is an earlier experiment. Read the amendments below before treating its findings as current.
The question
Thousands of test results had piled up. A small model could rank untested combinations of settings by risk.
What was tested
Two small models, one predicting whether a case breaks and one how many impacts, were trained on 2,058 results, with whole variants held back for testing. They then picked eight combinations, run on four layouts.
What was found then
The model predicted well which unseen variants break: a score of 0.87, where 0.5 is guessing and 1 is perfect. Its picks found 159 to 184 impacts per layout. Every affected case was at storm strength 10 or 14, and only 31 of the 682 impacts had no radio link. Retrained, it scored 0.91 but proposed no new kind of failure beyond storms on a slow link.
What we learned later
The next sweep (2026-09-06/07) found a failure from turbulence and tight spacing without any storm.
README.md:885, :919
What changed
Nothing in the simulator (a tool to choose what to test).
Corrections retained in the record
Earlier wording: Cross sweep 5: retrained model (3038 rows, AUC 0.88)
Record correction: No repo record of a 3,038-row / AUC 0.88 model; the README records the retrain on 3,486 rows at AUC 0.91.
README.md:885
Numbers and source references
- 2,058
- training rowsREADME.md:776
- 0.87
- AUC on held-out variantsREADME.md:776
- eight
- combinations pickedREADME.md:776
- four seeds
- seeds of cross sweep 4README.md:885
- 167, 184, 172 and 159
- impacts per seed, cross sweep 4README.md:885
- 0.91
- AUC after retraining on 3,486 rowsREADME.md:885
- 31 of the 682
- impacts outside any linkREADME.md:885
- 10 or 14
- storm strengths of every touched rowREADME.md:885