Not supported · Machine learning · 2026-09-26; timeframe gap fill 2026-10-11
Can a gradient-boosted model predict which 15m setups win?
What this means
A gradient-boosted model could not tell winning 15m setups from losing ones: its score (AUC 0.51) was within 1.4 standard deviations of models trained on shuffled outcomes. The same pipeline scored 0.56 on fake data with a planted signal, so it can find a signal when one exists. The 13 entry features therefore carry no usable information about which setups win. On the longer timeframes the model does a little better than chance - most on 4h, where it scores 0.54 and its top tenth of setups wins about 5 points more often - with distance from the 50-bar average as the most useful input. That is still far too weak to rank setups usefully, and on daily the setups it rated best actually won less often than average.
Explanation checked against the data.
By timeframe
| Timeframe | Verdict | n | Result |
|---|---|---|---|
| 15-minute | Not supported | 16,500 | AUC 0.5105 vs null 0.4988 (1.4 sigma) |
| 1-hour | Inconclusive | 13,208 | AUC 0.511 vs null 0.497 (2.5 sigma), top decile +2.4pp; q=0.02 gap fill G5, 2026-10-11 (trade logs) |
| 4-hour | Inconclusive | 3,990 | AUC 0.536 vs null 0.496 (3.1 sigma), top decile +5.1pp; q=0.006 gap fill G5, 2026-10-11 (trade logs) |
| Daily | Not supported | 2,098 | AUC 0.519 vs null 0.494 (2.6 sigma) but top decile -2.1pp; q=0.02 gap fill G5, 2026-10-11 (trade logs) |
- Hypothesis
- 13 entry features plus pattern carry enough information to rank setups.
- Method
- HistGradientBoosting, GroupKFold by instrument, out-of-fold AUC and permutation importance; 10 shuffled-label nulls; synthetic planted-effect control.
- Result
- OOF AUC 0.5105 vs shuffled null 0.4988 +/- 0.0083 (1.4 sigma); planted-effect control 0.5645 (5.3 sigma). Top-decile win rate 48.1% vs 42.6% base. Largest permutation importance +0.0037 (M100), within its fold SD. Gap fill (2026-10-11), same pipeline and 13 features + pattern, 20 shuffled nulls: 1h AUC 0.511 vs 0.497 +/- 0.005 (2.5 sigma); 4h 0.536 vs 0.496 +/- 0.013 (3.1 sigma), top-decile win 52.6% vs 47.5% base; daily 0.519 vs 0.494 +/- 0.010 (2.6 sigma) but its top decile won less often than average. None reaches the study's own bar for a signal (> 5 sigma and > +0.02 AUC).
- How much to trust it
- High: grouped CV, shuffled nulls and a positive control
- Did it change the live rule?
- No.
Scripts and data for this study will be linked here when the research code is published on GitHub.
Education and research only. Not financial advice. Backtests use today's index members (survivorship bias).