Not supported · Machine learning · 2026-09-26; timeframe gap fill 2026-10-11
Can a neural network do better than the tree model?
What this means
Small neural networks, logistic regression and gradient boosting all scored AUC 0.50-0.51, where 0.50 is a coin toss. The best model was 2.1 standard deviations above shuffled data, so it may not be pure noise, but an AUC of 0.51 is far too weak to pick trades with. A different model type does not get around the lack of information in the features. On 1h and daily no model beats shuffled outcomes. On 4h a plain logistic regression does a little better than the neural networks (0.55 vs 0.53), about two standard deviations above chance - consistent with the weak 4h result of the tree model, and still not enough to pick setups.
The explanation is our best reading; the data shows the result but not the cause directly.
By timeframe
| Timeframe | Verdict | n | Result |
|---|---|---|---|
| 15-minute | Not supported | 16,500 | best AUC 0.512 (+2.1 sigma) |
| 1-hour | Not supported | 13,208 | best (GBM) 0.512 vs null 0.501 (+0.9 sigma); q=0.31 gap fill G6, 2026-10-11 (trade logs) |
| 4-hour | Inconclusive | 3,990 | best (logistic) 0.550 vs null 0.500 (+2.2 sigma); NN 0.534; q=0.04 gap fill G6, 2026-10-11 (trade logs) |
| Daily | Not supported | 2,098 | best (GBM) 0.520 vs null 0.523 (-0.1 sigma); q=0.74 gap fill G6, 2026-10-11 (trade logs) |
- Hypothesis
- A small MLP on 29 encoded features predicts 15m setup outcomes.
- Method
- MLP (32,16) and (64,32,16), logistic regression and gradient boosting; split by instrument; shuffled-label null.
- Result
- AUC: NN(32,16) 0.5122 +/- 0.0031; logistic 0.5074; NN(64,32,16) 0.5066; GradBoost 0.5027; shuffled null 0.4950 +/- 0.0082 (best model +2.1 sigma). Gap fill (2026-10-11), same script and split: 1h best 0.512 (GBM) vs null 0.501 (+0.9 sigma); 4h best 0.550 (logistic) vs 0.500 (+2.2 sigma), NN 0.534; daily best 0.520 vs null 0.523 (-0.1 sigma).
- How much to trust it
- Medium: AUC 0.51 is of no practical use even if not pure noise
- Did it change the live rule?
- No.
Scripts and data for this study will be linked here when the research code is published on GitHub.
Education and research only. Not financial advice. Backtests use today's index members (survivorship bias).