Limitations & Known Issues
We believe radical honesty builds better research. Here is everything that is currently imperfect in FX Regime Lab v2.1.
⚠️ Risk Reversal Data is Synthetic
Our 25-delta risk reversal signal is currently a proxy derived from daily price action, not actual options market data. We are actively sourcing real OTC risk reversal feeds. Until then, Layer 3 execution skips skew-based rules entirely.
Status: Sourcing real data. Expected: v3.0 (July 2026).
⚠️ Confidence Scores Are Not Probabilities
The "confidence" score is a heuristic consistency metric, not a calibrated probability. A 60% confidence does NOT mean we believe the call has a 60% chance of being correct. We are building proper isotonic calibration for v3.0.
Status: Calibration models under development. Expected: v3.0.
⚠️ Weights Are Heuristic, Not Optimized
Signal weights (e.g., Rate 45%, COT 25%) are round-number heuristics based on macro intuition. They have not been walk-forward optimized. v3.0 will use ICIR-based dynamic weighting with purged cross-validation.
⚠️ Accuracy is Near Random
Our T+5 directional accuracy is ~49% over the full backtested history (1997–2026). Live out-of-sample performance since May 2026 is significantly lower (~15%) due to small sample size and recent market conditions. We publish both figures transparently because the goal is honest regime monitoring, not signals.
⚠️ Transaction Costs Not Included in Primary Metrics
Track record shows gross returns. Typical FX spot bid-ask is 0.1–1.0 bps for major FX and 5–20 bps for EM. Net returns may be substantially lower. We now report cost-adjusted metrics alongside gross.
⚠️ Layer 1 Thresholds Are Ad-Hoc
Regime gate thresholds (e.g., Z > 2.0, Δπ > 0.12) were set by inspection, not systematic optimization. v3.0 will replace these with a Hidden Markov Model.
⚠️ COT Data is Weekly, Treated as Daily
CFTC Commitment of Traders reports weekly. Between publication dates, our model uses stale data. v2.1 tracks days-since-COT; v3.0 will apply staleness decay.
If you find additional issues, please open a GitHub issue or email shreyash@fxregimelab.com. We will add them here with attribution.