Here's a story I tell anyone who's about to deploy a trading strategy.
In 2019 I built a backtest that looked like the holy grail. Bitcoin daily data, six years, simple rules: buy when X, sell when Y. The equity curve was beautiful. +840% with a max drawdown of -28%. I checked it three times. The math was clean.
I went live with $5,000. Six months later I had $4,100.
The strategy didn't break. The signals fired exactly as the backtest said they would. I held my discipline, executed every trade. And I still lost 18% in six months.
What happened? The same six things that happen to almost every strategy when it leaves the backtest and meets the real world. This is what they are, and what we do about them at BearBullRadar.
The Six Things Backtests Don't See
1. Slippage. Or: "the price you see isn't the price you get."
Backtest: BTC closes at $50,000. The bot buys at $50,000. Done.
Reality: BTC closes at $50,000. You place the order. It executes at $50,047. The next time it executes at $50,089. Over 50 trades a year, those tiny differences eat 1-2% of your return.
For a $10k paper portfolio nobody notices. For a $200k live account on a thinly-traded altcoin, slippage can cost more than the strategy makes.
Backtests pretend the price you see is the price you get. The real market disagrees.
2. Outages. Exchanges break.
In May 2022, Bybit's API was down for 47 minutes during a sharp BTC drop. If your bot wanted to sell during those 47 minutes, it couldn't. By the time the API came back, the price was 8% lower.
Solana has had three significant outages in its history. Binance has had several rolling restarts. Coinbase melted down completely on multiple high-volatility days.
Backtest assumes you can always trade. Reality assumes you can't.
3. Taxes. The boring killer.
Switzerland has a clean rule: if you trade like an investor, your gains are tax-free. If you trade like a professional, gains are taxed as income — up to 40-something percent depending on canton.
The line between "investor" and "professional" is fuzzy. But ~50 trades per year is in the danger zone. Most active strategies cross it.
Backtest shows pre-tax returns. Your actual wealth is post-tax. For a high-frequency strategy in Switzerland, that gap can be 30-40% per year. Your spectacular backtest +1,000% becomes a real-life +600%.
4. The future isn't the past.
Backtests use historical data. By definition.
What if the next five years are completely unlike the last eight? What if BTC stops having sharp 70% drawdowns? What if it goes sideways for years like gold did 2013-2019?
A strategy optimized for one regime can be useless in another. That's not a flaw in the strategy — it's the definition of the problem. We can't backtest the future. Nobody can.
5. You.
This one nobody talks about.
Imagine your bot is in a 30% drawdown. Three months in. You're losing sleep. The urge to "pause it for a few weeks until things stabilize" is enormous.
Backtest assumed the bot would run untouched, no matter what. That's not a real assumption. You will, at some point, second-guess the bot. Maybe you pause it. Maybe you turn it back on after it's recovered most of the loss. You just realized your "automatic" strategy was actually you-with-extra-steps. And you got the timing wrong.
The backtest didn't model your emotions. The real strategy must.
6. Disclosure decay.
The more public a strategy is, the faster its edge evaporates. If everyone uses MACD crossovers, MACD crossovers stop working — too many bots fire on the same signal at the same time, the move gets pre-empted, the alpha vanishes.
We publish our bot rules openly (well, the paper-tier ones). If thousands of readers ever consistently follow our signals, the edge would degrade just from being followed. We accept this as the cost of transparency.
Backtest can't model this. Live can.
What Real Quant Funds Have Learned The Hard Way
This isn't theoretical. Big professional funds have lost billions to the gap between backtest and reality.
LTCM (1998). Long-Term Capital Management had Nobel laureates and 25 years of perfect convergence-trade backtests. Russia defaulted, correlations broke that no historical model had ever seen, and the fund collapsed in months.
AQR's factor funds (2018-2020). Strategies with 30 years of academic backtest evidence sat in a multi-year drawdown. AQR's own analysis: factor correlations shifted in ways the historical data never showed.
The 2022 quant crypto wipeout. Several funds with strong 2019-2021 backtests blew up when correlations across crypto sectors shifted.
The pattern: backtests describe what happened. Reality is what hasn't happened yet.
What We Do About It At BearBullRadar
We can't eliminate the gap. We can manage it.
Step 1: Backtest is the entry exam, not the verdict.
Every strategy has to pass three tests before it sees paper money: full-period return that beats just-holding the asset, plus performing in three different historical periods, plus showing similar results across nearby parameter values. About 95% of strategies fail at least one of these. Those go to /post-mortems immediately.
Step 2: Paper money for at least 45 days.
Strategies that pass the entry exam earn a paper-trading slot. That means: real-time data, real signals, real Telegram alerts, real fees and timing — but virtual capital. We watch. After 45 days we ask: did the live trades match what the backtest predicted?
If yes, the bot has earned its lineup spot. If no, we look for which of the six factors above caused the gap.
Step 3: 90 days minimum before any real money.
Even with 45-day live data matching backtest, real money is a different conversation. We need broker selection, tax review, capital sizing, and a manual sign-off. No bot graduates to real capital based on backtest alone.
This three-step filter sounds slow. It is slow. That's the point. Most retail traders skip steps 2 and 3 entirely. The gap between their backtest and their actual portfolio is wide and quiet.
Right Now: A Live-vs-Backtest Experiment You Can Watch
We have three momentum bots running in parallel: Tactician 2.0, Rotator, and Tri-Rotator. They all trade BTC-related momentum strategies. Backtest predicts:
- Tri-Rotator wins
- Rotator second
- Tactician 2.0 last
The live data over the next 45 days will either confirm or contradict that ranking. We're documenting the comparison openly. If Tactician 2.0 surprises us and outperforms despite the backtest prediction, we'll publish the divergence and try to explain why.
Either outcome teaches us something. That's the whole point of the live-verification step.
What This Means For You
When you see Bot Cards on /bots, the backtest stats panel tells you what the historical data says. Sharpe, Calmar, walk-forward, beat-rate — all backtest-derived. They are predictions, not promises.
When a bot has been live for 90+ days, we'll show the comparison: what the backtest predicted vs what actually happened. If our methodology is sound, those numbers will roughly agree. If they don't, that's news, and we'll publish it.
For strategies you're personally evaluating — ours, your own, anybody's — the rule is: demand to see backtest evidence AND live performance AND honest disclosure of the gap when they differ. A pretty backtest without live data is research. A pretty live record without backtest disclosure is luck. Both together, with the gap honestly reported, is signal.
Related reading:
- Methodology page — How we test strategies before deploying
- Beat HODL or Don't Bother — The minimum bar every strategy must clear
- Benchmark Tunnel Vision — Why one number isn't enough
- The Tactician 2.0 — Currently running as our first explicit Live-vs-Backtest experiment
- Post-Mortems — Strategies that didn't make it
Not financial advice. The verification protocol reduces — but does not eliminate — the chance that a deployed bot underperforms its backtest. Past performance, live or backtest, does not guarantee future results.
