Defense First Backtesting: Why QuantAnalyzer Belongs in Every EA Evaluation

By

Stephanie Kurek

|

backtesting
  1. What “Defense First” Means (and Why We Use QA to Enforce It)
  2. 1) Performance Summary (After Costs)
  3. 2) Monte Carlo: Randomness & Cost Stress
  4. 3) In-Sample vs. Out-of-Sample (OOS) Split
  5. 4) Regime Robustness: Year/Month Heatmaps
  6. 5) What-If: Cost Sensitivity
  7. 6) Trade Distribution & Expectancy
  8. 7) Drawdown & Stagnation (Underwater Curve)
  9. 8) Correlation & Portfolio (If You Combine Pairs)
  10. 9) Session / Market Dependency (Optional, but Great for Anti-Overfit)
  11. 10) Reproducibility Pack (Process Proof)
  12. Putting It Together: A Defense-First Checklist You Can Reuse
  13. How We Use QuantAnalyzer for Kraitos Elite (and Why It Matters)
  14. Frequently Asked (Beginner-Friendly) Questions
  15. Your Next Step: See the Proof, Not Just the Curve

If you’re new to expert advisors (EAs), it’s tempting to judge them by one thing: how much money the backtest made. That’s understandable—but it’s also how people get blindsided by large drawdowns, over-fit strategies, and results that look great on paper and melt in real trading.

A defense-first approach flips the script. We still care about returns, but we rank risk control, robustness, and reproducibility higher. And that’s where QuantAnalyzer (QA) shines. It’s not “just another report generator.” It’s a toolkit that helps you validate an EA across time, regimes, and cost assumptions—so you’re not betting your account on a pretty equity curve.

In this article, we’ll walk through why QA matters, then go step-by-step through a report stack that proves whether an EA is genuinely robust or merely curve-fit. By the end, you’ll know exactly what to ask for (and what to ignore) when evaluating any strategy—including ours.

Quick note on assumptions: In our workflow, we run backtests on tick data with historical bid/ask, and we apply commissions and conservative slippage. This matters because QA can reflect those same costs in every analysis below. Realistic inputs → realistic expectations.

backtesting

What “Defense First” Means (and Why We Use QA to Enforce It)

“Defense first” means you prefer steady, repeatable edges over fragile profit spikes. The fastest way to spot the difference is to ask:

  • Does the strategy hold up after costs?
  • Does it stay profitable when you shuffle trades (Monte Carlo)?
  • Does it work outside the period it was tuned on (OOS)?
  • Does it survive wider spreads or extra slippage?
  • Does it degrade gracefully across regimes (COVID shock, rate cycles, etc.)?

QuantAnalyzer tackles each of those questions with a focused report. One report won’t tell the whole truth—but together, they give you a 360° perspective that minimizes unpleasant “surprises.”

Below is the exact stack we run for every profile. You can mirror it yourself or use it as a checklist for any EA you’re considering.

1) Performance Summary (After Costs)

What it is: Your baseline facts: Max Drawdown (DD), Profit Factor (PF), Sharpe, trades, stagnation, and more—after spread, commission, and slippage.

Why it matters (for beginners):

  • DD tells you the deepest equity drop (the pain).
  • PF (gross profit ÷ gross loss) hints at edge quality.
  • Sharpe balances return vs. volatility (smoother ≈ easier to hold).
  • Trade count & stagnation preview patience and sample size.

How to produce it:
 Import your backtest → open Reports → Overview / Summary. Make sure QA cost settings match your test (Settings → Commissions/Slippage). Export to PDF/HTML.

What to look for:

  • PF ≥ 1.3 (decent), ≥ 1.5 (strong), after costs.
  • Drawdown at a level you can tolerate.
  • Sharpe ≥ 1.0 (smoother).
  • Stagnation not absurdly long relative to the test.

Defense-first lens: A “good” PF with tiny trade count or huge stagnation is a yellow flag. Don’t let a single number charm you; always read the whole card.

2) Monte Carlo: Randomness & Cost Stress

What it is: QA can reshuffle trade order, perturb outcomes slightly, and vary costs. You get distributions (not just a single line) for PF, DD, and worst weeks.

Why it matters:
 Backtests are one historical path. But real trading contains randomness. If a strategy only looks good in one specific sequence, it’s brittle. Monte Carlo shows whether the edge survives many alternate paths—especially when you increase costs (wider spread, extra slippage).

How to produce it:
 Open your strategy → Monte Carlo.

  • Simulations: 1,000–5,000
  • Randomize trade order and trade outcomes (±10–20%).
  • Add slippage/spread variance (+25–50%).
    Export the percentile table and charts.

What to look for:

  • PF/Sharpe stay positive in lower percentiles (5th–10th).
  • DD doesn’t explode when you widen costs.
  • Use the 5–10th percentile to size risk, not the median. (That’s defense.)

3) In-Sample vs. Out-of-Sample (OOS) Split

What it is: Split your history into training (IS) and held-out (OOS) periods. If the strategy only works in the period it was tuned on, it’s likely curve-fit.

Why it matters:
 This is the most direct “anti-curve-fit” test. OOS performance should degrade some, but it should remain net positive.

How to produce it:
 Set Date Ranges in QA (e.g., IS: 2019–2022; OOS: 2023→). Go to Reports → By Periods / IS-OOS and export the side-by-side metrics and equity overlay.

What to look for:

  • OOS PF/Sharpe slightly lower than IS (that’s normal).
  • Still profitable after costs.
  • OOS DD in line with your risk tolerance.

Defense-first lens: If OOS collapses—or improves dramatically—ask why. Both can reveal instability or data quirks.

4) Regime Robustness: Year/Month Heatmaps

What it is: A breakdown of performance by year and month (and custom windows). Think of it as a regime stress test: COVID shock, war headlines, rate cycle, “normalization.”

Why it matters:
 Real markets don’t behave like one smooth line. If an EA only works during a specific trend and dies elsewhere, you’ll live through the “elsewhere.” Heatmaps make this painfully obvious.

How to produce it:
 Reports → Breakdown → By Year / By Month.
 Also create custom ranges for major events (e.g., 2019–2020 COVID, 2022→ war, 2022–2024 rate hikes). Export the heatmap/table.

What to look for:

  • A mix of greens with manageable reds.
  • No single “hero month/year” hiding a sea of red.
  • Performance that persists across multiple regimes
backtesting

5) What-If: Cost Sensitivity

What it is: A grid showing performance if you increase costs—bigger spreads, more slippage, higher commissions.

Why it matters:
 Backtest costs are assumptions. If a small nudge in spread or slippage destroys results, you’re looking at a fragile system. Defense first means results survive realistic worst-case friction.

How to produce it:
 Open What-If / Sensitivity (name varies by QA build). Test scenarios like Commission +50%, Slippage +0.2–0.5, Spread ×1.25/×1.5. Export the table/grid.

What to look for:

  • PF stays >1.0 under stressed costs.
  • DD increases are modest.
  • Your public assumptions are not cherry-picked best case.

6) Trade Distribution & Expectancy

What it is: Expectancy per trade, R-multiple distribution, win/loss streaks.

Why it matters:
 You want quality of edge—not just “lots of trades.” Expectancy forces you to ask, “On average, how much does each trade earn, after costs?” Distribution shows whether losers are capped and whether winners have a healthy tail.

How to produce it:
 Go to Reports → Trade Analysis. Export the histogram(s) and key tables.

What to look for:

  • Positive expectancy after costs.
  • A thinner left-tail (loss control) and a usable right-tail.
  • Streaks that are psychologically tolerable at your chosen risk.

7) Drawdown & Stagnation (Underwater Curve)

What it is: The underwater chart plots every equity dip from peak; stagnation tracks time between new equity highs.

Why it matters:
 Numbers are one thing; shape of pain is another. You should know what a “bad month” and “long flat stretch” actually look like—before you size live risk.

How to produce it:
 Charts → Underwater / Drawdown and Stagnation. Export both.

What to look for:

  • Max DD that matches your risk tolerance.
  • Stagnation durations you can realistically stick through.
  • A curve that recovers—not one that sinks and never returns.

8) Correlation & Portfolio (If You Combine Pairs)

What it is: Build a portfolio from multiple strategies and see the correlation matrix and portfolio-level equity.

Why it matters:
 Even strong single strategies can sync their drawdowns. If you plan to run multiple pairs, diversification is your second line of defense.

How to produce it:
 Select multiple strategies → open Portfolio view. Export the correlation matrix and portfolio metrics (PF, DD, Sharpe).

What to look for:

  • Correlations that aren’t all ~1.0; lower is better.
  • Portfolio DD that’s lower than the sum of parts.
  • Portfolio PF/Sharpe holding up after costs.

9) Session / Market Dependency (Optional, but Great for Anti-Overfit)

What it is: Performance by hour of day and day of week.

Why it matters:
 Over-fit systems sometimes rely on a tiny time window. If all the edge lives in 1–2 hours and everything else is noise, you’ll need tight execution to match the backtest—and even then it’s fragile.

How to produce it:
 Reports → Breakdown → Hour of Day / Day of Week. Export both charts.

What to look for:

  • Some clustering is normal, but not a razor-thin dependency.
  • If dependency exists, it should be deliberate (e.g., news avoidance) and explainable.

backtesting results

Supercharge your trading with our expert advisors. Start your journey today!

UNLOCK YOUR EDGE

10) Reproducibility Pack (Process Proof)

What it is: A zipped bundle per strategy: QA Summary PDF, Monte Carlo PDF, IS/OOS table, cost sensitivity, heatmap, underwater chart—plus the tester settings file (.set/JSON).

Why it matters:
 Trust comes from transparency. If someone can re-run your process, they’ll believe the results. That’s the opposite of curve-fit marketing.

How to produce it:
 After exporting each report above, drop them into a folder named by your Strategy ID (e.g., KE-GBPUSD-117/), include the settings file, then zip it.

What to look for (as a buyer):

  • Costs match what’s listed on the site.
  • IS/OOS split is disclosed and reasonable.
  • Monte Carlo and What-If don’t crater the edge.
  • Heatmaps/regimes line up with the public timeframe.

Putting It Together: A Defense-First Checklist You Can Reuse

When you evaluate any EA (ours included), ask for:

  1. Summary after costs (PF, DD, Sharpe, trades, stagnation).
  2. Monte Carlo with trade order/outcome randomization and cost variance.
  3. IS vs OOS results with a sensible split.
  4. Regime heatmaps (years/months, custom macro windows).
  5. What-If costs (spread ×1.5, slippage +0.2–0.5, commission +50%).
  6. Expectancy & distribution (quality of trades).
  7. Underwater & stagnation (the shape of pain).
  8. Correlation & portfolio (if running multiple pairs).
  9. Session dependency (avoid razor-thin windows).
  10. Reproducibility pack (so you or a third party can validate).

If any step is missing or avoids costs, proceed carefully. A defense-first product shouldn’t fear scrutiny—it should invite it.

How We Use QuantAnalyzer for Kraitos Elite (and Why It Matters)

Our philosophy is simple: ship profiles we’d personally run. For Kraitos Elite, we:

  • Backtest on Dukascopy tick data (bid/ask) and apply commissions + conservative slippage.
  • Run IS→OOS splits to ensure the edge survives outside the tuning period.
  • Stress the system with Monte Carlo and What-If costs.
  • Verify across multi-year regimes (2019→Today; COVID shock; war; rate cycle; normalization/mixed).
  • Bundle each profile’s QA PDFs + settings so you can inspect or re-run.

This is more work than posting a pretty equity curve—but it’s also why we can talk about drawdown control with a straight face. Defense comes first.

Frequently Asked (Beginner-Friendly) Questions

“What numbers should I focus on?”
 Start with PF, DD, and Sharpe after costs. Then check Monte Carlo lower percentiles, OOS metrics, and underwater shape. If you only remember one rule: size for the 5–10th percentile Monte Carlo outcome, not the best case.

“What if OOS is lower than IS?”
 That’s normal. Markets change. We care that it stays net positive and that degradation is reasonable (think 20–30%, not 80–100%).

“Why all the cost fuss?”
 Because many backtests quietly assume frictionless fills. Real spreads/commissions/slippage always sap edge. If results collapse with slightly worse costs, the strategy is fragile.

“Can I combine pairs to smooth results?”
 Yes—if correlation is modest. Always inspect the correlation matrix and portfolio DD. Diversification is a defense-first superpower.

backtesting
backtesting

Your Next Step: See the Proof, Not Just the Curve

If you’ve read this far, you’re already ahead of most EA shoppers. You know that realistic costs, OOS validation, Monte Carlo stress, and regime robustness are the difference between hope and probability.

We built Kraitos Elite—and our broader library of strategies—on that defense-first foundation. We publish the backtests, the assumptions, and QuantAnalyzer reports so you can evaluate us like a pro, even if you’re just getting started.

➡️ Explore our Backtesting & Proof Page (in our GoHighLevel funnel) to see:

  • Strategy cards with summary after costs
  • IS vs OOS and Monte Carlo snap-shots
  • Regime heatmaps and underwater curves
  • Downloadable QA PDFs and settings files

When you’re ready, use the Risk Modeling Tool to size the strategy to your account and tolerance. No hard sell—just receipts.

Past performance, including backtests, is hypothetical and does not guarantee future results. All projections include cost assumptions and may differ in live trading.

See How Automated Trading works

Schedule a demo call and get your questions answered by an expert. See live results showing consistent profits.

Discover Our Verified Results

Verified results, consistent profits — view multiple strategies running Kraitos Elite and GearBox now on Myfxbook!

widget

Kraitos Elite Reviews

Kraitos Elite Reviews

The More You Learn, The More You Earn