In Sample Optimization Training Data
It is important that the basic buy-and-sell rules are successful across a range of calculation periods and similar markets. If this fails, then you need to rethink your idea. I don’t believe in adding rules to make the strategy more complex in the hope that one rule will turn a losing idea into a profitable one. It is necessary that the basic concept is robust before moving forward. Then we can try to make it better. — Perry Kaufman
Show the code
df_train, df_test = main.train_test_split(screened_df, test_size=0.20)
optimize = bt.Optimization(TrendStrategy, df_train, df_train[('BTCUSDT', 'Close')], cash=100_000, commission=0.0)
values = [
lookbacks
]
main.timer.start()
optimize.run(values)
main.timer.stop()
[*] Total runtime: 13.30 s
Show the code
main.plot_comparative_graph(optimize.equity_curves())
Visually inspection on the “optimized” strategy equity curves do reveal that the 2 latest tested lookbacks (320 & 640) do underperfom heavily vs shorter lookbacks. Lookbacks to 80 days have overperformed the Benchmark in returns terms.
Show the code
strats_performance = pd.concat([tools.strategy_performance_metrics(optimize.equity_curves(), steps_per_year=365, rf=0),
optimize.optimization_trades_stats()],
axis='index').fillna('-')
strats_performance
| CAGR % |
177.49 |
212.29 |
210.23 |
212.03 |
225.96 |
222.57 |
164.46 |
74.06 |
53.10 |
| Ann. Vola % |
145.21 |
76.80 |
73.80 |
73.70 |
76.21 |
80.26 |
79.04 |
75.47 |
74.32 |
| Max Drawdown % |
-93.07 |
-79.09 |
-62.99 |
-64.26 |
-64.91 |
-70.28 |
-84.13 |
-90.43 |
-97.79 |
| VaR 5% |
-6.73 |
-3.90 |
-3.88 |
-3.95 |
-4.23 |
-4.55 |
-4.64 |
-5.05 |
-5.05 |
| Time in Market % |
99.52 |
64.28 |
61.46 |
58.34 |
58.25 |
58.64 |
60.01 |
59.81 |
66.89 |
| Sharpe |
1.22 |
2.76 |
2.85 |
2.88 |
2.96 |
2.77 |
2.08 |
0.98 |
0.71 |
| Sortino |
2.25 |
3.46 |
3.28 |
3.26 |
3.25 |
3.04 |
2.22 |
0.98 |
0.70 |
| Calmar |
1.91 |
2.68 |
3.34 |
3.30 |
3.48 |
3.17 |
1.95 |
0.82 |
0.54 |
| Efficiency |
1.78 |
3.30 |
3.42 |
3.63 |
3.88 |
3.80 |
2.74 |
1.24 |
0.79 |
| Nr trades |
- |
539.00 |
335.00 |
209.00 |
125.00 |
73.00 |
40.00 |
36.00 |
18.00 |
| Win Rate % |
- |
38.78 |
33.23 |
31.58 |
32.00 |
32.88 |
27.50 |
25.00 |
16.67 |
| Profit Factor |
- |
1.21 |
1.29 |
1.37 |
1.89 |
1.88 |
3.47 |
3.36 |
3.41 |
| Avg. Win % |
- |
13.13 |
22.73 |
34.94 |
58.10 |
100.60 |
245.71 |
185.43 |
1075.39 |
| Avg. Loss % |
- |
-3.45 |
-3.79 |
-3.83 |
-4.13 |
-4.20 |
-5.09 |
-8.94 |
-12.95 |
| Reward/Risk |
- |
3.81 |
6.00 |
9.12 |
14.06 |
23.95 |
48.24 |
20.75 |
83.02 |
| Expectancy % |
- |
2.98 |
5.02 |
8.41 |
15.78 |
30.26 |
63.88 |
39.66 |
168.44 |
| Avg. Bars Held |
- |
4.00 |
7.00 |
12.00 |
20.00 |
35.00 |
68.00 |
74.00 |
168.00 |
The results tell a clear story: trend following works well on Bitcoin, but only when trades are allowed to run long enough to capture major trends. Short holding periods generate many trades but relatively weak performance due to noise and volatility.
The strongest performance appears around 20–80 bars, where risk-adjusted metrics peak (Sharpe near 3, with strong Sortino and Calmar ratios).
The system operates with a low win rate (~30%), but remains profitable due to the large reward-to-risk asymmetry. Losses average around -5%, while winners range from roughly 20% to over 200%, reflecting the classic trend-following principle: cut losses short and let profits run.
As lokback increases, holding periods increase and trade quality improves as expectancy rises sharply.
Show the code
tools.parameter_sensitivity_bar_plot(strats_performance, 'CAGR %')
All lookbacks are profitable, pointing out that these (Entry & Exits) signals are something good to start with. The goal is not to trade/invest on the highest CAGR strategy, but to choose a strategy parameter that is robust to stand the test of time. As stated before, long Lookbacks (160, 320 and 640) have shown to underperform shorter lookbacks.
Show the code
tools.parameter_sensitivity_bar_plot(strats_performance, 'Ann. Vola %')
The trend following filter have managed to cut Bitcoin volatility in half in all of the tested parameters.
Show the code
tools.parameter_sensitivity_bar_plot(strats_performance, 'Max Drawdown %')
In terms of Maximum Drawdowns the trend following filter have delivered the best results on the medium-term 20-80 bars lookback horizons. After 80 bars lookback, the longer the lookback, the longer is the Maximum Drawdown.
Show the code
tools.parameter_sensitivity_bar_plot(strats_performance, 'VaR 5%')
All tested parameters managed to deliever a lower historical VAR5% than Bitcoin Buy & Hold, highlighting Trend Following usefullness for more sophisticated investors. VAR has increased with the increase of lookback period, the longer the lokback is, the bigger the VAR is.
Overall, the results suggest Bitcoin is a highly trend-dominated market, where the strategy’s edge comes from cutting losses quickly and allowing large trends to compound, with the best balance occurring at medium-term holding horizons.
Out of Sample Validation
Since we discussed earlier that the most reliable lookback horizon was between 20-80 bars, let’s go with the middle figure and test out of sample, including 0.2% cost per trade (Binance Cost Structure).
Show the code
main.timer.start()
os_backtest = bt.Backtest(TrendStrategy, df_test, df_test[('BTCUSDT', 'Close')], cash=100_000, commission=0.002)
os_result = os_backtest.run(sma_lookback=40)
main.timer.stop()
[*] Total runtime: 362.47 ms
Show the code
| Start Period |
2023-01-03 |
| End Period |
2026-02-18 |
| Duration (Months) |
37 |
|
|
| CAGR % |
56.80 |
| Ann. Vola % |
35.02 |
| Max Drawdown % |
-24.01 |
| VaR 5% |
-2.41 |
| Time in Market % |
60.30 |
| Sharpe |
1.62 |
| Sortino |
2.09 |
| Calmar |
2.37 |
| Efficiency |
0.94 |
|
|
| Nr trades |
36.00 |
| Win Rate % |
36.11 |
| Profit Factor |
3.13 |
| Avg. Win % |
17.61 |
| Avg. Loss % |
-2.40 |
| Reward/Risk |
7.35 |
| Expectancy % |
4.83 |
| Avg. Bars Held |
18.00 |
The Out of Sample period spans 37 months from january 2023 until february 2026. In this period, the Trend Following strategy using 40 days lookback managed to deliver a 56% annualized returns figure with roughly 35% annualized volatility leading to a whoping 1.6 raw Sharpe Ratio, 2.1 raw Sortino Ratio and 2.4 Calmar Ratio. Its only been invested ~60% of the time, with a total of 36 trades averaging ~1 trade per month. Consistent with previous In Sample results, its win rate is around 30% with Huge wins and short losses, boosting the Reward/Risk Ratio to ~7.
Show the code
main.plot_strategy_performance(os_result.returns, os_result.benchmark)
Trend Following Strategy has been trailing Buy & Hold for most of the time until just recently due to the recent fall Bitcoin has taken since 10/10 Trump Tariffs first announcement. Highlighting once again the superiorness of Wealth preservation vs Buy & Hold.
Returns Analysis
Show the code
tools.strategy_performance_metrics(pd.concat([os_result.returns, os_result.benchmark], axis=1), steps_per_year=365, rf=0)
| CAGR % |
56.80 |
55.86 |
| Ann. Vola % |
35.02 |
47.97 |
| Max Drawdown % |
-24.01 |
-49.74 |
| VaR 5% |
-2.41 |
-3.68 |
| Time in Market % |
60.30 |
100.00 |
| Sharpe |
1.62 |
1.16 |
| Sortino |
2.09 |
1.78 |
| Calmar |
2.37 |
1.12 |
| Efficiency |
0.94 |
0.56 |
In the out of sample period, Strategy has outperformed Bitcoin Buy & Hold in all fronts like returns, volatility, Maximum Drawdown and Risk Adjusted Returns.
Show the code
tools.metrics_time_window_analysis(pd.concat([os_result.returns, os_result.benchmark], axis=1), lookbacks=[3, 6, 9, 12, 18, 24])
| 3-month |
48.6% |
62.9% |
| 6-month |
75.0% |
78.1% |
| 9-month |
96.6% |
93.1% |
| 12-month |
92.3% |
84.6% |
| 18-month |
100.0% |
100.0% |
| 24-month |
100.0% |
100.0% |
Up until 9 months, BH proved to be more reliable in terms of wealth preservation than the trend following strategy. On a 12 month rolling scenario, trading strategy delivered a profit 100% of the time vs Buy & Hold ~85%.
Show the code
main.monthly_pos_neg_returns(os_result.returns)
Trades Analysis
Show the code
| 0 |
BTCUSDT |
2023-01-07 |
2023-03-01 |
53 |
16952.12 |
23150.93 |
5.89 |
36493.61 |
36.57 |
472.19 |
136021.42 |
1 |
None |
| 1 |
BTCUSDT |
2023-03-02 |
2023-03-04 |
2 |
23647.02 |
22362.92 |
5.74 |
-7371.58 |
-5.43 |
528.26 |
128121.59 |
1 |
None |
| 2 |
BTCUSDT |
2023-03-14 |
2023-04-22 |
39 |
24201.77 |
27265.89 |
5.28 |
16188.80 |
12.66 |
543.84 |
143766.54 |
1 |
None |
| 3 |
BTCUSDT |
2023-04-28 |
2023-05-02 |
4 |
29481.01 |
28087.18 |
4.87 |
-6783.60 |
-4.73 |
560.35 |
136422.59 |
1 |
None |
| 4 |
BTCUSDT |
2023-05-03 |
2023-05-08 |
5 |
28680.49 |
28450.46 |
4.75 |
-1092.02 |
-0.80 |
542.42 |
134788.16 |
1 |
None |
| 5 |
BTCUSDT |
2023-05-29 |
2023-05-30 |
1 |
28075.59 |
27745.12 |
4.79 |
-1583.38 |
-1.18 |
534.91 |
132669.87 |
1 |
None |
| 6 |
BTCUSDT |
2023-06-20 |
2023-07-25 |
35 |
26841.66 |
29178.97 |
4.93 |
11529.51 |
8.71 |
552.68 |
143646.70 |
1 |
None |
| 7 |
BTCUSDT |
2023-09-19 |
2023-09-25 |
6 |
26760.85 |
26253.78 |
5.36 |
-2716.45 |
-1.89 |
568.01 |
140362.24 |
1 |
None |
| 8 |
BTCUSDT |
2023-09-26 |
2023-09-27 |
1 |
26294.76 |
26209.50 |
5.33 |
-454.21 |
-0.32 |
559.42 |
139348.61 |
1 |
None |
| 9 |
BTCUSDT |
2023-09-28 |
2024-01-13 |
107 |
26355.81 |
42799.45 |
5.28 |
86767.34 |
62.39 |
729.82 |
225386.14 |
1 |
None |
| 10 |
BTCUSDT |
2024-01-30 |
2024-02-01 |
2 |
43300.23 |
42569.76 |
5.19 |
-3794.62 |
-1.69 |
892.16 |
220699.36 |
1 |
None |
| 11 |
BTCUSDT |
2024-02-02 |
2024-02-05 |
3 |
43077.64 |
42577.62 |
5.11 |
-2556.63 |
-1.16 |
875.92 |
217266.80 |
1 |
None |
| 12 |
BTCUSDT |
2024-02-07 |
2024-04-03 |
56 |
43090.02 |
65446.67 |
5.03 |
112500.83 |
51.88 |
1092.33 |
328675.30 |
1 |
None |
| 13 |
BTCUSDT |
2024-04-04 |
2024-04-13 |
9 |
65975.70 |
67188.38 |
4.97 |
6029.22 |
1.84 |
1324.14 |
333380.39 |
1 |
None |
| 14 |
BTCUSDT |
2024-05-16 |
2024-06-14 |
29 |
66256.11 |
66747.57 |
5.02 |
2467.94 |
0.74 |
1335.80 |
334512.54 |
1 |
None |
| 15 |
BTCUSDT |
2024-07-16 |
2024-08-03 |
18 |
64784.42 |
61414.81 |
5.15 |
-17364.16 |
-5.20 |
1300.65 |
315847.73 |
1 |
None |
| 16 |
BTCUSDT |
2024-08-24 |
2024-08-28 |
4 |
64103.87 |
59507.93 |
4.92 |
-22599.59 |
-7.17 |
1215.67 |
292032.47 |
1 |
None |
| 17 |
BTCUSDT |
2024-09-14 |
2024-09-17 |
3 |
60569.12 |
58192.51 |
4.81 |
-11435.89 |
-3.92 |
1142.93 |
279453.65 |
1 |
None |
| 18 |
BTCUSDT |
2024-09-18 |
2024-10-10 |
22 |
60309.00 |
60581.93 |
4.62 |
1262.15 |
0.45 |
1118.11 |
279597.69 |
1 |
None |
| 19 |
BTCUSDT |
2024-10-12 |
2024-12-23 |
72 |
62444.62 |
95099.39 |
4.47 |
145920.91 |
52.29 |
1408.00 |
424110.61 |
1 |
None |
| 20 |
BTCUSDT |
2024-12-25 |
2024-12-27 |
2 |
98675.91 |
95704.98 |
4.29 |
-12743.65 |
-3.01 |
1667.57 |
409699.39 |
1 |
None |
| 21 |
BTCUSDT |
2025-01-04 |
2025-01-08 |
4 |
98106.99 |
96924.16 |
4.17 |
-4929.69 |
-1.21 |
1625.67 |
403144.03 |
1 |
None |
| 22 |
BTCUSDT |
2025-01-16 |
2025-02-03 |
18 |
100505.30 |
97681.10 |
4.00 |
-11305.72 |
-2.81 |
1586.75 |
390251.57 |
1 |
None |
| 23 |
BTCUSDT |
2025-02-04 |
2025-02-05 |
1 |
101398.72 |
97878.01 |
3.84 |
-13523.06 |
-3.47 |
1530.84 |
375197.67 |
1 |
None |
| 24 |
BTCUSDT |
2025-04-13 |
2025-04-14 |
1 |
85279.47 |
83694.52 |
4.39 |
-6959.25 |
-1.86 |
1483.88 |
366754.54 |
1 |
None |
| 25 |
BTCUSDT |
2025-04-15 |
2025-06-06 |
52 |
84539.70 |
101574.37 |
4.33 |
73753.20 |
20.15 |
1611.60 |
438896.15 |
1 |
None |
| 26 |
BTCUSDT |
2025-06-07 |
2025-06-15 |
8 |
104390.65 |
105464.84 |
4.20 |
4507.29 |
1.03 |
1761.10 |
441642.35 |
1 |
None |
| 27 |
BTCUSDT |
2025-06-17 |
2025-06-18 |
1 |
106794.12 |
104602.07 |
4.13 |
-9047.02 |
-2.05 |
1744.95 |
430850.38 |
1 |
None |
| 28 |
BTCUSDT |
2025-06-26 |
2025-07-02 |
6 |
107375.07 |
105703.10 |
4.00 |
-6695.51 |
-1.56 |
1706.57 |
422448.30 |
1 |
None |
| 29 |
BTCUSDT |
2025-07-03 |
2025-08-02 |
30 |
108845.02 |
113320.39 |
3.87 |
17335.12 |
4.11 |
1721.09 |
438062.33 |
1 |
None |
| 30 |
BTCUSDT |
2025-08-04 |
2025-08-07 |
2 |
114223.92 |
115030.05 |
3.83 |
3085.44 |
0.71 |
1754.92 |
439392.85 |
1 |
None |
| 31 |
BTCUSDT |
2025-08-08 |
2025-08-19 |
11 |
117505.50 |
116241.86 |
3.73 |
-4715.75 |
-1.08 |
1744.63 |
432932.47 |
1 |
None |
| 32 |
BTCUSDT |
2025-09-11 |
2025-09-23 |
12 |
113961.43 |
112757.48 |
3.79 |
-4564.61 |
-1.06 |
1719.14 |
426648.71 |
1 |
None |
| 33 |
BTCUSDT |
2025-09-30 |
2025-10-11 |
11 |
114396.52 |
113236.43 |
3.72 |
-4318.00 |
-1.01 |
1694.55 |
420636.15 |
1 |
None |
| 34 |
BTCUSDT |
2025-10-27 |
2025-10-28 |
1 |
114479.85 |
114129.09 |
3.67 |
-1286.25 |
-0.31 |
1676.61 |
417673.28 |
1 |
None |
| 35 |
BTCUSDT |
2026-01-03 |
2026-01-21 |
18 |
89945.05 |
88326.51 |
4.63 |
-7500.96 |
-1.80 |
1652.36 |
408519.97 |
1 |
None |
At a first glance, it appears to be several trades lasting only 1 day, adding up unncessary Transaction Costs due to fakeouts.
Show the code
tools.metrics_trades(os_result.trades, full_metrics=True)
| Nr trades |
36.00 |
| Win Rate % |
36.11 |
| Profit Factor |
3.13 |
| Avg. Win % |
17.61 |
| Avg. Loss % |
-2.40 |
| Reward/Risk |
7.35 |
| Expectancy % |
4.83 |
| Avg. Bars Held |
18.00 |
| Commissions |
43979.81 |
| Max Bars Held |
107.00 |
| Min Bars Held |
1.00 |
| Best Trade % |
62.39 |
| Worst Trade % |
-7.17 |
| Avg. Open Positions |
1.00 |
| Max Open Positions |
1.00 |
Show the code
tools.plot_trades_signals(df_test['BTCUSDT'][['Close', 'SMA40']], os_result.trades)
As stated previously, there are consolidation periods that lead to several in and out trades in a short period of time, adding up transaction costs.
Show the code
clipped_returns = main.cap_series(os_result.trades['change_pct'], percentile=0.005)
ax2 = clipped_returns.plot.hist(bins=50, title="Trades Return Distribution", figsize=(10, 5))
Strategy is working as intended, cutting losses short and letting profits run.
Show the code
ax1 = os_result.trades['bars_held'].plot.hist(bins=30, title="Bars Held Distribution", figsize=(10, 5))
ax1.axvline(os_result.trades['bars_held'].mean(), color='red', linestyle='dashed', linewidth=2)
print("Avg. Bars Held:", int(os_result.trades['bars_held'].mean()))
Average trade lasted 18 days, there are several very short trades, this may not be ideal.
Show the code
import plotly.express as px
ax2 = px.scatter(os_result.trades,
x='bars_held',
y='change_pct',
width=900,
height=500,
trendline="ols",
trendline_color_override="black",
title='Trade duration VS Trade Profit',
template='seaborn'
)
ax2.show()
Trend is clear, the longer the trade is the more profit it delivers.
Statistical Analysis
Show the code
main.correlation_map(pd.concat([os_result.returns, os_result.benchmark], axis=1))
The tested Trend Following Strategy as a 0.7 correlation with Bitcoin Buy & Hold.
Is it significantly different from 0?
Show the code
main.compute_t_test(os_result.returns)
P-value: 0.009881528305754408 Reject the null hypothesis (H₀). The strategy average return is significantly different from zero.
The out-of-sample returns exhibit a statistically significant positive mean with a p-value of 0.0099. This suggests that the strategy’s expected return is unlikely to be zero.
Does it pass Monte Carlo Permutation?
This will shuffle the given equity curve returns in random order, all output equity curves start and end at the same place, only the path will vary, to obtain Confidence Intervals for Max Draw Down Figures.
Show the code
mcpyt = main.monte_carlo_permutation_yt(os_result.returns, nr_simulations=1000)
Show the code
| 2026-02-14 |
4.12 |
4.15 |
3.66 |
3.95 |
4.23 |
3.45 |
4.08 |
4.04 |
4.11 |
4.19 |
... |
3.99 |
4.17 |
4.06 |
4.11 |
3.76 |
4.16 |
4.02 |
3.99 |
4.24 |
4.09 |
| 2026-02-15 |
4.18 |
4.10 |
3.70 |
3.97 |
4.23 |
3.54 |
4.06 |
4.04 |
4.09 |
4.19 |
... |
4.06 |
4.03 |
4.06 |
4.07 |
3.93 |
4.22 |
4.12 |
3.87 |
4.26 |
4.01 |
| 2026-02-16 |
4.24 |
4.05 |
3.82 |
4.05 |
4.23 |
3.54 |
4.09 |
4.09 |
4.09 |
4.19 |
... |
4.06 |
4.12 |
4.06 |
4.07 |
4.00 |
4.20 |
4.16 |
4.07 |
4.11 |
4.01 |
| 2026-02-17 |
4.24 |
4.07 |
3.92 |
4.05 |
4.09 |
3.73 |
4.09 |
4.09 |
4.09 |
4.09 |
... |
4.01 |
4.14 |
4.09 |
4.07 |
4.00 |
4.23 |
4.16 |
4.09 |
4.09 |
3.98 |
| 2026-02-18 |
4.09 |
4.09 |
4.09 |
4.09 |
4.09 |
4.09 |
4.09 |
4.09 |
4.09 |
4.09 |
... |
4.09 |
4.09 |
4.09 |
4.09 |
4.09 |
4.09 |
4.09 |
4.09 |
4.09 |
4.09 |
5 rows × 1000 columns
Show the code
main.mcpyt_analysis(mcpyt, os_result.returns, ci=95)
The observed maximum drawdown lies within the 95% Monte Carlo confidence interval derived from return permutations. The p-value of 0.17 suggests that the realized drawdown is not statistically different from what would be expected under random ordering of returns. While the realized path exhibits somewhat smaller drawdowns than the average randomized path, the result does not provide strong statistical evidence that the strategy benefits from favorable return sequencing.