Dutch Algotrading

Strategy League Top 15 performance test 2022 - Deep dive (Patreon)

Published:

2023-01-24 14:41:02

Imported:

2026-06

Tags:

backtest data strategy league top15

haven't archived this post yet. have a subscription? use the importer!

Downloads

2023_top15_test_complete_dataset.7z

Missing 1 file, as this post has not yet been imported.

Content

In this post I am going to show you into the whole process I went through, when I analysed the performance of the Top 15 algo strategies over the 2022 bear market. I will show you each step I took and each command I used to get to these results.

I also am going to tell you what my decision is on the crypto pairs I am going to test in my future (yes, there will be a change here…), based on the conclusions of this analysis.

Preparations

To prepare myself for these Top15 tests over 2022 I gathered the strategies from my archive and added them to the /user_data/strategies directory. There, I added all the Top15 strategies I posted in the article Top 15 algo strategy in 2022 – How would they perform?.

To show the list of strategies in the strategies directory I used the command:

<code><code><code>sudo docker-compose run --rm freqtrade list-strategies</code></code></code>

Which gave me the following output:

<code><code><code>+-----------------------+--------------------------+----------+----------------+--------------+---------------+
|                  name |                 location |   status |   hyperoptable |   buy-Params |   sell-Params |
|-----------------------+--------------------------+----------+----------------+--------------+---------------|
|       ADXMomentumHopt |       ADXMomentumHopt.py |       OK |            Yes |            5 |             0 |
|           AwesomeMacd |           AwesomeMacd.py |       OK |             No |            0 |             0 |
| DemaSmaCrossoverHopt | DemaSmaCrossoverHopt.py |       OK |            Yes |            3 |             2 |
| NostalgiaForInfinityX | NostalgiaForInfinityX.py |       OK |             No |            0 |             0 |
| ReinforcedSmoothScalp | ReinforcedSmoothScalp.py |       OK |            Yes |            8 |            10 |
|                 Scalp |                 Scalp.py |       OK |             No |            0 |             0 |
|            SimpleHopt |            SimpleHopt.py |       OK |            Yes |            2 |             1 |
|      SmaCrossoverHopt |      SmaCrossoverHopt.py |       OK |            Yes |            2 |             0 |
|            SmaRsiHopt |            SmaRsiHopt.py |       OK |            Yes |            2 |             1 |
|             bb_riding |             bb_riding.py |       OK |             No |            0 |             0 |
|           keltnerhopt |    keltnerchannel-rsi.py |       OK |            Yes |            3 |             0 |
|            powerxhopt |            powerxhopt.py |       OK |            Yes |            2 |             0 |
|        slope_is_dope2 |        slope_is_dope2.py |       OK |             No |            0 |             0 |
|      smamacdcrossover |    sma-macd-crossover.py |       OK |            Yes |            1 |             0 |
|            vulcanhopt |            VulcanHopt.py |       OK |            Yes |            2 |             1 |
+-----------------------+--------------------------+----------+----------------+--------------+---------------+</code></code></code>

These strategies are backtested with the Top 50 pairs of 2022. If you followed my other article "My update of backtesting data and trading pairs", then you would know how the configuration of the Freqtrade bot for spot and futures pairs is configured and why I made these choices.

With this baseline it is time for me to test the 15 best strategies (up to december 2022) and see how they would perform if I used these with the 50 most popular Crypto assets of Januari 2022. Would they bring me fortune in 2022?, get me rekt?, or just hold my money for future trading opportunities?

Let's find out…

Backtesting commands

To backtest each strategy, I created a list of commands that had to execute. All backtest outputs are then saved in the user_data/backtest_results directory so that I can copy the necessary data to my analysis sheets.

Note that the whole 2>&1 | tee user_data/backtest_results/SmaCrossover_BtScreenOutput.txt part of this command saves the screen output to a file so that I can archive the output and also read this more easy.

<code><code><code>freqtrade backtesting -c spot_config.json --timerange=20220101-20230101 --export trades --export-filename=user_data/backtest_results/ADXMomentumHopt_export.json -s ADXMomentumHopt --timeframe=1d 2>&1 | tee user_data/backtest_results/ADXMomentumHopt_BtScreenOutput.txt
freqtrade backtesting -c spot_config.json --timerange=20220101-20230101 --export trades --export-filename=user_data/backtest_results/AwesomeMacd_export.json -s AwesomeMacd --timeframe=1d 2>&1 | tee user_data/backtest_results/AwesomeMacd_BtScreenOutput.txt
freqtrade backtesting -c spot_config.json --timerange=20220101-20230101 --export trades --export-filename=user_data/backtest_results/DemaSmaCrossoverHopt_export.json -s DemaSmaCrossoverHopt --timeframe=1d 2>&1 | tee user_data/backtest_results/DemaSmaCrossoverHopt_BtScreenOutput.txt
freqtrade backtesting -c spot_config.json --timerange=20220101-20230101 --export trades --export-filename=user_data/backtest_results/NostalgiaForInfinityX_export.json -s NostalgiaForInfinityX --timeframe=15m 2>&1 | tee user_data/backtest_results/NostalgiaForInfinityX_BtScreenOutput.txt
freqtrade backtesting -c spot_config.json --timerange=20220101-20230101 --export trades --export-filename=user_data/backtest_results/ReinforcedSmoothScalp_export.json -s ReinforcedSmoothScalp --timeframe=15m 2>&1 | tee user_data/backtest_results/ReinforcedSmoothScalp_BtScreenOutput.txt
freqtrade backtesting -c spot_config.json --timerange=20220101-20230101 --export trades --export-filename=user_data/backtest_results/Scalp_export.json -s Scalp --timeframe=1d 2>&1 | tee user_data/backtest_results/Scalp_BtScreenOutput.txt
freqtrade backtesting -c spot_config.json --timerange=20220101-20230101 --export trades --export-filename=user_data/ba in 2022cktest_results/SimpleHopt_export.json -s SimpleHopt --timeframe=1d 2>&1 | tee user_data/backtest_results/SimpleHopt_BtScreenOutput.txt
freqtrade backtesting -c spot_config.json --timerange=20220101-20230101 --export trades --export-filename=user_data/backtest_results/SmaCrossoverHopt_export.json -s SmaCrossoverHopt --timeframe=1h 2>&1 | tee user_data/backtest_results/SmaCrossoverHopt_BtScreenOutput.txt
freqtrade backtesting -c spot_config.json --timerange=20220101-20230101 --export trades --export-filename=user_data/backtest_results/SmaRsiHopt_export.json -s SmaRsiHopt --timeframe=1d 2>&1 | tee user_data/backtest_results/SmaRsiHopt_BtScreenOutput.txt
freqtrade backtesting -c spot_config.json --timerange=20220101-20230101 --export trades --export-filename=user_data/backtest_results/bb_riding_export.json -s bb_riding --timeframe=1d 2>&1 | tee user_data/backtest_results/bb_riding_BtScreenOutput.txt
freqtrade backtesting -c spot_config.json --timerange=20220101-20230101 --export trades --export-filename=user_data/backtest_results/keltnerhopt_export.json -s keltnerhopt --timeframe=1d 2>&1 | tee user_data/backtest_results/keltnerhopt_BtScreenOutput.txt
freqtrade backtesting -c spot_config.json --timerange=20220101-20230101 --export trades --export-filename=user_data/backtest_results/powerxhopt_export.json -s powerxhopt --timeframe=1d 2>&1 | tee user_data/backtest_results/powerxhopt_BtScreenOutput.txt
freqtrade backtesting -c spot_config.json --timerange=20220101-20230101 --export trades --export-filename=user_data/backtest_results/slope_is_dope2_export.json -s slope_is_dope2 --timeframe=4h 2>&1 | tee user_data/backtest_results/slope_is_dope2_BtScreenOutput.txt
freqtrade backtesting -c spot_config.json --timerange=20220101-20230101 --export trades --export-filename=user_data/backtest_results/smamacdcrossover_export.json -s smamacdcrossover --timeframe=1d 2>&1 | tee user_data/backtest_results/smamacdcrossover_BtScreenOutput.txt
freqtrade backtesting -c spot_config.json --timerange=20220101-20230101 --export trades --export-filename=user_data/backtest_results/vulcanhopt_export.json -s vulcanhopt --timeframe=4h 2>&1 | tee user_data/backtest_results/vulcanhopt_BtScreenOutput.txt</code></code></code>

If I wanted to do another individual backtest of a strategy for additional information I could always use the next command (e.g.):

<code><code><code>sudo docker-compose run --rm freqtrade backtesting --config user_data/spot_config.json --strategy SmaCrossover --timerange 20220101-20230101 --timeframe 1h</code></code></code>

All the screen output (which was logged in a file) was then carefully inspected and I copied the following information to the spreadsheet for performance scoring:

# pairs with profit / # pairs with loss
Number of wins / draws / losses
Average stake amount
Final balance
Total profit %
Absolute drawdown of account

And as you probably have seen in my video I got very surprising (and unrealistic) results back with the "Golden cross / Death cross" (SmaCrossover) strategy. I decided to dig deeper to see what's was going on here because if these backtests are wrong, then future test were probably also compromised. Could there be something wrong with the data or are there bugs in the program? That is important to know.

Backtest result analysis of golden/death cross strategy

Here are the summary metrics of the Golden/Death cross backtest:

<code><code><code>=================== SUMMARY METRICS ===================
| Metric                      | Value                 |
|-----------------------------+-----------------------|
| Backtesting from            | 2022-01-01 00:00:00   |
| Backtesting to              | 2023-01-01 00:00:00   |
| Max open trades             | 10                    |
|                             |                       |
| Total/Daily Avg Trades      | 403 / 1.1             |
| Starting balance            | 1000 USDT             |
| Final balance               | 1125335.879 USDT      |
| Absolute profit             | 1124335.879 USDT      |
| Total profit %              | 112433.59%            |
| CAGR %                      | 112433.59%            |
| Profit factor               | 2.04                  |
| Trades per day              | 1.1                   |
| Avg. daily profit %         | 308.04%               |
| Avg. stake amount           | 77563.434 USDT        |
| Total trade volume          | 31258063.751 USDT     |
|                             |                       |
| Best Pair                   | LUNA/USDT 1995879.98% |
| Worst Pair                  | XTZ/USDT -61.80%      |
| Best trade                  | LUNA/USDT 1995904.00% |
| Worst trade                 | MATIC/USDT -25.45%    |
| Best day                    | 1397070.564 USDT      |
| Worst day                   | -59000.442 USDT       |
| Days win/draw/lose          | 46 / 186 / 119        |
| Avg. Duration Winners       | 10 days, 4:47:00      |
| Avg. Duration Loser         | 3 days, 15:37:00      |
| Rejected Entry signals      | 125669                |
| Entry/Exit Timeouts         | 0 / 0                 |
|                             |                       |
| Min balance                 | 670.292 USDT          |
| Max balance                 | 1461301.049 USDT      |
| Max % of account underwater | 32.30%                |
| Absolute Drawdown (Account) | 23.95%                |
| Absolute Drawdown           | 349994.853 USDT       |
| Drawdown high               | 1460301.049 USDT      |
| Drawdown low                | 1110306.196 USDT      |
| Drawdown Start              | 2022-08-17 18:00:00   |
| Drawdown End                | 2023-01-01 00:00:00   |
| Market change               | -78.87%               |
=======================================================</code></code></code>

And everybody knows that these results are not realistic at all. So I had to find out what this caused.

So I looked at the individual pairs and found the performance of LUNA:

This pair is now my main suspect for further analysis.

Creating 'Pickled' backtest information

A somewhat newer feature to analyse backtest results further is the ability to create a 'pickled' dictionary of candles and signals. This dictionary holds additional information for further analysis in a terminal window and can provide more insights in a strategies performance. You can read more about this functionality on the Freqtrade site.

To create such a 'pickled' file, I used the following option in the backtesting command: --export=signals.

<code><code><code>freqtrade backtesting -c spot_config.json --timerange=20220101-20230101 --export trades --export-filename=user_data/backtest_results/SmaCrossover_export.json -s SmaCrossover --timeframe=1h --export=signals 2>&1 | tee user_data/backtest_results/SmaCrossover_BtScreenOutput.txt</code></code></code>

This output is also saved in the 'backtest_results' directory (note that other results should be removed according to the documentation) and can be used for deeper analysis.

Pickled output

To show more detailed output of the backtest results to the screen I used the following command:

<code><code><code>freqtrade backtesting-analysis -c spot_config.json</code></code></code>

And I got a small issue here, my output looked something like this (note the 'nan' reason):

<code><code><code>+------------+----------------+---------------+
| pair       |   enter_reason | exit_reason   |
|------------+----------------+---------------|
| BTC/USDT   |            nan | exit_signal   |
| XMR/USDT   |            nan | exit_signal   |
<snip>
| LTC/USDT   |            nan | force_exit    |
| ATOM/USDT |            nan | force_exit    |
| LUNA/USDT |            nan | roi           |
+------------+----------------+---------------+</code></code></code>

If you get this output as well, then your strategy file probably does not have the "enter_tag" and "exit_tag" configured in the "populate_buy_trend" and "populate_sell_trend" functions. This is necessary to get the correct output to the file/screen.

This is not clearly written on the Freqtrade site and it took me a while to figure this out. Also because almost all of the available strategies do not really use these enter/exit tags as well.

Anyway, I solved this by changing the code of the buy and sell functions of the 'SmaCrossover.py' file to the following:

<code><code><code>def populate_buy_trend(self, dataframe: DataFrame, metadata: dict) -> DataFrame:
      dataframe.loc[
          (
              (qtpylib.crossed_above(dataframe['quick_sma'], dataframe['slow_sma']))
          ),
          # Added an enter tag to this code to check if this is visible in advanced backtesting
          # See: https://www.freqtrade.io/en/stable/strategy-advanced/#enter-tag
          ['enter_long', 'enter_tag']] = (1, 'quick_sma above slow_sma')
      return dataframe

   def populate_sell_trend(self, dataframe: DataFrame, metadata: dict) -> DataFrame:
      dataframe.loc[
          (
              # Buy when SMA 50 crosses above SMA 200
              (qtpylib.crossed_below(dataframe['quick_sma'], dataframe['slow_sma']))
          ),
          ['exit_long', 'exit_tag']] = (1, 'quick_sma bwlow slow_sma')
      return dataframe</code></code></code>

And after recreating the pickled output (remove old files first) I used the following command to get me the complete backtest-analysis output:

<code><code><code>freqtrade backtesting-analysis -c spot_config.json --analysis-groups 4 --enter-reason-list all --exit-reason-list all --indicator-list quick_sma slow_sma | tee user_data/backtest_results/SmaCrossover_BtAnalysisOutput.txt</code></code></code>

Now the output looks like this:

<code><code><code>2023-01-10 10:11:38,414 - freqtrade.data.entryexitanalysis - INFO - Loaded signal candles: /opt/freqtrade/user_data/backtest_results/SmaCrossover_export-2023-01-10_10-00-31_signals.pkl
2023-01-10 10:11:38,414 - freqtrade.data.entryexitanalysis - INFO - Processing SmaCrossover : 50 pairs
+------------+--------------------------+--------------------------+------------+------------------+---------------------+-------------------+---------------------+-------------------+--------------------+
| pair       | enter_reason             | exit_reason              |   num_buys |   profit_abs_sum |   profit_abs_median |   profit_abs_mean |   median_profit_pct |   mean_profit_pct |   total_profit_pct |
|------------+--------------------------+--------------------------+------------+------------------+---------------------+-------------------+---------------------+-------------------+--------------------|
| MATIC/USDT | quick_sma above slow_sma | quick_sma bwlow slow_sma |         12 | -48433.2         |      -739.644       |   -4036.1         |        -56.9083     |       -3.01155    |        -4.74236    |
| XEC/USDT   | quick_sma above slow_sma | quick_sma bwlow slow_sma |          6 | -48356.2         |     -8723.39        |   -8059.37        |         -4.69772    |       -6.89663    |        -0.782953   |
| BCH/USDT   | quick_sma above slow_sma | quick_sma bwlow slow_sma |         12 | -47413.4         |     -1853.87        |   -3951.12        |        -42.0051     |       -4.02457    |        -3.50042    |
| XTZ/USDT   | quick_sma above slow_sma | quick_sma bwlow slow_sma |          8 | -44147.5         |     -2650.41        |   -5518.44        |        -61.8024     |       -7.60045    |        -7.7253     |
| LRC/USDT   | quick_sma above slow_sma | quick_sma bwlow slow_sma |          9 | -34397.2         |     -2157.73        |   -3821.91        |        -48.052      |       -5.01641    |        -5.33911    |
| LUNA/USDT | quick_sma above slow_sma | quick_sma bwlow slow_sma |         16 | -32311.7         |       -10.6119      |   -2019.48        |        -24.0136     |       -3.85622    |        -1.50085    |

<snip>

| UNI/USDT   | quick_sma above slow_sma | quick_sma bwlow slow_sma |          8 | 24424.9         |      2683.32        |    3053.11        |         12.769      |        2.12235    |         1.59612    |
| KLAY/USDT | quick_sma above slow_sma | quick_sma bwlow slow_sma |          9 | 40312.5         |      -959.379       |    4479.17        |         18.1445     |       -3.6662     |         2.01605    |
| QNT/USDT   | quick_sma above slow_sma | quick_sma bwlow slow_sma |         10 | 72638.8         |     -2128.71        |    7263.88        |         52.5445     |       -3.67932    |         5.25445    |
| DOGE/USDT | quick_sma above slow_sma | quick_sma bwlow slow_sma |         10 | 127422           |        -2.46317     |   12742.2         |         82.5235     |       -1.57583    |         8.25235    |
| LUNA/USDT | quick_sma above slow_sma | roi                      |          1 |      1.39707e+06 |         1.39707e+06 |       1.39707e+06 |          1.9959e+06 |        1.9959e+06 |         1.9959e+06 |
+------------+--------------------------+--------------------------+------------+------------------+---------------------+-------------------+---------------------+-------------------+--------------------+
```

```
+------------+--------------------------+--------------------------+-----------------+-----------------+
| pair       | enter_reason             | exit_reason              |       quick_sma |        slow_sma |
|------------+--------------------------+--------------------------+-----------------+-----------------|
| ATOM/USDT | quick_sma above slow_sma | force_exit               |     8.9171      |     8.91603     |
| BNB/USDT   | quick_sma above slow_sma | force_exit               |   244.784       |   244.77        |
| ICP/USDT   | quick_sma above slow_sma | force_exit               |     3.7028      |     3.70117     |
| LTC/USDT   | quick_sma above slow_sma | force_exit               |    67.4334      |    67.4263      |
| XMR/USDT   | quick_sma above slow_sma | quick_sma bwlow slow_sma |   147.226       |   147.161       |
| XMR/USDT   | quick_sma above slow_sma | quick_sma bwlow slow_sma |   131.864       |   131.825       |

<snip>

| LTC/USDT   | quick_sma above slow_sma | quick_sma bwlow slow_sma |   100.374       |   100.36        |
| LTC/USDT   | quick_sma above slow_sma | quick_sma bwlow slow_sma |   109.568       |   109.554       |
| LINK/USDT | quick_sma above slow_sma | quick_sma bwlow slow_sma |     7.0308      |     7.02815     |
| AMP/USDT   | quick_sma above slow_sma | quick_sma bwlow slow_sma |     0.0034788   |     0.00347845 |
| LUNA/USDT | quick_sma above slow_sma | roi                      |     5e-05       |     5e-05       |
+------------+--------------------------+--------------------------+-----------------+-----------------+</code></code></code>

So as you can see, much better output here.

LUNA analyis

To check what LUNA has done in this list, I had to filter out this pair with the 'grep' command. Which resulted in the following output:

<code><code><code>cat SmaCrossover_BtAnalysisOutput.txt | grep LUNA</code></code></code>

Here I see the enter and exit reasons of the pair. Apparently the sma crossover exit signal adds to the losses, but there was one HUGE trade that exited on the ROI which had a stupendous profit. Remember here that the ROI is set to: minimal_roi = {"0": 100.00} in this strategy…

<code><code><code>+------------+--------------------------+--------------------------+------------+------------------+---------------------+-------------------+---------------------+-------------------+--------------------+
| pair       | enter_reason             | exit_reason              |   num_buys |   profit_abs_sum |   profit_abs_median |   profit_abs_mean |   median_profit_pct |   mean_profit_pct |   total_profit_pct |
|------------+--------------------------+--------------------------+------------+------------------+---------------------+-------------------+---------------------+-------------------+--------------------|
| LUNA/USDT | quick_sma above slow_sma | quick_sma bwlow slow_sma |         16 | -32311.7         |       -10.6119      |   -2019.48        |        -24.0136     |       -3.85622    |        -1.50085    |
| LUNA/USDT | quick_sma above slow_sma | roi                      |          1 |      1.39707e+06 |         1.39707e+06 |       1.39707e+06 |          1.9959e+06 |        1.9959e+06 |         1.9959e+06 |</code></code></code>

Then there are the indivual trades here (unfortunately no dates of these trades (yet?). All the trades with exit reason 'quick sma below slow sma' sum up to be a losing strategy (maybe some trades are winners, but I cannot see that here…)

Again, the one trade with an exit reason of 'ROI' is the winner here. But look at the quic and slow sma values here… They are super low. This has to be further examined in the charts to get a visual perspective of these occurences.

<code><code><code>+------------+--------------------------+--------------------------+-----------------+-----------------+
| pair       | enter_reason             | exit_reason              |       quick_sma |        slow_sma |
|------------+--------------------------+--------------------------+-----------------+-----------------|
| LUNA/USDT | quick_sma above slow_sma | quick_sma bwlow slow_sma |     1.27943     |     1.2788      |
| LUNA/USDT | quick_sma above slow_sma | quick_sma bwlow slow_sma |     1.62909     |     1.62805     |
| LUNA/USDT | quick_sma above slow_sma | quick_sma bwlow slow_sma |     1.6384      |     1.63738     |
| LUNA/USDT | quick_sma above slow_sma | quick_sma bwlow slow_sma |     2.5186      |     2.51778     |
| LUNA/USDT | quick_sma above slow_sma | quick_sma bwlow slow_sma |     2.50858     |     2.50851     |
| LUNA/USDT | quick_sma above slow_sma | quick_sma bwlow slow_sma |     2.48141     |     2.48119     |
| LUNA/USDT | quick_sma above slow_sma | quick_sma bwlow slow_sma |     2.57648     |     2.57387     |
| LUNA/USDT | quick_sma above slow_sma | quick_sma bwlow slow_sma |     1.7404      |     1.73775     |
| LUNA/USDT | quick_sma above slow_sma | quick_sma bwlow slow_sma |     1.8829      |     1.88273     |
| LUNA/USDT | quick_sma above slow_sma | quick_sma bwlow slow_sma |    91.1736      |    91.1341      |
| LUNA/USDT | quick_sma above slow_sma | quick_sma bwlow slow_sma |    83.6868      |    83.6338      |
| LUNA/USDT | quick_sma above slow_sma | quick_sma bwlow slow_sma |    89.891       |    89.8686      |
| LUNA/USDT | quick_sma above slow_sma | quick_sma bwlow slow_sma |    87.7184      |    87.6162      |
| LUNA/USDT | quick_sma above slow_sma | quick_sma bwlow slow_sma |    52.863       |    52.7847      |
| LUNA/USDT | quick_sma above slow_sma | quick_sma bwlow slow_sma |    54.5446      |    54.5403      |
| LUNA/USDT | quick_sma above slow_sma | quick_sma bwlow slow_sma |    74.7008      |    74.4984      |
| LUNA/USDT | quick_sma above slow_sma | roi                      |     5e-05       |     5e-05       |
(.env)</code></code></code>

Plot analysis

After looking at the plot of LUNA, the big winning trade here started on May 21, 2022 and ended on May 31, 2022. Between these trades was a profit of 1995904% percent…

But after another look I think there is something wrong with this trade because trading LUNA was not possible on the Binance exchange, as far as I can remember. And besides, checking this on Trading view I cannot reproduce the same chart on Binance because there is a data gap there. Looking at charts on other exchanges like Bitfinex shows that prices stayed down.

My suspicion here is that, because Freqtrade adds its own input to missing data between points, the strategy 'goes berserk'. This is because the calculations are still made and the (faulty) results of these calculations can cause buy or sell signals. Especially because at the end of May Binance resumed trading again on the pair and it had a small spike in price here (possibly because of this event).

So because of these events, and the way Freqtrade handles missing data, I think that I should neglect this trade.

However I want to verify my findings and check the next highly profitable pair and that is DOGE/USDT.

DOGE/USDT Analysis

Doge eventually had 10 trades in 2022 and all together they were pretty profitable. Also all the exits came from the quick sma that went below the slow sma.

<code><code><code>$ cat SmaCrossover_BtAnalysisOutput.txt | grep DOGE
+------------+--------------------------+--------------------------+------------+------------------+---------------------+-------------------+---------------------+-------------------+--------------------+
| pair | enter_reason | exit_reason | num_buys | profit_abs_sum | profit_abs_median | profit_abs_mean | median_profit_pct | mean_profit_pct | total_profit_pct |
|------------+--------------------------+--------------------------+------------+------------------+---------------------+-------------------+---------------------+-------------------+--------------------|
| DOGE/USDT | quick_sma above slow_sma | quick_sma bwlow slow_sma | 10 | 127422 | -2.46317 | 12742.2 | 82.5235 | -1.57583 | 8.25235 |</code></code></code>

And here the table with the individual trades (no dates though…)

So let's check the plot to see where these trades occured and where the big winner(s) are:

In the chart of Doge there are only two major wins and 8 losing trades. The two winning trade had a 6.46% win (Jun 20, 2022) and a 110% win (Nov 6, 22).

And again, curcumstances that are not quite logical are in force here. In october there likely was a short squeeze to liquidate short positions so that they had to buy Doge to cover their positions. See: https://www.coindesk.com/markets/2022/10/31/dogecoins-102-spike-steals-the-spotlight-as-october-ends/

But all in all this chart looks quite 'normal' behaviour.

So, let's take a third and final look at another remarkable winner in 2022, QNT/USDT.

QNT/USDT backtest analysis

The total performance of QNT looks like this. 10 trades with a mean profit of 7263 USD. And a median of only 52 USD. My first guess is that one or two pumps are also responsible for this performance…

<code><code><code>$ cat SmaCrossover_BtAnalysisOutput.txt | grep QNT
+------------+--------------------------+--------------------------+------------+------------------+---------------------+-------------------+---------------------+-------------------+--------------------+
| pair | enter_reason | exit_reason | num_buys | profit_abs_sum | profit_abs_median | profit_abs_mean | median_profit_pct | mean_profit_pct | total_profit_pct |
|------------+--------------------------+--------------------------+------------+------------------+---------------------+-------------------+---------------------+-------------------+--------------------|
| QNT/USDT | quick_sma above slow_sma | quick_sma bwlow slow_sma | 10 | 72638.8 | -2128.71 | 7263.88 | 52.5445 | -3.67932 | 5.25445 |</code></code></code>

Ok, nothing to see here…

Let's also take a look at the chart in this case to see what was going on here.

If I filter out all the noise and only show the trade starts again, I can check if there are 10 trades on the chart and where they occured.

The 10 blue circles indicate the trades, and this time I can see that the general trend is upward.

There are three profitable trade exits. Around 7% (Jun 29, 22), one with 73% Jul 22, 22) and one with 11% (Oct 20, 22).

And ofcourse 7 losing trades as well.

Funny enough there seem to be some good opportunities at the beginning of the year also. But I guess that all trading slots were full at that time so there were no trades.

Conclusion

So in the end, what should I conclude?

The first thing that comes up in my mind is the combination of the black swan LUNA event, how the exchange reacted to it (stop trading and then start trading again) and how Freqtrade bot handles these missing candles (filling the gaps). These circumstances can disturb the analysis of a strategy pretty good. Backtest results can be done but have to be looked at very critical. Behaviour like this can destroy a strategies average performance or make it unrealisticaly good, like this case.

All I can do is to make another backtest without LUNA and see how this performs.

Therefore I have to adjust my config files and remove LUNA from the whitelist. Just to see how this Golden/Death cross behaves without LUNA.

<code><code><code>freqtrade backtesting -c spot_config.json --timerange=20220101-20230101 --export trades --export-filename=user_data/backtest_results/SmaCrossover_export_NOLUNA.json -s
SmaCrossover --timeframe=1h --export=signals 2>&1 | tee user_data/backtest_results/SmaCrossover_BtScreenOutput_NOLUNA.txt</code></code></code>

The test results in the following output:

<code><code><code>============================================================= BACKTESTING REPORT =============================================================
|       Pair |   Entries |   Avg Profit % |   Cum Profit % |   Tot Profit USDT |   Tot Profit % |     Avg Duration |   Win Draw Loss Win% |
|------------+-----------+----------------+----------------+-------------------+----------------+------------------+-------------------------|
| DOGE/USDT |        10 |           8.25 |          82.52 |            33.575 |           3.36 | 4 days, 13:54:00 |     2     0     8 20.0 |
|   QNT/USDT |        11 |           4.68 |          51.43 |            25.581 |           2.56 | 6 days, 2:38:00 |     3     0     8 27.3 |
| MANA/USDT |         4 |           5.49 |          21.95 |            20.278 |           2.03 | 7 days, 20:15:00 |     2     0     2 50.0 |
|   XRP/USDT |        11 |           1.96 |          21.52 |            16.855 |           1.69 | 6 days, 2:33:00 |     3     0     8 27.3 |
|   XMR/USDT |        18 |           0.93 |          16.68 |            14.745 |           1.47 | 7 days, 21:10:00 |     8     0    10 44.4 |
|   TRX/USDT |        16 |           1.08 |          17.31 |            13.731 |           1.37 | 6 days, 15:04:00 |     6     0    10 37.5 |
|   AXS/USDT |         9 |           1.07 |           9.67 |            11.804 |           1.18 | 3 days, 23:40:00 |     1     0     8 11.1 |

<snip>

|   BTC/USDT |        16 |          -2.75 |         -44.07 |           -31.428 |          -3.14 | 3 days, 3:00:00 |     2     0    14 12.5 |
|   LRC/USDT |         9 |          -5.34 |         -48.05 |           -32.091 |          -3.21 | 3 days, 12:13:00 |     1     0     8 11.1 |
|   AMP/USDT |         9 |          -5.21 |         -46.93 |           -32.545 |          -3.25 | 3 days, 11:00:00 |     0     0     9     0 |
|   BCH/USDT |        14 |          -3.70 |         -51.78 |           -34.307 |          -3.43 | 4 days, 13:30:00 |     4     0    10 28.6 |
| MATIC/USDT |        12 |          -4.74 |         -56.91 |           -39.873 |          -3.99 | 4 days, 14:05:00 |     3     0     9 25.0 |
|   XTZ/USDT |         8 |          -7.73 |         -61.80 |           -40.099 |          -4.01 | 3 days, 12:00:00 |     0     0     8     0 |
| THETA/USDT |        10 |          -6.11 |         -61.11 |           -51.356 |          -5.14 | 5 days, 9:42:00 |     2     0     8 20.0 |
|      TOTAL |       396 |          -1.57 |        -620.18 |          -493.048 |         -49.30 | 5 days, 9:05:00 |   100     0   296 25.3 |
===================================================================== ENTER TAG STATS =====================================================================
|                      TAG |   Entries |   Avg Profit % |   Cum Profit % |   Tot Profit USDT |   Tot Profit % |    Avg Duration |   Win Draw Loss Win% |
|--------------------------+-----------+----------------+----------------+-------------------+----------------+-----------------+-------------------------|
| quick_sma above slow_sma |       396 |          -1.57 |        -620.18 |          -493.048 |         -49.30 | 5 days, 9:05:00 |   100     0   296 25.3 |
|                    TOTAL |       396 |          -1.57 |        -620.18 |          -493.048 |         -49.30 | 5 days, 9:05:00 |   100     0   296 25.3 |
========================================================== EXIT REASON STATS ===========================================================
|              Exit Reason |   Exits |   Win Draws Loss Win% |   Avg Profit % |   Cum Profit % |   Tot Profit USDT |   Tot Profit % |
|--------------------------+---------+--------------------------+----------------+----------------+-------------------+----------------|
| quick_sma bwlow slow_sma |     392 |     98     0   294 25.0 |          -1.61 |        -631.64 |          -498.939 |         -63.16 |
|               force_exit |       4 |      2     0     2 50.0 |           2.86 |          11.45 |             5.891 |           1.15 |
========================================================== LEFT OPEN TRADES REPORT ==========================================================
|      Pair |   Entries |   Avg Profit % |   Cum Profit % |   Tot Profit USDT |   Tot Profit % |     Avg Duration |   Win Draw Loss Win% |
|-----------+-----------+----------------+----------------+-------------------+----------------+------------------+-------------------------|
| ICP/USDT |         1 |           7.42 |           7.42 |             3.824 |           0.38 | 8 days, 2:00:00 |     1     0     0   100 |
| ATOM/USDT |         1 |           4.82 |           4.82 |             2.458 |           0.25 | 7 days, 22:00:00 |     1     0     0   100 |
| BNB/USDT |         1 |          -0.08 |          -0.08 |            -0.039 |          -0.00 | 2 days, 4:00:00 |     0     0     1     0 |
| LTC/USDT |         1 |          -0.71 |          -0.71 |            -0.352 |          -0.04 |          9:00:00 |     0     0     1     0 |
|     TOTAL |         4 |           2.86 |          11.45 |             5.891 |           0.59 | 4 days, 15:15:00 |     2     0     2 50.0 |
================== SUMMARY METRICS ==================
| Metric                      | Value               |
|-----------------------------+---------------------|
| Backtesting from            | 2022-01-01 00:00:00 |
| Backtesting to              | 2023-01-01 00:00:00 |
| Max open trades             | 10                  |
|                             |                     |
| Total/Daily Avg Trades      | 396 / 1.08          |
| Starting balance            | 1000 USDT           |
| Final balance               | 506.952 USDT        |
| Absolute profit             | -493.048 USDT       |
| Total profit %              | -49.30%             |
| CAGR %                      | -49.30%             |
| Profit factor               | 0.56                |
| Trades per day              | 1.08                |
| Avg. daily profit %         | -0.14%              |
| Avg. stake amount           | 65.175 USDT         |
| Total trade volume          | 25809.185 USDT      |
|                             |                     |
| Best Pair                   | DOGE/USDT 82.52%    |
| Worst Pair                  | XTZ/USDT -61.80%    |
| Best trade                  | DOGE/USDT 110.23%   |
| Worst trade                 | MATIC/USDT -25.45% |
| Best day                    | 56.523 USDT         |
| Worst day                   | -38.801 USDT        |
| Days win/draw/lose          | 47 / 188 / 116      |
| Avg. Duration Winners       | 10 days, 10:16:00   |
| Avg. Duration Loser         | 3 days, 16:09:00    |
| Rejected Entry signals      | 120816              |
| Entry/Exit Timeouts         | 0 / 0               |
|                             |                     |
| Min balance                 | 500.41 USDT         |
| Max balance                 | 990.02 USDT         |
| Max % of account underwater | 49.45%              |
| Absolute Drawdown (Account) | 49.45%              |
| Absolute Drawdown           | 489.61 USDT         |
| Drawdown high               | -9.98 USDT          |
| Drawdown low                | -499.59 USDT        |
| Drawdown Start              | 2022-01-16 06:00:00 |
| Drawdown End                | 2022-12-29 02:00:00 |
| Market change               | -78.46%             |
=====================================================</code></code></code>

Now, these numbers look much more realistic. The overall winrate stays roughly the same, as is the list of most succesfull pairs.

You might wonder why DOGE, QNT and other pairs also have sudden drops in the profitabilty of their trades (the amount that was gained).

I think this is because in the faulty backtest the LUNA 'trade' added such a big amount of money to the trading account, that this was also used for the consequtive trades.

This is shown by the average trade amount that dropped from 77563.434 USDT to only 65.175 USDT. Which looks much more realistic to me, given the fact that this strategy only has a winrate of around 25%.

So, what to do?

I want to stay realistic and events like LUNA can still occur in the future. When it will happen and with which pair is a question nobody can answer.

One side of the coin says that you cannot change history and LUNA is still available to trade on the Binance exchange. Also Before the collapse LUNA was a tradable coin where people profited from. I can consider this part of the risks of trading Crypto and things you have to take into consideration.

It would also be unfair to the strategies that I will test in the future and have the possibility to go short to deny these circumstances.

On the other hand, it is arbitrary which coins to include in the strategy analysis and which not. Profits only tell you so much and if I concentrated only on that result, I could better add a bunch of low cap sh*tcoins to the list which sometimes have insane pumps. That would be cherry picking just to make attractive posts and video's. It is better to use pairs that have reasonable 'normal' behavious (what's normal in crypto?) and not use these outliers in the analysis so that I can concentrate better on the probability that a strategy is profitable in the future. Also the behaviour of the bot and the exchange with the missing data helped me with this decision.

A new pair

So what I will do is remove the LUNA/USDT pair and add the next Coinmarketcap Top50 coin that is traded on Binance to the whitelist.

This way I prevent the constant checks and balances in future strategy analysis (I do not have to check the LUNA influence on the results all the time), which makes the final conclusions more reliable and trustwothy.

Therefore I end this post with the new *_config.json whitelist for strategy analysis:

<code><code><code>"pair_whitelist": [
       "BTC/USDT","ETH/USDT","BNB/USDT","SOL/USDT","ADA/USDT","XRP/USDT",
       "DOT/USDT","AVAX/USDT","DOGE/USDT","SHIB/USDT","MATIC/USDT","UNI/USDT",
       "ALGO/USDT","LTC/USDT","LINK/USDT","NEAR/USDT","BCH/USDT","ATOM/USDT","TRX/USDT",
       "XLM/USDT","FTM/USDT","MANA/USDT","AXS/USDT","HBAR/USDT","VET/USDT","SAND/USDT",
       "ICP/USDT","FIL/USDT","THETA/USDT","ETC/USDT","XMR/USDT","XTZ/USDT","AAVE/USDT",
       "KLAY/USDT","ONE/USDT","GRT/USDT","GALA/USDT","EOS/USDT","CAKE/USDT","STX/USDT",
       "FLOW/USDT","LRC/USDT","CRV/USDT","KSM/USDT","MKR/USDT","ENJ/USDT","QNT/USDT",
       "XEC/USDT","AMP/USDT",
       "ZEC/USDT"
       // "LUNA/USDT"
       ],</code></code></code>

With this new list I took all strategies and backtested these again. This time the results looked much more realistic for the 2022 market circumstances.

You can see all the results I got in my Youtube video

I hope you learned something from this article and see you in the next post!