The reinforced average strategy (Patreon)

Introduction

Hello, and welcome to another blog post where I will discuss the profitability of yet another algorithmic trading strategy. In this post I will continue with the strategies that are provided by the Freqtrade developers. I will continue to do this until they all are tested or analyzed on their usefullness.

The strategy

In this post I will go over the results of the tests I did on the ‘reinforced average strategy’. And as said earlier, this strategy is located in the github repository of the Freqtrade developers. You can find this strategy by clicking on this link.

This strategy is created by Gert Wohlgemuth. So he gets all the credits of course.

This time he did his best to create an algo strategy that is a combination of the EMA and Bollinger bands indicators at first sight. Based on the name of this file, it is probably an improvement or variation of the average strategy that was tested earlier.

Let’s just run over the code to see what he build.

and by the way he indicates in the comments that this strategy does not perform well, but we will see about that…

Default parameters

Let’s begin with the default parameters each strategy for the Freqtrade bot has.

ROI

To begin with the ROI setting. The strategy will sell when there is a profit reached of 50%. No matter how much time it took.

# Minimal ROI designed for the strategy.

# This attribute will be overridden if the config file contains "minimal_roi"

minimal_roi = {

"0": 0.5

}

Stop loss

The stop loss is set to sell the position if there is a loss reached of 20%.

# Optimal stoploss designed for the strategy

# This attribute will be overridden if the config file contains "stoploss"

stoploss = -0.2

Time frame

This strategy was originally build for the 4 hour timeframe. I will test this on multiple timeframes so we will see if it performs better of worse on other timeframes.

# Optimal timeframe for the strategy

timeframe = '4h'

Trailing stop loss

There is no trailing stop loss set. So moving on, nothing to see here…

# trailing stoploss

trailing_stop = False

trailing_stop_positive = 0.01

trailing_stop_positive_offset = 0.02

trailing_only_offset_is_reached = False

Using exit signal

And this strategy also will use the exit signal also. This will be set a little bit further in the code.

# run "populate_indicators" only for new candle

process_only_new_candles = False

# Experimental settings (configuration will overide these if set)

use_exit_signal = True

exit_profit_only = False

ignore_roi_if_entry_signal = False

Indicators configured

In the populate indicators function, the EMA and Bollinger bands are configured to be used in this strategy.

There are two EMA’s configured, the EMA 8 and EMA 21. And this is the same setting in the previously tested ‘Average’ strategy.

def populate_indicators(self, dataframe: DataFrame, metadata: dict) -> DataFrame:

dataframe['maShort'] = ta.EMA(dataframe, timeperiod=8)

dataframe['maMedium'] = ta.EMA(dataframe, timeperiod=21)

Then the bollinger bands from the qtpylib library is used with a lenght of 20 and a standard deviation of 2. Which are pretty much the default settings for this indicator.

# required for graphing

bollinger = qtpylib.bollinger_bands(dataframe['close'], window=20, stds=2)

dataframe['bb_lowerband'] = bollinger['lower']

dataframe['bb_upperband'] = bollinger['upper']

dataframe['bb_middleband'] = bollinger['mid']

Then there is a small section which I think is the ‘reinforced’ part of this strategy. In this section the following things occur:

1. It calculates the a new functon variable resample_interval by calling the timeframe_to_minutes() function to convert a given timeframe to minutes and then multiplying it by 12.
   So for the 4 hour timeframe this strategy was built for, the resample interval gets to be (4 * 60) * 12 equals 2880.
2. It resamples the dataframe to the resample_interval by calling the resample_to_interval() function. This will then be a new dataframe called dataframe_long.
3. It computes the simple moving average (SMA) of the resampled dataframe’s close price over the last 50 periods using the SMA() function from the ta (technical analysis) library.
4. It adds a new column ‘sma’ to the dataframe_long (the resampled dataframe) and assigns the calculated SMA values to this new column.
5. And finally it merges the original dataframe with the resampled dataframe_long by calling the resampled_merge() function and fills any missing values in the merged dataframe with the most recent non-missing value using the fill_na argument.

self.resample_interval = timeframe_to_minutes(self.timeframe) * 12

dataframe_long = resample_to_interval(dataframe, self.resample_interval)

dataframe_long['sma'] = ta.SMA(dataframe_long, timeperiod=50, price='close')

dataframe = resampled_merge(dataframe, dataframe_long, fill_na=True)

return dataframe

The dataframe will look like this for each pair that is calculated:

{'pair': 'BTC/USDT'}

resample interval: 2880

date     close       maShort      maMedium  resample_2880_close  resample_2880_sma

11728 2022-12-23 20:00:00+00:00  16778.50  16811.550174  16809.723159             16821.43         18421.3504

11729 2022-12-24 00:00:00+00:00  16840.65  16818.016802  16812.534690             16821.43         18421.3504

11730 2022-12-24 04:00:00+00:00  16848.26  16824.737513  16815.782446             16821.43         18421.3504

11731 2022-12-24 08:00:00+00:00  16824.41  16824.664732  16816.566769             16821.43         18421.3504

11732 2022-12-24 12:00:00+00:00  16836.35  16827.261458  16818.365244             16821.43         18421.3504

11733 2022-12-24 16:00:00+00:00  16843.59  16830.890023  16820.658404             16821.43         18421.3504

11734 2022-12-24 20:00:00+00:00  16836.12  16832.052240  16822.064004             16836.12         18364.0352

11735 2022-12-25 00:00:00+00:00  16832.14  16832.071742  16822.980003             16836.12         18364.0352

11736 2022-12-25 04:00:00+00:00  16821.97  16829.826911  16822.888185             16836.12         18364.0352

11737 2022-12-25 08:00:00+00:00  16821.51  16827.978708  16822.762895             16836.12         18364.0352

11738 2022-12-25 12:00:00+00:00  16793.09  16820.225662  16820.065359             16836.12         18364.0352

11739 2022-12-25 16:00:00+00:00  16794.07  16814.413293  16817.702145             16836.12         18364.0352

11740 2022-12-25 20:00:00+00:00  16832.11  16818.345894  16819.011950             16836.12         18364.0352

11741 2022-12-26 00:00:00+00:00  16894.19  16835.200140  16825.846318             16836.12         18364.0352

11742 2022-12-26 04:00:00+00:00  16843.65  16837.077887  16827.464835             16836.12         18364.0352

11743 2022-12-26 08:00:00+00:00  16864.04  16843.069467  16830.789850             16836.12         18364.0352

11744 2022-12-26 12:00:00+00:00  16809.51  16835.611808  16828.855318             16836.12         18364.0352

Here the BTC/USDT resample close on 2022-12-25 00:00:00+00:00 is 16836.12.
Which is the close price of the last 4hr candle on 2022-12-24 20:00:00+00:00.

Then this close price will be used as the resampled close over all the candles on that day (25 december).

So in other words, the close price of the previous day timeframe is used as a reference on the candles of the lower timeframe that is used (here the 4 hr timeframe).

And do not forget that the 50 SMA is calculated over that resampled close price!

Testing on other timeframes

The caviat when backtesting is that, if I use this exact strategy on lower timeframes, this resample calculation on the daily timeframe close will be broken. Or at least use the resampled close price of the timeframe that is 12 times larger.

For example, if I wanted to test this strategy on the hourly timeframe without altering the code, then the resampled close price would be that of the 12 hour timeframe (60 minutes (1 hr tf) * 12 = 720 minutes).

And If I did this on the 30 min timeframe it would use the close on the 6 hour timeframe. Etcetera…

Buy and sell signals

Now how will this resampled and merged dataframe be used to create buy and sell signals?

Buy signals

This strategy will buy when the following events happen:

The exact moment the short EMA8 crosses above the medium EMA21 AND
when the close price of the current (1d, 4 hr, 1hr, 30m etc) candle is above the resampled SMA.

So in the case of the 4 hour strategy: the close price of the 4 hour candle should be above the 2 day SMA AND the EMA crossover (on the 4 hour) should occur to get a valid buy signal.

dataframe.loc[

(

qtpylib.crossed_above(dataframe['maShort'], dataframe['maMedium']) &

(dataframe['close'] > dataframe[f'resample_{self.resample_interval}_sma']) &

(dataframe['volume'] > 0)

),

'enter_long'] = 1

Sell signals

The sell signal is triggered when the medium EMA21 crosses above the short EMA8 of the tested timeframe. Or in other words, the short EMA should cross below the medium EMA. And volume should be higher than 0 to make this a valid sell signal.

dataframe.loc[

(

qtpylib.crossed_above(dataframe['maMedium'], dataframe['maShort']) &

(dataframe['volume'] > 0)

),

'exit_long'] = 1

And notice that the Bollinger bands are not used for buy and sell signals.

How it looks on a chart

To try to make this visual on a chart I’ll open Tradingview and add the indicators on the chart.

I will use a snippet of the backtest output of Bitcoin here:

And the purple line is 25 december 00:00 on the 4 hour chart below. I already have the EMA8 and EMA21 on the chart from my previous video.

Then I will add the SMA50 indicator as well and make it look like in the dataframe. The only thing here is that this indicator in Tradingview cannot show the 2 day SMA. So instead I will use the 1 day SMA as an example.

Unfourtunately no 2 day timeframe here, but you get the point I hope

Then the chart should look something like this:

Also another remark. The SMA50 on the 1 Day timeframe on this chart looks pretty smooth here.
But if I would use the dataframe data, and would plot it on a chart. It would look more like stairsteps. This is because the resampled SMA value is the same for all the 4 hour in each day. So they would produce small horizontal lines then.

Initial backtest results

The initial backtests show that the 4 hour timeframe indeed has the best chance to gain profits and keep them as well. The 1 hour seems to be much more profitable with an backtest run that has a profit percentage of 2939%.

The 30 minute, 15 minute and 5 minute timeframe did not show any results unfortunately (no output). I have seem this behavior more often with this updated dataset. I think that it is simply too much for my computer to calculate with and I think that in the future I will try to use smaller datasets for these time periods to see if they work better.

Initial backtest results No output on the 30 minute timeframe…

Now back to the results.
The 4 hour and 1 hour timeframe score the same with my scoring method so I will use the Calmar ratio, profit pairs ratio and winrate here to determine which timeframe has the best gains and the least chance to give everything back to the market. And in this case it is the 4 hour timeframe.

And that is a good thing because the strategy was also originally intended for this timeframe.

Hyperopt

Now I have also done a Hyperopt session to find out if I could squeeze more out of this strategy by letting the optimization tweak the ROI and stoploss settings.

But this time unfortunately the optimization session gave overall results that I am not happy with. Indeed the Sortino ratio was optimized. But at a cost that is dissapointing to say it mildly.

Hyperopt session logged

Over the complete backtesting period there was only an 85% profit which summed up to a CAGR of only 12%. That’s more that you will get on your bank account, but I expect more when trading crypto with these risks involved. Also the winratio is only 16% and that is simply too low. The Calmar ratio says the same. It is only 0.4 whereas the original 4 hour timeframe has a 1.51 ratio.

After optimization

Now I was curious if the optimized settings for the AverageStrategy, which I tested the last time, would provide better results to this slightly improved Reinforced one. So I took the other JSON file and adjusted it to make it work with this code.

To my surprise those optimized ROI and Stoploss settings proved to be much more profitable. To find out why there are so many differences I made a comparison between the exit reasons of these three tests.

Original test

If I compare the exit reasons between the initial backtest, the hyperopted backtest and the backtest where I used the optimization file of the Average Strategy.

Then I see that in all three cases the ROI is the main cause for the profits. In the original file an ROI with takeprofit on 50% profit looks to be the best way to earn nice gains. The exit signal is a pain and is also the main cause for drawdown. But at least you get to keep a lot of these profits.

Initial backtest results

The initially optimized settings are not that optimal at all as you can see. The stoploss is getting hit too many times and causes a lot of losses on the initial, and not so great winning trades. Psychological factors come into play here. Trading bots can take a lot of psychological stress of real live trading away, but if you check on your bot and see the strategy have so many losses, you’ll be quickly thinking about changing algo strategies here.

Hyperoptimized strategy results

Then there are the exit reasons of the backtest with the optimization JSON file of the AverageStrategy. This is just an example out of curiousity. But it shows that the optimization session for some reason did not go well and provides us with sub optimal settings. Even though we always think that it always improves the parameters.

Let this be a lesson to us to always check these optimized parameters as well.

AverageStrategy json file backtest results on the Reinforced strategy

To round this test up. Let’s see where this strategy ends up in the strategy league.

Strategy league

Even though the name says it is "reinforced" that does not always mean it is better. In comparison to it’s simpler brother, this strategy does not perform that great and with optimization even does a worse job.

Maybe the additional check on the resambled simple moving average is it’s Achilles heel. It does have a higher total profit, but also has too many signals that give more risks to losing trades as well. That is why the winrate is pretty low and also causes a higher drawdown in comparison to the top 3 algo strategies.

Anyway, thanks for visiting my site and I will test another algo strategy soon.

In the mean time I will keep you posted on my other efforts as well.

Files of this test

All files will be posted on my Patreon site for my supporters. With these files you can check and simulate this test on your own PC or according to your own methodology. If you want to be a supporter, then you will receive these additional benefits.

Thank you!