Freqtrade Lookahead Bia (Patreon)

Look-ahead  bias is a subtle yet critical concept to understand when analyzing  financial markets, whether you're dealing with equities, stocks,  options, forex, or cryptocurrencies. This bias occurs when a strategy is  tested using information that would not have been available at the time  of trading. In other words, it's like unintentionally peeking into the  future when making a decision in the present.

Imagine you're a time traveler with tomorrow's newspaper in your  hands today. If you use that future information to make decisions today,  you have an unfair advantage, right? That's essentially what look-ahead  bias introduces in trading algorithms and financial models. It makes a  strategy appear more effective than it truly is because it uses  information that wouldn't be accessible in real-time trading.

For example, if an algorithm incorporates the end-of-day price as a  signal to make trades at the start of the day, it's indulging in  look-ahead bias because, in reality, the end-of-day price wouldn't be  known at the start. This bias can lead to overly optimistic backtesting  results, making a strategy seem profitable on paper when it might not be  in live markets.

It's crucial for traders and analysts to meticulously ensure that  their models only use information that would be available at the moment  of decision-making to avoid this bias. Proper data alignment, careful  construction of historical databases, and strict adherence to  chronological order in simulation and backtesting processes are  essential steps to prevent look-ahead bias and ensure the integrity and  reliability of trading strategies.

By understanding and avoiding look-ahead bias, traders can develop  more accurate and realistic models, which is essential for navigating  the complex and fast-paced world of financial markets with confidence  and success.

The reason for this video is the feedback I got from my last strategy testing video.

There I tested the Machete strategy and people from the community  notified me that these unrealistic spectacular results were caused by  lookahead bias that this algo has.

And although I explicitly warned about this in the video this  feedback made me wonder if there is any way to detect if an algorithm  has lookahead bias.

Luckily for us there is a function in Freqtrade bot that helps us with detecting this bias.

It has been introduced in version 2023.6.

I consulted the Freqtrade documentation on this lookahead-analysis and there it also states that it is easy to introduce lookahead bias in your algorithm. But it is hard to decect that you created this bias afterwards. Except for results that maybe look too good to be true.

The documentation even literally states that "If the programmer is  not careful or oblivious how things work internally (which sometimes can  be really hard to find out) then it will just look into the future  making the strategy amazing but not realistic."

So, when testing someone elses algorithm with lots and lots of buy and sell signals, it is even harder to detect lookahead bias.

Hopefully the Freqtrade lookahead-analysis function provides enough  information to add an extra warning to the results i have when testing  future trading strategies.

Reading the documentation states that:

It will start with a backtest of all pairs to generate a  baseline for indicators and entries/exits. After the backtest ran, it  will look if the minimum-trade-amount is met and if not cancel the  lookahead-analysis for this strategy.After setting the baseline it will then do additional runs for every  entry and exit separately. When a verification-backtest is done, it will  compare the indicators as the signal (either entry or exit) and report  the bias. After all signals have been verified or falsified a  result-table will be generated for the user to see.

So what I understand from this is that the function starts by  performing a regular backtest, considering all available data to  establish a baseline for your strategy's performance.

And after the initial backtest, the function then conducts a series  of additional backtests. but, it slightly alters the data each time to  simulate real-time conditions. It removes future data that wouldn't be  available at the time of trade execution to see if your strategy's  performance changes.

The function compares the results of these lookahead backtests with  the initial baseline. And if your strategy performs significantly better  in the initial backtest (where it had access to all data), it might be  because it's using future information, indicating lookahead bias.

Strategies without bias

So what happens if I use this command with a strategy that I suspect does not have lookahead bias?

Since this command is very similar to the backtesting function I used  the following command to find out if the next algo code has bias.

Well I had to test out a little bit what the exact command should be  to give me the correct output and I came up with the following command:

So as you see, this command can use a lot of commands that are similar of Backtesting.

The command also has the following settings to avoid users accidentally generating false positives.

A nice addition is that lookahead analysis also supports the analysis of freqai strategies.

So let's show the first example where there is no bias detected...

You can see that the output shows that this algorithm has no bias detected.

So in this case I am happy that this algo has been correctly build.

Strategies with bias

Let's now look at an example where a strategy does suffer from lookahead bias.

What does this command show where this bias is located.

Again, I'll use the same command, but this time use another strategy to test.

Export file contains the following information:

According to the analysis this algorithm unfortunately suffers from lookahead bias.

And the results show there are 17 entry signals and 12 exit signals  biased. Now this algo does not have 17 different types of entry signals  and also not 12 exit signals configured. So I guess that theses amounts  refer to the actual trading signals where the bias has been discovered.

The 'wt1, wt2, slowk and def' indicators seem to have this bias in their code.

So let's open this code and see if there are any hints or clues that  provide me with information where this bias has been implemented.

If I take a look at the code, and if i'm correct, then there seems to  be a problem in the wavetrend oscillator because that is where the  problem indicators are used.

Let me first take a look at the buy and sell signals.

At first sight there seems to be nothing wrong with these signals.

These are just based on where the position is of the indicators in  relation to minimum and maximum values. And since a lot of strategies  are coded like this, I assume there is no mistake here. Or I should be  terribly wrong.

So what rests is to check the indicators themselves and hopefully see what seems to be the problem here.

wt1 seems to be derived from the TA libraries exponential moving  average and some calculations where candles dataframes has been used..

The wt2 is derived from the TA library Simple moving average but also from the wt1 indicator so it seems.

The slowk is derived from the TA libaries Stochastics indicator. And, as far as I can tell not much more.

And the def indicator value is derived from the slowk and wt1 again.

So without delving too deep into this specific problem, the error could be from the way the indicators are created in this code.

And at first I suspected that the creation of the wt1 indicator has  something to do with the bias. Perhaps because it is also derived from  the dataframe itself through the ap variable. Since the dataframe holds  future data as well.

Now since we all have the availability of AI, I also asked ChatGPT where this potential problem could be located in the code.

And there I was warned that the MinMaxScaler could be the cause of this bias.

Potential Lookahead Bias

The potential lookahead bias in this function could arise from the scaling operation:

How to Fix

To avoid lookahead bias when  scaling, you need to ensure that the scaling is done in a rolling or  incremental manner that only uses past and current data available at the  time of each trade decision. One approach could be to use a rolling  window to calculate the min and max values for scaling, ensuring that  only historical data up to the current point in time is used for  normalization. This way, the strategy remains realistic and doesn't  inadvertently "peek" into the future.

So this looks to be very good additional information to go and fix this issue.

Now there was one final thing that I found strange and a bit  worrysome. It seems that the Stochastics slowk also suffers from bias  according to this check.

This could mean that using indicators from the TA library also might be incorrect.

This is something to keep into the back of my head and I'll watch  this closely if I will use this lookahead detection function more often.

Besides this wicked problem, this lookahead analysis feature also has  some caveats to be aware of, which are described in the documentation.

Caveats¶

lookahead-analysis can only verify / falsify the trades  it calculated and verified. If the strategy has many different signals /  signal types, it's up to you to select appropriate parameters to ensure  that all signals have triggered at least once. Not triggered signals  will not have been verified. This could lead to a false-negative (the  strategy will then be reported as non-biased).lookahead-analysis has access to everything that backtesting has too.  Please don't provoke any configs like enabling position stacking. If  you decide to do so, then make doubly sure that you won't ever run out  of max_open_trades amount and neither leftover money in your wallet.

So please remember this when looking at the results and before making any final conclusions.

Not all strategies work

There is one final remark that I want to make here and that is that not all strategies can be easily checked with this function.

It might be that the algo runs perfectly fine with normal backtests but with the lookahead analysis they error out.

In these cases you will have to figure out where the error is and  that can be a problem if you test out complex algorithms that you did  not create.

So please be aware that you can experience these issues.

End

Now maybe I did not use the lookahead function correctly or am just plain wrong with some of my conclusions.

And I dont mind being corrected, as long as it is criticism that also  contains information that can help me and the community understand  things better and prevent future mistakes.

But anyway, I think we now have another welcome addition to our  toolbox to determine if our algorithmic trading strategy performs  without errors and biases.

I am still figuring out how to incorporate this additional  information into my backtests and until that moment I keep a close  critical eye on these strategies that seem too good to be true. And I  hope that you, the viewer, will do this too. You should always be  sceptic and do your own research.

And with this conclusion I'm at the end of this video.

Thanks for watching. If you like this content, then please click like, subscribe and give a comment.

And I will see you in the next one.

Goodbye!