Running multiple bots with Docker and Docker compose (Patreon)

Hi all, 

Here's a post that contains the answer of a good a Patron asked (thanks Isaac!).

How do you run multiple Docker bot instances with the same Docker file.

Actually I run Docker compose for this since it allows to use a specific docker-compose.yml file to be configured with everything the bot environment needs and then simply run it as a service in the background with one command. 

Btw. For my setup I assume the following

You are running a Linux system here. It should also work with Windows but I have not tested this. 

I also assume that you already have some experience with running freqtrade with docker. If not, you can always follow their procedure on: https://www.freqtrade.io/en/stable/docker_quickstart/  

You already have a config.json file and a strategy.py file that are both tested and working. In the config the web environment is enabled in my case.

Finally you also have installed Docker and Docker-compose on your system. 

My exact setup is as follows:

I have a directory called /opt/freqtrade. You can make this by doing:

cd /opt

mkdir freqtrade

cd freqtrade

Then create a docker-compose.yml file with your favorite text editor and add the following to the file:

docker-compose.yml file

---
version: '3'

services:

  bot1:

    image: freqtradeorg/freqtrade:stable

    restart: always

    container_name: ft_bot1

    volumes:

      - "/opt/freqtrade/user_data:/freqtrade/user_data"

    ports:

      - "0.0.0.0:8181:8080"

    environment:

      - TZ=Europe/Amsterdam

    command: >

      trade

      --logfile /freqtrade/user_data/logs/ft_bot1.log

      --db-url sqlite:////freqtrade/user_data/ft_bot1.sqlite

      --config /freqtrade/user_data/config_bot1.json

      --strategy skz_f

  bot2:

    image: freqtradeorg/freqtrade:stable

    restart: always

    container_name: ft_bot2

    volumes:

      - "/opt/freqtrade/user_data:/freqtrade/user_data"

    ports:

      - "0.0.0.0:8282:8080"

    environment:

      - TZ=Europe/Amsterdam

    command: >

      trade

      --logfile /freqtrade/user_data/logs/ft_bot2.log

      --db-url sqlite:////freqtrade/user_data/ft_bot2.sqlite

      --config /freqtrade/user_data/config_bot2.json

      --strategy ClucMay72018

Explanation

Now that do these lines mean:

--- version: '3' # Version of the docker-compose file format

services: # The services section where you define all docker running containers

bot1: # Name of the service

image: freqtradeorg/freqtrade:stable # The docker image that you are going to use for this container

restart: always # Determining the restart after a reboot or errors 

container_name: ft_bot1 # The container name for this instance

volumes: # here you determine the location where docker should keep its persistend data. NOT DOING THIS WILL LEAD TO LOSS OF DATA AFTER A RELOAD

- "/opt/freqtrade/user_data:/freqtrade/user_data" # Local data location (in my case absolute): container data location

ports: # when running the freqtrade website determine here port forwarding

- "0.0.0.0:8181:8080" # 0.0.0.0 = reachable from everywhere! 8181 = remote port 8080 = interna; docker container port. This way you dont have to change port numbers in the config.json file

environment: # environment variables

- TZ=Europe/Amsterdam # My timezone

command: > # this is the actual command that docker will use to start the bot in the container

trade # We are going to trade!

--logfile /freqtrade/user_data/logs/ft_bot1.log # an external logfile that I can monitor for bot action

--db-url sqlite:////freqtrade/user_data/ft_bot1.sqlite # the database name that is going to be used

--config /freqtrade/user_data/config_bot1.json # The config that I will use for this bot instance

--strategy skz_f # The strategy that I am using in the bot instance

You see that these last command lines are very similar to starting a bot without Docker. The difference here is that the location of all files seem to be located internally in the bot image, but because of the volume section above, their actually located in a local and persistent data directory.

Adding your configs and strategies

If you have working configs and strategy files. You will have to add them to the correct directories. Assuming that you did not start the bot with docker-compose or docker yet, you will still have to add these to the correct locations.

Therefore you will have to check the following before actually starting the bot(s):

Is there already a user_data directory in the /opt/freqtrade directory? If not, make it since docker-compose will look for a config file over there (according to my compose file above...).

mkdir user_data

It should now be: /opt/freqtrade/user_data

Move your config.json file over there and change the docker-compose file accordingly.

Is there a strategies directory in your /opt/freqtrade/user_data/ directory?

If not create that one as well with:

mkdir strategies

It should now be: /opt/freqtrade/user_data/strategies

And then add your strategy to that directory and change the docker-compose file accordingly.

Starting the bots

To start the bots, you should be in the same directory as where the docker-compose.yml file resides!

Start this file and see its output to the screen, use the command:

docker-compose up

If you haven't already downloaded the image, this will be the first time you will see it downloading. After this download, the bot instances will start and show the output to the screen.

If everything goes well, then you already could see some trading action on your screen. If not, then investigate the output on your screen carefully to see where things go wrong. It could be that the configuration files or strategy file is not seen. Check the directories to see if everything os configured correctly

Now it is time to start the docker images in the background with daemon mode. 

Stop the bots with CTRL+C and start them start them again, but this time in the background:

docker-compose up -d

The bots will start in the background and you can check the output of these by looking in their respective logs like this:

tail -f /user_data/logs/ft_bot1.log

If everything goes well you will see the output of the bot instance to this log. If you have errors, again check what's happening and fix these errors.

And always ALWAYS test this with the dry-run option enabled in the config json file of your bot!!

Last tips

I gave you my personal docker-compose file here with my strategies that I am currently testing. 

Additional settings are in the config.json file itself to keep things simple for me.

Your specific setup is not the same as mine so keep the correct datastructure and filenames in mind. Otherwise this will not work (correctly).

Also, do not make 10 bots in one single file but be aware of the capabilities of your system. Otherwise your system wil halt and you wil have to restart it (personal experience). Start with 2 bot instances and then slowly increase the amount of bots used. The system should still be responsive and capable of handling data quickly. 5 bots configured but none of them working swiftly has no use here.

I am trying out multiple different strategies here so that I diversify my portfolio. My older Freqtrade instruction videos on youtube also explain how you can set up your bots to use specific amounts/parts/percentages of your account balance to that bot's do not get in each others way and all think they have the complete account balance to their disposal. This could lead to very unpredictable behaviour.

Good luck with your BOT ARMY!