LEAN Engine (on AWS) and Interactive Brokers Guide

The article “LEAN Engine (on AWS) and Interactive Brokers Guide” first appeared on AlgoTrading101 blog.

Table of contents:

• What is the LEAN Engine?
• What is LEAN Engine used for?
• What is QuantConnect?
• What is Interactive Brokers?
• Why should I use the LEAN Engine?
• Why shouldn’t I use the LEAN Engine?
• How to get started with the LEAN Engine and AWS?
• How to set up your AWS cloud server for the LEAN Engine?
• How to run a bot on Interactive Brokers with LEAN Engine?
• How to stop a LEAN Engine bot from running?

What is the LEAN Engine?

The LEAN Engine is an open-sourced algorithmic trading engine. The LEAN Engine is built by QuantConnect and seamlessly integrated with it.

Link to the repository: QuantConnect/Lean: Lean Algorithmic Trading Engine by QuantConnect (Python, C#) (github.com)

What is LEAN Engine used for?

The LEAN Engine is mostly used by algorithmic traders who want to backtest their trading strategies and/or deploy them in a live market.

We will demonstrate how to connect the LEAN engine to your Interactive Brokers’ account.

What is QuantConnect?

QuantConnect is an algorithmic trading platform that lets you develop, test and execute strategies. It integrates with many brokers and exchanges and is often used by algorithmic traders. You can read more about it below:

What is Interactive Brokers?

Interactive Brokers (IBKR) is an online brokerage that offers trading solutions for traders, investors, and advisors, with global access to stocks, options, futures, currencies, bonds, and funds.

Website: https://www.interactivebrokers.co.uk/en/home.php

Why should I use the LEAN Engine?

• LEAN Engine is easy to use.
• LEAN Engine is fast and performant.
• Has a highly modular design.
• Allows the use of custom data.
• Has Visual Studio and JupyterLab integrations.
• Has access to live data streams.
• Costs you less to self-host it.
• Higher sense of privacy.

Why shouldn’t I use the LEAN Engine?

• LEAN Engine might be harder to use for beginners.
• LEAN Engine uses your local resources vs the QuantConnect Web UI that offers external ones.
• Requires setup and a running server.
• Requires more overhead.

How to get started with the LEAN Engine and AWS?

To get started with the LEAN Engine, we will need to have a few things ready and those are the following:

• A QuantConnect account
• QuantConnect account User ID and Token
• An IBKR account
• An AWS Account and EC2 instance
• Python and Docker

You can follow our QuantConnect article to get a sense of the overall platform and get your account ready. To obtain the User ID and Token, navigate to your profile settings by clicking the Account icon in the upper right corner of your screen.

When there, scroll down until you see hyperlinked text that states “Request Email With Token and Your User-Id for API Requests”, click and you will get an email from QuantConnect with the information you need. Make sure to keep this secret.

For an IBKR account, navigate to their landing page and click the red “Open Account” button. Then, follow through their guides and UI to have the account verified and set up. Make sure to know your Username and Password as we will need them.

To set up your Amazon Web Services (AWS) account you can follow this blog post. When it comes to spinning up an EC2 server, you can pick a good location as shown in the blog post, and an Ubuntu t2.small instance should be enough for this article.

One important detail is to provision at least 50GB or so of memory to your EC2 instance as the docker containers that LEAN uses are very bulky and not well optimized for size.

In the following section, I’ll show you how to set up Python and Docker.

How to set up your AWS cloud server for the LEAN Engine?

To set up your cloud server for the LEAN Engine you will need to perform 5 steps which are the following:

1. Create a User
2. Install Docker
3. Create a Python virtual environment
4. Install LEAN
5. Configure LEAN Engine

It is always good to start with updating and upgrading your fresh servers as they will often have new security updates to be made. To do this, we run:

$ sudo apt-get update && sudo apt-get upgrade

After this has been completed, we move on to create a new user. I will call the user “algotrader” but you can choose any name. To create a user, we write:

$ adduser algotrader

Then, you will be prompted to fill in some information about your user, you can keep most things blank or as detailed as you wish. I will keep most things blank for this article. We will add the user to the superuser group:

$ usermod -aG sudo algotrader

Now, let us install Docker by running the following set of commands:

$ sudo apt install apt-transport-https ca-certificates curl software-properties-common
$ curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg
$ echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
$ sudo apt update
$ apt-cache policy docker-ce
$ sudo apt install docker-ce
$ docker

Once this is ready, we will want to install the dependencies that we need into our virtual environment. But first, let’s create a folder for our main workspace:

$ mkdir qc-ib
$ cd qc-ib/

I recommend having the latest version of pip on the ready:

$ sudo apt install python3-pip

Now, we will create a Python virtual environment that will contain all of our dependencies:

$ sudo apt install python3.10-venv
$ python3 -m venv venv
$ source venv/bin/activate
$ pip install lean

Now, we are ready to configure the LEAN Engine. To do this we run:

$ lean init

We need to log into our QuantConnect account through LEAN and pull our projects into the server. To do this, we write:

$ lean login

This will prompt you for your User ID and Token that was emailed to you in one of our previous steps.

Now we can run lean init again and it will get the Docker image on our server and start the setup process. Make sure to select Python as your language of choice when prompted. At the end of this process, you will see two new things in your directory which are the lean.json file and the data folder.

Once this is done, you can run this command to pull your scripts:

$ lean cloud pull

Nice! We have our server working and the LEAN Engine pumping, now let’s connect IBKR to it.

How to run a bot on Interactive Brokers with LEAN Engine?

To run a bot[1] on Interactive Brokers with LEAN Engine, all you need to do is start your bot go through the prompts that will ask you a few questions, and get it going. To do so you write:

$ lean live 'QC-SCRIPT-NAME'

[1] If you don’t know how to code a QuantConnect bot, read this article first.

Although this works, it does require you to have a subscription to IBKR data feeds and similar. For the purpose of this article, I’ll request delayed market data and run the following command that will answer these prompts directly:

$ lean live "QC-SCRIPT-NAME" --brokerage "Interactive Brokers"  
    --data-feed "Interactive Brokers" 
    --ib-enable-delayed-streaming-data true 
    --ib-user-name yourusername 
    --ib-account youraccount 
    --ib-password yourpassword

Take note that this will likely require you to have 2FA on IBKR disabled or only be using the mobile phone version of it. You can do this reconfiguration inside your IBKR account.

If your 2FA and credentials for IBKR were correct, you will see the bot running and showing its logs. If you update your script, you will need to run the pull command again to get the updated version. Don’t forget to stop the bot before pulling.

How to stop a LEAN Engine bot from running?

To stop a LEAN Engine bot from running, you will need to press CTRL+C on your keyboard to stop the LEAN Engine. You can also write lean live stop "QC-SCRIPT-NAME".

Another thing that you can check is the docker container directly and stop it:

$ sudo docker ps

$ sudo docker stop <CONTAINER-ID>

Go Source