Blog: Automating Trading and Market Making With Artificial Intelligence
A general ML approach to creating liquidity and profits.
The goal is to capture information in a market’s order books and use that information to predict market movement/direction. That prediction can enable repricing of orders and more efficient market making. Such an approach allows the market maker to provide liquidity whilst making profits at the same time.
Market makers are essential to modern markets. They provide the markets with necessary liquidity and make sure the bid/ask spread is reasonably narrow to allow efficient purchasing. This can be taken a step further by market makers that do more than simply provide a constant bidding and asking price. Some market makers trade at higher frequencies and constantly take advantage of inefficiencies as well as small swings in asset prices.
Market makers with this strategy for making profits can usually be classified as high frequency traders, performing many trades in short time-frames. It is worth point out that markets for assets like blue chip stocks have banks, hedge funds, and other market participants market making in this exact fashion, with billions of dollars of volume attributed to these actions.
Cryptocurrency markets, especially in their younger days, had many lower net worth funds market making in this fashion. Generally, the smaller the market, the easier it is for lower net worth individuals and funds to partake in market making. Similarly, smaller volume assets in stock markets, bond markets, etc can be places that lower net worth market makers can provide liquidity and turn a profit. Note: by lower net worth I’m usually still meaning in the millions of dollars. It is difficult to quantify what amount of funds are necessary to partake in what markets. If one is high frequency trading but is not in the game of providing liquidity, they are probably just high frequency traders, not market makers.
At its core, this approach fundamentally involves capturing information in the order book, establishing a directional view, creating liquidity, and taking small profits on each trade. This can be repeated many times in very short time-frames.
In this article, we will break down each of the above steps in depth.
As mentioned, the goal is to capture and process information from the order book(s). This data can come from a multitude of different sources. In this article, we will be market makers for a cryptocurrency market. The fundamentals are the same for any other market, i.e. stocks, bonds, etc. The first part of capturing information is always finding a proper data source. A great first data source for cryptocurrencies is the order book of an exchange. The Coinbase API is well developed and easy to use. Here is an example of using the Coinbase REST API to get order book data in real time.
This GET will retrieve level 2 data from the order book. That is, it will retrieve the top 50 bids and asks from the Coinbase order book for the desired product.
Providing Liquidity and Making Money
So what can we do with the data we have retrieved? First, we should try and understand how market makers can profit in financial markets. At its core, the market maker’s profits are gathered like so:
Why is this a good outcome? The market maker has allowed the asking party to make more from the trade than they would have otherwise ($99.50 vs $99). The market maker has allowed the bidder to pay less than they would have otherwise ($100.50 vs $101). And the market maker has bought the asset for $99.50 and sold it for $100.50, netting a profit of $1.00. Depending on the volume and liquidity in a market, this process can be repeated many times over. In a market with higher existing liquidity, a market maker may buy the stock from the asking party for 100.50 and sell it to the bidding party for 100.51. This seems like minuscule profits, but when thousands of these trades can be executed each second or every few seconds, the profits quickly begin to build.
It is fair to point out that this may not always work. The bidding party may adjust their desired price just as the market maker goes to sell them the asset. As long as more trades are in the green than the red, this strategy is profitable. This strategy can be especially profitable if know what tiny values the market will move in either direction. We obviously cannot know this for sure, but if we can guess this with some probabilistic model, we can optimize our decision making.
There are many approaches for building this market view. There are many simple (and not so simple) statistical and machine learning approaches that sophisticated firms will utilize to maximize profits.
AI Approach to Optimizing Market Making
Perhaps the most interesting approach out of modern methods is the Reinforcement Learning approach. Reinforcement Learning (RL) is about training an agent to take actions that maximize the reward function. Naturally, the reward in our market making example is profits. The agent can be trained on prior data from previous market conditions, order book(s), etc. You can feed many states to this agent and have the agent optimize profits.
The agent needs to identify current state at each moment in time. Some useful parameters for state will primarily involve signals from the order book, but state can also involve parameters such as economic data, financial data, and technical data. Most notably, a baseline model will probably include the bid-ask spread, mid price (avg of best bid and ask prices), market imbalance, volume, volatility, etc.
It is also crucial to place bounds on the agent that keep it from optimizing the reward function in strange ways (potentially unprofitable in the long term). For example, the agent may decide using 100% of its allocated funds at all times is optimal. You may want to limit this, limit risk, or limit a number of factors that the learner may not be directly considering when simply trying to optimize a path to profits. The infamous OpenAI RL approach from a few years ago is a great example of how RL can optimize a reward function, but not necessarily in the way we desire.
Keras is a great way to not only build your first neural networks, but Keras is also a great place to try things like Reinforcement Learning.
The abstract environment class that is used by all agents. This class has the exact same API that OpenAI Gym uses so…keras-rl.readthedocs.io
A model-free approach which relies on trial and error for training is likely the best first approach to try out. The agent fundamentally will need to learn the actions it can take, will need to be fed a state, and have a well-defined reward.
Actions: buy + range of amounts + deviation from mid-price, sell + range of amounts + deviation from mid-price, hold
State: bid-ask spread, mid price (avg of best bid and ask prices), market imbalance, volume, volatility, etc.
Reward: P/L from previous action
Supervised Learning Approach
Supervised Learning can also be used for analysis. The primary difference here is our supervised approach will be providing a market view, not specifically providing us with more complex information like sizing or other necessary information for optimizing the reward function.
We can set the market view up as a classification problem where our outputs include buy, sell, and hold.Train the model on past market conditions using prior order book data and then feed present market conditions and use the provided signals to decide which direction to buy from the mid-price point. Of course, if the market view is bullish, you can comfortably buy just above the mid-price and take profits shortly after. If bearish, put your buys in below the mid-price or sell your current holdings.
Basics of Supervised Learning: https://medium.com/machine-learning-for-humans/supervised-learning-740383a2feab
Market makers use complex analysis for optimizing the decision making of their high frequency systems. One element of market making we did not discuss is compensation for creating liquidity. This can come in the form of fees and certainly aids in profitability. For example, an exchange may partake in market making, providing more liquidity and increasing volume. This increase in volume can create an increase in fees and returns for the exchange. Someone without fees is simply returning P/L of trades whereas the house is receiving P/L of trades + fees. The house always wins…or so it seems.
A market maker or liquidity provider is a company or an individual that quotes both a buy and a sell price in a…en.wikipedia.org