Interview Traders Magazine – How to create a trading system (Cover Story)
- Von Thomas
The recognized and at the same time highest-circulation trader magazine in the German-speaking world “Traders Magazin” has set a focus on the topic of trading robots in the December issue. Of course, an interview with Thomas Vittner is a must in this context.
For reasons of space, Traders Magazin has published the edited but abridged version. The unedited full version can be found here.
Unfortunately, the article in Traders Magazine is not freely accessible. Nevertheless, we link to the e-paper
Below you will find the table of contents of the issue with the marked article (red arrow).
What advice would you give a beginner looking for a suitable trading system?
A beginner must first get an overview of the different asset classes and products that he wants to trade. They should then look for software solutions that can be used to develop trading systems in the desired target market. This is because not all software is suitable for every market.
For example, if a trader wants to backtest share portfolios, they need software that is capable of portfolio backtesting. Not all programs available on the market can do this. It is therefore necessary to research the basics before developing the strategy itself.
And only then can you start developing the system. Here I would first look for the basics on social media channels. For example, on my channel. But beware of people who promise the best of everything on Facebook, YouTube and the like and advertise intensively. There’s usually not much behind it.
Where do you get your strategy ideas to construct a trading system?
There are several approaches here. On the one hand, I look at price movements with a naked eye to identify patterns. If I think I’ve found something interesting, I think about whether there are rules that reflect this behavior and are unambiguous. Rules that allow me to carry out a statistical analysis (backtest) with it. So I ask myself: can I somehow quantify/backtest this idea? Are there any indicators that reflect the pattern?
Not every idea you have can be backtested. However, you should only trade things that you can quantify. If I cannot test an idea, either because the rules are not clear enough or because there is simply a lack of reproducibility, then I discard the idea.
Because I only want to trade what I can test and what has been proven to work after testing.
And then there is the other way – the computer-aided way. Today, I have the opportunity to use various machine learning algorithms in system development. And you don’t even have to know how to program.
For example, my Wealth Lab backtesting software includes the extension of neural networks, as well as genetic optimizers that improve on their own from system generation to system generation or various indicator profiling tools that I can use to measure the alpha of an indicator.
The algorithm therefore finds good indicator combinations and parameter settings by using different machine learning algorithms. However, it is important to understand that AI will not replace the trader in the area of system development either. These processes need to be observed properly.
And last but not least, the trader decides which system they actually want to trade live based on various metrics. Regardless of whether they have “invented” it themselves or whether the computer is the creator of the trading logic.
Translated with www.DeepL.com/Translator (free version)
What is most the difficult part for you when designing a trading system?
One of my favourite sentences with regard to system development is: we optimize even when we don’t optimize. By this I mean that despite the separation of “seen data” and “unseen data”, which can also be called “in-sample” vs. “out of sample”, we often over-optimize. For example, our Quant Master 1 training is dedicated exclusively to this topic of over-optimization because it is so hugely important.
Now to answer the question: over-optimization is the biggest problem in the development of trading systems and therefore one of the most difficult parts. And you could probably write a whole book about it.
Over-optimization is usually not intentional. The developer simply wants too much. To force one percent more performance in the backtest. A little less drawdown or a slightly better sharp ratio: and poof – the system has been – unknowingly – over-optimized.
And as I said: in sample vs. out of sample is a good start to keeping over-optimization in check. But here too, unknown data quickly becomes known data. Namely, when you look at the out-of-sample period after the in-sample development and realize that you might not like what you see so much after all.
Because of course it is usually the case that out of sample performs worse than in sample. After all, it is not difficult to optimize the system on seen or known data so that the results are good. It is much more difficult to ensure that the system also performs well on unseen out-of-sample data.
So if I now realize that out of sample is not good enough for me, I go back to the in sample period with this information and optimize again. And I’m already in the cycle. I have over-optimized to some extent. And if I repeat this a few times now, I have systems that are perfect in the backtest but fail miserably in live trading.
Until recently, there was little or nothing you could do about over-optimization. In most cases it was not even noticed. The best I can do is a walk forward analysis to see if my system is over-optimized. But Walk Forward only gives me a rough indication of the over-optimization – with markings. Walk Forward gives me no way of reducing the over-optimization.
But with my backtesting software, I can now visualize the over-optimization in step 1 and slowly reduce it in the following steps. I will show you three graphs of three trading systems in different stages of (over)optimization. The performance in sample (blue curve) compared with the performance out of sample (red curve) in different colors visualizes this very well.
Over-optimized system (graphic 1)
We can see in the graph above that the genetic optimizer used always finds better and better settings in the in-sample period (blue curve) and thus the performance increases massively. However, the red curve, our out-of-sample period, cannot keep up, in fact it even tends slightly downwards and the gap between the two curves is increasing.
This is a prime example of strong over-optimization. This system can be safely put to one side. There is no point in investing any more time in reducing the over-optimization. The parameters are far too strongly adapted to the known data right from the start.
Basic system (graphic 2)
What we see here is common normality. The system development process has been completed and the developer is pleased with the excellent returns in the backtest. However, the visualization of the returns split into an in sample vs. out of sample period shows that the red curve (out of sample) “keeps up” with the blue curve (out of sample) up to a maximum of two thirds of the way. Everything after that is therefore over-optimized.
On the far right are the trades that the optimizer finds in sample, which will never be found again. This means that the system is over-optimized to a certain extent, but this is not a problem. This over-optimization must now be gradually reduced in the next system development step by readjusting parameters one by one. And you repeat this until you get a picture like the one in the next case.
Over-optimization reduction process completed (chart 3)
Finally, a graphic of a trading system in which the red curve (out of sample) runs upwards almost to the right edge of the image with the blue curve (out of sample). This is the ideal case. The system has been cleaned up as much as possible after intensive efforts regarding over-optimization.
As I said, every backtested trading system has a slight over-optimization, but here it has been significantly minimized. I would describe this system as no longer over-optimized with regard to the indicators and parameter settings used. Whether the displayed return is realistic is another question that is not directly related to the optimization or over-optimization of the parameters.
How do you optimize the settings within the trading system? Is there a key figure that is particularly good?
Various optimization algorithms are available to me for optimization, which are used depending on the situation. It is important to understand that you have to optimize carefully. For example, the best-performing trading system is not always used; things like stability play a much more important role, even if this is at the expense of returns in the backtest.
As far as the importance of key figures is concerned, there are a few that we pay attention to. In addition to performance and drawdown, exposure, the sharp ratio or the average profit per trade also play a decisive role, to mention just a few key figures.
It is therefore a combination of different metrics that ultimately decide whether a system makes it into live trading or not.
Which is easier to design - a trend trading system or a cyclical oscillator system?
You can’t say that across the board. The fact is that every target market has its own tendencies as to which type of system works better. In our Trader Masterclass we work with our clients on 4 different system types: Reversion, Momentum/Breakout, Rotation Systems and Limit Dip Buyer. We separate trading systems into pro-cyclical and anti-cyclical models.
What tends to work well with shares, for example, may work differently or not at all with currencies. And, of course, the time unit selected and, in particular, the distinction between “intraday” and “end of day” plays a not insignificant role.
Each target market therefore has its own tendency and once this tendency has been identified, it is relatively easy to find system logics that basically work. If, on the other hand, you want to develop a system that runs counter to this tendency, the system development process can become laborious and tough.
How many parameters can a trading system have so that it remains controllable?
Here, too, there is no one-size-fits-all answer. I think it should and must be neither too simple nor too complicated. Each additional parameter increases the risk of over-optimization and if I had to choose between two systems today that perform approximately well and system A) consists of 18 entry parameters and system B) of 12, then I would probably tend towards the simpler system.
But in the end, I need to know what my parameter contributes to my desired target metric. If every parameter is important, there can and may be many parameters as long as I have the over-optimization under control. If, on the other hand, parameters are “dragged along” that have little or no influence on my desired metric (e.g. APR – annual percentage return), I can remove the parameter under certain circumstances.
Here are two examples that show the influence of the individual parameters on a system. (Figure 4)
Above is a trading system with many parameters. Each parameter is significant and contributes more or less to my desired target metric. Here the metric APR (Annualized percentage return – annual performance in %) is chosen.
In the next case, things are a little different (Figure 5)
The last two parameters are apparently not important for my target value APR. I should therefore check whether I can omit these parameters, which of course belong to certain indicators. If this is not possible because the indicator consists of several parameters and the other parameters of this indicator do have a great influence on my target metric, I can at least omit further parameter optimizations and fix this parameter, which saves time.
Ultimately, however, everything depends on the question: am I over-optimizing or not? And fortunately, today we can visualize over-optimization, measure it and – with various work steps – reduce it. We already discussed the topic of over-optimization earlier.
What do you think of trading systems that contain an AI component? Are these systems superior to others in the long term?
Various machine learning algorithms relieve the system developer because parts of the system development process can be automated and thus optimized. We also enjoy working with them.
Ki is a buzzword today, but neural networks have been around in stock market trading since the late 1980s, for example. So this is nothing new, but due to the public debate about AI, it has now become fashionable again in trading.
These systems are certainly not superior. Neither today nor in 10 years’ time. If you consider the computing power that large corporations such as Google would have to supposedly crack the stock market code, but don’t do it because their actual core business seems to be more profitable, you already have the answer.
Due to the millions of market participants, whether human or AI, the stock market and therefore trading will always remain a largely random walk. The next price movement is always purely random and there is a lot of noise. With our statistical analyses (backtests) we try to capture this noise and find patterns.
These patterns exist and some of our systems have been in use for years and are still working. But no AI will crack the stock market permanently. Otherwise the markets would have to close forever due to the interplay between supply and demand that no longer exists.
Are your current trading systems subject to continuous monitoring, a maintenance interval so to speak?
What you are talking about here is probably one of the most difficult things with trading systems: monitoring in live use. In my experience, it also depends on the target market as to how often adjustments need to be made. For example, in my experience, trading systems tend to work much longer in the equity markets than in currency pairs, for example. The equity markets are significantly less efficient than forex.
But even with equity systems, we regularly check whether the system is doing what it should according to the backtest. This starts with daily checks of slippage through to periodic performance observations.
Here we pay attention to several key figures. Above all the drawdowns, although not the maximum drawdown, which has no relevance from a statistical point of view because it is a single event. Drawdown times of 10% or 20%, for example, are more suitable and are monitored alongside many other key figures.
So it is not just one key figure that tells me whether my system is no longer working and needs to be revised or even shut down. Rather, it is a bundle of key figures and an overall impression that decide whether a system should continue to be traded despite temporarily poor results or whether it needs to be adapted.
Danke für das interessante Gespräch!
With pleasure, anytime again!
