Hi Howard.
I was going to ask something similar to this on your website, think you have opened up a place to discuss system development topics.
Just finished reading your "Quantitative Trading Systems" , lots of great info, actually about to re-read the last few chapters, Walk Forward Testing, Statistical Test and Monte Carlo especially, a lot to take in for someone with no formal education in quantitative analysis. Actually got a few more topics I'd like to ask but save it for later after I give those chapters a good read again. Then can't wait to get into "Mean Reversion Systems".
I think I know what tim_testpat is trying to ask.
From my very basic knowledge and lack of great terminology, my understanding of working with price data, and trying to optimize a trading system, you need a in sample series of data and out of sample series of data. The goal is to develop a profitable system in the in sample series that will duplicate itself in the out of sample series.
I myself have used the optimize features a bit, I can see the troubles of easily over fitting the data depending on what variable is being optimized.
But from what I know, the idea is to take the optimized parameters that best fit your objective function and to use this in the system and forward test it on the out of sample data.
Example: In Sample period is 2008-01-01 to 2012-01-01 and Out of Sample is 2012-01-01 to present.
When I get confused is the idea of walk forward testing. In this case your using some sort of variable that can be optimized and your looking for a parameter to best fit your objective function, and producing a series of In Sample and Out Of Sample results to observe if the parameter being optimized has discovered a pattern in the price series that proves to be a profitable model.
Now to me in my head this is over fitting as I understand it but as I said, I'm very novice at this quantitative trading approach.
The reason I say that is because for example in your book, page 292, figure 20.26, you show a table of a walk forward result list. The in sample is in steps of 2 years and the out of sample is the following year. The next step is a year after the beginning point and continues in increments of 2 years of in sample and 1 year of out of sample.
Each in sample optimize run provides different results for your optimization of "Periods" and "AMAAvg".
What I don't get and I believe the original poster is how does one use these changing optimizations or validate the results.
What I've always done prior to learning about walk forward analysis is backtest in sample data which is optimized and built to provide a profitable model, then to backtest the out of sample, the variables don't change between the two.
Does this walk forward optimizng not fall into mining the data and overfitting? Does one need to be aware of what their optimizing exactly and how it will work in the model? How would one trade such a system going forward if it was proven to be a robust system after using walkforward optimization.
Using the sample above, the system was optmizing the two previous years and using those optimized parameters for the system the following year. So going live, say this year with such a system, would one optimize 2014-01-01 to 2016-01-01 and use these parameters in the system for 2016?
Posted by: brandon.richard03@yahoo.com
| Reply via web post | • | Reply to sender | • | Reply to group | • | Start a New Topic | • | Messages in this topic (4) |
This group is for the discussion between users only.
This is *NOT* technical support channel.
TO GET TECHNICAL SUPPORT send an e-mail directly to
SUPPORT {at} amibroker.com
TO SUBMIT SUGGESTIONS please use FEEDBACK CENTER at
http://www.amibroker.com/feedback/
(submissions sent via other channels won't be considered)
For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG:
http://www.amibroker.com/devlog/
EmoticonEmoticon