Hey Howard,
Why do you recommend python for beginners instead of something like AFL or the other languages with other software?
Thanks
Hey Howard,
Why do you recommend python for beginners instead of something like AFL or the other languages with other software?
Thanks
Great thread and good to see Howard instigating a meaningful discourse . Id love to dig in and say my bit but its xmas and there is plenty of time for that . Watching with interest .
statistical based empirically proven rules based systems can have defined expectancy and by their very nature can be exploited to maximize returns exponentially J curve style . This is the realm of quants , dismiss at your peril ...
rock on
Ohh and MERRY XMAS to all , make 2017 matter
J-Curves in Equity Funds
In private equity funds, the J-curve effect occurs when funds experience negative returns for the first several years. This is a common experience, as the early years of the fund include capital drawdowns and an investment portfolio that has yet to mature. If the fund is well managed, it will eventually recover from its initial losses and the returns will form a J-curve. Losses in the beginning dip down below the initial value, and later returns show profits above the initial level.
Got a question for Howard regarding "data". Data availability, data reliability, data latency and cost of data are some factors that limit data analysis. I notice on Quantopian they claim 4 million data sets are available through Quandl and Yahoo Finance. What is a data set, please?
A data set is a series of values associated with a tradable issue, such as a stock, fund, future; or with an economic, econometric, or demographic indicator, such as gross domestic product or population. Being a series implies that there are dates associated with the data values, such as daily closing price for a stock, quarterly GDP, or annual population.
Best, Howard
Greetings --So Howard I've been meaning to ask, with all this talk of machine learning etc. do you actually have/run any ML systems yourself? I imagine that you must have a few systems that earn you an income. what success have you had with ML and trading?
Greetings --
The short answer is yes, I do.
I do follow my own advice, most of which is outlined in the presentations I have made and writings I have published. I'll break my response into a few parts.
1. Most of the development work I am doing now is with machine learning. The flexibility and power of the Python base, coupled with the variety of models available through scikit-learn and other libraries, the ability to determine hyperparamters, fit models, test whether learning took place, and calculate the risk and profit potential all in a single program run gives a great deal of control over the development process. I continue to work in AmiBroker -- particularly with some of my clients.
2. The sweet spot in terms of risk-normalized profit potential really is high accuracy and short holding periods. Using one-day lookahead targets fits naturally into machine learning. Anticipating market-to-market daily management of trades after the system has moved on from development, using day-to-day state signals rather than impulse signals works well.
3. I am making fewer trades these days. One of the reasons to stop trading is when you have enough. At my stage of life, I do not know what I would do with more money.
I highly recommend following the scientific method of learning then validating. And becoming competent in machine learning for trading. Pay particular attention to learning, validation, and aspects of modeling and simulation unique to time series. There is a lengthy learning curve and it includes becoming competent in both mathematics and programming. Do not be fooled by suggestions that these skills are unnecessary.
The best traders have already moved to machine learning. They would not have done that unless it gave an advantage over all other trading techniques -- including subjective trading and quantitative trading using only the decision tree model available in traditional development platforms. Large trading organizations are already employing thousands of highly educated, highly skilled system developers. There is only one trading arena. Trading is zero-sum at best. In order to be profitable, every trader must be able to compete profitably along side David Shaw, James Simons, and Goldman Sachs.
Best, Howard
3. I am making fewer trades these days. One of the reasons to stop trading is when you have enough. At my stage of life, I do not know what I would do with more money.
The best traders have already moved to machine learning. They would not have done that unless it gave an advantage over all other trading techniques -- including subjective trading and quantitative trading using only the decision tree model available in traditional development platforms. In order to be profitable, every trader must be able to compete profitably along side David Shaw, James Simons, and Goldman Sachs.
Hi again,
I'll number the parts I can address:
1. Is it better to just passively invest, if there is no possibility of alpha given the strong competition?
2. How do you explain the success of buffet type strategies who generally are not Quants?
3. What is the risk and reward profile of these quant type strategies?
4. The problem was that the insurer went bankrupt and COUNTERPARTY FAILURE ensued.
5. Until we reach/if we reach the singularity.
my twocents
My thoughts -- some that can be justified using the principles of mathematics, modeling, simulation, and statistics. Others not so easily.
1. Review my thoughts on investing versus trading versus expenses. If I make an investment, I have my lawyer write the details and disposition into my will. If there is a reasonable chance I will close out that transaction sometime before I die, it is a trade. Trades need rules. I need to know what conditions will cause the exit. If there are so few examples that I cannot use the scientific method of evaluating many in-sample examples followed by testing previously unused out-of-sample examples, then that transaction is subjective and no analysis is possible.
Passive investing, to me, means buying a stock or fund, putting it in my will, and forgetting about it. Not even another GFC would cause a sale. Passive trading, to me, means buying a stock or fund, expecting to sell it sometime, but not having clear rules. I am terrible at that. The first book I recommend to everyone is Daniel Kahneman's "Thinking, Fast and Slow." Dr. Kahneman explains how we fool ourselves.
Passive holding of any asset implies relatively long holding periods. Long enough that there will be substantial drawdown. An assumption of passive holding is that the drawdown will be short enough and shallow enough that it can be tolerated -- held through. Some people even think that is an opportunity to increase position size. In some cases, that works. The drawdown in 2009 was over 50%. Easy money policies by central banks bought a quick recovery. We will learn the true consequences of those policies in the next year or two. I expect another very steep drawdown. The drawdown in 1932 was about 80% and took over 25 years to recover.
Passive holding assumes that returns will compensate for risk and inflation. Much of the period those of us alive and trading today remember is that following WWII. There are many signs that we cannot expect the conditions that brought those gains to continue. For example, increasing globalization, increasing unrest, and serious climate changes.
2. Warren Buffett acts more like the president of a corporate conglomerate than a trader. Berkshire Hathaway buys large positions in companies expecting to hold them long periods. They experienced drawdowns of more than 50% in 2009. My risk tolerance is much lower. To my thinking, a trader holding through steep drawdowns is making a mistake. A better technique is exiting and returning later.
3. By quant strategies, I'll assume those similar to James Simons and David Shaw. When I was working in a hedge fund, we, and most funds, charged "2 and 20" -- 2% annually of notional funds under management and 20% of profits -- and our customers were profitable. Some of the best funds charged higher fees -- as high as 5 and 45 -- and still made customers wealthy. But funds that profitable are rare. And the 2 and 20 model is disappearing.
4. I recommend not trading anything that is not cleared by an independent central clearing agent. In over-the-counter trades, when I buy from you, you are my counterparty. If you fail to meet your obligations, I must deal with you directly. When trades are cleared by an independent central clearing agent, the clearing house is the counterparty to both traders. Be very wary of trading anything OTC.
5. I began working with artificial intelligence and machine learning in the 1960s. Some of the research I did in graduate school contributed to early work with neural networks and nearest neighbor analysis. Alan Turing posed the Turing Test in 1950, anticipating increasing capabilities of computer-based applications. The Turing Test has been convincingly passed; computers are now champions of every board and card game; artificial intelligent applications are better than human experts in almost all professions, including medicine, law, and logic.
Some people estimate that the singularity -- computer abilities surpassing human abilities in almost everything -- will happen soon. I agree with them. I believe the singularity is near.
I also believe the singularity will be harmful to the world, including to people. I imagine the similarities with being discovered by a superior society. Consider the Incas being discovered by Pizarro, or the US plains Indians by European settlers, or ...
--------------
I know -- this is more than you expected to hear. I hope it helps in convincing readers of the importance of following the scientific method in trading system development and in trading.
Thanks for listening.
Best, Howard.
My thoughts -- some that can be justified using the principles of mathematics, modeling, simulation, and statistics. Others not so easily.
1. Review my thoughts on investing versus trading versus expenses. If I make an investment, I have my lawyer write the details and disposition into my will. If there is a reasonable chance I will close out that transaction sometime before I die, it is a trade. Trades need rules. I need to know what conditions will cause the exit. If there are so few examples that I cannot use the scientific method of evaluating many in-sample examples followed by testing previously unused out-of-sample examples, then that transaction is subjective and no analysis is possible.
Passive investing, to me, means buying a stock or fund, putting it in my will, and forgetting about it. Not even another GFC would cause a sale. Passive trading, to me, means buying a stock or fund, expecting to sell it sometime, but not having clear rules. I am terrible at that. The first book I recommend to everyone is Daniel Kahneman's "Thinking, Fast and Slow." Dr. Kahneman explains how we fool ourselves.
Passive holding of any asset implies relatively long holding periods. Long enough that there will be substantial drawdown. An assumption of passive holding is that the drawdown will be short enough and shallow enough that it can be tolerated -- held through. Some people even think that is an opportunity to increase position size. In some cases, that works. The drawdown in 2009 was over 50%. Easy money policies by central banks bought a quick recovery. We will learn the true consequences of those policies in the next year or two. I expect another very steep drawdown. The drawdown in 1932 was about 80% and took over 25 years to recover.
Passive holding assumes that returns will compensate for risk and inflation. Much of the period those of us alive and trading today remember is that following WWII. There are many signs that we cannot expect the conditions that brought those gains to continue. For example, increasing globalization, increasing unrest, and serious climate changes.
Great post again.
I haven't seen a lot of evidence to show that passive doesn't work in the long term.
I suppose it is a bet on the future of the country.
Growth by population, efficiency from technology, better regulations etc etc
cheers
I'll comment just on passive versus active.
My concern is that, in my opinion, the future is unlikely to resemble the past. The US economy grew strongly following WWII, up through about 2000. The period since 2000 has been much weaker. .....
1. BY this thought process almost all historical back-testing is invalid.
2. As the assumption of back-testing is that the future will follow similar rules.
3. But don't forget we are not playing blackjack.....No one knows what cards are in the deck.
cheers
Greetings --
1. Correct. Almost all historical backtesting is without value in estimating future performance. The phrase I used was: profitable backtesting is necessary, but not sufficient.
2. Let me make a slight change in the terminology: Successful trading requires that the future resemble the past. That is one of the tenets of technical analysis, as well as machine learning.
In order for a trading system to be profitable in live trading, the system must be profitable for the out-of-sample validation period and into the live trading period. (It will have been profitable for the in-sample period too, but that is always true of a system that enters validation testing.) We rely on the conditions being stationary. That is, the future must resemble the past with respect to the signals and trade generated by the system. Watch my YouTube presentation on "The Importance of Being Stationary."
http://www.blueowlpress.com/video-presentations
3. Trading is like some gambling games.
Trading is similar to blackjack. Blackjack has a memory. The cards already played change the distribution of the cards yet to be played, and change the players odds of winning. An expert blackjack player does know the cards that remain in the deck, can compute the probability of winning and losing given those yet-to-be-dealt cards, can identify when the player has an advantage over the house, and can bet accordingly. My "Modeling" book has an analysis of blackjack as a business and compares trading as a business. Both require capital adequate to stay solvent through drawdowns.
But trading is not similar to roulette. Roulette has no memory. No history or sequence of recent winning pockets changes the probability of winning a bet on any pocket on the next play. The odds of a player winning is never in his or her favor. And no money management scheme can turn that negative-expectation game into a winning system. The "Modeling" book also describes roulette.
Best, Howard
My concern is that, in my opinion, the future is unlikely to resemble the past.
The deck in poker does not change.
Roulette is beatable and not always a negative ev game.
Edward O Thorp one of the main contributors to basic blackjack strategy
beat roulette as well...
cheers
We use cookies and similar technologies for the following purposes:
Do you accept cookies and these technologies?
We use cookies and similar technologies for the following purposes:
Do you accept cookies and these technologies?