Python - getting started

[howardbandy]

Hi guys,

The next step would be to use this arrays to generate signals and try create a backtesting model. Again I don't think it will be do difficult to generate signals. The back test model may be more difficult.

Hi JJ --

Replicating the capabilities of AmiBroker is a worthwhile project if the goal is refreshing programming skills. But not if the goal is to improve on AmiBroker. The python replica of AmiBroker will be less efficient than Tomasz' AmiBroker by an order of magnitude or worse. It will not have all of the AmiBroker features. And it will still only use the decision tree model.

Anecdotally -- Years ago, in the dark ages of the 1990s, I wrote a trading system development platform that was sold commercially. I was in the process of designing another one, to be better in many ways. A colleague asked me one day whether I wanted to spend my time and energy supporting development platforms -- or developing trading systems and trading. Each is a full time job in its own. He introduced me to AmiBroker. Which already included 95% of my list. After a few conversations with Tomasz, AmiBroker had everything I needed and 98% of what I wanted. I could focus on trading systems and leave the platform maintenance (and customer service) to someone else.

As you work in python, be sure to check out the pandas libraries. Pandas arrays ("Series" and "DataFrames") are based on numpy arrays. They handle many of the data access and data manipulation routines that are commonly used in trading systems -- reading data from a variety of sources, missing data, many time-related functions, moving window functions, etc. In my opinion, pandas DataFrames are essential to python-based trading system platforms. Pandas is standard in the Anaconda distribution of python (which is the distribution I recommend). A web search will locate many excellent tutorials, video presentations, and books. Any material by Wes McKinney or Jake VanderPlas is excellent.

If it would be of use to you -- everything you outline you are doing in python to replicate AmiBroker has been published in my QTA book, and is ready for download and use as a template for improvement.

After the AmiBroker-replica project, I highly recommend continuing trading system development using machine learning. One of the major advantages is the ability to use any of several dozen different models, including but not limited to decision tree. Also explained in QTA.

Best regards, Howard

 

[howardbandy]

I second everything Dave says.

 

[lmz4005]

Hi guys,
Interesting post indeed.
Anyone here from Brisbane Region that could help me to get started in a basic Python trading system back-testing (which I have got the Python code for) on SP500?
Thanks
Laurent

 

[Trembling Hand]

Hi guys,
Interesting post indeed.
Anyone here from Brisbane Region that could help me to get started in a basic Python trading system back-testing (which I have got the Python code for) on SP500?
Thanks
Laurent

Where did it come from?

 

[lmz4005]

Clenow SP500 Momentum trading

I have managed to convert from Premium data the SP500 but it does create one file per stock in the SP500 (about 505 files). Not sure how to import them all in Panda!

I have managed to import 1 file in panda, but not very efficient process

Should I look at Quandl datas?

 

[Trembling Hand]

So you are trying to do a portfolio test in python. That is test a system on a basket of stocks (S&P500) with position sizing and performance metrics etc?

 

[Trembling Hand]

To get all data from multiple csv files in one data frame you would use something like this,

import os
import glob
import pandas as pd


path = r'C:\Users\Yoda\Documents\Data'
all_files = glob.glob(os.path.join(path, "*.txt"))
df = pd.DataFrame()
for file_ in all_files:

file_df = pd.read_csv(file_,sep=',', parse_dates=[1], infer_datetime_format=True,header=None )
df = df.append(file_df)​

 

[lmz4005]

yep

 

[lmz4005]

hummm this is what I get:
path = r'C:\Trading Data\Stocks\'
^
SyntaxError: EOL while scanning string literal

 

[lmz4005]

code is
path = r'C:\Trading Data\Stocks\'
all_files = glob.glob(os.path.join(path, "*.csv"))

 

[Trembling Hand]

Most probably there is a problem above that line. I have sent you a PM. The chances of debugging your setup line by line is.......

 

[lmz4005]

fixed. now I need to find a way to download all prices for all stocks from SP500. any ideas?

 

[Trembling Hand]

I really don't understand why you are persisting with a language you don't know and one that has little infrastructure for portfolio testing while you do know AFL and is actually what you are set up for, you even have the paid data? It's like someone offers you a job in the next village you can get there with a flat walk 10 ks up the road or hike over an icy mountain with vertical rock faces and you are saying, "cool I'll take the mountain way"!!. What am I missing?

 

[Gringotts Bank]

How long before there's a point-and-click retail version of machine learning software? Something like Watson's Analytics for financial markets?
https://www.fastcompany.com/3062951/a-new-point-and-click-revolution-brings-ai-to-the-masses

 

[Trembling Hand]

How long before there's a point-and-click retail version of machine learning software? Something like Watson's Analytics for financial markets?
https://www.fastcompany.com/3062951/a-new-point-and-click-revolution-brings-ai-to-the-masses

There already is. Weka and orange. Been around for some time too.

But scalpels have been around for a long time as well, doesn't make everyone a brain surgeon though.

 

[lmz4005]

I think so too Gringotts Bank. I am only learning this the hard way because I find it interesting from a brain challenge point of view. However, the ROI is so poor that the days of the nerds might be counted thanks -or because- to AI capabilities and technology.

 

[fiftyeight]

I really don't understand why you are persisting with a language you don't know and one that has little infrastructure for portfolio testing

I thought you were a fan of Python?

 

[Trembling Hand]

I thought you were a fan of Python?

Yeah I'm really getting into it but the more i do the more you learn what you didn't know before. You then realise how unrealistic and improbable the chances are of people without domain knowledge being able to use the tools correctly and framing and asking the right questions.

But thats not unique to ML.

 

[fiftyeight]

But thats not unique to ML.

This is just getting started in Python though, ML is next level as I understand it.

I may never be an amazing programmer but my aim is not write amazing programs. I want to be able frame a question and then have a look in to it. If I have a light bulb moment this could change though I guess.

Is Python the wrong tool for this?

 

[Trembling Hand]

Hard to say. If you are looking at 1 market at a time with data in a flat file then its pretty easy language to learn and use. But it could be argued that AFL or NT etc are the same but offer greater ease as you go deeper because they have the structure already sorted.