Tips to get started on automating

21 posts / 0 new
Last post
volexp
Tips to get started on automating

Hi all, I have what looks like a profitable trading strategy that I have backtested in Excel. While I am very proficient in Excel, I am at the opposite end of the scale in programming.

Does anyone have tips on how to get started on the automation phase from here?

Anything that you think will point me in the right direction will be very much appreciated.

Thanks,

Andrew

neilcaithness
Matlab App for automated sprint trading

I hope to have a complete automation tool available quite soon. It's a Matlab App so requires a Matlab license, which isn't as expensive as it used to be. It's designed specifically to work with the IG Forex Sprint Markets, and will be available by subscription with different rates for trading on your Demo vs Live accounts.

There are just two user defined functions: one specifies what data processing you want to happen when new tick data arrives, and the second generates a Buy, Sell, or DoNothing signal. These are Matlab functions with working examples, so it's very powerful but quite straightforward at the same time. The trade signal function for instance could act on a simple indicator like a MACD crossover, or it could be more complicated and apply a machine learning classifier trained on historical data collected by the App. All the rest, streaming, trading, etc., is handled in the background by the App. I've posted a screenshot preview here http://zebtrading.co.uk/

If you think you'd be interested I can let you know when it's ready for release.

f2calv
ermm

Neil,

Not wanting to ruin your day but the T&C of the IG API are pretty clear in that the Web API is for personal use only, so your subscriptions idea may well not work out.

I have also written a technical analysis application using the IG Web API and have recently been in communication with Emily Ashton at IG Institutional Sales regarding getting permission to allow other IG clients to use my TA app. And after quite a few emails back and forth the final response today was pretty much as expected;

"I’m really sorry for the mixed messages, I’ve spoken to a few different people in the business and got some conflicting information.

Essentially, at the moment we can’t give you permission to use our web API for purposes other than for your own personal use.

We have developed an option for regulated firms to do this but as an individual we don’t have a drive yet to allow this. However, in the future it may be available and if so we will try to update you."

Thankfully with a bit of rewriting I can release my TA code using a different brokers data feed.

Cheers and good luck!

neilcaithness
Hi Alex

Hi Alex

Thanks for your feedback on this. I had until now thought the interpretation was clear, it is of course very important to get it right. I think the question must be, who is using the API? To me the answer has always been crystal-clear, the IG account holder. If I write software that makes calls to the API, it is the user of the software who is then making use of the API with their credentials, so it's the user who must comply with the personal use condition.

It's clear why IG needs to have the personal use condition: as a non-regulated individual I can't provide professional financial services. So, in turn, IG can't provide API access to individual accounts for professional use. In my case I would be providing a piece of software that does not encapsulate any professional financial service - any trading algorithm is entirely defined and developed by the user, it's their intellectual property, their risk, their account, and they have to comply with their agreement on T&C's with IG. I can't see that it matters here whether the software is sold, made available by subscription, or given away.

If on the other hand, the software were to use the API to access my account, say to provide price data via the streaming API to a 3rd party user, then that certainly would not comply with the personal use condition.

I'll be very glad to hear your thoughts now.
Regards
Neil

f2calv
T&C

Neil,

First a link to T&C would of helped in my earlier post;

http://www.ig.com/content/dam/publicsites/igcom/uk/files/other/api_terms...

I think you make very sound points, I totally agree its the users risk when running any such software. To be honest I was dubious about letting others use my app for trading as the API isn't as robust as I'd like, however for pure technical analysis I don't see the issue. However the T&C are quite clear you can't distribute without permission;

"
(2) You warrant and represent the following to us:
(a) you will use the API Connection for your own personal use, for the purposes of entering into Transactions with us;
(b) you will not use the API Connection for any commercial purposes, including for the purposes of publishing or distributing Data available from the API Connection to third parties or for the purposes of communicating with us on behalf of other clients of ours;
(c) any application or software that you use in conjunction with the API will be software/application that is developed by you and not a third party; and
(d) you will not use an API Hub to communicate with us via the API Connection, unless you first ask our permission and unless
"

Point (c) is obviously the clear kicker here. Do email institutionalsales@ig.com and get further confirmation and keep me in the loop via this thread...

neilcaithness
Alex

Alex

Thanks. Point c. above is frankly absurd, to the extent that it is meaningless and threatens the validity of the rest of the T&C's. It says, clearly, that to use the API in software (what other way is there?) you must have written even the operating system yourself.

You can't even use any of the libraries or tools provided by IG because you didn't write them yourself. The statement says essentially that no shared-software can be used whatsoever. That is a statement so divorced from reality it can't be interpreted.

How are we going to get some clarification?

Neil

f2calv
clarification

Re: How are we going to get some clarification?

I would suggest raising the point with your IG account manager, get them on-side, and then contacting IG Institutional Sales yourself and getting the full details from them firsthand - and hopefully your situation is different to mine?

neilcaithness
Alex

Alex

Quick question: where do you see the statement regarding permission, "the T&C are quite clear you can't distribute without permission"?

Cheers
Neil

f2calv
permission

Neil,

The text below is taken from the IG T&C, all the points (a thru d) pretty much complement one-another towards developers being unable to distribute their applications - you can write it and use it yourself, but that's it.
And in addition as I already said I've been in contact with IG directly and they flat out refused me permission to distribute - *unless* I was a company regulated by the FCA.

Please don't take my word on it and ask IG yourself - the more people who directly ask them for permission then all the more likely they are going to change their position!

"
(2) You warrant and represent the following to us:
(a) you will use the API Connection for your own personal use, for the purposes of entering into Transactions with us;
(b) you will not use the API Connection for any commercial purposes, including for the purposes of publishing or distributing Data available from the API Connection to third parties or for the purposes of communicating with us on behalf of other clients of ours;
(c) any application or software that you use in conjunction with the API will be software/application that is developed by you and not a third party; and
(d) you will not use an API Hub to communicate with us via the API Connection, unless you first ask our permission and unless
"

djas
Matlab app

Hi Neil,

I am potentially interested in your Matlab app if it could be used for implementing an automated algorithm for spreadbetting shares. I have the specification of my algorithm but need a tool for operating it. Although I have some idea about programming I am just getting started in understanding the world of automated trading.

Please would you let me have a contact email so that we can discuss your offering

Many thanks, David

neilcaithness
Hi David

Hi David

Thanks for your interest. The Matlab App I described is I think preciously what you want: you have an algorithm, it's your intellectual property, but you need a shell environment in which it can operate. But let me be clear about an important point: I don't offer, and my program does not contain, any financial services, technical analysis or trading advice. All trading decisions are derived from your implementation of your own algorithm. What my App provides is the software shell that allows the program of trading decisions that you write, to operate.

Also, you may want to know that the current version of the App is designed to work with the IG Forex Sprint Markets only (i.e. the eight major pairs: AUD/USD, EUR/USD, GBP/USD, USD/JPY, EUR/JPY, GBP/JPY, USD/CHF and USD/CAD).

There are many advantages to this, a few of the significant ones as follows:
1. 24 hour trading.
2. Single entry with an automatic period exit at 1, 2, 5, 20 or 60 minutes.
3. Profit/loss determined only by the direction of price movement, not by the magnitude of price movement.

If you have an algorithm for spread betting on shares, perhaps you can see how it could also work for sprint binaries.

Regarding contact, I’m very happy for you to contact me, but the Community Guidelines for IG Labs says the following:

'Please use a community screen name/first name (no surnames).' Unfortunately I’ve been using my full name since I first signed up, and the profile page for users give First and Last names anyway.

And,

'Please don't give away specific personal details that make it easy for others to identify you in real life (such as your own specific business names/addresses/sites).' But just below the box I'm typing this reply in is a line inviting email addresses, 'Web page addresses and e-mail addresses turn into links automatically.' So I don't know if it's allowed or not really.

Regards
Neil

volexp
Thanks Neil,

Thanks Neil,

However, I'm looking for tips on how to do this myself and would like to be pointed in the right direction for starting.

Does anyone have any knowledge they can share? I want to create an automated trading strategy to tie into IG's spreadbetting market. What software is needed? What computer coding language do I need to learn? Is there any literature you can recommend?

Information like that would be a great help!

Thanks,

Andrew

sidewinder
My 2p on automation

I have used Python for all of my experiments with the IG API. Using the json and requests module make accessing the REST API trivial.

http://docs.python-requests.org/en/master/

On the prices front it might be worth asking if you need all prices and changes via the streaming API or a periodic snapshot using the REST API. The latter is much easier to get going but there are number-of-requests-per-minute restrictions, see the FAQ: http://labs.ig.com/faq

I use a Mac laptop and a Linux desktop and the same Python code runs on both within a terminal with no problems. It almost certainly will run on Windows as well but I have not yet tried.

volexp
Thanks

Hi sidewinder,

Thanks for your reply. I wonder if you could tell me before I start whether I can do the following using Python, json, IG API, etc. Basically, I would like to be able to automate a trading strategy which uses open-high-low-close data on highly liquid FX pairs on 30-minute to 4-hourly markets, creates simple indicators like Bollinger bands, momentum, etc. to identify entry and exit prices and will send trade requests automatically to open and close positions once certain, relatively simple criteria are met. I would also like to do this on the standard spreadbetting markets, not Sprint markets, etc.

I have this fully formulated in Excel where I choose my preferred market, indicator settings, timeframe, etc. from dropdown menus and out pops the full backtest with all the relevant stats and charts. It looks great, but is ultimately useless unless I can automate it!

Sorry for picking on you :)

Thanks,

Andrew

sidewinder
Excel/Python

Short answer is yes, you can do pretty much anything with something like Python.

From what you've said on this thread though it might be a better idea to explore leveraging Excel to automate things for you. I understand there are some Excel samples which I'm sure we could get working to bring data in to your existing Workbook. I'm guessing you have a sheet somewhere where you have the OLHC data which you'd like to have loaded in on a timer rather than being manually copied and pasted?

I'd be happy to give it a try at least to the point of a proof-of-concept. Maybe drop me a personal-message and we can discuss.

f2calv
how long is a peice of string?

To go from discretionary to semi/fully automated systematic trading is a massive undertaking - and an even bigger undertaking if you have no serious programming skills to start with in the beginning.

As far as languages are concerned, the IG API utilises REST over HTTP meaning you can pretty much use any programming language which can initiate a HTTP connection, i.e. Python, Ruby, c, c++, Microsoft .NET, PHP, Perl, Matlab, etc...

If you have thousands of hours spare to a) learn to code and b) learn to automate your strategies then I'd start reading the following blogs (both of whom are published authors of algorithmic trading books);

http://epchan.blogspot.co.uk/
http://qoppac.blogspot.co.uk/

If after reading those blogs you are still none the wiser about how to get started then I would suggest automated trading isn't something you should spend time on.

You can find more Q&A sessions on this forum (for a less IG-centric viewpoint);
http://www.elitetrader.com/et/index.php?forums/automated-trading.48/

Good luck!

neilcaithness
Nice answer. I was wondering

Nice answer. I was wondering how to respond.

volexp
Brilliant, that's very useful

Brilliant, that's very useful information and will give me the start I need. I have a profitable trading strategy, so why wouldn't I put significant effort into automating it? Nothing in the world is worth having or worth doing unless it requires effort. I have never envied a person who has led an easy life!

Thanks again for your help.

neilcaithness
Let me add to Alex's very

Let me add to Alex's very useful answer.

Don't take the choice of language lightly. All those mentioned (and many more) will do the job, but with very different development paths and very different end points.

You'll be developing three very different kinds of things:

1. A trading mechanism that acts on your signals and uses the REST Trading API to submit trades to your account.

2. A data streaming mechanism that uses the Streaming API to gather the relevant market and trade data in real-time that your strategy requires.

3. A mechanism for implementing your strategy/algorithm that uses the data stream in 2, and feeds a signal to your trading mechanism in 1.

You could (unlikely, but you could) use a different language for each of the above, but more likely you'll choose one. At the end of the day, 3 above will be the most important to you. That's where you'll be doing new things and trying new ideas. That's where you'll want the most high-level computational and language features.

To be honest, 1 and 2 above are where (initially) you'll write 99% of your code. But when it's done, neatly and elegantly implemented and fully documented (because you will forget how the thing works) then you'll want it to just sit in the background, work like a Trojan and never complain, while you get on with the interesting bit of making your strategy work.

So, I suggest you choose based (say 90%) on 3 above. I chose Matlab, and would do so again, even though the LightStreamer API in particular was a bit quirky to get going. For the REST API, it doesn't really matter what you choose, but your choice will definitely affect how you get on with 3. Try to login to your account using the API in several of the languages mentioned, and then think about how you would implement your strategy in each of them.

When you have questions, you'll find that folks on the forum are very helpful.

tusasirw
VolexP, VolexP!!

VolexP, VolexP!!

What you are thinking about is noble: I have ever thought about it myself.

But look. You said you are low on programming scale. As far as I understand what you are asking is far more complex. Why not begin by making simple programming like an API which is semi-automated. Whether you do it in JAVASCRIPT Or whatever language then u can think of making it fully automated.

Personally, I have a REST API Client in PHP. It works but could not achieve my original aims for a simple reason: it is hosted far away from IG servers. Simple. So it cant be fast enough. Hope you know sprint options: they need good speed.

Your idea I have ever thought about. And this was my conclusion: do the strategy in MT4....let it then send an http request to IG via the API. Hehehehe....you said you dont know programming. Already I have mentioned u need core skills in Javascript(like AJAX), maybe PHP; and now MQL4.

mvddoes
Hello Andrew,

Hello Andrew,

I noticed your message at this community and I am curious to hear about your profitable trading strategy.
I recently developed an application using the IG rest API webservices which makes it possible to automate trading strategies. In that application it is also possible to save all streams that are received, so all rate-changes (ask and bid) are saved. This way it is possible to test out a strategy and see with real data if it works.
Are you willing to share your trading strategy? I can test it for you then with real data.

Regards,
Maurijn van der Does
(the Netherlands)

Log in or register to post comments