LinkedIn Imports « KYRIAKOS ANASTASAKIS – ΚΥΡΙΑΚΟΣ ΑΝΑΣΤΑΣΑΚΗΣ

During the past few months I have been creating some EAs in MQL5. I have also created some microservice like functions for other FinTech related projects (e.g. Market News / custom indicators). All those services are exposed with REST APIs. So there is the need to reuse this functionality from MQL5 Expert Advisors.

There are numerous examples in MQL5 that use WebRequest, however it’s not possible to use them from within backtesting. An error is raised in such cases. The other obvious alternative was to use a custom DLL that will be used from within MQL5 and delegate the REST call to the DLL.

In this direction I have created an opensource library rest-mql and an example on how to use it through MQL5. The library can be found here: https://github.com/cyrus13/rest-mql

The example MQL code can be found here: https://github.com/cyrus13/rest-mql/blob/master/mql5-sample-code/testmqldll.mq5

It is worth mentioning that REST calls certainly slow down the EA and should be used judiciously.

I need to start a MySQL server in order to do some tests. Luckily MySQL has an official docker image. So in 3 simple steps:

Step 1: Download the Image

docker pull mysql/mysql-server:8.0.13

In this case I wanted to download a specific version of MySQL, 8.0.13.

Step 2: Run the Container

docker run -d --name mysql-8 -v /Volumes/USBHDD/mysql/data/:/var/lib/mysql/ -e lower_case_table_names=1 -e MYSQL_ROOT_PASSWORD=root -e MYSQL_ROOT_HOST=% -p 3306:3306 mysql/mysql-server:8.0.13

Explanation of params:

Parameter	Explanation
-d	Detached mode so that it runs the container in the background.
–name	Provide a friendly name to the container so that we can start/stop it in the future
-v	I want to store the MySQL data from the container to an external USB Volume mounted on the host at: /Volumes/USBHDD/mysql/data/. So all data saved by MySQL in will be available in the external USB drive. I believe the proper way is to do it using docker volumes, but this is a quick way as the title says.
-e lower_case_table_names=1	The -e parameter allows us to pass properties to the container process. lower_case_table_names=1 is required in a MacOS and Windows Host because the file system is case insensitive.
-e MYSQL_ROOT_PASSWORD=root	The password of the root user.
-e MYSQL_ROOT_HOST=%	By default the root user is only allowed to connect from ‘localhost’ (i.e. from within the container). I want to be able to connect as root from the host machine, that’s why this option is required. Note: This is obviously a security hole, but for a dev environment it should be ok.
-p 3306:3306	Map port 3306 (the default MySQL port) from container to host.

Step 3: Connect to container from host

Using MySQL Workbench from the Host machine and connection params:

Host: 127.0.0.1
Username: root
Password: root

we can connect to the MySQL running on the container and create a new DB.

That’s it! Now we may want to stop the container.

Step 4: Stop the container

docker stop mysql-8

Step 5: Start the container again

docker start mysql-8

Connecting again using the MySQL Workbench we can see our data and continue development.

Today is the day of the British EU referendum and I decided to express some random thoughts while waiting for the outcome. Polls and the bookmaker odds / market pricing (of the sterling, the FTSE 100) show mixed messages. Polls reveal a very close call (50/50), while the bookmaker odds and the pricing of the market reveal a strong belief that the remain vote will prevail (75% for remain/25% for exit).

These two mixed messages remind me of the famous line from Naked Gun. Nordberg (O. J. Simson) is severely injured in hospital and his colleagues discuss:

Doctors say that Nordberg has a 50/50 chance of living, though there’s only a 10 percent chance of that.

So the Brexit referendum is all about the polls, or should I say the Poles? It’s primarily about internal EU immigration. The Brexit supporters claim that currently people from poorer member states can easily move to richer states and work or claim benefits, essentially moving money out Great Britain. Talking about going into a country, taking its wealth and moving it back to your home country, I can’t help but think those Brexit supporters have never visited the British Museum.

The Brexit supporters also claim that EU is increasing the bureaucracy in Great Britain. Having lived in the UK for 5 years, I am a bit puzzled. I remember we had to fill in a risk assessment form every time we wanted to play basketball at Uni. I also remember lecturers at Uni. complaining they had to spend hours preparing and attending useless committees and doing less research. In Greece which is notoriously bureaucratic I when it comes to your dealings with the public sector things are much more straight forward in everyday life. So I don’t for the everyday bureaucracy in the UK, the EU is to blame.

Experts say if Great Britain decides to exit it will have to worry more about the Scots and Irish the next day, than for its future relations with the EU.

The EU is definitively broken and it needs fixing before it falls apart. All member states face similar issues and Brussels doesn’t react (at least effectively). If history has tough us anything though, leaving Germany as the only primary ruling power in Europe has never had a happy ending. The EU needs Britain to help shape its future.

Is there really a 25% chance of Brexit?

Going back to the fact the polls give 50% for Brexit, while the bookies only 25%. Either the polls, or the bookies (or both for that matter) have to be wrong. One argument is that the polls can’t be trusted because there is no history on such a referendum. But what about the bookies? In addition to the “fair odds” (what a bookie decides to be an objective probability for a result) they also adjust the odds based on their exposure to the outcomes, to minimize risk.

Assume that you are a bookie and at the start of two events A and B you think there are 50/50 chances. So (assuming you gain nothing for yourself) you give odds of 2 for A and odds of 2 for B. During the course of the event you have asymmetric wagering (e.g. $1M is wagered on A and $250K is wagered on B). Assume your view as a bookie is still 50/50 for A and B, but if the outcome is A, you owe: $2M (net loss of $750K) while if the outcome is B you owe $500K (net profit of $750K). So as more money is wagered on A, the bookie will naturally lower the odds on A and increase the odds on B, purely to minimize the risk and attract more money on event B. So for example the bookie may adjust odds for A to 1.5 and odds for B to 3. It doesn’t mean the bookie thinks there is a 66.6% change for A and 33.3% chance for B. The bookie merely lowers the odds of A for risk management purposes.

A blog post by Landbrokes that in essence the amount of money wagered on remain is responsible for the low odds of remain, validates this claim.

Let’s also have a flashback one year ago at another referendum, the Greek one. If you study both the polls and the bookies, it’s a deja vu. Polls showed it was a close call, while the bookies gave odds that attributed probabilities of around 64% on Yes (around 1.5 decimal odds). Guess what happened? Greeks voted No, bookies were “wrong” (as explained in the previous paragraph there is no right or wrong, it’s all about risk management) and whoever saw the valuebet and bet on No made some good money.

Let’s have another flashback, two years ago at the Scottish referendum. Again the polls gave it a close call, but bookmakers were much more certain that the remain vote would prevail. As explained by this financial times article, there is much resemblance in the betting patterns of the Brexit referendum with the Scottish referendum.

So, for the Brexit referendum will the bookies get it right like the Scottish referendum or will they get it wrong like the Greek one? Difficult to tell until the result is out. Will the pollsters get it right that it’s going to be a close call? A blog post by financial times suggests that (a) pollsters are subject to herding and (b) if you count the tendency of the undecided voters to vote for the status quo and (c) take into account that the leave campaigners are much more “loud”, the remain has more than 50% chances.

Odds of around 4 for the leave vote are certainly a value bet and even if you think that there is as low as 30% probability of leave, the Kelly criterion suggests you should bet 6.6% of your bankroll on leave. You should be betting on the poor 1.3 odds of remain only if you think it has at least 75% chance of happening. Does it? No idea! I personally think remain has more chances and there are decent odds for the handicap (50-55% or 55-60%) of the remain.

I have recently started using MongoDB for a very demanding task. Storing all Forex pair ticks (e.g. every change in bid/ask prices). I know there are frameworks designed for this task (e.g. kdb+), but since I wanted to avoid the learning curve. Besides I already use Spring Data in my project and it works with a minimal number of changes for Mongo.

In Mongo I have a collection with more than 3.5 billion records (and growing) and I want to find the latest date for each pair. I tried using the aggregation framework of Mongo, but it doesn’t seem to use the indexes and takes ages (didn’t finish after one day).

Relational Structure

In relational DB the table structure would look something like:

id	pair	dateTime	bid	ask
1	EUR/USD	2015-04-03 21:32:31.456	1.14141	1.14142
...		...	...	...

Then you would have to run the following query:

SELECT  t.pair, MAX(t.dateTime)
FROM tick_data t
GROUP by t.pair;

MongoDB Aggregation Framework

In MongoDB the document structure is the same. I am a very very novice user of Mongo, but I gather we could use the aggregation framework for this query:

db.tick_data.aggregate(
    {$group:{_id:"pair", "maxValue": {$max:"dateTime"}}}
);

However, this takes ages, even though I have used a composite index on pair and dateTime.

Very Fast Result Using MongoShell

I tried using a sort of iterative approach using MongoShell:

db.tick_data.distinct( "pair" ).forEach(function(per_pair) { 
  var lastTickPerPair =  db.tick_data.find({ "pair": per_pair }).sort({"dateTime": -1}).limit(1);
 var lastTickOfPair = lastTickPerPair.hasNext() ? lastTickPerPair.next() : null;
  print( "pair: " + lastTickOfPair.pair + ", dateTime:" + lastTickOfPair.dateTime); 
  } 
 );

This approach seems to use the composite index on pair and dateTime I defined and the results are lightning fast (for 3.5 billion records).

Maybe there are other ways, but after some digging around I couldn’t find any other method that would use indexes.

KYRIAKOS ANASTASAKIS – ΚΥΡΙΑΚΟΣ ΑΝΑΣΤΑΣΑΚΗΣ

Category Archives: LinkedIn Imports

Call REST endpoints from MQL5 during backtesting

Quickly starting a MySQL server using docker