Today is the day of the British EU referendum and I decided to express some random thoughts while waiting for the outcome. Polls and the bookmaker odds / market pricing (of the sterling, the FTSE 100) show mixed messages. Polls reveal a very close call (50/50), while the bookmaker odds and the pricing of the market reveal a strong belief that the remain vote will prevail (75% for remain/25% for exit).

These two mixed messages remind me of the famous line from Naked Gun. Nordberg (O. J. Simson) is severely injured in hospital and his colleagues discuss:

Doctors say that Nordberg has a 50/50 chance of living, though there’s only a 10 percent chance of that.

So the Brexit referendum is all about the polls, or should I say the Poles? It’s primarily about internal EU immigration. The Brexit supporters claim that currently people from poorer member states can easily move to richer states and work or claim benefits, essentially moving money out Great Britain. Talking about going into a country, taking its wealth and moving it back to your home country, I can’t help but think those Brexit supporters have never visited the British Museum.

The Brexit supporters also claim that EU is increasing the bureaucracy in Great Britain. Having lived in the UK for 5 years, I am a bit puzzled. I remember we had to fill in a risk assessment form every time we wanted to play basketball at Uni. I also remember lecturers at Uni. complaining they had to spend hours preparing and attending useless committees and doing less research. In  Greece which is notoriously bureaucratic I when it comes to your dealings with the public sector things are much more straight forward in everyday life. So I don’t for the everyday bureaucracy in the UK, the  EU is to blame.

Experts say if Great Britain decides to exit it will have to worry more about the Scots and Irish the next day, than for its future relations with the EU.

The EU is definitively broken and it needs fixing before it falls apart. All member states face similar issues and Brussels doesn’t react (at least effectively). If history has tough us anything though, leaving Germany as the only primary ruling power in Europe has never had a happy ending. The EU needs Britain to help shape its future.

Is there really a 25% chance of Brexit?

Going back to the fact the polls give 50% for Brexit, while the bookies only 25%. Either the polls, or the bookies (or both for that matter) have to be wrong. One argument is that the polls can’t be trusted because there is no history on such a referendum. But what about the bookies? In addition to the “fair odds” (what a bookie decides to be an objective probability for a result) they also adjust the odds based on their exposure to the outcomes, to minimize risk.

Assume that you are a bookie and at the start of two events A and B you think there are 50/50 chances. So (assuming you gain nothing for yourself) you give odds of 2 for A and odds of 2 for B. During the course of the event you have  asymmetric wagering  (e.g. $1M is wagered on A and $250K is wagered on B). Assume your view as a bookie is still 50/50 for A and B, but if the outcome is A, you owe: $2M (net loss of $750K) while if the outcome is B you owe $500K (net profit of $750K). So as more money is wagered on A, the bookie will naturally lower the odds on A and increase the odds on B, purely to minimize the risk and attract more money on event B. So for example the bookie may adjust odds for A to 1.5 and odds for B to 3.  It doesn’t mean the bookie thinks there is a 66.6% change for A and 33.3% chance for B. The bookie merely lowers the odds of A for risk management purposes.

A blog post by Landbrokes that in essence the amount of money wagered on remain is responsible for the low odds of remain, validates this claim.

Let’s also have a flashback one year ago at another referendum, the Greek one. If you study both the polls and the bookies, it’s a deja vu. Polls showed it was a close call, while the bookies gave odds that attributed probabilities of around 64% on Yes (around 1.5 decimal odds). Guess what happened? Greeks voted No, bookies were “wrong” (as explained in the previous paragraph there is no right or wrong, it’s all about risk management) and whoever saw the valuebet and bet on No made some good money.

Let’s have another flashback, two years ago at the Scottish referendum. Again the polls gave it a close call, but bookmakers were much more certain that the remain vote would prevail. As explained by this financial times article, there is much resemblance in the betting patterns of the Brexit referendum with the Scottish referendum.

So, for the Brexit referendum will the bookies get it right like the Scottish referendum or will they get it wrong like the Greek one? Difficult to tell until the result is out. Will the pollsters get it right that it’s going to be a close call? A blog post by financial times suggests that (a) pollsters are subject to herding and (b) if you count the tendency of the undecided voters to vote for the status quo and (c) take into account that the leave campaigners are much more “loud”, the remain has more than 50% chances.

Odds of around 4 for the leave vote are certainly a value bet and even if you think that there is as low as 30% probability of leave, the Kelly criterion suggests you should bet 6.6% of your bankroll on leave. You should be betting on the poor 1.3 odds of remain only if you think it has at least 75% chance of happening. Does it? No idea! I personally think remain has more chances and there are decent odds  for the handicap (50-55% or 55-60%) of the remain.

I have recently started using MongoDB for a very demanding task. Storing all Forex pair ticks (e.g. every change in bid/ask prices). I know there are frameworks designed for this task (e.g. kdb+), but since I wanted to avoid the learning curve. Besides I already use Spring Data in my project and it works with a minimal number of changes for Mongo.

In Mongo I have a collection with more than 3.5 billion records (and growing) and I want to find the latest date for each pair. I tried using the aggregation framework of Mongo, but it doesn’t seem to use the indexes and takes ages (didn’t finish after one day).

Relational Structure

In relational DB the table structure  would look something like:

idpairdateTimebidask
1EUR/USD2015-04-03 21:32:31.4561.141411.14142
............

Then you would have to run the following query:

 

MongoDB Aggregation Framework

In MongoDB the document structure is the same. I am a very very novice user of Mongo, but I gather we could use the aggregation framework for this query:

However, this takes ages, even though I have used a composite index on pair and dateTime.

 

Very Fast Result Using MongoShell

I tried using a sort of iterative approach using MongoShell:

This approach seems to use the composite index on pair and dateTime I defined and the results are lightning fast (for 3.5 billion records).

Maybe there are other ways, but after some digging around I couldn’t find any other method that would use indexes.