A Slightly Different Take on High Frequency Trading (HFT)
My guess is that HFT will soon become one of those bogeyman words that people automatically associate with "bad stuff" without ever actually understanding what it means. But it is worth understanding the underlying problem, and that problem is not high speed or frequency per se.
As I understand it, when an order to buy, say, 10,000 shares of Exxon gets placed, the purchase will get pieced together by searching across multiple servers where offers are listed and putting together the 10,000 shares in bits and pieces from these various servers. What HFT's are doing (and I am sure this is grossly oversimplified) is that once it sees this order pinging a server, it runs ahead at high speed to other servers and buys up blocks of Exxon at price A and then offers it up to the pokey buying search when it finally arrives at those servers at A+a bit more. That "a bit more" may be less than a penny, but the pennies add up and if done right, there is almost no trading risk.
This is bad, though generally not for us small investors but for our mutual fund companies. For my little trade of 100 shares that might be cleared on the first server, HFT's have no opportunity to play. Moreover, I may not even notice a penny or two difference in the price I get. This is a much bigger deal for mutual fund companies and large investors clearing larger trades, where a few pennies can add up to a lot of money.
An exchange always has to be really careful to maintain its image of fairness, and systematically allowing such behavior, called front-running, is not good for the health of the market. Which is why you are hearing a lot about this.
Here is what you are not hearing, and I will admit that it is all a hypothesis of mine. But it may well be possible that HFT's actually reduce the total cost of front-running to investors. It may be that HFT's real crime is that what they are doing is more transparent and visible than what market makers were doing in the past -- ie they are not increasing the volume of front-running, they are just making it more obvious. I would not be at all surprised if such front-running always existed in market-making (certainly Goldman Sachs has been accused of it) and that HFT's are actually the Wal-Mart or Amazon of front-running -- not doing anything new but doing it cheaper on tighter margins. Kind of ironically, I suppose this is what efficient markets theory would predict for the market in front-running.
If this is the case, while we would rather see front-running eliminated entirely, HFT's may actually be reducing the cost of front-running and making things more rather than less efficient.
Rondo:
Interesting video regarding HFT
http://www.zerohedge.com/contributed/2013-11-04/wall-street-code-released
April 2, 2014, 8:54 amShane:
It was my original understanding of HFT that the way that they were making money was between the bid/ask. By executing faster the HFT could become the market maker, thus making millions of transactions at fractions of a cent that would using the law of large numbers to put a lot of money in their owners pockets. As an amateur trader I have seen lots of little snippets of news that are saying that the market for this activity is flooded and it is hard to make money doing it unless you have an insanely fast execution speed. Hence the need to be close to the exchanges.
It is possible that a new variant for HFT would be a type of front running, but I don't see this providing enough volume to really have the law of large numbers kick in especially when big orders in the HFT world are a very rare event. Plus the speeds to other ECN's are going to be a bottleneck in my view and definitely a limiting factor.
I don't have facts to come right out and say you are off base, but what you are suggesting doesn't jive with my experience. I may be wrong. It wouldn't be the first time in my life.
Shane
April 2, 2014, 9:05 amXmas:
More Zerohedge stuff, an excerpt from Michael Lewis's book. I think you'll like this, as it explains the "problem" and why it's very bad.
http://www.zerohedge.com/news/2014-03-31/read-michael-lewis-flash-boys-wall-street-revolt-adapatation
April 2, 2014, 9:10 amcbyrneiv:
The HST aspects of HFT aren't the problem, it's the frequency driving price that's the problem, particulary in interaction with various other algorithmic/automated trending and trading.
HFT can have the effect of a pump and dump if exploited.
April 2, 2014, 9:13 amShane:
Indeed interesting. Could do without the conspiracy theory stuff. One thing that was said struck a nerve with me was "guaranteed profits". There is no such thing. See this. One of the owners of the company that I linked was none other than Myron Scholes. This guy was a quant before quants had computers.
April 2, 2014, 9:14 amXmas:
To sum up, there are three problems with HFT.
1) Phantom Orders - When the broker hits enter to buy or sell a lot of stocks larger than the lots available, HFT will swipe some of those lots and sell them at a small price higher or lower. (Public Frontrunning).
2) Exchange Rebate Arbitrage - certain exchanges offer rebates for using their system, other exchanges charge money. HFT will capture the rebate while forcing humans to exchanges that charge money.
3) Blackbox Exchange Frontrunning - The largest brokerage house run their own internal exchanges where their customers' orders are dropped first. These exchanges likely have HFT systems that frontrun their own customers.
Also keep in mind that the BATS exchange is physically the closest electronic trading exchange to the NYSE. BATS also houses the servers of HFT firms, so their "customers" HFT servers are the closest systems to that exchange. Millions of dollars a year for prime real-estate and server uptime are being spent to milk microseconds off of data transfer times. As a computer person, I'd call this a Race Condition Exploit.
April 2, 2014, 9:37 amMercury:
The only metric that matters in terms of trading costs is
implementation shortfall: the difference between the share amount/all-in
price level you made your buy/sell decision on and what you ended up
with when the order was fully executed or abandoned after a partial
execution. You can loose your shirt on a trade even when commissions and spreads are close to zero.
Was implementation shortfall larger pre-HFT dominance or is it
larger now? The answer may be matter to a greater or lesser extent for
different types of market participants but it’s really the crux of this
part of the controversy.
HFT can facilitate not just front-running but front-running and the manipulation of markets in the direction favorable to the HFT firm in question while utilizing one of the thousands of order types (not algos) that they have themselves designed and had blessed by regulators. There's plenty of crony-abbeted, government sanctioned, mega-complexity at work here that makes neat little loop-holes for the well connected and financed.
The bigger problem is that thanks to Reg NMS equity market structure is now totally fragmented and opaque which at some point is antithetical to what a market actually is. With the right regulatory vision today's technology is such that together they could have made this market system an extremely robust and near frictionless utility but it hasn't worked out that way. What we have now has made the whole system systemically vulnerable to stress as we have already seen several scary examples of.
HFT will be the Lehman-esque scape goat of the next (Fed induced) financial crisis even though it's the symptom, not the disease. You can see it coming a mile away.
April 2, 2014, 10:25 amMatthew Slyfield:
Of course, they always blame private parties for taking actions that are direct consequences of the last time the meddled in the market for the next major failure.
April 2, 2014, 10:42 amArrian:
I believe how you're thinking is how HFT started, but it has since grown? After all, if you're running HFT operations on 3 or 4 or a dozen exchanges, it's very natural to tie those separate systems together on your end and have them talk to each other.
So, the way I see it:
Wile E Coyote tells his to buy a million shares of Acme stock for $1.05/share.
His broker goes to exchange A, places a bid for a million shares at $1.05. There are offers for 300,000 shares at $1.03 and he buys them all. The HFT agorithms see this (or even participate in it) and let their counterparts on the other exchanges know that there should be a 700,000 share bid coming their way.
HFT algorithms on exchanges B and C bid $1.04 for 700,000 shares. They get 200,000 shares on exchange B and 500,000 shares on exchange C.
Wile E Coyote's broker then goes to exchange B, makes his $1.05 bid and gets and offer for $1.05 from the HFT algorithm that won on that exchange. The same happens on exchange C for the last 500,000 shares.
The HFT algorithms on each server are doing what they've always done, but they've also added a little bit of a crystal ball allowing them to see future bids.
Which leads to the question: Why don't buyers front run themselves? Why go to exchanges sequentially, why not hit all of the exchanges simultaneously and gather up all the lowest bids themselves?
April 2, 2014, 11:35 amShane:
Have read many books about how directional traders enter orders. What you have a described is not a good way to enter a large order, and I highly doubt an institutional trader would do this. A large order will typically be entered over days, in pieces. A large directional trade usually develops over a long time frame (in HFT terms) so time is on the directional traders side when he enters the order. Way of the Turtle describes the difficulty and solutions for entering very large orders.
April 2, 2014, 11:57 amDaublin:
Note there is a spy-versus-spy aspect to the story. If you are trying to make a large transaction, and you want to avoid being detected by these front runners, you can divide your bid up into a large number of randomly sized amounts and spread it out over a few days. However, your splitting algorithm better not be too simple. If you just buy 100 shares a minute, every minute, then the front runner can infer what you are doing and still intermediate in front of you. So you should randomize it, and maybe also wait a little bit if you see the price suddenly leap up.
This kind of stuff is clearly a stupid way to make what is essentially a simple transaction. I have no idea how to fix it, but one would hope that the exchanges themselves would take care of this kind of problem.
April 2, 2014, 12:00 pmskhpcola:
Zero Hedge is the Alex Jones of financial commentary. Conspiracy theories and black helicopters are 95% of the content there, and the commenters are certifiable lunatics.
April 2, 2014, 12:10 pmShane:
Front running has been around before time was time, and the ways to deal with it are many. Breaking it up is one way. Because large traders have an essentially zero transaction cost they can play all kinds of games to make it appear that the order is not the order. If a trader is known then he can push the trade across ECN's to provide anonymity. Head fakes are common i.e. push a large order in the opposite direction of what is intended then get a small fill and pull the rest of the order then turn the order around. Phantom orders to exaggerate market depth, and on and on ... In trading it is a mental war, and the smarter (luckier) warrior wins.
I don't know a ton about this stuff, but remember there are literally millions of people that look at the market ever second, trying to make a living off of what they see.
April 2, 2014, 12:39 pmArrian:
Right, then, it's good to be reminded that a little knowledge id a dangerous thing and just because you took a class and read a couple articles a decade ago doesn't mean you know what you're talking about. Time for more reading, less typing.
April 2, 2014, 12:42 pmShane:
Did I mention that I actually trade? That I have been trading for about a decade? That I actually have made large directional trades? Maybe I forgot to mention that. I know that this is the interwebz and all but please a modicum of politeness goes a long way.
April 2, 2014, 1:20 pmArrian:
That was meant at face value as a mea culpa, not snark. It apparently didn't come across that way.
April 2, 2014, 1:33 pmShane:
All good.
April 2, 2014, 1:36 pmSo much meaning is lost in text, I forgot that.
Sorry for the snappy comeback.
duanegran:
HFT is not the problem, but it is the vector by which the exchanges have been gamed recently. The newer IEX exchange is growing rapidly precisely because they have put into place safeguards against some brokerage firms pushing prices up by purchasing up shares in the nanoseconds between placing an order and securing it. The whole situation is quite complicated but it amounts to a needless tax on transactions. HFT firms defend themselves as creating liquidity and greasing the market. To this end they are pretty much right. When they start creating alpha out of having their offices positioned closer to the stock exchange and leveraging the speed of light in their favor it undermines their value on the exchanges.
The genie isn't going back in the bottle but I do find the IEX story fascinating because the people behind it didn't stop this practice with regulation, shaming or whatnot -- they simply created a level playing field and the market is rewarding them. HFT can continue, but it can't be used to make brokers outside of Wall Street (literally, the street) pay pennies more per share just because it takes nanoseconds for light to reach the servers.
April 2, 2014, 2:00 pmGdn:
So...what is the economic purpose of the stock market?
Does HFT benefit this purpose directly?
Does HFT benefit this secondarily?
Does HFT merely benefit a mechanism for a secondary attribute of the stock market?
My understanding of the purpose of the stock market is to efficiently get money from those who have it to an entity that will do something useful with it, and returning to the investor a compensation for the risk.
My purpose statement is undoubtedly clumsy, but I don't see HFT supporting the primary goal...but rather taking advantage of a mechanism. Does there have to actually be any intelligence applied to the end investment in HFT?
April 2, 2014, 5:09 pmmesaeconoguy:
Most of what you just posted is incorrect.
Due to my proximity to this topic, I cannot comment any further.
April 2, 2014, 5:29 pmmesaeconoguy:
I do enjoy much of zerohedge’s commentary, but on this one, they are very wrong, as is Michael Lewis, who is wrong about many financial
things (he only briefly worked at Salomon, and is an English major by training, I believe. Not exactly an institutional practitioner.)
As I state below, I am extremely close to this topic and cannot comment further, but additional missing information will be forthcoming.
April 2, 2014, 5:32 pmHarry:
Shane, great point.
This guy who wrote the book being promoted implies that "ordinary retail investors" like Coyote or me or you are being taken to the cleaners, further implying that investing is the same thing as trading. When trading anything, whether it be two oranges for a peanut butter sandwich, or money for 10,000 shares of Exxon, there are transaction costs which both the buyer and seller seek to minimize and, if there is a party in between, that party wants to maximize the transaction cost.
Your point is most pertinent, because it also implied that the people engaged in this computer trading win all the time, at the expense of poor people who have no money.
Only one time in my life did I have the inside track on a sure thing. A guy I went to school with was a harness driver, and I visited with him and his family at the racetrack. He told me to bet on a horse in the next race, and that the horse was going to win in the next race by seven lengths, and she did. Had I had any money at the time, and had I gone to the window, surely would I have profited, and if I had done that a thousand times in a row, at 2:1 odds, well, you do the math.
This is not to say that one cannot make money as an arbitrageur or a rug merchant, but when trading for your own account you have risk.
April 2, 2014, 6:06 pmHarry:
I did not mean you implied that there was riskless trading; rather, the opposite.
April 2, 2014, 6:09 pmScott:
Markets emerge spontaneously. To ask what the purpose' of the stock market is, or any market, is the wrong way to think about markets. There is not central planning entity that determines the 'purpose' of a market. Markets exists because parties can voluntarily exchange and both end up wealthier than before the exchange, and will continue to exist as long as the opportunity for both parties to increase wealth remains.
April 3, 2014, 5:50 amMercury:
Well, Reg NMS is a product of central planning and without it there would be no HFT.
April 3, 2014, 6:16 amRuss R.:
"Why don't buyers front run themselves? Why go to exchanges sequentially, why not hit all of the exchanges simultaneously and gather up all the lowest bids themselves?"
My firm does exactly that.
We place our stock trades (typically in the order of millions and occasionally tens of millions of dollars in value) with an electronic broker who uses a routing algorithm that confounds the HFT algos by staggering order release times so that they arrive simultaneously at all exchanges, leaving the HFT algos insufficient time to front-run our trades.
Our fill rates are virtually 100%.
April 3, 2014, 6:49 amArrian:
Is there a good place to read up on HFT that isn't wrong?
April 3, 2014, 9:36 ammesaeconoguy:
Well, that’s tricky, since there are so many competing viewpoints.
A good survey of the industry view can be had here
http://www.tradersmagazine.com/
and here
http://www.institutionalinvestor.com/
with a lot of background and side-discussions about market structure.
Any complete explanation will involve an in depth discussion of Reg NMS, and I’m not going to do that in this forum.
The net result of all this Sturm und Drang will probably additional regulation, resulting in higher costs to investors.
I have not yet read Michael Lewis’ book, but plan to do so in the near future. But I’ll probably have to refrain from commenting, for industry reasons (completely idiotic, in my opinion).
April 3, 2014, 4:01 pma_random_guy:
Your inability (or unwillingness) to provide details demonstrates the lack of transparency, and implies even more severe abuses than those described.
April 5, 2014, 4:02 ammesaeconoguy:
Sure it does.
Actually, you can thank regulation for that.
Just wait until Eric Holder fixes this.
April 5, 2014, 8:59 amFred Mertz:
HFT adds volume to the exchanges. Volume is not liquidity. Volume often receives benefits in the form of rebates. What troubles me most is that the rules are not published for most quasi-exchanges - Dark Pools. Stocks originally were used to provide investors with dividend income and or price appreciation. Today, while that still may be true in some cases, the markets are more focused upon in and out, short term trading, day trading and not investing. The reason for this is that exchanges generate revenue from transactions. The more volume the better for the exchanges.
April 8, 2014, 6:37 amFred Mertz:
Yes, spreads between bids and offers have been reduced as a direct result of electronic trading. Commissions and fees have also gotten lower. All are good things. However the leveling of the playing field is untrue. What results from HFT is gaming the system through speed. How can exchanges accurately verify the true sequence of trading in a time and sales of transactions to determine if front running has occurred? They don't seem capable of doing so.
April 8, 2014, 7:44 amSeekingfactsforsanity:
So you have not read Michael Lewis' book but you state that he is wrong. Hmmm. I have read the book. My hat is off to Michael Lewis for writing the book, and think he did a great job with the story of Brad, his crew, and the reasons for developing IEX to counter the front running games of the HFT. He did a great job of describing how the HFT version of electronic trading operates, as well as the downside of dark pools, specialized trade orders, undisclosed rules, etc.
May 7, 2014, 5:46 ammesaeconoguy:
Um, no. Lewis is a technical amateur, and has in the past oversimplified many aspects of not only markets, but institutional money management. He is a good storyteller.
I still haven't yet read the book, but have read many, many reviews, in addition to having significant firsthand industry experience, far beyond Lewis'.
A little more info:
Lewis confuses 2 concepts in the book: payment for order flow and co-location. Payment for order flow has been around since I can remember, and was expressly codified within Reg NMS. It is disclosed under Rule 605 and 606 disclosures by all major broker-dealers.
Co-location is the practice of speed arbitrage - certain HFT firms have located their servers as close as possible to execution venues and centers to exploit price differences and capture spreads.
Does that constitute front-running? You would have to parse the rule very, very finely to arrive at that conclusion. You would also have to explain how, if HFTs price "disimprove," the reported metric of price improvement is valid under Rule 605.
For most investors, the result of HFT has been increased liquidity, tighter spreads, and lower transaction costs, and is not a material factor -
http://online.wsj.com/news/articles/SB10001424052702303978304579475102237652362
Lewis also omits the fact that various broker-dealers may be contractually obligated to send order flow to various
execution centers and venues. Such arrangements almost always carry performance clauses to ensure execution performance within certain parameters, to protect execution quality.
One of the major problems with Reg NMS is that it protects top-of-book, not depth-of-book. By executing intermarket sweep orders (ISOs), firms may then trade thru certain resting limit orders using the Rule 611 exception. That is a perfectly acceptable practice under Reg NMS, and if you want things like that to go away, Reg NMS will need an overhaul.
The existence of HFT, as well as dark pools and current market structure, is a direct result of Reg NMS.
The likely outcome of all this uproar - most of it misplaced - is this:
Eric Holder (and various state Attorneys General) will "investigate" the current arrangements, errantly conclude that various parties have unfairly exploited Reg NMS, fine various firms for perfectly legal arrangements prior to Lewis' book, and a new, more convoluted rule scheme will replace Reg NMS, causing transaction costs to rise and retail customers to pay more to execute trades.
May 7, 2014, 4:41 pmSeekingfactsforsanity:
Technical amateurs are incompetent and therefore unknowingly dishonest whereas Technical experts are competent and honest. Is that your point? I agree that our central authority deceives all pretending open and fair competition in just about everything these days including healthcare, energy, education, and on and on. I am curious if some of the shovel ready projects included laying HFT communication channels through all those public right of ways?
May 10, 2014, 8:33 ammesaeconoguy:
No, my point is that Lewis is wrong, the market is not “rigged,” he doesn’t understand current market structure, and neither do you.
May 10, 2014, 2:23 pm