The Great Order Flow Debate May 25, 2006 URL: http://www.financetech.com/showArticle.jhtml?articleID=188500216
As transaction cost analysis (TCA) tools emerge on the desktop, buy-side traders increasingly are scrutinizing the quality of their executions - both to dissect their own trading performance and rank their brokers. But, while many firms use TCA to help direct their order flow, there are questions about the prudence of relying solely on these metrics to select brokers.
Meanwhile, the trend toward measuring execution quality is accelerating as buy-side traders take more control over their executions through direct market access (DMA) and algorithmic trading strategies. While there are regulatory pressures from pension plan sponsors and the need to comply with best-execution obligations from the SEC, it is, in large part, this shift in control from the sell-side block desk to the buy side's own trading desk that is driving the current TCA momentum.
"The buy side has so many options for trading - they can do DMA, algorithmic trading; they can use traditional markets through the sell side," relates Peter Bergan, senior consultant at Citisoft, a financial services consultancy that works with 75 percent of the top 50 asset-management firms. "They're using TCA to measure how they are doing as they continually tweak their processes," he explains. "Maybe an algorithm is the correct tool; maybe they need to adjust the setting to be more or less aggressive," Bergan adds. "They're also using the TCA tools to rate internal traders, as well as the external traders, not necessarily to punish anyone, but as a learning and evaluation tool."
At Boston-based Putnam Investments, Richard Block, head of global trading, utilizes TCA metrics for many things, including assessing broker quality. While Putnam's traders use the tools to analyze trading results the day after the trade is completed, Block scrutinizes the numbers on a monthly basis. "We do look at [TCA] as a guidepost," he says. "If there are firms where the cost numbers are significantly higher than forecasted costs or versus their peers, we need to drill down further. We look at broker costs in concert with other factors that drive trading strategies."
However, Block cautions against relying entirely on TCA models to rank brokers and route order flow. "The broker is not the one pulling the trigger on the trade, so we have to bear that in mind," he points out. "He is the third decision maker on a trade." There's a series of steps that occur before the broker receives the trade, Block explains. "Our portfolio manager selects what names to trade. Our [internal] trader selects strategy, pace and venue, and then selects a broker to handle the order under his direction," he says. "To just take the numbers as a metric and decide order flow and evaluate a broker based on that, I think, is a wrong decision."
In fact, Putnam takes many other factors into consideration. "The factors that drive our traders to select brokers basically haven't changed a whole lot," Block continues. "It's capital usage, it's market intelligence, it's order handling and responsiveness, and access to position traders."
However, other sources confirm that many buy-side traders indeed use only the pre-trade and post-trade statistical models to rank their brokers, which helps them determine how to direct their order flow. Peter Weiler, SVP of the Investment Manager Group at Abel Noser, suggests that this reliance on TCA arose out of necessity. In the late '80s and early '90s, there wasn't any type of grading system to measure broker performance, Weiler explains. "There were very few traders and brokers being scrutinized," he says. Now, according to Weiler, the industry has changed from "a pass/fail scenario to a report card scenario."
For example, at mutual fund complexes, which "have had onerous regulations thrust upon them," traders have taken ownership of the TCA tools, says Weiler. "It may verify some instinct that this particular broker is not performing well. And, on the compliance side, they will ask questions of head traders, and if there are numbers that are wild and woolly, they will ask for explanations," he says.
But are these metrics determining which brokers get the order flow? Or, does the buy-side relationship with the sell side still reign supreme? "Both play a role," says John Feng, a consultant with Greenwich Associates in Stamford, Conn. "As long as this is a business that has human beings involved, relationship will play a role," he says. But "execution quality is a key ingredient," he adds. When the research and consulting firm asked institutions how they evaluate the overall performance of the sell side, they mentioned execution quality, minimum market impact and anonymity. But they also mentioned a good relationship, adds Feng. "The two, in a sense, cannot be completely de-linked."
According to a Greenwich Research study conducted earlier this year, nine out of 10 institutions in the U.S. have a formal TCA process in place. "It varies a bit from investment managers to mutual funds to hedge funds, but the usage is pretty much spread across the board," notes Feng.
Robert Shapiro, SVP of Abel Noser's Advanced Strategies Trading Group, says, "TCA empowers the buy side to grade the sell side - at least when it comes to this execution dynamic - with a degree of granularity." A former head trader for Iridian Asset Management on the buy side, Shapiro uses advanced trading analytics and strategies at Abel Noser, an agency-only broker that provides TCA to pension sponsors and investment managers, to help buy-side firms optimize their trading styles.
"The more sophisticated buy-side desks have adopted a leave-no-basis-point-behind approach to trading. They use TCA as a tool to capture more alpha [or extra market returns] on a regular basis, and that leads to a logical extension on how they actually allocate trades," says Shapiro. "When used correctly," he continues, "TCA is highly objective. It is data that is simply crunched and analyzed in its purest form to determine execution quality."
Pulling Rank
According to Joe Gawronski, COO of Rosenblatt Securities, an institutional agency-only broker, the trend of using metrics to rate brokers began among quantitative asset managers, who historically did not use much, if any, Wall Street research, and consequently were able to use execution-only brokers that offered unbundled rates. "It's the quant shops that began rating their brokers because they could unbundle," he says. "For a long time, we've been ranked by our quantitative buy-side clients, usually quarterly rankings, sometimes monthly," Gawronski adds. Some clients "have a strict formula for how often you get ranked and your rank determines directly how much business you get," he says.
While rankings based on execution quality started with quants, "It's beginning to permeate the rest of the buy side," observes Gawronski. Now, traditional asset management shops and mutual funds have started to get involved in ranking their brokers, "not only to reward them for giving great research, but also to reward those that give great execution," he continues.
Meanwhile, buy-side traders say they are studying the data from post-trade analysis reports to spot trends and outliers - trades that fall outside the range of what's statistically expected for a given stock - but they note that brokers can look horrible in an isolated quarter, so they look at the results over time. "You can find trends in a quarter, but I don't think one quarter's performance makes or breaks a relationship," says Brian Williamson, VP at The Boston Company Asset Management (BCAM), who trades small cap stocks, a $300 million highly illiquid sector. Williamson says he has been monitoring TCA for about nine years. "It's a giant process," says the equity trader, noting that BCAM reviews its brokers and does extensive in-depth analysis with third-party services - including Abel Noser and ITG - on a pre- and post-trade basis.
Whereas buy-side traders used to rely on paper-based reports reviewing trades monthly and quarterly, the new, so-called real-time TCA tools are integrated into the trader's front end, Williamson notes. "Our management and senior traders want to know what is our real cost and how are we affecting alpha in the portfolio," he says. "Traders want to know how much alpha are we missing - are we missing 30 basis points because the transaction costs were 30 basis points?"
Like Putnam's Block, however, Williamson cautions against putting too much weight on TCA analysis when rating broker performance. Basing broker selection on TCA numbers may not be fair because most orders are not market orders; rather, they're limit orders, he contends, suggesting instructions given by the buy-side trader to the broker could influence the results. But there are other times when a broker has discretion to be more or less aggressive in handling the order and giving instructions to the floor, Williamson explains. As a result, "TCA can be a muddy water to look at," he says. "We like to look at this as a partnership with the broker - we check our performance together."
Improving the Investment Process
Other industry sources stress that TCA is being used for more than ranking brokers. "[While] there certainly are shops that use TCA in part to rank brokers and make educated choices on where to send order flow, TCA looked at more broadly is not strictly for analyzing the broker ranking," says Ian Domowitz, CEO of ITG Solutions Network, a subsidiary of ITG Group. "Broadly diversified, it's for improving the entire investment process."
A major reason for TCA's current upswing is that there's a general recognition on the buy side that lowering transaction costs can help boost investment returns. "This issue wasn't as significant when we were in the go-go late '90s," says Scott Burrill, a former buy-side trader who is now director of product development and analytics at Rosenblatt Securities. "When your index is returning 30 percent a year, there's a lot of room for slippage," he adds. "Now that the equity risk premium has shrunk so much, there isn't a lot of room."
On top of that, as the industry starts grappling with soft-dollar research payments and the notion of unbundling executions from research, TCA will become more important. "You're going to have some qualitative and quantitative value assessment of who you trade with and why," says Abel Noser's Shapiro.
Even if a buy-side firm "is going to trade for 2 cents per share, it's still going to need to assess the implicit costs of trading, which we know are much more significant," says Shapiro, referring to more-elusive factors such as market impact and opportunity cost. "These costs should determine who gets the 2 cents of the unbundled trade," he says, while acknowledging that, sometimes, qualitative offerings such as mission-critical market information and an agency-only model are factored in to order flow decisions.
"If a buy-side firm is sitting down with a client or the SEC and they ask why broker X does 70 percent of your volume, you have to be able to answer the 'why,'" says Shapiro. "Any prudent man or woman would want to see some form of analysis to lead you to conclude that a broker has earned that 70 percent market share. Ideally, that has to be substantiated by an independent third party."
Buy-side traders have been measuring executions since the 1980s when independent service providers Abel Noser, Elkins McSherry and Plexus Group came on the scene and began providing these services, including extensive monthly and quarterly paper-based reports, ranking investment management firms against their peers and benchmarks such as volume weighted average price (VWAP). "On the heels of a Department of Labor release in 1985, we started to market this VWAP analysis. Then it was based on un-time-stamped data analysis from banks," recalls Abel Noser's Weiler. "If you fast-forward into the 1990s, we started to have the advent of the [order management system], which was able to dissect and decompose costs by allowing us to capture relevant data points right away," he explains.
Around 2000, more-sophisticated, high-tech measurement services began to emerge from players such as ITG, which launched a TCA suite of products; Abel Noser, which created Trade Zoom; and Quantitative Services Group, which joined the fray more recently.
"There's a drive from a lot of these vendors to get more sophisticated and to try to figure out how much volume these stocks can take before they move," says BCAM's Williamson. According to Williamson, each TCA system has its method for determining implicit costs - the costs that result from trading. "Based on volatility, spreads, momentum, these systems came up with an expected cost measure," he says.
The more-sophisticated buy-side traders would be looking at the metrics on a daily basis to determine, "How did the brokers do yesterday, or at the end of the day?" says Burrill. "The reason for that is the buy side wants to know, 'Are we employing the right tactic for this strategy?'"
But, despite all the statistical models that are available and the technological advances in order management systems (OMSs) for tagging and time-stamping orders, the science of measuring execution quality still is fraught with complexity and uncertainty. And this may be another reason why the buy side should not rely solely on TCA numbers for ranking their brokers.
Cloudy Numbers
"There's a lot of factors that can cause cloudiness in the numbers and, because of that, I look for trends, I look for patterns, I look for outliers - both for traders and for brokers," relates Putnam's Block. "For example, if one broker's or trader's costs are five basis points lower than somebody else's, it's not meaningful."
The two components of trading costs are explicit costs - trading commissions and investment management fees, which are known; and implicit costs - factors such as market impact and opportunity cost resulting from waiting to trade versus trading immediately. "Implicit costs are elusive and controversial. The controversy centers on myriad benchmarks, measurements and methodologies, all of debatable accuracy," writes TABB Group senior consultant Adam Sussman in "Trading Under a Microscope: The Buy-Side Perspective on Trading Cost Research," published in November. "Implicit costs are harder to track because doing so involves a large set of data-complex calculations," he writes.
There also has been much debate over whether to use VWAP or implementation shortfall as the appropriate benchmark to measure trading costs. "A lot of the buy side didn't think VWAP accurately measures the trader," says BCAM's Williamson. "There are so many inputs that go into what moves the stock in the day - it might be momentum, news, major demand/supply imbalances and multiple portfolio managers buying the stock at the same time," he explains. "It just wasn't enough for someone to come into and say, 'Did you beat the median price?'"
Now, Abel Noser's Shapiro says, arrival price and implementation shortfall have become the buzzwords of choice. "Implementation shortfall has become the metric that a lot of people feel is the purest benchmark to measure trading costs, but also one of the most difficult to measure," he asserts.
One of the problems with the arrival price (sometimes referred to as the decision price or strike price) is that it must be accurately defined and time-stamped by the OMS. "It may be the price that existed when the portfolio manager made the decision to buy the stock - that price is never really recorded no matter how sophisticated the OMS is," says ITG's Domowitz. "Some people think of it as the price at which the trading desk actually submitted it to the broker," he notes.
Comparing Algorithms
Regardless of the uncertainties inherent in analyzing trading costs, experts say adoption of pre-trade tools is growing, and they predict the buy side increasingly will rely on these tools to discriminate among the various algorithmic trading solutions being thrown at them.
"I expect to be able to effectively compare broker algorithms when measuring execution quality," says Putnam's Block. His firm has "kicked the tires" a lot on which algorithms it prefers, and has been working with a few firms on building them out to its specifications, notes Block.
"The buy-side desks are now in control, and in order to control [their trades] they need both measurement and the analysis," according to ITG's Domowitz. "As they grow more sophisticated, they begin thinking about [analysis] as more than a post-trade exercise," he says. Then, Domowitz predicts, firms will think about incorporating it before the trade and during the trade. "You are adding value to the investment process," he says.
Yet others criticize the accuracy of the pre-trade models. "The models that forecast the costs are not infallible. They are exactly that - they're models," says Putnam's Block.
Block also cautions against focusing on the transaction costs too closely without analyzing the post performance of the stock. "There are times when a high-cost trade may turn out to be a good thing. If the stock is $20 and we buy it for $20.50, resulting in a 2.5 percent market impact, it may at first glance look like a poor trade. If, however, a month later the stock is at $27, our shareholders have benefited from the trade," Block says.
Ultimately, the purpose of TCA is to help make better traders, contends BCAM's Williamson. "It's one piece of the puzzle, and one bad trade doesn't break the relationship," he says. "You use these tools to provide some sort of feedback that can make you perform at a top level, whether it's in a machine or whether it's with a broker."
Drilling Through the Data
With the advent of DMA technology and algorithms, traders and portfolio managers are integrating transaction cost measurement tools into their daily workflow. "We're looking at it on a pretty grand scale," says Brian Williamson, VP at The Boston Company Asset Management (BCAM), who spends 50 percent of his day on transaction cost analysis (TCA).
A trader of small cap stocks, Williamson - who also executes for a long, short (market-neutral) and micro-cap hedge fund - starts with ITG's pre-trade expected cost numbers versus the number of shares he wants to buy or sell. From there, BCAM traders can overlay market factors - such as correlations to the floor, news, volume and how the stock's trading - and relay that information back to the portfolio manager, who also has access to these tools. Williamson explains that the portfolio manager may look at TCA numbers from Plexus (acquired by ITG earlier this year) and ITG, and see that a stock "is raising a red flag and talk to the traders about it." But, Williamson points out, "One of the problems with pre-trade tools is that you don't have dynamic data - your data is based on data up until yesterday."
On the post-trade side, BCAM subscribes to Abel Noser's service and has done a lot of custom work with the firm. "We have a simple platform set up [showing] how the brokers did on a VWAP basis, how we did on a VWAP basis and how we did on arrival price," Williamson says. Abel Noser also performs different analysis with its own formulas and pre-trade analytics, he adds, noting, "We can get a number on how these stocks typically trade."
The equity trader looks at data quarterly on a year-over-year basis to find trends by drilling through different sets of data by trader, capitalization, mutual fund group, country or product. Using this, Williamson says, he can analyze how a particular portfolio manager likes to trade or how a particular partnership with a broker works in a particular country.
Within these subgroups, "You can look at how these orders work over one day or the first 30 minutes of the day, and you can really slice the data down to the increments to find if the trade is costing you," he says. Because the post-trade data tends to be "pretty sporadic between quarters," Williamson says he also looks at the TCA daily to see what worked.
As a note of caution, Williamson says, "Your pre-trade analytics should match up with your post-trade to have an apples-to-apples comparison." -I.S.
What's New in TCA?
Below are some of the TCA providers and the products they have on the horizon.
NYFIX is introducing Sidekick, a nonintrusive database recall and filtering application that enables buy-side traders to measure trading performance in real time. Because NYFIX has more than 225 buy-side traders routing their orders through its network to brokers, the system has all the exchange prints and real-time prints from the trader as to how he or she did, and plots all the trades on a chart. "It enables the trader to see in real time what their portfolio managers see in a month's time," says Andy Wilson, global head of the NYFIX Network Division. And the trader has the ability to go back and replay the data, he adds. Further, the trader can enter free-form text reminders describing trade details so they can "defend their work" later on, says Wilson.
Institutional agency broker TradeTrek Securities is working on a product that measures a trader's consistency of execution. It lets the buy-side trader or portfolio manager take all the trades and normalize them for risk analysis and consistency, and then do a comparison among the brokers or algorithms. The new service will analyze the market impact versus the risk that a broker took to enter the position, as well as examine the liquidity of the stock, among other factors. "It boils it down to one number to measure consistency of a trader's execution," says George Rodriguez, managing director at Algorithm Trading Solutions, a division of TradeTrek.
Investment Technology Group (ITG) is in the process of writing data extracts from the Macgregor system that will parse out the various algorithmic trading strategies from brokers. "TCA wrestles with the issues of apples-to-apples comparison," says Ian Domowitz, CEO of ITG Solutions Network. Previously, ITG examined algorithmic trades being sent through the Macgregor order management system. "The one thing we were not happy about was the fact that the individual strategy was not defined," says Domowitz. "Using performance statistics to differentiate between algorithms is one of the coming things," he predicts.
Abel Noser is launching what it claims is the first computer-based fixed-income TCA tool. "We've been able to leverage our existing clientele on the pension fund side," says Peter Weiler, SVP, Abel Noser's Investment Manager Group. "We've asked them to send us data." The firm already has calculated the measurement results for about 50 of them, he says. This will likely bring more transparency to fixed income, Weiler predicts.
For more TCA products, look for the Aug.-Sept. 2006 issue, in which Advanced Trading will publish a TCA directory. To be included in the directory, submit your information at wallstreetandtech.com/transaction