The Micro to Macro Edition
On charts, markets, and the importance of getting your arms around the shape of data
|Guest Contributor||May 18||10|
Josh Reich (JR) is a med school drop out, former CEO of the online bank, Simple, and former goat and cattle farmer. He recently moved back to Australia after 17 years in America. When not spending time with his wife and two toddlers, he dabbles in electrical and mechanical engineering projects.
Josh here. A few weeks ago a piece in the Times about the UK retailer Timpsons caught my eye. The CEO talks about how he has eschewed the modern trend for big-data analytics to focus on two numbers: sales and customer satisfaction.
“The businesses we bought were often collecting vast amounts of data from their fancy tills, yet the managers were actually reading very little of it, and it rarely helped colleagues give better customer service. As sales plummeted, they analyzed more data, and brought in more finance experts and consultants to work out where the problems were.”
It’s easy to feel safe in a cocoon of charts and graphs. Many of us look at numbers every day and we’re generally good at making up stories to explain them. In business, competing departments have very plausible explanations of why numbers change. It’s easy to congratulate yourself for when things are going up and to the right, and equally easy to convince yourself that external factors explain any dips.
As Fred Brooks put it in his classic technology management book The Mythical Man Month, “Show me your flowchart and conceal your tables, and I shall continue to be mystified. Show me your tables, and I won't usually need your flowchart; it'll be obvious.”
Why is this interesting?
Like a lot of lessons in life around startups, it’s taken me a few tries to internalize this idea.
I walked into my mandatory first-year business school data analysis class with a lot of confidence. By that point I had already been the CTO of a marketing analytics company and had tested out of all the other introductory statistics and math classes. One of our first assignments was to analyze a largeish data set and develop a predictive model. I ran into the project convinced I was going to nail it and wrote up a far longer report than required.
What I missed was that a small percentage of the rows in the original file were improperly formatted, so I discarded them from the set, figuring it was uncorrelated noise. It turned out that those rows were key to understanding the underlying problem. The fact that I can’t even remember what the data was about tells the entire story. Without understanding the micro, it is very easy to completely misinterpret the macro.
Of course, I wrote the lesson off as a trick by the professor—only to have to learn it again in the real world just a few years later.
After graduating I was working at a startup that was running a marketplace where advertising companies would source leads for mortgage companies, who would then bid for those leads in the market. People applying for mortgages aren’t completely fungible. There are many dimensions that need to match between the mortgage companies and the applicants. As such, the market was complicated to run. It was interesting from a mathematical and financial standpoint. I read books about market microstructure and wrote a paper about market making price algorithms for multidimensional markets.
As the tide was turning on the mortgage market, the board moved to replace the founder and CEO with a seasoned bond market veteran. On her first day, she asked me to produce a report of market operations. I was excited to nerd out and provided a series of charts that I looked at every day to track market depth, clearing prices, supply & demand mismatches, etc. She had a quick look at my charts and promptly asked me to walk her through every single bid and ask that was placed on the market the day before.
I thought she was nuts. I told her that her request would take forever and she told me that she had no plans for the evening and neither should I.
She asked me to load up as much as I could into Excel and we began the drudgery of looking at each bid and ask. When the trades didn’t match, she’d ask me why. I took the exercise as an attempt to find faults in the software I was responsible for that executed the matching and trades. She wasn’t really looking for bugs, though, she just wanted to ensure she understood the business she was now running. As I found myself at a loss for explanations of individual trades, I realized my grasp on the market was a lot less firm than I thought.
In producing the charts that I used to manage the business, I had assumed that I was operating at the right level of abstraction. But it turns out that I had assumed too much about how the market should work. High level reports miss the nuance. They aggregate up to the measures that you assume matter. Doing the grunt work of piece by piece analysis was at first boring, then enlightening.
By the end of the night we had a plan to improve the business significantly. By the end of the year Lehman was bankrupt and the mortgage market we were running was no longer relevant.
But the lessons I learned have stuck with me. You need to immerse yourself in the micro before you can even pretend to know what’s important about the macro. (JR)
Story of the Day:
25 edits that define internet video (CJN)
A Blackbirdspyplane retrospective (CJN)
A new menswear designer for JCrew (CJN)
Thanks for reading,
Noah (NRB) & Colin (CJN) & Josh (JR)
Why is this interesting? is a daily email from Noah Brier & Colin Nagy (and friends!) about interesting things. If you’ve enjoyed this edition, please consider forwarding it to a friend. If you’re reading it for the first time, consider subscribing (it’s free!).