Analysis 101: Make Your Assumptions Explicit

My younger brother is a big college football fan, far more than I. Today, he spotted an argument made by some ESPN football analysts about my beloved BYU Cougars that I thought would be a good springboard for talking about another fundamental principle of analysis — Make Your Assumptions Explicit.

First, watch this. My apologies for that and what follows if you're not a football fan.

A bit of context: the analysts were discussing BYU's chances of getting into the new NCAA College Football Playoff (CFP) tournament which replaces the deeply flawed (and deeply corrupt) Bowl Championship System that, for 15 years, picked who got to play for the NCAA college football championship. Here's the kicker — there's no mathematical formula for picking who gets invited to play in the CFP. Instead, a panel of 13 experts will choose the teams.

Now, after watching that 2.5-minute clip, my brother's immediate reaction to the analysts'  comments was: 

"...BYU can't go undefeated because the teams it has left are too tough. But if BYU does go undefeated, its schedule was weak. ???"

To understand his complaint, you have to understand the concept of strength of schedule. The idea is that beating a strong team should count for more than beating a weak team. Okay, fair enough. The guy who beats Goliath should get more glory than the guy who stomped on a weakling. If BYU only plays weak teams and goes undefeated, should it get into the tournament over a team that played some strong teams and lost a few games? Arguably not.

Here's the problem — strength of schedule can't be settled until the season is finished. At season's end, the 13 experts will have the benefit of reviewing the full season records of all NCAA teams to evaluate . But those ESPN analysts? Despite only being three weeks into the season, despite BYU being so-far undefeated, with a 41–7 butt-whoopin' inflicted on the Texas Longhorns, a perennial #25 team, they've already concluded that BYU can't get into the tournament. "So sorry. Strength of schedule."

On to the analytical lesson.

Sooner or later, everyone gets asked which possible future outcome seems most likely in a given situation; but the world is so amazingly complex, with so many different events occurring every single second, that it's impossible to be aware of them all, much less figure out how to factor them in properly to any prediction. Any and every prediction of a future outcome is based on wildly incomplete information.

For our football example, none of those analysts predictions is worth anything because no one can know how hard a team's schedule really was until the season ends and they can see the final tally of all NCAA games played and see who lost to who by how much. The evidence doesn't exist yet. Sporting events are exercises in chaos theory and any one random event can destroy the delicate foundation on which any given prediction rests. One-hard-tackle-and-a-compound-fracture later and a team's Heisman candidate is in traction, with the analysts' pick-to-win-it-all collectively praying it can go 4–4 for the rest of the season.

So, lacking hard facts, we make assumptions. We assume, consciously or not, that X will stay constant and therefore Y will occur...or that X will not stay constant and therefore Z will occur. Those ESPN analysts are assuming that strong teams are going to win the games they're supposed to win and no key players will end up in the hospital.

Making assumptions isn't a bad thing to do. In fact, we couldn't function on a daily basis if we didn't do it. We can't prepare for the future if we can't depend on some degree of predictability. But some things are so random or complex (or both) that all we can do is flat-out guess that things won't change or will at least change in at a constant rate and act on that guess.

So how do we handle assumptions? By doing the following:

1. Identify your key assumptions and declare them up front. Key assumptions are the ones so critical to your argument that if they fail, your argument collapses. Not being aware of your assumptions is a recipe for surprise. Not being aware of your key assumptions is a recipe for disaster. Being aware of them but not making your audience aware of them is a recipe for watching your credibility get gutted when your analysis doesn't pan out. So tell your audience. As Colin Powell liked to say, "Tell me what you know. Tell me what you don't know. Then tell me what you think. Always distinguish which is which." Your audience will usually forgive an "I don't know" answer. An audience will be less forgiving of an "I know" answer that proves false. They'll feel misled.

2. Review your key assumptions regularly. You have to revisit your assumptions regularly to see if they're still valid. The world changes. Yesterday's solid assumption might fall apart today. Good assumptions become bad assumptions.

So if the ESPN talking heads really wanted to be "analytically correct," the best they could say is, "If there are no major upsets or injuries for the remainder of the season and our current assumptions about how all NCAA teams will finish in the standings hold up, BYU's strength of schedule would not compare favorably enough with other likely candidates to secure a tournament berth."

But I'm sure that wouldn't be nearly as entertaining as yelling "BYU HAS NO CHANCE! NO CHANCE AT ALL!"

Where I Was

Two months before 9/11/01, an office colleague and I attended a business meeting in the World Trade Center. When the meeting was finished, our hosts asked whether we'd like to go up to the WTC observation deck to look out over lower Manhattan. We checked the time and realized that catching our Acela train back to WDC was already going to be iffy. "We'll check it out it the next time we're back in town."

There never was a next time.

Two months later, on 9/11/01, thirteen years ago this morning, I was on an airplane flying to Pittsburgh. Some colleagues and I took off from Washington National Airport around 0745 and I distinctly remember how the Pentagon filled my window as we banked over it. I've never thrown away the boarding pass stub.

My wife worked in the fitness center of FDIC, across the street to the west of the White House complex. I almost had a panic attack when I saw the White House being evacuated...my mind blanked and I couldn't remember whether it was her day on the early shift or the late shift. It was the latter. She was still at home. Small blessings. We'd been married for all of four months. Had I been on a different plane, she might've been left a widow.

My colleagues and I drove back to WDC in a rental car that we managed to get before they all disappeared once all flight over US airspace were grounded. It took us five hours. We turned on the radio and just listened to the news reports. Nobody said a word.

A week later, my wife and I stood in the dark on a knoll across I-395 across from the Pentagon. We stared at an unfurled flag and a massive hole in the largest building I've ever seen. It was evil made tangible in a way I'd never seen outside of the Holocaust Memorial.

There truly are evil men in this world. It's been one of the great privileges of my life to help fight them. But someday, somewhere, someone else will manage to do something just as horrifying on US soil. Don't live your life in fear, but don't ever think for a minute that we're entirely safe. Evil never truly surrenders...you either destroy it or it falls back to lick it wounds and wait for another day. Just be grateful that there are men and women who've made it their life's work to stand between you and such evil. They have protected you and those you love more times than you'll ever know.

Amazon and Hachette: The Dispute in 13 Easy Steps

A great piece in the LA Times explaining why the Amazon-Hachette dispute matters. The best quote:

"What's ultimately at stake is whether Amazon is going to be able to freely and permanently bully publishers into eventual nonexistence." -- John Green, author of The Fault in Our Stars.

BTW, #10 explains why Hachette is in Amazon's crosshairs, and why the other four major publishing houses can expect the same treatment if something doesn't change.

Update: my literary agent tells me that I really need to turn on the commenting feature in my blog. I'd kept it turned off because I've had some really bad experiences with Internet trolls. But, at his behest, I'm going to give it a test run and turn on comments for a few selected blog entry just to see what happens. Please justify my tenuous faith in humanity's capacity for civility.

Militarized Police

What's going on in Ferguson really shouldn't surprise anyone.

Even if things turn around quickly, though, it won't erase the memories from this past week or end the debate about tactics. Chief among them are decisions like deploying heavily armed officers and using military equipment, which some experts say helped to make a bad situation even worse.

"The tactics they are using, I don't know where they learned them from," [Retired Lt. Gen. Russell] Honore said Thursday on CNN Newsroom. "It appears they may be making them up on the way. But this is escalating the situation."

This is just a real-life demonstration of the Law of the Instrument, aka Maslow's Hammer -- "If all you have is a hammer, everything looks like a nail."

Strategy is shaped by tactics and tactics are shaped by tools, i.e. your weapons' capabilities dictate what you can do. So if you arm your police with military-surplus weapons and equipment, why is anyone surprised when the police start using military-style tactics?

"Authors to rebuke Amazon over Hachette dispute with full-page NYT ad"

From AppleInsider:

Nearly 1,000 authors affected by the ongoing e-book spat between Amazon and machete have signed their names to a letter — set to run as a full-page advertisement in the Sunday New York Times later this week — imploring the online retailer to settle the dispute and calling on readers to voice their displeasure directly to Amazon CEO Jeff Bezos.

To my literary brothers and sisters, I say, "booyah." You can read the letter here; and here's a good perspective from the New York Times on how this is affecting individual authors.

Apple vs. DOJ e-Book Case: Judge Cote's Guilty Verdict "Impossible"

On Friday, Judge Denise Cote reversed course and signed off on the Apple-DOJ plea bargain, thereby eliciting the single best explanation I've seen as to why the Appeals Court should overturn Cote's guilty verdict against Apple in the e-book anti-trust case.

"Antitrust experts in particular were outrage at the verdict as under that section of the law it should be impossible for a 'vertical' reseller (like Apple) to join a 'horizontal' conspiracy amongst publishers. Cote also ignored evidence that other booksellers had previously urged publishers to force Amazon to use the 'agency' model (where publishers set prices) before Apple even began planning its iBookstore business as Amazon's predatory pricing at the time was preventing competitors from joining the market."

Note the word "impossible." Analysts love words like "impossible" because they set up a binary choice that usually isn't hard to evaluate — something is either impossible or it's not. 

Never mind the verdict's other problems which I've listed off and linked to before — anti-trust experts are asserting that, as a matter of black-letter law, a vertical reseller like Apple can't be part of a horizontal conspiracy like the one DOJ says existed among the publishing houses...legally, it's "impossible." That's no mere technicality. If that's true, here's the binary choice:

1. DOJ prosecutors and Judge Cote were incompetent and didn't understand the relevant case law; or

2. They did know the law and ignored it for reasons I won't speculate on here.

Incompetence or bias. If the anti-trust experts are right, there are no other possible conclusions.

Amazon's Faulty Math on the Literary Market

Amazon posted a statement explaining its motives in trying to strong-arm Hachette (and, inevitably, other publishers if it succeeds) into slashing the wholesale prices of e-books. A few choice bits:

"If publishers lowered prices, they would sell more books, which would create more readers, which in turn would sell more books, it argues...The argument of profit-through-volume rings hollow coming from Amazon, however, with its price-to-earnings ratio of 500. The company has yet to be able to achieve any sort of meaningful profit despite growing to nearly $20 billion in quarterly sales across a huge range of products."

By it's own profit-through-volume logic, Amazon should be the richest company on the planet. Instead, Amazon has rarely ever posted a profit in its entire 20-year corporate history and just reported a $126 million loss for the last quarter alone.

"Amazon, by its own admission, was previously losing $3 to $5 per e-book when it sold e-books at the $9.99 price point before the Department of Justice stepped in to force publishers to adhere to a wholesale model. However, profits of the publishers did not go up significantly during the same period, suggesting that in fact publishers cannot sustain a $10 e-book price when it would only see $3.50 of each sale as Amazon proposes. In part, this is due to the fact that lower e-book pricing was previously demonstrated to have a deleterious effect on hardback and other printed book sales, as well as harming physical bookstores. These outlets continue to be the main profit centers for publishers, and the main way readers find and buy new books, despite the growth of e-books."

Yep. Steve Jobs called it four years ago.