Saturday, June 26, 2010

The Problem With Numbers

On the early afternoon of December 19, 1675, about 1,100 New England colonial soldiers and friendly Natives marched on a large, fortified Narragansett village located in the Great Swamp in present-day South Kingstown, Rhode Island.  A few hours later the fort was in flames while hundreds of Narragansett men, women and children lay dead.

Not long after, Puritan ministers sang the praises of this great victory—one desperately needed by the colonists, who had suffered a series of defeats to Wampanoag and Nipmuc warriors in the opening months of King Philip’s War.

Two days later, one of the English chaplains involved in the Great Swamp Fight wrote that about 200 Narragansett men had died.  His sources were two of the captains involved in the thick of the battle, at least one of whom tried to do some counting.  About a month after that a captive from the Narragansett side confirmed 150 to 200 of his fellow warriors killed.

Later, a spy overhead the Narragansett estimate 40 fighting men had been killed along with 300 old men, women and children.  Another captain at the fight estimated “300 fighting men” were killed.

You see where this is going?  In a world of imprecision, we always have nice, precise numbers to fall back upon.  Watch.

The following year, a merchant in New England claimed 600 Narragansett had been killed in the battle—introducing the concept that the Narragansett removed their dead (in the heat of the battle) so that the English could not count them.  By the time the Puritan ministers wrote their histories a few years after King Philip’s War ended, one claimed seven hundred fighting men had been killed, along with another 300 that later died of their wounds.

The dead count had risen from 200 to a 1,000 in about 24 months, depending upon who was doing the counting (and what they were trying to prove).

That’s one of the problems with numbers. 

In Saturday’s Wall Street Journal, “The Numbers Guy” discussed how difficult it has been to estimate the amount of oil spewing into the Gulf of Mexico from the BP disaster.  One week it was 12K to 19K barrels a day; a few weeks later, 35K to 60K.  The Feds in charge of estimating—obviously influenced by lawyers unavailable to Puritan ministers--have made clear that these estimates are made only to aid recovery planningOther estimates will be prepared for purposes of fines and litigation, which run $1,100 to $4,300 a gallon spilled.

BP understands the problem with numbers.

Here’s a surprise from The Numbers Guy: We still don’t know today how much oil really spilled into Prince Edward Sound from the Exxon Valdez.  Exxon estimated 10.8M gallons by subtracting the total in the Valdez from the total offloaded.  (Seems like a word problem in school—pretty straightforward.)  Except we now know that a lot of seawater was recovered along with the oil.  Some scientists estimate that perhaps 30M gallons, not 10.8M, spilled into the Sound.

Exxon is sticking with 10.8M as the “number that was agreed upon at the time.”  Exxon understands the problem with numbers.

Peggy Noonan had a smart way of saying it when she described President Obama’s major briefing on Afghanistan, the one that set the current strategy.  She said the president seemed to think his government experts might have answers.  Instead, “They have product.  They had factoids.  They have free-floating data.  The have dots in a pointillist picture, but they’re not artists, they’re dot-makers.”

President Obama knows the problem with numbers.

I see it in the annual reports about First Night in Boston, the celebration of New Years that features free concerts and ice sculptures and lots of people wandering around downtown.  I don’t know how to estimate crowds, but I do understand it helps both Boston’s pride and police overtime to suggest “hundreds of thousands” at the event.  OK, maybe. . .but I’ve been there, and there seemed to be plenty of elbow room.

I see it when we arrive at work and compare snowfall levels in our backyards after a February storm.  “We had eight inches.”  “Eight inches?  We had at least 10.”  “Pshaww—10 inches?  That’s nothing.  We had at least 14.  And it snowed sideways.”

(That’s nothing.  We lived for three months in a paper bag in a septic tank. We used to have to get up at six in the morning, clean the paper bag, eat a crust of stale bread, go to work down t' mill, fourteen hours a day, week-in week-out, for sixpence a week, and when we got home our Dad would thrash us to sleep wi' his belt.)

One of my favorite “business books” of all time is Innumeracy by John Allen Paulos.  In it, he argues that we ought to at least take a stab at being numerate, given that our society often hinges on science and math.  What Paulos doesn’t warn is that even the numerate have a trial ahead of them.  Understanding the subtlety and logic of numbers is a fine skill, unless you can’t trust the numbers, or they change depending upon who is presenting them.

Old historian-consultant joke: Q: How many Narragansett died in the Great Swamp Fight? A: What point are you trying to make?

When there are lots of precise answers to the same problem, we can pick the one we like best.

That’s one of the problems with numbers.  But you already knew that.

For example, how much do you weigh? 

Come again?

See, you did already know that.