“The fox knows many things, but the hedgehog knows one big thing.”
The quote is one Nate Silver is fond of. It can be found in his 2012 book The Signal and the Noise as well as in an article at his new site FiveThirtyEight. Silver says he wants to be a fox but some are, rightly, questioning whether he needs to know his subject matter a little more in depth.
Silver made a name for himself during the last few election cycles by predicting the outcomes through the use of statistics rather than the gut feelings, and wishful thinking most media pundits use. Before that he created a system of using baseball statistics for projecting player performance. In The Signal and the Noise he branched out a bit and covered several topics like the financial meltdown, weather forecasting, global warming, and even terrorism.
The book was pretty good I thought, if looked at as a primer on how statistics are used in various fields, and how they often fail. As investigative reporting however it wasn’t quite as good. The notion that there might have been malfeasance at work in the financial meltdown got merely a short mention before moving on to discuss faulty financial models with no relevant historical data to draw on. He drew the ire of global warming researchers by too easily falling for the critiques of deniers. He was clearly out of his element in either of these fields.
At the new site it appears that he is unaware of his shortcomings. The new site intends to delve into many topics outside his expertise. The thing is though, election prediction isn’t all that difficult a task, comparatively. You have lots of polls that are conducted by mostly competent firms. You can track the history of each firm as to their reliability. Throw in a few social factors (state of the economy, etc.) and you have a pretty robust model. Not everything is so clean cut.
The word model is fairly important there. If you don’t have a model how can you know what data is relevant in the first place? Paul Krugman:
Source: Tarnished Silver
It’s not the reliance on data; numbers can be good, and can even be revelatory. But data never tell a story on their own. They need to be viewed through the lens of some kind of model, and it’s very important to do your best to get a good model. And that usually means turning to experts in whatever field you’re addressing.
Unfortunately, Silver seems to have taken the wrong lesson from his election-forecasting success. In that case, he pitted his statistical approach against campaign-narrative pundits, who turned out to know approximately nothing. What he seems to have concluded is that there are no experts anywhere, that a smart data analyst can and should ignore all that.
Similarly, climate science has been developed by many careful researchers who are every bit as good at data analysis as Silver, and know the physics too, so ignoring them and hiring a known irresponsible skeptic to cover the field is a very good way to discredit your enterprise. Economists work hard on the data; on the whole you’re going to do better by tracking their research than by trying to roll your own, and you should be very wary if your analysis runs counter to what a lot of professionals say.
The link embedded in the Krugman quote offers a good example of just blindly looking at the data. Roger Pielke Jr. is the man in question and he wrote an article questioning the notion that more extreme weather is causing more damage to society. His data was that the increase in damage costs tracks the increase in Gross Domestic Product. Therefore, there hasn’t been an increase in damage, just an increase in wealth. Sounds pretty straight forward right? Well maybe not.
It turns out over the same time some of that increase in wealth has gone into building things that can better resist disasters. Because of that should we be seeing a decrease in disaster costs relative to the increase of wealth? Probably so. Researchers in the field seem to think so. At any rate it should show you why the model you use is so important. It tells you want data you need to pay attention to. It seems to me factoring in how well your buildings can withstand storm damage is something that belongs in the model.
Krugman is not the only critic. Long time Silver fan and libertarian(ish) economic commentator Tyler Cowen, along with several others, has been critical as well. Krugman points us to an article by a staffer about corporate cash hordes.
One of the early narratives of the economic recovery was that companies were “hoarding” cash. The story line made sense: Companies, burned by the credit crisis and cautious amid economic and political uncertainty, were stockpiling money. The data backed up the story: The Federal Reserve in 2011 reported that American companies had more than $2 trillion stashed away in overflowing vaults (or, in most cases, bulging bank accounts). As a share of all assets, cash holdings were at their highest level in nearly half a century.
Then the Fed revised its data. New figures released in early 2012, based on more complete tax filings, showed that American companies actually had close to half a trillion dollars less cash than previously thought. Companies did hoard cash in the early months after the financial crisis, but only up to the point that they had rebuilt savings they’d had to spend during the credit crunch. After that, cash holdings as a share of assets leveled off and have remained pretty much flat since. The revision didn’t just change the numbers—it undermined the whole narrative.
As Krugman points out, it is silly to say that it destroys any narrative. Why should a revision from $2 trillion to $1.5 trillion refute that narrative? What did the corporations spend the money on? Was it on their own businesses? Did they invest in Treasury bills? The article doesn’t say.
We’re told that the “whole narrative” is gone; which narrative? Is the notion that profits are high, but investment remains low, no longer borne out by the data? (I’m pretty sure it’s still true.) What is the model that has been refuted?
It seems to me that Silver has overestimated his ability to just jump into any field and look at a few numbers and give us all the “straight scoop.” There is a reason people go to school for years and years to master fields like climate science and economics. He isn’t dealing with a bunch of hacks that are paid to say stupid and controversial things on a TV news program.
He’s in way over his head and should stick to political forecasting which he’s good at and where things aren’t looking so great for Senate Democrats in November.