If you can’t measure it, you can’t manage it

The problem with Scope 3 underscores the weakness of current emissions data providers in general, a new study finds

Kevin Doncaster/flickr.

(Mark Hulbert, an author and longtime investment columnist, is the founder of the Hulbert Financial Digest; his Hulbert Ratings audits investment newsletter returns.)

CHAPEL HILL, N.C. (Callaway Climate Insights) — If you like legislation or sausages, the old saying goes, don’t look too closely into how they are made.

We should add greenhouse gas emission data to this list. The more you dig into the GHG databases, the more you discover that the data often are not what they seem to be on the surface. If one of your investment goals is pressuring companies to reduce their GHG emissions, therefore, you need to exercise great care before relying on the data from carbon data providers.

These are the conclusions reached by a recently completed study titled, “Green Data or Greenwashing?,” which began circulating in academic circles at the beginning of November. Its authors are Vitali Kalesnik, a partner at Research Affiliates; Marco Wilkens, dean of the faculty of business and economics at the University of Augsburg in Germany, and Jonas Zink, a Ph.D. candidate at that institution. They reached their conclusions after exhaustively examining the databases of four different carbon data providers (whose names they did not reveal), measuring how comprehensive each was and the extent of disagreement between the four.

In an interview, Kalesnik emphasized that he and his co-authors intend no criticism of the carbon data providers themselves, which on the whole are making the best of a bad situation. Polluters are not required to report their GHG emissions, and there is no single accounting methodology that can be used by companies that volunteer to nevertheless report. The carbon data providers use estimates to fill in the gaps in their databases, but no one has yet discovered a consistently reliable way of estimating a company’s GHG emissions.

I can confirm what the researchers found from personal experience. This summer I embarked on what I thought would be the straightforward task of creating a database containing the GHG emissions of the companies in the S&P 500 index. I quickly discovered that it was impossible. As Kalesnik confirmed: “Some companies aren’t even aware of their own GHG emissions.”

I told you not to look too closely . . .

The GHG protocol

To illustrate the challenges faced by carbon data providers, consider the GHG Protocol, which is perhaps the most widely-accepted accounting standard for reporting GHG emissions. The Protocol calls for reporting emissions in three categories, or “scopes:”

• Scope 1: GHG emissions over which the company has direct control

• Scope 2: Either direct emissions from sources owned or controlled by the company, or indirect emissions from the production of electricity that the company uses

• Scope 3: All other indirect emissions

By far the largest share of GHG emissions is in Scope 3, but it is the category for which the fewest companies choose to report. And even among those firms that do, reporting is hit or miss: There are 15 individual reporting categories within Scope 3, and among those companies that do report Scope 3 emissions, the majority report in just a minority of those categories.

Reporting of Scope 2 emissions is also fraught with difficulties. That’s because the GHG Protocol allows companies to choose, at their own discretion, between two distinct methods for reporting the Scope 2 emissions that derive from their purchase of electricity. “The choice of reporting appears arbitrary and leads to considerable differences in reported Scope 2 emissions between data providers,” the researchers report.

Faulty estimates

To overcome some of these obstacles, many of the carbon data providers will estimate the GHG emissions of those companies that choose not to report or which report incomplete data. Given that the data is already suspect when companies choose to report their emissions themselves, it is hardly surprising that the carbon data providers’ estimates often turn out to be even less reliable.

The statistics the researchers reached involved are complex. To illustrate the investment implication of their findings, the researchers imagined an investor wanting to avoid the 500 worst emitters from a universe of 10,000 stocks. This hypothetical situation is quite realistic, by the way, since 5% of companies are responsible for 80% of GHG emissions.

Given the imprecision of the GHG emission estimates, the researchers calculate that this hypothetical investor would have to exclude a lot more than 500 companies in order to avoid the bulk of those worst emitters. And I mean a lot more: This hypothetical investor would need to exclude 1,250 companies from consideration in order to avoid at least 95% of those worst emitters.

That’s a high price to pay.

There’s another unfortunate consequence of these imprecise estimates from the carbon data providers: It makes it difficult for us to identify the one or two “green” companies within a “brown” industry. That’s because the data providers’ estimates are largely based on industry averages.

Consider an energy company that emits significantly less GHG than other firms in its industry. If it doesn’t report the data itself, carbon data providers will estimate its GHG emissions, in large part by assuming that it was just as bad as the industry average. We therefore would never know that the company was better than its peers, and that’s a shame: It would be the company that many climate-conscious investors would want to favor in their portfolios.

Problematic as estimates are of companies’ past emissions, projections of companies’ future emissions are even less reliable. That’s important for investors to know, since many of them orient their portfolios around companies that carbon data providers rate highly for their plans to reduce their GHG emissions in coming years.

That’s a laudable thing to do, of course, since we want to encourage companies to do that. But the researchers found that the carbon data providers’ forward-looking scores “contain no useful … information.” Since there typically is little year-to-year change in a company’s GHG emissions, “investors are well served by taking the latest reported emissions as the proxy for next year’s emissions.”

The next steps

The solution to this database nightmare is straightforward: Companies should be required to report audited GHG emissions data that are calculated using a single, uniform standard. Period.

Until that universal reporting standard is in place, it will be next to impossible to know which companies are the best and worst emitters of GHG, as well as which are making the most progress towards reducing their emissions. And that will continue to be a big obstacle to mitigating climate change.

As the adage goes, “if you can’t measure it, you can’t manage it.”