This is the third in a series of posts regarding “grade inflation” for the EPA’s Energy Star building benchmarking score.  The first article looked at Medical Office buildings and the second looked at Dormitories/Residence Halls.  Here we look at evidence for inflation in scores for Office buildings.

Energy Star benchmarking was first introduced for office buildings back in 1999.  Office buildings are the largest single building type in the commercial building stock.  Since its introduction the EPA’s Office model has been revised twice, first in 2003 (based on 1999 CBECS data) then again in 2007 (based on 2003 CBECS data).  The CBECS data on which the present model is based are now 10 years out of date.  The model applies to office buildings, financial centers, banks, and courthouses – but for brevity I will simply call it the Office Model.

The current version of the Office Model used office building data from the 2003 CBECS to define its parameters — that is, model parameters were obtained from a regression applied to office building data extracted from CBECS 2003.  Therefore these data cannot be used to independently verify the distribution of Energy Star scores in the building stock.

But the 1999 CBECS data do provide an independent snapshot of the building stock that can be used to test whether or not Energy Star scores, as defined by the current Office model, are appropriately distributed.  While the building stock certainly changed somewhat from 1999 to 2003, there is no evidence that it experienced significant changes in energy consumption or efficiency.

I have extracted all office building data from the 1999 CBECS database, omitting buildings that are outside the scoring parameters of the Office model.  (For instance, the model only applies to buildings 5,000 sf or larger.  And CBECS energy consumption data are inaccurate for any of its buildings that utilize district chilled water.)  After extracting CBECS 1999 data for eligible buildings I used the 2007 Office model to calculate their Energy Star scores, then using the CBECS weights for each sampled building, produced a histogram of Energy Star scores for the entire (eligible) office/finance/bank/courthouse building stock.  The resulting histogram is shown below.

This histogram represents Energy Star scores from an estimated 314,000 buildings occupying a total of 9.7 billion gsf!  The average ES for the 314,000 buildings is 62.  The graph clearly shows that the scores are not uniformly distributed and, instead, there is an overabundance of scores higher than 50.  A salute to lake Wobegon!

The above graph provides convincing evidence that 1) Energy Star scores for Office buildings do not represent their percentile ranking in the office building stock, and 2) that a score of 75 – required for Energy Star certification – does not mean that a building is using 30% less (source) energy than the average office building.

Admittedly, this is a test of the 1999 Office Building stock, not the 2013 building stock.  But it is not plausible that the commercial office building stock has gotten so much less efficient since 1999 that the histogram for 2013 buildings is uniform.  In any case, in 2014 when CBECS 2012 data are released we will have another independent survey to test this hypothesis.

To recap – this last week I have looked at the distribution of Energy Star scores for 1) medical office buildings, 2) dormitories/residence halls, and 3) offices/financial centers/banks/courthouses – and in all cases have found evidence that the mean scores for the commercial building stock is significantly higher than 50 – means were 65, 70, and 62, respectively.  This provides convincing evidence that the scoring system itself is biased to high scores – and hence the score does not represent a building’s energy efficiency percentile ranking in the population.

It would appear that the reason that the mean score for all buildings whose data are entered into Portfolio manager is 62 has a rather simple explanation — the scoring system is biased that way.  Sometimes you can’t judge a book by its cover — other times you can.

It is worth mentioning that having a uniform distribution of Energy Star scores in the building stock is a necessary, but not a sufficient condition, that the Energy Star score is legitimate.  The “bias problems” indicated above can be easily fixed – but it will represent disruptive shifts in the Energy Star score.  (How does this impact LEED and other external organizations that have adopted the Energy Star score as their metric for energy efficiency?)  After this bias is addressed – there are still legitimate questions to ask about the regression model itself.  It turns out that the definition of building energy efficiency is contained in the way the regression model is constructed – choices on regression variables impact the definition of energy efficiency.  These issues will be addressed in subsequent posts.