This month I am beginning a series of articles to discuss the science (or lack thereof) behind the US Environmental Protection Agency’s building Energy Star benchmarking score.  Energy benchmarking has become very popular these days with eight or more major US cities having passed ordinances requiring commercial buildings to benchmark their energy data.  The EPA’s Energy Star Portfolio Manager is being used by all these cities for this effort.  In addition, both the US Green Building Council and Green Globes have adopted the building Energy Star score as the metric for energy efficiency success in their green building certification programs.

What is Benchmarking?

Benchmarking is a process by which you compare the energy used by your building with that used by other buildings in order to learn how you stand relative to “the pack.”  The energy used by your building is easily quantified by simply recording monthly energy purchases, combining data for twelve consecutive months to determine your annual energy consumption.  Anyone interested in lowering operating costs or improving the operation of a specific building might decide to to track their own annual energy consumption, comparing annual usage for successive years.  Simply comparing annual energy use for successive years of the same building can guide a building manager in making equipment and operational changes intended to improve energy efficiency.

But it is also useful to know how your energy use compares with energy used by other, similar buildings.  This is really what benchmarking is all about.  If you learn that your building uses much more energy than most other similar buildings – that would suggest there are some changes you can make to significantly lower your own energy consumption (and cost).  If, on the other hand, your building uses much less energy than most other buildings – then it probably does not make sense to invest a lot of time and energy in making further energy efficient improvements to your building.

the Commercial Building Energy Consumption Survey

So how do you find out how much energy other buildings use?  The basic tool for this is the Commercial Building Energy Consumption Survey (CBECS) usually conducted every 3-4 years by the Energy Information Administration (EIA).  The US commercial building stock consists of about 5 million buildings with 70 billion sf of floor space.  CBECS is designed to gather data from a small fraction of these buildings (about 6,000) specifically chosen to accurately represent the entire building stock.  In addition to recording size and annual energy purchases for these buildings the survey gathers numerous other pieces of information to characterize these buildings and how they are used.  Strict confidentiality is maintained for the 6,000 or so sampled buildings.  Nevertheless, sufficient data are gathered to perform queries on the data to learn average properties for various kinds of buildings broken down by climate region, function, size, age, and use.  The last CBECS to be performed was in 2003 and data for the next survey (2012) are to be released in 2014.

The Energy Star Building Score

In 1999 the EPA first introduced its Energy Star building score for office buildings, the most common building type.  The score is a number ranging from 1-100 that is intended to represent a particular building’s percentile ranking with respect to energy consumption as compared with similar buildings nationally.  So, if your building receives a score of 75 that is supposed to mean that, if you were to look at all similar buildings across the country, your building uses less energy than 75% of them, adjusting for indicated operating conditions.  Office buildings are the most common type of building.  Presumably if it were possible to determine the Energy Star score for every office building in the country you would find that half of them have scores ranging from 1-50 and the rest from 51-100.  Similarly you expect 10% of office buildings to have scores ranging from 91-100 and another 10% would have scores from 1-10, etc.  In general, you would expect a histogram of Energy Star scores for all office buildings to look like this.

The Problem with Energy Star Scores

In the last 8 years or so more and more building studies have published the Energy Star scores for fairly large sets of buildings.  For some reason the mean Energy Star scores for these buildings sets always seems to be greater than 50.  It is, of course, possible that, in each case, the buildings studied represented “better than average” buildings.  But it also raises the question – how do we know that the Energy Star scores for all US buildings are distributed as expected?  What evidence has the EPA ever offered to demonstrate the validity of these scores?  So far as I can tell the answer is none.  There are no peer-reviewed articles and no masters or Ph.D. theses describing these models and the numerous tests undertaken to demonstrate their validity.  All we have are rather short technical descriptions of algorithms used to define the models.  In fact, the EPA has known for years that the mean Energy Star score for all buildings whose data were entered into Portfolio Manager was 60 (now 62).  You would think they might want to investigate why?

One obvious way to test this is to conduct a random sample of a large number of US commercial buildings, use EPA algorithms to calculate their Energy Star scores, and see how these scores are distributed.  But the only such sample is CBECS!  When the 2012 CBECS data become available this will afford an excellent opportunity to conduct such a test – that should be sometime in 2014.  (Meanwhile, thousands of commercial buildings in major US cities are benchmarking their buildings using these Energy Star models.)  For many building types the CBECS 2003 data were the basis for the associated Energy Star model – this is the case for the current model for office buildings.  In these cases the 2003 CBECS data cannot provide independent confirmation of the Energy Star models.

But there are a few building types for which the Energy Star models are based on 1999 CBECS data.  One such building type is “Medical Office Buildings.”  In this case we can extract data for medical office buildings from CBECS 2003, calculate their Energy Star scores using the EPA’s model, then generate a histogram to show how these scores are distributed for all medical office buildings contained in the 2003 US commercial building stock.  The distribution is expected to be uniform as shown in the Figure above with some random uncertainty, or course.

I have done just that and the results are graphed below.  The graph clearly demonstrates that the scores are not uniformly distributed, and therefore the score cannot have the stated mathematical interpretation.  The mean Energy Star score is 65 well above the expected value of 50.  Nearly 45% of US medical office buildings have Energy Star scores from 81-100 – significantly higher than the expected 20%. and only 8% have scores ranging from 11-40, well below the expected 30%!  It is highly unlikely that US medical office buildings saw massive improvements in energy efficiency from 1999 to 2003.  The explanation is simpler — the model is based on faulty assumptions.

This graph clearly calls into question the validity of the Energy Star Medical Office building model.  This model was developed in 2004 and has been in use for nearly a decade.  Is it possible that the EPA never conducted this simple test to check the validity of this model?   It would appear that for a decade now the EPA has employed a flawed building model to generate Energy Star scores for medical offices and to draw conclusions about the amount of energy the Energy Star program has saved.

If this one model is wrong — and the error went undetected so long — what confidence can we have in Energy Star models for other building types?

In my next issue I will look at the distribution of Energy Star scores for Dormitories/Residence Halls.