EPA Energy Star Review Report Fails to Account for Uncertainties

The other day a friend emailed me a link to the EPA’s April 2019 report of its Review of the Energy Star Hotel benchmarking score.  In a nutshell, after suspending Energy Star Certification for the last six months or so pending a review of its revised methodology, the EPA has issued a report saying their revised methodology is correct and they are resuming operations.  But the statistics reported in this “Analysis and Key Findings” simply confirms what I have documented earlier in my book, that the Energy Star staff do not understand the difference between real trends and random noise.

On page 1 of their report the EPA Energy Star folks publish this table demonstrating how U.S. Lodging buildings have evolved between 2003 and 2012.

The EPA’s text accompanying this table says, “Between 2003 and 2012, the estimated number of hotel buildings in the United States increased by 14%. During that period, the average site EUI decreased by 3% while the source EUI increased by 7%.”

Presumably these statements are made in order to justify changes in Energy Star scores for Hotels/Motels — the building stock has changed so the relative ranking of a particular building with respect to this building stock will change.  Unfortunately the two EPA claims are false.

The table they used to justify this statement is not for Hotels — it is for all buildings classified by CBECS as Lodging.  This includes hotels, motels, inns, dormitories, fraternities, sororities, nursing home or assisted living, and “other lodging.”  Moreover, when you include the EIA’s relative standard errors (RSE) for both the 2003 and 2012 statistics you find these differences are absolutely meaningless.  In particular, the Site EUI figures for 2003 and 2012 in the above tables are uncertain by 17% (2003) and 8% (2012), respectively.  The differences between the 2003 and 2012 SiteEUI are just as likely to be due to random sampling errors as they are real trends!

The EPA’s Hotel Energy Star model applies only to Hotels and Motels/Inns.  When you look at these CBECS data for 2003 and 2012 you find even larger RSE that swamp any differences.  The relevant statistics are shown in the Table below.  The EIA did not calculate statistics for these two categories in 2003; these numbers are calculated by me using CBECS 2003 microdata.  The EIA did perform the calculations for these categories in 2012.  SourceEUI figures are calculated by me using the EPA’s 2012 site-to-source energy conversion figures (3.14 for electric).  The percentages listed are the RSE’s for each statistic.

The number of Hotels increased by 50% from 2003 to 2012.  During this same time the number or motels/inns decreased by 23%; their combined number showed no significant change and their combined floor area increased by 8%, hardly resolvable given the uncertainties in these quantities.  The Site and Source EUI for these two types of facilities did not change in any significant way.  The uncertainties in the survey far exceed the small changes in these EUI.  It is impossible to know whether the changes reflect real trends or just sampling errors.

Joelle Michaels, who oversees the CBECS operation, is well aware of the limitations of the CBECS methodology.  It must drive her nuts to see the Energy Star staff present such silly numbers and reports based on CBECS data.

This gets at the heart of my criticism in my book, Building Energy Star scores: good idea, bad science.  The numbers employed by the EPA in their statistical analysis are so uncertain that in most cases they are studying noise and reading into it things that cannot be found.  The science is sophomoric.  It is the result of teaching a little statistics to people who lack the mathematical and scientific knowledge to use it properly.

 

 

 

 

Hotel at Oberlin — poster child for “Green Wash”

In May 2016 Oberlin College opened its newly constructed Hotel at Oberlin.  The New York Times ranked the Hotel third in its list of 5 Hotels and 5 Tours for the Eco-conciousTraveler.  It is all part of the ongoing marketing effort to paint Oberlin College as a sustainable and green institution.  Hard to believe that any amount of eco-spin can convince people that a view of Oberlin’s Tappan Square is  environmentally rewarding.

Of course what makes the Hotel at Oberlin a green destination is not it surroundings — it is the building itself.  Like the Taj Mahal, committed environmentalists will simply swoon in the presence of this green wonder.  The second (and larger) of Oberlin College’s highly-publicized green buildings, the College has claimed that the Hotel is the first 100% solar powered hotel in the world and one of only five Hotels in the world to win the coveted LEED Platinum rating.  In addition to claims of solar power the building is said to be heated by a geothermal well field and to include other green technologies — including radiant-cooled rooms.  Its web site boldly claims that it has achieved the LEED platinum rating.

Truth is the hotel is not powered by the sun nor is it LEED-certified at any level.

I  wrote about this Hotel nearly two years ago when it opened.  The main focus of that post was to address the solar claim.  I will not rehash the evidence here — please read the blog.  The claim is a brazen and clever lie — Donald Trump would admire its creativity!  Simply stated, the Hotel is no more solar powered than is my century-old home.  There is not one solar panel on the building site.  The 2.2 MW OSSO array that is claimed to power the Hotel was built years before the hotel, is located a mile away, and, by contract, sends all of its electricity to the City of Oberlin until 2037 at a price of $85/MWh.

Today I write to share the Hotel’s energy-performance data and to discuss its LEED rating.  The Hotel is well into its second year of operation and we now have 21 months of utility data.

In my 2016 post I suggested that the Hotel would use two million kWh annually, more than double the 800,000 kWh used by the Oberlin Inn it replaced.  For 2017 the Hotel actually used 1,400,000 kWh of electric energy.  This is 75% more electric energy than was used by the former Oberlin Inn, but less than my estimate.  It is consistent with the annual electric use projected for the Hotel by its design team.

But the Hotel also uses natural gas.  The marketing literature for the Hotel says that the building is heated with ground-source heat pumps.  Natural gas, we are told, is primarily for heating domestic water (laundry, showers, etc.) — available, but not anticipated for backup heat.  The design team projected the annual gas use to be 8,350 therms (Ccf).

In fact, for 2017 the Hotel at Oberlin used 39,000 therms (Ccf), nearly 5X that predicted by the design team.  This is more natural gas than is used by any other Oberlin College building save one — the 130,000 sf Science Center!  The Science Center, constructed 17 years ago, contains numerous research and teaching laboratories and chemical hoods and has never been described as a green building.  It used 58,000 therm of natural gas in FY2017.  The natural gas use of the Hotel at Oberlin exceeds that of any other College building including the Firelands Dormitory (26,000 therm), the new Austin E. Knowlton complex (26,000 therms) and Stevenson Dining Hall (23,000 therms).

How does the Hotel at Oberlin’s energy performance compare with that of other hotels?  Consider its Energy Star score.  This can be estimated using the EPA’s Target Finder web site that allows quick data entry to estimate scores.  Entering the Hotel’s floor area (103,000 sf), number of guest rooms (70), cooking facility (Yes), 100% of the space heated and cooled, and actual FY2017 energy use, and accepting other default parameters, the Hotel at Oberlin is awarded an Energy Star score of 56.  According to the EPA — just a bit above average.  Don’t get me wrong — I am a huge critic of the Energy Star benchmarking score.  But it is one way to compare energy use with other hotels.

The monthly gas usage for the Hotel at Oberlin is shown below.  The excessive use in months Nov. – Feb. is clear evidence that significant gas is used for heating.  But even if you eliminate this heating use, the remaining use is nearly 3X the design estimate.

Finally, let me address the claim that the Hotel at Oberlin is certified LEED Platinum.  It simply is a lie.  I downloaded the USGBC LEED project database today.  The Hotel at Oberlin was registered on March 8, 2013 as “Confidential.”  Its LEED project ID is 1000031165.  As of today, February 23, 2017 the Hotel at Oberlin is not LEED-certified at any level.  The LEED project database says it has achieved 53 points — not enough to even achieve certification at even the Gold level.

Perhaps one day the claims being made for the Hotel at Oberlin will become true.  There is a lesson to be learned by looking at Oberlin’s Green building, generation-I, the Adam Joseph Lewis Center.

Oberlin College’s Adam Joseph Lewis Center  opened in 2000 to much acclaim.  Its proponents claimed it was a zero energy building (ZEB) for more than a decade when it just wasn’t true.  The claims were repeated by two Oberlin College presidents, College literature, and the College web site. The College never issued a retraction — it spent hundreds of thousands of dollars to correct flaws in the building’s HVAC design hoping to lower building energy use to a level that could be met by its 45 kW rooftop PV array.  The College eventually switched from “sticks” to “carrots” and in 2006, with the gift of a million dollars, built a second, 100 kW PV array over the adjacent parking lot and, with tripled electric production, renewed its ZEB claim for the building.  The building continued to use more energy than all of its arrays generated through 2011.  Even when faced with incontrovertible evidence that the claim was false the College continued to print the claim for another year in admissions literature distributed to students.  The College has never issued a public retraction or correction.  In 2012, after hiring a full-time building manager, the building finally used less energy that year than its PV arrays generated.  These arrays now feed two buildings, the AJLC and its adjacent annex.  Energy-intensive functions have been located in the annex and, collectively, these two buildings use more energy than the arrays produce.

Maybe in the next decade the College will build a parking garage next to the Hotel at Oberlin and put a huge PV array on it.  This could make the Hotel at Oberlin solar-powered — but not 100%.  Not sure how it will solve its natural gas problem — but clever minds will think of something.

The era of Donald Trump is here.  It is not illegal to lie, and no lie is too big to sell.

The bottom line is this.  The Hotel at Oberlin is just a normal, expensive hotel that purchases both electricity and natural gas from the local utility companies.  It uses more energy than the hotel it replaced.  It is the perfect symbol of modern green wash — 20 % substance, 60% exaggeration, 20% lies.

USGBC gives new meaning to Energy Star Score

This weekend I have been gathering data regarding LEED certified buildings made available at the Green Building Information Gateway.  In browsing through the web site I ran across a page that described Top Performing Buildings.  On that page I read this statement:

“One percent of buildings earned an Energy Star Score of 90+”

I don’t know if this statement is true or not — but I am humored by its implications.

According to the EPA, the building Energy Star score is a ranking of a building’s energy efficiency as compared with similar buildings in the U.S. commercial building stock.  It is assumed that the mean or median building score is 50 — simply reflecting the inescapable fact that half U.S. buildings are better than average and half are worse.  This is a necessary consequence of the meaning of a cumulative population distribution!

It also follows that 10% of the buildings necessarily have scores below 11 and 10% have scores higher than 90.

Perhaps it is true that only 1% receive scores that are 90 and higher.  But if true, the score clearly cannot reflect the meaning given it by the EPA.  Perhaps the author of that gbig web page needs to reflect on the meaning his/her/their statement.

NYC’s building energy grade discredits both Energy Star and LEED

I receive occasional newsletters from HVAC consultant Larry Spielvogel concerning building energy and the HVAC industry.  Yesterday he sent out a link to an editorial that appeared in Crains New York Business concerning a recent ordinance passed in New York City that “forces large buildings to post letter grades reflecting their energy use.”  These grades will apparently be based upon a building’s Energy Star score.

The Crains’ editor is aghast that a fine building like One World Trade Center which is LEED-gold certified, receives only a B grade.  Worse yet, the highly-acclaimed, LEED-platinum One Bryant Park building receives a C grade.  In closing the essay the editor writes, “Slapping a C next to a LEED Platinum rating will discredit both metrics, confuse the public and accomplish nothing.”

He is right on the first two counts but wrong on the third.  This will accomplish something very important, it will further the cause of truth!

It is better that the public be confused by the truth than to be told lies that bring clarity.  Confusion may lead to investigation and resolution.  The City’s new grade is based on the EPA’s building Energy Star score.  As I have shown in multiple venues, this score is largely garbage.  (See, for instance, earlier blogs from 2016-11-21, 2016-12-14, 2015-09-19, or 2014-08-22.)  The scoring system is mostly ad hoc, made up by non-engineers with a political agenda.  Armed with the knowledge acquired in a semester college statistics course they have developed scores that lack basis in building science or engineering .  They mean well — they want to help the environment.  Their approach is to condense building energy efficiency into a single metric that masses can understand and they can control.  But the score is largely meaningless and the DOE building scientists who helped develop the score 15 years ago have long since distanced themselves from this runaway system that lost its connection with reality.

LEED building certification, a system also born with good intentions, has been shown to have little average impact on building energy use!  LEED-certified office buildings in NYC use just as much energy as do other NYC office buildings.  Similar results have been uncovered in Chicago building energy benchmarking data.

The excessive energy used by One Bryant Park (aka The Bank of America Building) has been discussed before.  (See my earlier post and the New Republic article by Sam Roudman.)  For 2016, One Bryant Park had an annual site energy intensity of 211 kBtu/sf, more than twice that of the average NYC office building for 2015 (94 kBtu/sf). (For 2015 its energy use was somehow omitted from NYC’s public disclosure.)  The energy use of One World Trade Center (aka The Freedom Tower) has not appeared in the 2014, 2015, or 2016 NYC disclosures, despite the fact that the building opened in November 2014.  No doubt the Port Authority keeps its energy use secret as a matter of national security.

Nature does not care what awards these buildings have won or the clever technologies their owners have employed.  Nature only cares about total GHG emission and fossil fuel consumption and, by these measures, these buildings are not exemplary.

No doubt these building owners believe they are not responsible for the excessive energy use — it is their tenants.  True or not, it does not matter.  The building and its occupants are judged together.  If the owner is embarrassed — find different tenants.

 

The Illusions of EUI in Calculating Energy Savings

In the last month I have found the time to begin looking at the 2012 CBECS data released by the EIA last May.

Today I am writing about something I just learned concerning U.S. Worship Facilities.  Here I am looking at the subset of Worship Facilities that meet the criteria stated by the EPA for performing their multivariate regression for the Worship Facility ENERGY STAR model (about 80% of all U.S. Worship Facilities).

In comparing the 2012 and the 2003 CBECS data for Worship Facilities we see there was an estimated 2% increase in the number of these buildings.  As there is an 8-9% uncertainty in the estimated number of these facilities, this increase is  not statistically significant.  The EIA data show that the mean site energy use intensity (EUI) for these facilities actually went down by 15% from 48 to 41 kBtu/sf — and this reduction is statistically significant as it exceeds the 6-8% uncertainty in these figures.  No doubt some government agency will use this reduction to claim success in programs to promote energy efficiency.

But nature is not impressed because total energy used by these buildings actually went up.  The reason — the buildings are, on average, getting bigger!  From 2003 to 2012 the total gross square footage contained in this filtered subset of Worship Facilities increased from 3.2 to 3.8 billion sf, a whopping 23%.  Thus the total site energy used by Worship Facilities grew by 5%.  A similar conclusion can be made for source energy, even with the improved efficiency of the electric power sector over this last decade.

It should be noted that statistics show that the number of Americans who actually go to church declined by about 7% from 2007-2014.  So in a decade when religious worship is decreasing the amount of energy used by Worship Facilities has grown by about 5%.

Bottom line — don’t be fooled by decreases in building EUI.  It is total energy that matters.

New CBECS Data confirm EPA’s K-12 School ENERGY STAR score is nonsense

As I have written before — indeed, the subject of my recent book — my work shows that the EPA’s ENERGY STAR benchmarking scores for most building types are little more than placebos.  The signature feature of the ENERGY STAR benchmarking scores is the assumption that the EPA can adjust for external factors that impact building energy use.  This adjustment is based on linear regression performed on a relatively small dataset.  For most building types this regression dataset was extracted from the Energy Information Administration’s 2003 Commercial Building Energy Consumption Survey (CBECS).  The EPA has never demonstrated that these regressions accurately predict a component of the energy use of the larger building stock.  They simply perform their regression and assume it is accurate in predicting EUI for other similar buidings.

In the last three years I have challenged this assumption by testing whether the EPA regression accurately predicts energy use for buildings in a second, equivalent dataset taken from the earlier, 1999 CBECS.  In general I find these predictions to be invalid.    For one type of building — Supermarkets/Grocery Stores — I find the EPA’s predictions to be no better than those of randomly generated numbers!

In May of this year the EIA released public data for its 2012 Commercial Building Energy Consumption Survey.  These new data provide yet another opportunity to test the EPA’s predictions for nine different kinds of of buildings.  These new data will either validate the EPA’s regression models or confirm my earlier conclusion that they are invalid. Over the next year I will be extracting 2012 CBECS data to again test the nine ENERGY STAR benchmarking models based on CBECS data.

This week I performed the first of these tests for K-12 Schools.  539 records were extracted from the CBECS 2012 data for K-12 Schools representing 230,000 schools totalling 9.2 billion gsf.  After filtering these records based on EPA criteria, 431 records remain, representing a total of 137,000 schools with 8.0 billion gsf.

I performed the EPA’s weighted regression for K-12 Schools on this final dataset and obtained result totally inconsistent with those obtained by the EPA using CBEC 2003 data. Only 3 of the 11 variables identified by the EPA as “significant predictors” of building Source EUI for K-12 Schools demonstrated statistical significance with the 2012 data. Numerous other comparisons confirmed that the EPA’s regression demonstrated no validity with this new dataset.

The EPA will no doubt suggest that their model was valid for the 2003 building stock, but not for the 2012 stock — because the stock has changed so much in the intervening 9 years! While this seems plausible, this explanation does not hold water.  First, CBECS 2012 data do not suggest significant change in either the size or energy use of the K-12 School stock.  Moreover, this explanation cannot also explain why the EPA regression was not valid for the 1999 building stock — unless the EPA is to suggest that the stock changes so much in just 4 years to render the regression invalid.  And if that is the EPA position — then why would they even attempt to roll out new ENERGY STAR regression models for K-12 Schools based on 2012 CBECS data more than 4 years after these data were valid?  You can’t have it both ways.  Either the stock changes rather slowly and a 4 year delay is not important or this benchmarking methodology is doomed to be irrelevant from the start.

 

The more plausible explanation — supported by my study — is that the EPA’s regression is simply based on insufficient data and is not valid — even for the 2003 building stock.  I suggest a regression on a second, equivalent sample from the 2003 stock would yield results that differ from the EPA”s origina regression.  The EPA’s ENERGY STAR scores have not more validity than sugar pills.

 

“Building ENERGY STAR scores – good idea, bad science” book release

After more than three years in the making I have finally published my book, Building ENERGY STAR scores — good idea, bad science.  This book is a critical analysis of the science that underpins the EPA’s building ENERGY STAR benchmarking score.  The book can be purchased through Amazon.com.  It is also available as a free download at this web site.

rotated

I first began looking closely at the science behind ENERGY STAR scores in late 2012. The issue had arisen in connection with my investigation of energy performance of LEED-certified office buildings in New York City using 2011 energy benchmarking data published by the Mayor’s office.  My study, published in Energy & Buildings, concluded that large (over 50,000 sf) LEED-certified office buildings in NYC used the same amount of energy as did conventional office buildings — no more, no less.  But the LEED-certified office buildings, on average, had ENERGY STAR scores about 10 points higher than did the conventional buildings.  This puzzled me.

So I dug into the technical methodology employed by the EPA for calculating these ENERGY STAR scores.  I began by looking at the score for Office buildings.  Soon thereafter I investigated Senior Care Facilities.  Over the next three years I would dig into the details of ENERGY STAR models for 13 different kinds of buildings. Some preliminary findings were published in the 2014 ACEEE Summer Study on Energy Efficiency in Buildings.  A year later I would present a second paper on this topic at the 2015 International Energy Program Evaluation Conference (IEPEC)  Both of these papers were very limited in scope and simply did not allow the space necessary to include the detailed analysis.  So I decided to write a book that contained a separate chapter devoted to each of the 13-types of buildings.  In time the book grew to 18 chapters and an appendix.

This book is not for the general audience — it is highly technical.  In the future I plan to write various essays for a more general audience that do not contain the technical details. Those interested can turn to this book for the details.

As mentioned above the printed copy of the book is available through Amazon.com. Anyone interested in an electronic copy should send me a request via email with their contact information. Alternately an electronic copy may be downloaded from this web site.

Incidently, the book is priced as low as possible — I do not receive 1 cent of royalty.  The cost is driven by the choice of large paper and color printing — it was just going to be too much work to re-do all the graphs so that they were discernable in black and white!