When will the US Senate conduct hearings on “energy loss” programs?

Yesterday a Senate Committee grilled “Dr. Oz” about the promotion of weight-loss products on his show.  At issue are the unsupported claims made for these products and the false hopes of millions of viewers who are looking for quick ways to lose weight.  I would like to know when the Senate will grill proponents of green buildings in the same way.

Don’t get me wrong — I know that Americans need to lose weight, and there are very clear ways to do that with slow, determined change in behavior.  The same is true for improving building energy efficiency.  There are clear ways to cost-effectively improve buildings so that they use 10-20% less energy without any loss of performance.

But Americans want quick solutions — ways to lose 30 pounds in one month without pain or suffering.  And there is an entire industry out there selling products which promise to achieve these very results.  But there is no scientific evidence to support such claims, and mostly people spend their money on these products and never reap their promised benefits.  The few who do achieve the desired weight loss do it because of their regular exercise and reduction in caloric intake — perhaps coincident with the use of some new product, but having no other connection to it.

America’s energy-guzzling buildings have much in common with its overweight population.  And a government-sponsored industry – not unlike the one promoted by Dr. Oz – has emerged promoting green buildings, zero energy buildings, and high-performance buildings — all promising great energy savings for those who adopt their strategies. The US Green Building Council claims that its LEED-certified buildings are achieving 47% energy savings.  The EPA claims that its ENERGY STAR benchmarking program yields significant energy savings.  The New Buildings Institute promotes Zero Energy Buildings as the ultimate “weight loss program.”  The US Federal government pours millions of dollars into GSA, DOE and EPA programs that prumulgate these ideas.  My own state of Ohio has spent millions on LEED-certified schools without a single scientific study to demonstrate that these buildings actually save energy.  The list of organizations and claims goes on and on.

Yet the above claims are, at best, outrageous exagerations.  The USGBC claim is made for a “cherry picked” subset of its buildings and is based on ENERGY STAR scores which have no scientific credibility [see earlier post].  The EPA claims are similarly based on ENERGY STAR scores and do not stand up to close inspection.  And close inspection of data gathered by NBI shows that, at most, about 10 US commercial buildings in the country have demonstrated net-zero performance.  Amercia is spending millions on these green and high performance buildings efforts with little data to demonstrate efficacy.

Don’t get me wrong — I am a stong advocate of cost-effective energy efficiency and energy conservation.  I am also a strong advocate of exercise and sensible nutrition.

 

 

DC Benchmarking data show modest energy savings for LEED buildings

A few months ago Washington DC released its 2012 energy benchmarking data for private commercial buildings 150,000 sf and larger.  Credible energy and water consumption data for some 400 buildings were released, of which 246 were office buildings.  A recent article — stemming from the web site LEED Exposed — has claimed that these data show LEED buildings use more energy than non-LEED buildings.  Specifically it is claimed that LEED buildings have an average weather normalized source EUI of 205 kBtu/sf whereas non-LEED buildings have an average EUI of 199 kBtu/sf.   No details are provided to support this claim.

My students and I have cross-listed the DC benchmarking data with the USGBC LEED Project directory and identified 94 LEED-certified buildings in the 2012 DC benchmarking dataset — all but one being classified as office buildings.  The unweighted mean weather normalized source EUI for these 94 LEED certified buildings is 202 kBtu/sf.   The unweighted mean weather normalized source EUI for remaining 305 buildings is 198 kBtu/sf.  No doubt this is the basis for the claim that LEED buildings use more energy than non-LEED.  However, the difference is not statistically significant.

Moreover, the non-LEED dataset, in addition to 154 office buildings, contains 64 (unrefrigerated) warehouses and 90 multifamily housing buildings — all of which use significantly less energy than do office buildings.  The comparison of these two average EUI is not useful — just a meaningless sound bite.

It should also be noted that the unweighted mean EUI for a collection of buildings is an inappropriate measure of their total energy consumption.  The appropriate measure of energy consumption is their gross energy intensity — their total source energy divided by the total gross square footage.  This issue has been discussed in several papers [2008 IEPEC; 2009 Energy & Buildings].

Note that an apples-to-apples comparison of energy consumed by one set of buildings to that consumed by another requires that the two sets contain the same kinds of buildings in similar proportions.  When possible this is best accomplished by sticking to one specific building type. Since office buildings are far and away the most common in both datasets it makes sense to make an office-to-office comparison — pun intended. 

93 of the LEED-certified buildings are offices.  But many of these buildings were not certified during the period for which data were collected.  Some were certified during 2012 and others were not certified until 2013 or 2014.  Only 46 of the office buildings were certified before Jan. 1, 2012 and are then expected to demonstrate energy and GHG emissions savings for 2012.

The 2012 gross weather-normalized source energy intensity for the 46 LEED certified office buildings is 191 kBtu/sf.  This is 16% lower than the gross weather-normalized source energy intensity for the 154 non-certified office buildings in the dataset, 229 kBtu/sf.  These modest savings are real and statistically significant, though much lower than the 30-40% savings routinely claimed by the USGBC.

Note that similar savings were not found in 2011 or 2012 NYC energy benchmarking data. Analysis of these data showed that LEED-certified office buildings in NYC used the same amount of primary energy and emitted no less green house gases than did other large NYC office buildings.  So the 2012 results from Washington DC are significantly different.  It should be noted that NYC office buildings certified at the gold level were found to exhibit similar modest energy savings.  Perhaps this is a clue as to why Washington DC LEED buildings show energy savings.  More analysis is required.

For the last few years the USGBC has pointed to ENERGY STAR scores for LEED certified buildings as evidence of their energy efficiency.  While ENERGY STAR scores have two important characteristics — they use source rather than site energy and they are based on actual energy measurements — they simply do not have sound scientific basis.  The science has never been vetted, and my own analysis shows these scores are little more than placebos to encourage energy efficiency.  They certainly do not have any quantitative value.

So to summarize, in 2012 LEED offices in Washington used 16% less source energy than  did other office buildings in DC.  What this means and whether such savings justify the added costs of LEED are open questions.

Another look at the analysis by Pollock and Rosiak

A few months ago I called attention the Washington Examiner article by Richard Pollock and Luke Rosiak.  On first read it provided more evidence confirming a trend I have seen in several data sets — that LEED-certified buildings, on average, are not saving primary energy — in this particular case, as measured by ENERGY STAR scores.

But 24 hours later I pulled my original post.  I am not convinced that this cursory study is sufficiently rigorous to stand up to scrutiny.  There are two key reasons for my skepticism.  The first is that over the last year I have leaned that the EPA’s ENERGY STAR building rating system is built on a “house of cards.”  It may encourage building energy efficiency, but it is not founded on good science and there is little reason to believe that a higher ENERGY SCORE means a more energy efficient building.

The second reason is more complicated.  Comparing the energy use of one group of buildings with another is actually difficult to do correctly.  Several of my papers have concluded that other researchers have gotten it wrong in the past.  It is certainly not an activity to be left to people who begin the study with a stake in the outcome — either those promoting LEED or those “dug in against it.”  And, despite the publicity surrounding Thomas Frank’s study in USA Today, it is an activity best left to building professionals and researchers — not reporters.

In this particular case it appears to me that many of the buildings identified by Pollock and Rosiak as “LEED buildings” are not actually LEED-certified at all — they are mostly LEED registered projects.  Anyone can “register” a LEED project — simply stating intentions and requiring a modest fee.  But only minority of those projects that register actually see the process to completion and become LEED-certified.  As critical as I have been of the USGBC’s past claims about energy savings — I cannot hold the USGBC responsible for energy consumption of buildings that have never completed LEED certification. Maybe the registered LEED buildings in this article will soon complete certification.  If and when that occurs then we should look at their subsequent performance.

Washington DC has just released benchmarking data for all its large commercial buildings.  One of my students is in the process of parsing this list to identify LEED and ENERGY STAR certified buildings.  The goal will then be to compare the performance of LEED certified buildings with other buildings.  Only then can we draw any conclusions.

USGBC Continues to “cherry pick” LEED energy data

At the 2007 GreenBuild Conference the USGBC released the results of their first major study of energy consumption by LEED-certified buildings.  Then they presented conclusions from the now infamous study conducted by the New Buildings Institute (paid for by the USGBC and EPA) which, based on data “volunteered by willing building owners” for only 22% of the eligible buildings certified under LEED NC v.2, concluded that LEED certified buildings, on average, were demonstrating the anticipated 25-30% savings in (site) energy.

NBI’s analysis and conclusions were subsequently discredited in the popular media by Henry Gifford and in the peer-reviewed literature by me [see IEPEC 2008 and Energy & Buildings 2009].  NBI’s analytical errors included:

  1. comparing the median of one energy distribution to the mean of another;
  2. comparing energy used by a medium energy subset of LEED buildings with that used by all US commercial buildings (which included types of buildings removed from the LEED set);
  3. improper calculation of the mean (site) energy intensity for LEED buildings and comparing this with the gross mean energy intensity from CBECS;
  4. NBI looked only at building energy used on site (i.e., site EUI) rather than on- and off-site energy use (i.e., source EUI).

To NBI’s credit they made their summary data available to others for independent analysis with no “strings attached.”  In the end even the data gathered by NBI, skewed towards the “better performing” LEED buildings by the method for gathering data, when properly analyzed demonstrated no source energy savings by LEED buildings.  LEED office buildings demonstrated site energy savings of 15-17% — about half that claimed by NBI, the difference being associated with NBI’s improper averaging method.  This site energy savings did not translate into a source energy savings because LEED buildings, on average,  used relatively more electric energy, and the off-site losses associated with this increased electric use wiped out the on-site energy savings.

The lack of representative building energy data was addressed in LEED v.3 (2009) by instituting a requirement that all LEED certified buildings supply the USGBC with annual energy consumption data for five years following certification.  Never again would the USGBC have to publish conclusions based on data volunteered by 1 in 5 buildings.  Expectations were high.

But what has this produced?  The USGBC has learned from their experience with NBI — not to hand over such an important task to an outside organization because you can’t control the outcome.  NBI’s analysis was scientifically flawed — but it was transparent, and such transparency gave critics ammunition to reach different conclusions.  Nowadays the USGBC simply issues carefully packaged sound bites without supplying any details to support their conclusions.  There isn’t even a pretense of conducting scientifically valid analysis.

Consider the most recent claims made by the USGBC at the 2013 Greenbuild conference, summarized by Tristan Roberts in “LEED buildings above average in latest energy data release.”  Roberts asserts the following:

  1. The USGBC has received energy data from 1,861 certified buildings for the 12-mos period July 2012 – June 2013;
  2. About 70% of these were certified through LEED-EBOM (existing buildings);
  3. 450 of these buildings reported their data through the EPA’s Portfolio Manager;
  4. the “building-weighted” (or un-weighted) average source EUI for these 450 buildings is 158 kBtu/sf;
  5. this average is 31% lower than the national median source EUI;
  6. 404 (of the 450) buildings above were eligible for (and received) ENERGY STAR scores;
  7. the average ENERGY STAR score for these 404 buildings was 85.

In addressing the above claims it is hard to know where to begin.  Let’s start with the fact that the USGBC only provides energy information for 450 (or 24%) of the 1,861 buildings for which it has gathered data.  Is this simply due to the fact that it is easier to summarize data gathered by Portfolio Manager than data collected manually?  If so I willingly volunteer my services to go through the data from all 1,861 buildings so that we can get a full picture of LEED building energy performance — not just a snapshot of 24% of the buildings which “self-select themselves” to benchmark through Portfolio Manager.  (The EPA has previously asserted that buildings that benchmark through Portfolio manager tend to be skewed towards “better performing” buildings and are not a random snapshot of commercial buildings.)

Next, consider the “un-weighted” source EUI figure for the 450 buildings.  This is a useless metric.  All EUI reported by CBECS for sets of buildings are “gross energy intensities” equivalent to the gsf-weighted mean EUI (not the un-weighted or building-weighted mean EUI).  This was a major source of error in the 2008 NBI report — leading NBI to incorrectly calculate a 25-30% site energy savings rather than the actual 15-17% site energy savings achieved by that set of LEED buildings.

Consider the assertion that the 158 kBtu/sf source EUI figure is 31% lower than the median source EUI (presumably for all US commercial buildings).  To be correct this would require the median source EUI for all US commercial buildings be 229 kBtu/sf.  This is rubbish.  The best way to obtain such a median EUI figure is from the 2003 CBECS data.  The Energy Information Administration (EIA) does not report source energy figures in any of its CBECS reports.  But the EIA does report site and primary electric energy used by buildings, and these may be combined to calculate source EUI for all 2003 CBECS sampled buildings.  This results in a median source EUI for the estimated 4.9 million commercial buildings to be 118 kBtu/sf.  If you instead restrict this calculation to all buildings with non-zero energy consumption you find these estimated 4.6 million buildings have a median source EUI of 127 kBtu/sf — way below the 229 kBtu/sf figure asserted by the USGBC.  This USGBC claim is patently false.  Of course the USGBC may be referring to the median source EUI of some unspecified subset of U.S. buildings.  By choosing an arbitrary subset you can justify any claim.  And if you don’t specify the subset — well, the claim is nothing more than noise.

What about the average ENERGY STAR score of 85?  Is this impressive?  The answer is no.  Even if you believed that ENERGY STAR scores were, themselves, meaningful, such an average would still mean nothing.  ENERGY STAR scores are supposed to represent percentile rankings in the U.S. building population.  Since there are 4.8 million buildings, by definition we would expect 10% of these (or 480,000) to rank in the top 10% and we would expect another 480,000 of these to rank in the bottom 10%.  That means that if 1,861 buildings are chosen at random from the building population, we expect 10% of these to have ENERGY STAR scores from 91-100.  Similarly, we expect 30% of these (or 558) to have ENERGY STAR scores ranging from 71-100.  Guess what — the average ENERGY STAR scores of these 558 buildings is expected to be 85.  Only those who are mathematically challenged should be impressed that the USGBC has found 404 buildings in its set of 1,861 that have an average ENERGY STAR score of 85.  If you cherry pick your data you can demonstrate any conclusion you like.

And, of course, these 1,861 buildings are not chosen at random — they represent buildings whose owners have a demonstrated interest in energy efficiency apart from LEED.  I would guess that the vast majority of the 404 buildings were certified under the EBOM program and have used Portfolio Manager to benchmark their buildings long before they ever registered for LEED.  LEED certification is just another trophy to be added to their portfolio.  No doubt their ENERGY STAR scores in previous years were much higher than 50 already.  What was the value added by LEED?

I openly offer my services to analyze the USGBC energy data in an unbiased way to accurately asses the collective site and source energy savings by these LEED buildings.  How about it Brendan Owens (VP of technical development for USGBC) — do you have enough confidence in your data to take the challenge?  Which is more important to you, protecting the LEED brand or scientific truth?

ENERGY STAR energy benchmarking is not ready for prime time

I recently had occasion to read an old paper by Janda and Brodsky describing the “first class” of ENERGY STAR certified office buildings.  This is one of only a handful of papers in the peer-reviewed literature regarding ENERGY STAR building scores.  Janda and Brodsky describe the brand name ENERGY STAR as

a set of voluntary partnerships between the U.S. government and product manufacturers, local utilities, home builders, retailers, and businesses.  These partnerships are designed to encourage energy efficiency in products, appliances, homes, offices, and other buildings.

This was the basis for the EPA’s building ENERGY STAR scoring system.  It was a “game” that building managers voluntarily agreed to play with rules (methodology for scoring buildings) set by the EPA in consultation with those playing the game.  There was no scientific vetting of the “rules of the game” — nor did there need to be — it was just a game designed to “encourage energy efficiency.”  No one was forced to play the game.  Data submitted to Portfolio Manager (the EPA’s web-based tool for calculating scores) and ENERGY STAR scores issued by the EPA were confidential — unless a building sought and received ENERGY STAR certification.  Participation was entirely voluntary.  Building managers disappointed with their ENERGY STAR scores could just walk away from the game — no harm, no foul.

But this has all changed.  In recent years 1) the EPA has published specific claims regarding energy savings associated with its ENERGY STAR benchmarking program (real savings not just fantasy football), 2) external organizations like the USGBC have adopted the ENERGY STAR score as their metric for energy efficiency in green building certification programs and are using these scores to make energy savings claims of their own, and 3) major U.S. cities have passed laws requiring commercial building owners to use Portfolio Manager to benchmark their buildings and, in many cases, the resulting ENERGY STAR scores are being made public.  With federal, state, and local governments requiring LEED certification for public buildings this is no longer a voluntary game — it is mandatory and real (testable) energy claims are being made based upon ENERGY STAR scores.  Now the science behind such claims actually matters — and this science has never been vetted.

Its kinda like a small, “mom and pop” operation that has been selling chicken soup using “grandma’s recipe” without obtaining proper license or FDA approval.  Now imagine Walmart decides to market and sell the soup — the scrutiny changes.

As a voluntary game with no connection to reality it is OK that the EPA negotiates rules for its ENERGY STAR ratings with different constituents — like allowing Washington DC office buildings to ignore their “first floors” in seeking ENERGY STAR certification.  After all, who am I to interfere in the activities between consenting adults when these activities do not affect me?  But for ENERGY STAR — these days are gone.

In the next year we will learn much about the science that underpins the EPA’s ENERGY STAR benchmarking system — and the results are likely to be very disappointing.  This benchmarking system is not ready for prime time.

Hero Stories and Learning Stories

Kathryn Janda and Marina Topouzi published an interesting paper at the 2013 Summer Study on Energy Efficiency sponsored by the European Council for an Energy Efficient Economy (ECEEE).  The title of their paper is “Closing the loop: using hero stories and learning stories to remake energy policy.”  Kathryn Janda spent a few years on the faculty at Oberlin College and refers to the College’s “trophy building” — the Adam Joseph Lewis Center as one of her examples.

Janda and Topouzi argue that “hero stories” describe something that is bigger than life and not realistic.  They draw parallels with narratives offered for energy efficient buildings and projects — the “stories” or claims are unrealistic and, all too-often, reality falls short of the promise.  In the end they argue that there is an interesting “learning story” in the reality — one that is all too-frequently left untold.

The paper is interesting and well-written — not at all like most scientific papers.  Rather than provide a poor summary of their paper let me provide this quotation from their abstract:

This hero story, where we are saved by clever technologies, is inspiring, positive, and familiar. In this story, we don’t need to do anything because the technology will do it for us. But how real is it? The counterpart to the hero story is the learning story, where things are not quite as simple as they first seemed. In a learning story, protagonists are normal people who need to rise to a challenge. They are not saved by Superman, they have to save themselves. The learning story in energy policy lies in the between the technical potential and what is achieved in practice.

The piece is a refreshing read.

Pertamina Energy Tower to generate all its energy — really?

Fast Company has a short article by Adele Peters that describes some design features for the 99-story Pertamina Energy Tower that is planned for construction in Jakarta.
The building, designed by Skidmore, Owings & Merrill, will be the “first supertall tower in the world to generate its own power” apparently by capturing wind power at the top of the building and sucking geothermal from the bottom.  Excuse me if I am a bit skeptical.

3023870-inline-pertamina-energy-tower1tower-from-roundaboutsom

Flash back 15 years to a different architect — William McDonough – whose firm designed the Oberlin College’s Adam Joseph Lewis Center.  Bill is the standard bearer for architects that make fantastic claims backed up by little substance [Green guru gone wrong - Fast Company].  This success requires a media willing to publish such claims years before we ever see the fruit of the work.  When the fruit finally arrives — shriveled and disappointing — it matters little because the architect has moved on.

Consider Oberlin College’s Adam Joseph Lewis Environmental Center.  Long before its construction in 2000 McDonough was claiming he had designed a zero energy building — one that would be powered by solar cells mounted on its roof.  Through the years his claims never wavered.  Never mind that, following construction, the building consumed three times the energy his firm claimed it would [Early performance of a green academic building - ASHRAE] or that years later, after hundreds of thousands of dollars of HVAC modifications, it consumed double his initial projections — less than half the energy provided by its rooftop array.  Now, after 12 years of false claims the building has finally moved “into the black” powered by a $1,000,000 PV array constructed over a nearby parking lot which, combined with its $400,000 rooftop array, produces 3X as much energy as the the design team projected for the building.  [A paler shade of green - HPAC Mag]  I wonder how much attention McDonough would have gotten in 1998 had he told the world that he had designed, at a cost of $500/sf, a 14,000 sf building that used 20% less energy than conventional buildings and was to be powered by PV panels covering its roof and a parking lot, costing $1,400,000.  (That is an additional $100/sf just for the solar arrays.)  Excuse me Mr. McDonough — what was your “value added” — publicity?

But this post is about another project — the Pertamina Energy Tower.  My point is this.  Talk is cheap.  There is little doubt that this building will not deliver on its promises.  But for now the architects can spin their exciting story to media who don’t ask the hard questions — like “wouldn’t a shorter, fatter building be cheaper and use less energy? Or, “how much rental space is lost to provide space for the wind turbines and what is the cost/benefit of this?”  Physicists are good at “back of the envelope” calculations.  A 99-story building seems like an expensive way to mount a wind turbine.