NYC Energy Benchmarking Report Over-estimates Energy Savings

The Mayor’s Office in New York City has recently released their annual report looking at the 2013 energy data for commercial buildings.  This is the fourth such report.  Each annual report appears to take longer and longer to prepare suggesting it is easier to gather energy data than to analyze and understand it.

The lead line in this report is that those preparing the report conclude that over a four-year period (2010-2013) green house gases associated with NYC building energy has decreased by 8% and energy use by buildings has decreased by 6%.  They cannot resist suggesting that NYC’s energy benchmarking program can take credit for this reduction.

My analysis of these data show the savings is only half this amount. The other half of the claimed savings is an artifact of the EPA’s having lowered its national, site-to-source energy conversion factor for electricity in Summer 2013.  The same mistake was made by the Washington DC Department of the Environment a year ago.

NYC does not live in a vacuum.  Over the last 10 years expanded use of natural gas and retirement of coal plants has cleaned up the entire U.S. electric grid — of which NYC is a part.  In fact, the purchase of fracked natural gas from Pennsylvania (fracking is outlawed in NY State) is the primary driver of reduced green house gas emission in NYC.  It has little to do with NYC building policies!

The NYC analysis apparently comes from adding up the annual greenhouse gas emission and weatherized source energy use of some 3,000 properties that submitted benchmarking data for all four years.  Using 2010 figures as a baseline the relative annual reductions for these selected properties are graphed below.

report figure 1

Here I want to focus on the source energy curve.  As compared with 2010, energy use went up slightly in 2011, then dropped by nearly 4% in 2012 and another 2.5% in 2013.  The drop in 2012 is easy to understand — hurricane Sandy brought the City to a grinding halt affecting tourism and many operations.  This reduction in energy use should be viewed with great skepticism.  But the continued reduction into 2013 seems like a sign of increased energy efficiency.  Or does it?

Until 2013 the EPA used a site-to-source energy conversion factor for electric energy of 3.34.  In summer 2013 the EPA adjusted this number by 6% to 3.14.  When it generated the 2013 report for NYC it used this reduced site-to-source energy conversion factor.  In other words, the 2013 reduction in NYC’s weather normalized source energy has little to do with building operation and everything to do with the EPA adjusting source energy down for the entire nation!  And this reduction does not reflect the single year improvement in the electric grid.  The EPA made no adjustment to this factor for many years prior to 2013, then in 2013 made a one-time-adjustment to reflect a 5-year average.

The NYC report is based on confidential data — no public benchmarking data were released for 2010.  Nevertheless, I can mimic the analysis by looking only at public NYC benchmarking data for 2011, 2012, and 2013.  In these data I find about 1200 buildings that reported energy data for each of these three years. About 1000 buildings remain after removing any that have questionable data for any of these three years (i.e., site EUI >1000 or <10 kBtu/sf).  The total weather normalized source energy for these buildings is graphed in blue below for each of the three years.  This graph mirrors the trend displayed in the NYC report.  The total site energy for these buildings is graphed in red.  The change in site energy matches the change in source energy for 2012 but not for 2013.  This confirms what I have explained above — that the EPA’s changing site-to-source energy conversion factor for 2013 is responsible for most of the change.  The graph below shows that 2013 site energy was actually higher than 2012 site energy.  It did not go down at all.

relative-energy-savings-scofield

The simple fact is that over the three year period shown below the site energy use of these 1000 buildings went down by only 3.5% — a figure which is highly uncertain given the sample size.  The 6% energy savings claimed by the Mayor’s Office is obtained through faulty analysis.

 

Energy harvesting — the siren’s allure

My wife, Deborah Mills-Scofield monitors dozens of media outlets and forwards articles to me that might be of interest.  One recently came my way about an effort in Portland, ME to harvest hydroelectric energy from its water pipes.  A company, LucidEnergy, has developed turbines that can be installed for this purpose.  The basic idea is to capture free energy in municipal water pipes that would otherwise be wasted.

While I applaud such innovation and creativity, I find the effort is misplaced.  I predict these turbines, like solar panels of the 1970’s and green roofs of this last decade — will soon be removed and abandoned.  This kind of energy harvesting is a fool’s errand.

About a decade ago I learned about another energy harvesting project in Israel — to install piezo-electric tranducers in highways to capture energy from passing trucks.  As heavy vehicles passed over these tranducers the truck weight would cause the transducers to compress and produce electricity.  The promoters of this energy argued that normal road compression represented lost energy — their technology would capture energy that would otherwise be lost.  The installed transducers did, in fact, produce electricity.  But I am confident that careful analysis would show that this energy comes from slight increase in fuel consumption of the vehicles that pass over the transducers.  Highway rolling resistance is mostly due to compression of the tires, not the road surface!

I am not aware of any evidence that water passing through municipal pipes arrives at end destinations with excessive kinetic energy.  Therefore any energy harvested along the way is likely to have to be re-injected by pumps.

And the maintenance issues must be significant.  I envision a few years of testing at the end of which it will be concluded that the cost of maintaining these units far exceeds the value of the energy they generate.  And what about the maintenance of pipes which get plugged due to low flow velocity?

Nature has handed us sunlight, wind, and hydo energy.  Harvesting these abundant resources is proving to be a challenge.  Harvesting efforts should focus on these well-understood and low-maintenance options.

Humans clearly waste a terrific amount of energy.  And there are many different ways that this wasted energy might be harvested.  The problem is cost-effectiveness.

 

 

San Francisco PUC Building not so green

SFPUC photoThe San Francisco Public Utilities Commission Administration building, constructed in 2012, has been billed as the greenest office building in North America.  Yesterday the San Francisco Examiner published an article which suggests the declaration was a bit premature.   According to its author, Joshua Sabatini, the $202 million dollar, LEED Platinum building has not performed up to expectations.  The building included integrated photovoltaic panels and wind turbines — enough to provide 7% of the building’s energy (not sure if that is total energy or just electric energy).  The energy produced by the wind turbines was never metered and the wind turbines have already been decommissioned; the company that installed them has filed for bankruptcy.  While the PV panels are reported to have satisfactory performance the inverter room was over-heating, requiring the installation of an auxiliary cooling system.  We will have to take the SFPUC’s word for this result as nowhere can I locate specific information about the expected PV electric generation.  It is so much easier to control the story when you don’t share the facts.

But Sabatini’s article does not discuss the energy performance of this building which is also rather disappointing.  According to 2014 energy benchmarking data published by San Francisco for municipal buildings the 277,511 sf SFPUC building had a measured site EUI of 54 kBtu/sf, just 10% lower than the mean for SF office buildings (60 kBtu/sf).  This is hardly the 32% energy savings claimed on the sfwater.org web site.  Moreover, the source EUI for this building is 153 kBtu/sf, which is 10% higher than the mean for the other 38 municipal office buildings whose 2014 energy data were disclosed.  This “greenest office building in North America” uses 10% more primary energy than used for other municipal office buildings — most of them constructed many years ago.

In other words this LEED Platinum building, the greenest office building in North America, uses 10% more primary energy than its counterparts in the San Francisco municipal building stock.  Sounds like a real winner.

Previously in 2009 I found that LEED-certified office buildings demonstrated modest (about 10%) site energy savings but, owing to their greater reliance on electric energy, demonstrated no significant source energy savings.  The result for the SFPUC building is even worse.

2012 CBECS show building energy use up from 2003

Last week the U.S. Energy Information Administration (EIA) released summary energy use data from its 2012 Commercial Building Energy Consumption Survey (CBECS).  The EIA reports that, as compared with 2003 results, the energy use intensity (EUI) for all U.S. commercial buildings has decreased by 12%.  They also report that for office buildings and educational buildings EUI have decreased by 16% and 17%, respectively.  These numbers, taken at face value, would appear to be encouraging.

But dig a little deeper and you find there is not much to celebrate.  The first thing to note is that mother nature does not care about energy use intensity.  This is a man-made metric for comparing energy use between buildings of different size. What really matters is total green house gas emission and total fossil fuel consumption.  To arrest global climate change, or at least to stabilize it, will require a global reduction in annual green house gas emission.

The 2012 CBECS data show that the total gross square footage (gsf) of the U.S. commercial building stock has expanded by 21% since 2003.  Its total (site) energy consumption has expanded by 7%.  That’s right — U.S. buildings are using more (not less) energy.  During this same time the U.S. population grew by 7.6%.  If world energy consumption and green house gas emission continues to grow with world population we are doomed!  Energy use in undeveloped countries will grow much faster than population as they increase their standard of living.  This growth is especially notable in India and China.  Developed countries like the US — which already use 5X-10X more energy per capita than non-developed countries — must decrease their energy consumption and green house gas emission.  Yet the U.S. is not even holding steady.

The above figures are based on site energy — not primary or source energy which is what really matters.  Building source energy — which includes the off-site energy use associated with energy generation and transportation — is a better indicator of primary energy consumed by buildings.

I have made crude source energy calculations based on the 2012 CBECS summary data and find that for all U.S. commercial buildings source EUI decreases by only 7% and source EUI for offices and educational buildings decreased by 12 and 13% respectively.

But again, what matters is total primary, or equivalently, source energy consumption.  When you combine these figures with the 21% growth in building gsf you find that the total source energy for all buildings increased by 13% — faster than the rate of population growth!  For offices and educational buildings the increases in source energy were 15 and 8%, respectively.  For offices that is double the rate of U.S. population growth and for educational buildings it is about the same as population growth.

2003 to 2012 is the decade of ENERGY STAR and LEED building certification.  These programs both provide cover for building owners to “feel good” about their ever-growing buildings that consume more energy and produce more green-house gas emission — yet are judged to be “green” and “energy-efficient.”  Proponents of these programs will claim that, while their accomplishments are disappointing, things would be far worse if these programs and their goals did not exist.  I doubt the truth of this assertion.  There is no evidence that ENERGY STAR and LEED-certified buildings are performing any better, on average, than other commercial buildings.  These programs are pretty much a distraction from the important societal goals to reduce green house gas emission.

The 2012 CBECS data also put the EPA’s claims that ENERGY STAR benchmarking is saving energy into perspective.  In 2012 the EPA published marketing literature which claimed that 35,000 buildings that used Portfolio Manager to benchmark for the consecutive years 2008, 2009,, 2010, and 2011 demonstrated a 7% reduction in source EUI over this same time period.  The analysis is sophomoric because they literally average the EUI for these 35,000 buildings rather than calculate their total gross source EUI (as does CBECS) which is the sum of all their source energy divided by the sum of their gsf.  It is entirely possible that the gross EUI for these buildings did not decrease at all while their average showed 7% reduction. The 35,000 buildings in the EPA study is dominated by office buildings — by far the largest set of buildings that use their benchmarking software.  Hence their claim of 7% reduction in source energy over the three year period must be seen in a context in which all U.S. office buildings saw a reduction in source EUI of 12% over a 9 year period.  There is simply little reason to believe that buildings that benchmark perform any better than those that don’t.

Once again real energy performance data cast doubt on energy savings claims for U.S. buildings.

 

ENERGY STAR building models fail validation tests

Last month I presented results of research demonstrating that regressions used by the EPA in 6 of the 9 ENERGY STAR building models based on CBECS data are not reproducible in the larger building stock.  What this means is that ENERGY STAR scores built on these regressions are little more than ad hoc scores that have no physical significance.  By that I mean the EPA’s 1-100 building benchmarking score ranks a building’s energy efficiency using the EPA’s current rules, rules which are arbitrary and unrelated to any important performance trends found in the U.S. Commercial building stock.  Below you will find links to my paper as well as power point slides/audio of my presentation.

This last year my student, Gabriel Richman, and I have been devising methods using the R-statistics package to test the validity of the multivariate regressions used by the EPA for their ENERGY STAR building models.  We developed computer programs to internally test the validity of regressions for 13 building models and to externally test the validity of 9 building models.  The results of our external validation tests were presented at the 2015 International Energy Program Evaluation Conference, August 11-13 in Long Beach, CA.  My paper, “Results of validation tests applied to seven ENERGY STAR building models” is available online.  The slides for this presentation may be downloaded and the presentation (audio and slides) may be viewed online.

The basic premise is this.  Anyone can perform a multivariate linear regression on a data set and demonstrate that certain independent variables serve as statistically-significant predictors of a dependent variable which, in the case of EPA building models, is the annual source energy use intensity or EUI.  The point in such regressions, however, is not to predict EUI for buildings within this data set — the point is to use the regression to predict EUI for other buildings outside the data set.  This is, of course, how the EPA uses its regression models — to score thousands of buildings based on a regression performed on a relatively small subset of buildings.

In general there is no a priori reason to believe that such a regression has any predictive value outside the original data on which it is based.  Typically one argues that the data used for the regression are representative of a larger population and therefore it is plausible that the trends uncovered by the regression must also be present in that larger population.  But this is simply an untested hypothesis.  The predictive power must be demonstrated through validation.  External validation involves finding a second representative data set, independent from the one used to perform the regression, and to demonstrate the accuracy of the original regression in predicting EUI for buildings in this second data set.  This is often hard to do because one does not have access to a second, equivalent data set.

Because the EIA’s Commercial Building Energy Consumption Survey (CBECS) is not simply a one-time survey, there are other vintages of this survey to supply a second data set for external validation.  This is what allowed us to perform external validation for the 9 building models that are based on CBECS data.  Results of external validation tests for the two older models were presented at the 2014 ACEEE Summer Study on Energy Use in Buildings and were discussed in a previous blog post.  Tests for the 7 additional models are the subject of today’s post and my recent IEPEC paper.

If the EUI predicted by the EPA’s regressions are real and reproducible then we would expect that a regression performed on the second data set would yield similar results — that is, similar regression coefficients, similar statistical significance for the independent variables, and would predict similar EUI values when applied to the same buildings (i.e., as compared with the EPA regression).  Let the EPA data set be data set A and let our second, equivalent data set be data set B.  We will use the regression on data set A to predict EUI for all the buildings in the combined data se, A+B.  Call these predictions pA.  Now we use the regression on data set B to predict EUI for all these same buildings (data sets A+B) and call these pB.  We expect pA = pB for all buildings, or nearly so, anyway.  A graph of pB vs pA should be a straight line demonstrating strong correlation.

Below is such a graph for the EPA’s Worship Facility model.  What we see is there is essentially no similarity between these two predictions, demonstrating the predictions have little validity.

Worship pBvspA

This “predicted EUI” is at the heart of the ENERGY STAR score methodology.  Without this the ENERGY STAR score would simply be ranking buildings entirely on their source EUI.  But the predicted EUI adjusts the rankings based on operating parameters — so that a building that uses above average energy may still be judged more efficient than average if it has above average operating characteristics (long hours, high worker density, etc.).

What my research shows is this predicted EUI is not a well-defined number, but instead, depends entirely on the subset of buildings used for the regression.  Trends found in one set of buildings are not reproduced in another equally valid set of similar buildings.  The process is analogous to using past stock market values to predict future values.  You can use all the statistical tools available and argue that your regression is valid — yet when you test these models you find they are no better at picking stock winners than are monkeys.

Above I have shown the results for one building type, Worship Facilities.  Similar graphs are obtained when this validation test is performed for Warehouses, K-12 Schools, and Supermarkets.  My earlier work demonstrated that Medical Office and Residence Hall/Dormitories also failed validation tests.  Only the Office model demonstrates strong correlation between the two predicted values pA and pB — and this is only when you remove Banks from the data set.

The release of 2012 CBECS data will provide yet another opportunity to externally validate these 9 ENERGY STAR building models.  I fully expect to find that the models simply have no predictive power with the 2012 CBECS data.

 

 

Jay Whitacre wins 2015 MIT Prize

Today it was announced that Oberlin College physics alumn (and my former student) Jay Whitacre (OC’94) has been awarded the MIT Prize for his inventive work on batteries.  His company, Aquion Energy, has attracted funds from some pretty important investors.  Not bad for a kid who didn’t take calculus in high school.

Jay-Whitacre-Lemelson-MIT_0

Congrats Jay!

Mounting evidence that LEED certified buildings do not save energy

Two recent publications provide corroborating evidence that LEED-certified buildings, on average, do not save primary energy.  One of these looks at energy consumption for 24 academic buildings at a major university.  The other looks at energy consumption by LEED-certified buildings in India.  In both cases there is no evidence that LEED-certification reduced energy consumption.

The study of academic buildings is found in the article entitled “Energy use assessment of educational buildings: toward a campus-wide susainability policy” by Agdas, Srinivasan, Frost, and Masters published in the peer-reviewed journal Sustainable Cities and Societies.  These researchers looked at the 2013 energy consumption of 10 LEED-certified academic buildings and 14 non-certified buildings on the campus of the University of Florida at Gainesville.  They appear to have considered site energy intensity (site EUI) rather than my preferred metric, source energy intensity.  Nevertheless their conclusions are consistent with my own — that LEED certified buildings show no significant energy savings as compared with similar non-certified buildings.  This is also consistent with what has been published now in about 8 peer-reviewed journal articles on this topic.  Only one peer-reviewed article (Newshem et al) reached a different conclusion — and that conclusion was rebutted by my own paper (Scofield).  There are, of course, several reports published by the USGBC and related organizations that draw other conclusions.

The second recent publication comes out of India.  The Indian Green Building Council (IGBC) — India’s equivalent of the USGBC — of its own accord posted energy consumption data for 50 of some 450 LEED certified buildings.  Avikal Somvanshi and his colleagues at the Centre for Science and the Envionment took this opportunity to analyze the energy and water performance of these buildings, finding that the vast majority of these LEED-certified buildings were underperforming expectations.  Moreover, roughly half of the 50 buildings failed even to qualify for the Bureau of Energy Efficiency’s (BEE) Star Rating (India’s equivalent of ENERGY STAR).  The results were so embarrassing that the IGBC removed some of the data from their website and posted a disclaimer discounting the accuracy of the rest.  In the future no doubt the IGBC will follow the practice of the USGBC of denying public access to energy consumption data while releasing selected tidbits for marketing purposes.

How long will the USGBC and its international affiliates be afforded the privilege of making unsupported claims about energy savings while hiding their data?

The Fourth Great American Lie

There is this standing joke about the three great Amercian lies:  1) “the check is in the mail;” 2) “of course I will respect you in the morning;”, and 3) well … let me skip the last one. I think it is time to add a fourth lie to the list — this green project will lower energy use.

In my last post I mentioned that my home town of Oberlin, OH recently purchased new, automatic loader trash/recycling trucks and spent an extra $300,000 so that three of them included fuel-saving, hydraulic-hybrid technology.  Town leaders claimed these trucks would save fuel and reduce carbon emissions.  Simple cost/benefit calculations using their cost and fuel savings figures showed that this was an awful investment that would never pay for itself (in fuel savings) and that the cost per ton of carbon saved was astronomical.

A few weeks ago I requested from the City fuel consumption data for the first six months of operation of the new trucks.  The City Manager and Public Works Director, instead, asked me to wait until after their July 6 report to City Council on the success of the new recycling program.  They both assured me that fuel usage would be covered in this report.  I was promised access to the data following their presentation.

Last Monday, in his presentation to Council, the Public Works Director highlighted data which showed that for the first six months of operation the City recycled 400 tons — as compared with the 337 tons it had recycled in the comparable period prior to acquisition of the new trucks.  This represents a 19% increase in recycling. Unfortunately there was no mention of fuel usage or savings.

Yesterday I obtained fuel consumption data from the Public Works Director for Oberlin’s new garbage/recycing trucks along with comparitive fuel data from previous years using the old trucks. The new trucks are on track to use 2,000 gallons MORE diesel fuel than were used by the old trucks, annually.  That’s right, not less fuel, but MORE fuel.  This is a 19% increase in fuel usage.  Gee what a surprise!

Soon the spin will begin.  City Adminisrators will point out that fuel usage would be even worse were it not for their $300,000 investment in the hybrid technology.  They will point out that the increased fuel usage is due to the new, automatic loading technology included in these trucks (though they failed to mention any expected increased fuel usage when the project was being sold to the public) — which enabled the use of larger recycling containers and the improvement in recycling.  What they will fail to tell us is that they could have achieved the same increase in recycling using the older style truck without automatic loaders.

This is the second recent City project for which the public has been mislead regarding expected enegy savings. The first was the LEED-certified Fire Station renovation.  This green building was supposed to save energy.  It, of course, is bigger and better than the building it replaced — oh yes, and it uses more energy.  But the increase in energy use wasn’t as much as it might have been because it was a green building.  Now we have the same result for the trash and recycle trucks.

Oberlin College is in the process of constructing a new, green hotel — called the “Gateway Project” as it will usher in a new era of green construction.  But people should understand, this new green hotel will use more energy than the old hotel —  it will be bigger and better, and its energy use won’t be as big as it might have been — and this should make us feel good.

And in the next few months Oberlin residents will be asked to approve additional school taxes to construct new, green, energy-efficient public school facilities.  But don’t be surprised when these new facilities actually use more energy than did the old ones.  Don’t get me wrong — they will be more energy efficient than the old facilities, but they will be bigger, and better and — use more energy.

This is the new lie — that our new stuff will use less energy than our old stuff.  But it isn’t true.  Fundamentally we want bigger and better stuff.  People like Donald Trump just build bigger and better stuff and proudly proclaim it.  But isn’t pallitable for most of us — we feel guilty about wanting bigger and better stuff.  So instead we find a way to convince ourseles that our new stuff will be green, it will lower carbon emission, it will make the world a better place — oh, and yes, it will be bigger and better.

We need our lies to make us feel good about doing what we wanted to do all along.  Don’t get me wrong — sometimes the check is in the mail and sometimes the green project does save energy.  But more often than not these lies are offered for temporary expediency,  And, of course, I really will respect you in the morning.