Groves Green IT

There is no Dr. Kervorkian for dead servers!

 

Before I get to the topic of The Uptime Institute, let me recommend that you read this exuberant blog entry by Dean Nelson from Sun Microsystems, wherein he describes his own relatively new data center: http://blogs.sun.com/geekism. It is a good overview of Sun’s goals and achievements, and you read Dean’s enthusiasm for the project through his description. Thanks Dean for bringing it to my attention.

And what does that cryptic title refer to: You may remember that Dr.Kervorkian was an avid evangelist for the "right to die" movement and willing to stand by his moral beliefs to the point where he was jailed for aiding elderly and terminally ill patients to commit suicide.  Apparently, in any large enterprise data center you can discover that 10% of your server base is in fact not being used, sometimes referred to as comatose servers. But nobody comes along along to disconnect the power, hence the metaphor for servers needing assistance to be turned off.

Michael Zatz, also of the EPA Energy Star program spoke next. Andrew works with the product appliances and Michael works on the buildings side of EPA. Building owners told the EPA they would like to better understand the data centers in their buildings. More than 70,000 buildings measure and track their energy performance using Energy Star Portfolio Manager, an online tool. 11 billion square feet of space, nearly 20% of all buildings across the USA, are covered in the portfolio.

 This kind of peer to peer comparison seems to provide an incentive to improve.  When the building owners become aware of their energy use through this program, they generally improve their performance; today they use an average 40% less than buildings that are not in the program.  The rating is based on actual as-billed (12 months) energy data for the entire building.

The template tracks you company’s energy use accounting for weather and operational changes over time. You can update your account by adding your utility bill information each month. Buildings that perform better than 75% of all similar buildings earn the ENERGY STAR label.  (You can see these on the exterior walls of many buildings you enter (hotels, office buildings) nationwide – since learning about this program I have paid more attention to this sign as I walk around San Francisco and elsewhere.)

 Now the same program is being extended to data centers – by popular demand of Energy Star building participants, as described above. By building on the same platform, useable for standalone data centers as well as those within office buildings, it is a way to assess performance within the building. It does not say why a building performs in a certain way, just how it measures up. The unit of analysis is IT Energy/Total Energy. This measures the infrastructure efficiency and it does not capture IT efficiency but does capture the impact of cooling and support systems. Because the IT industry players is still discussing how to define “useful work” and that definition  is still fuzzy, it is important to start tracking, measuring, and improving energy efficiency given the constraints of the definitions we have.

 
USA are encouraged to ask their customers to participate in the data collection process. Read more information at www.energystar.gov/datacentersUSA. Statistically valid models can then be developed. The data collection starts in June 2008, and runs for 12 months through June 2009. The analysis will be completed by December 2009. So far, the co-los have been the first to sign up and commit to sending data. Vendors throughout the Energy Star administrators estimate that they need good data from at least 125 - 150 data centers in order to develop the rating, and data is required from a wide variety of facilities, such as large and small, stand-alone and within larger buildings, and in every geographic location across the

Paul Scheihing, Technology Manager of the US Department of Energy also addressed the DOE Data Center/IT program overview. Their programs have the tagline “Save Energy Now”. The national initiative goal is drive a 25% reduction in industrial energy intensity (including data centers) by 2017. Achieving the “25- in-10” goal will save an amount of energy equal to that consumed across all sectors of California, about 8% of the energy requirement for the entire USA!

 
The DOE Save Energy Now program sets standards, does training, provides tools, and does assessments (500 to date). The information on the website includes case studies, tip sheets, and emerging technologies. If we were to save 10% energy in all data centers by 2011, that would save about $1 billion in energy costs. To do that, DOE is planning to offer 3000 data center operators training on energy efficiency, and certify 200 qualified specialists in data center best practices. Then we will all “know what it means to be GREEN”. I guess we’ll have to wait 2 more years to find out what it means to be green – never too late…

 Distributed energy will be one important factor to consider since that could increase the efficiency from its measly 15% today (7 watts generated at source result in 1 watt to be used by a server, because of losses along the way). A DC Pro recommendation could include, for example,:

• Eliminate redundant power supplies
• Supply DC voltage to the it rack;
• Locate transformers outside the data center, etc.

 The ingredients for data center energy efficiency education programs before 2011 include:

• Assessment protocols
• Training
• Case studies
• Best in-class guidelines
• Technology demonstrations
• DC Pro Tool
  A good place to start reading about this program is from the core document you can find here:

Andrew Fanara, also of the EPA provided an update on the server Energy Star – (see my blog from last November 6, 2007 to get a more complete overview of this process if it is unclear).

 The initial focus is on volume servers and blade servers. The EPA is interested in covering DC powered servers and inclusion will require market information, interest on behalf of DC powered server manufacturers, and test procedures. The first draft for serves include power management, virtualization hooks and idle power consumption as part of their evaluation.

 Energy star products will be accompanied by a performance data sheet that is a data sheet for every server sold, making it easier for customers to decide they are getting what they want. This will include a link to savings calculator.

 If a lack of a standard protocol is not an existing barrier to effective management, ability to measure and output data may be sufficient. The specifications were released on February 14, 2008 and Andrew expects that in December EPA will be able to announce the final specification effective date. All of you can get more info at www.energystar.gov/productdevelopment (click on new specs in development). The revised definitions and scope document was released for review and comment on April 25 with comments due on May 9, 2008.

* * *

5 things to do with your IT assets

1. Remove dead servers
2. Upgrade older equipment
3. Deploy virtualization for existing and      new demand
4. Reduce demand for new servers: cut off      the longtail and stop buying
5. Introduce more power efficient server      and enable power savings features.

5 things to do with your facilities

1. Implement free cooling,
2. Increase cold aisle temperature,      eliminate dueling cooling units and increase chilled water temperature.
3. Selectively turn off core components to      increase remaining unit efficiency
4. Reduce cooling loss through sealing      cable cutouts and rationalizing cables 
5. Create data center energy dashboard to      identify pain points.

This concise list comes from Will Forrest of McKinsey & Company who  just completed a report together with the Uptime Institute tiled Revolutionizing Data Center Efficiency - Key Analyses.  He also gets credit for the title of this blog.  Have a green day!

Tuesday at the Uptime Institute 2008 Symposium

Another full day in Orlando beginning with a breakfast with the women participants at the Symposium.  We got together to talk about whether we had different agendas, perceptions,  and contributions to make as presentors at future Symposium events.  Participants from Allstate, Sun, Bank of Montreal, Northwest Airlines, Deloittes, CSRWare, and a few other companies attended.   Among the topics discussed, we thought that this division between facilities and IT in teaming to create data center energy efficiency solutions could be resolved with good Organization Development consultants - after all, this isn't the first time one silo in a company hasn't understood the needs and priorities of another silo - it happens all the time and OD's are called in to rebalance the conversations based on mutual goals. 

I  had the chance to speak with Lynda Haake, program manager for energy efficiency program for the Americas group at IBM as she was staffing a booth. Lynda was working with the System x Intel server product line before she was tagged to manage Project Big Green, the energy efficiency initiative. For people like Linda who feels passionate about the environment, she shared that it is difficult to compute why companies don’t embrace “green” faster.  Customers won’t really change their data center policies until they are either forced to do so by their C level staff or from federal regulation.  As an example,

Raleigh is suffering a drought, and her peers at Raleigh’s Triangle Parkare all taking steps as individuals and as IBM employees to lower their water consumption.  IBM’s Green Solution Showcase in Raleigh invites leading industry figures such as Roger Schmidt, one of the pioneers who along with the Uptime institute helped defined data center energy efficiency, and the utility companies (Raleigh is served by Duke Power and Progress Energy), in their monthly national customer briefings with 20 - 30 or more companies in the audience. 

Attendees at the Uptime are validating her own experience about the IT and facilities people to partner on solutions. IT people keep on adding servers without realizing the implications on capacity. Lynda remarked that the presentations at the Uptime, though technical, were engaging the audience of a caliber similar to her own customers.  “Its really affirming to watch the audience absorb all these new ideas for improved efficiency,” she said.

I also enjoyed taking part on a panel co-moderated with John Koomey of Stanford on sustainability issues and data centers.  Panelists represented perspectives from Yahoo, Hong Kong Shanghai Bank, IBM, the EPA, and APC. Following that I heard a great overview of the SPEC industry association mission and what steps are in place to create standards for servers. SPEC is working with Energy Star to create multiple testing protocols for benchmarking server energy efficiency. 

I realize that one of the reasons this is a great conference is because participants come from different geographic areas, the presentations are both simple and detailed, so that if you are strong in IT but not in Facilities you can still make out the gist of it, and vice versa, and the meal breaks are long enough to have 3-4 solid far-ranging conversations with peers.  Tomorrow is the last day and I look forward to the content as well as the informal "case studies" I am listening to from data center operations managers that I sit next to at speeches, panels, or meals. Have a green day!

Green = Savings = Efficiency, said Ken Brill

Monday morning at TUI: 442 participants from 13 countries are
sitting in the audience today.

 “Emerging regulatory measures” is a term that came up again today as yesterday evening. Let’s fix our own house, is the mantra among vendors, and users seem to agree: we know our own challenges and what will work in-house, and we want to address them before any federal agencies step in.

 Server consumption is expected to double between 2005 and 2010, resulting in a need for 30 extra power plants over the next 10 years. We are still growing so rapidly it is hard to get our arms around the problem. In 2006 and 2007, according to Ken Brill, the top 1/3 are running at a compound growth rate of 27% per year, up from a CAGR of  7% in the previous time period of 1999 – 2005.

Enterprise data centers greater than 5,000 sq feet represent 39% of the total landscape.

 For those of you doing your carbon footprint discovery, the Green House Gas (GHG) produced per server is four tons, whether it sits in a Tier II, Tier III, or Tier IV data center. The total annual expense per server in a Tier II is $1,320 or 1,870 in the Tier III, or $2,020 in Tier IV data centers.

 Ken Brill said that the 2007 Design Charrette concluded:

 1- “Industrial plants consuming similar amounts of energy have serious engineering competence”, said Ken Brill. That means, they don’t tolerate the waste that we (IT professionals) do. This requires a top-down corporate governance policy: bottoms up won’t accomplish a 50% increase in efficiency, unfortunately.

 2- “The potential savings from IT energy efficiency far overwhelm anything that can be done by facilities.” And the silver bullet is not to build a new data center. Start making the changes in the 3-5 year appliances, not the 8 – 10 year CAPEX investments such as air handlers.

 3- Lack of leadership and lack of empowerment are the most serious challenges we face today. Senior level executives don’t understand the future implications of current cost trends for data centers. It should be part of your daily job, not a special project, to work toward energy efficiency in your data center.

 Ken asked of the audience: Uptime wants to work with 10 companies who will make a commitment to reduce their data center energy consumption by 50% over the next 12 – 18 months. Sign up now!

 

The first keynote today was about DC power: a presentation from www.validusdc.com by CEO Rudi Kraus. Just transformation of 10% of US data centers from AC to DC would save three gigawatts of power. 

 What follows is a basic overview of DC power as provided in that talk. Usually I abbreviate data centers to DC, but today, I am using DC as in Direct Current, and spelling out ‘data center’. Hope it is not too confusing.

 Data centers today are no longer commercial properties, really, they are more like industrial properties based on their capacity requirements and consumption of energy.  All of the production lines in industrial plants run on DC. IT can  benefit from DC, an enabling technology, in the data center because:

· all server and storage devices are DC;

· all energy storage and renewable energy systems are DC; and

· higher voltage distribution uses less copper and distribution; and

· at less than 600V, DC remains within UL and National Electric Code (NEC) guidelines.

 DC in newly design data centers is straightforward and easy to implement. It is more difficult in retro-fits where you have a mix of traditional (AC) equipment.

 It is cost effective and simple to go straight from your utility to DC in your data center but it isn’t the norm because of conventions – by that I mean how the industry has evolved. 

 The power grid supplies AC power that goes through your UPS and converts it to DC power. It charges your back-up (DC) batteries. (Backup batteries need to be there to make up for the minor “sags” in electricity that happen continually, and to bridge the time between power outages and the generator startups – usually less than 30 seconds.) 

 Power is converted back to AC a 2^nd time, and transformed to a lower voltage AC. At that point it is fed into the power supply of the server, which converts it back to DC, because all servers run on DC power.

 All of these transformations result in efficiency losses and additional heat that needs to be removed from the data center. Having a DC data center results in two conversions instead of five: fewer equipment components, and single points of failure – always higher reliability, and lower Total Cost of Ownership.   Power, cooling and processing are simplified in this process. You can run air conditioning off the DC as well as all your IT equipment (routers, storage, etc.).

 

So where is the problem? Old myths live on.

 The perception of risk (safety) is the biggest reason people are not jumping up to install DC in their data centers. Safety is always a question in customer minds, but all of the endpoints are at 48 volts, a touch-safe voltage, according to Underwriters Laboratory.  All of the 600 volts DC cabling is distributed in conduits with coordinated breakers that clear any short-circuits(faults).

 There is one more aspect of this conversation about DC we haven’t touched upon yet. That is the volume of copper, steel, and plastic that make up the components that could be eliminated if AC were replaced by DC throughout the data center. Any company looking to have a lower carbon footprint would be interested in the impact on their supply chain: it would be “lighter” if you used less copper and steel.

 

Following the DC presentation, Mark Monroe of www.sun.com said computer processing power will increase 100,000 to 1 million times over the next 25 years and bandwidth is growing faster than processing power. GDP has been growing at 2-3% p.a. but recent companies launched, such as YouTube are growing at tremendously faster rates than conventional companies. YouTube served 2.5 billion videos in one month, as an example. Those are renamed ‘Redshift’ companies within Sun, expanding capabilities beyond traditional technologies,, making business out of IT – online auctions, social networks, etc. In contrast, ‘Blueshift’ companies grow at GDP rates, producing “real” goods like bread or transporting goods. Shifting electrons is the “light” part of the economy.  You can leverage IT to replace the physical, achieving on average a 10:1 overall efficiency. For every 1 watt you spend on electricity, you save 10 watts of power in the real world.

 How many companies can be in Redshift? Lots, going forward. The applications they do, such as high performance computing, discovering genomes, calculating derivatives, will consume 60 – 75% of compute cycles over the next five years. Infinite work exists for them. Scientific computing is becoming affordable so that more enterprises can consume some of it. Frontier airlines, for example,  take financial modeling to a new high with internal programs to take advantage of geographic differences in fuel prices. Spinoffs from the space program (such as Teflon pans) help the rest of us: spinoffs from the Redshift companies, such as fresh air cooling, DC power systems, will also filter down to the enterprise.

 

‘What can IT Execs do right now to increase energy efficiency?’ Is the name of the roundtable I attended right before lunch. The conversations meandered along many different paths:

 1. Turn off servers: Finding zombie servers is a high priority. In some data centers 8% of the servers turned out to be just junk – nobody ever got fired for leaving a server running although the reverse is true, they have been fired for turning off a server. Interviewing users, application support staff, and otherwise investigating low utilization servers needs to be done. ‘Mini-links’ among business users support access to data rather than doing productive work, for example. Mercury interactive was named as a good detection tool: a summer intern with a clipboard is another solution.

 2. “your ox gets gored!”: Implement chargebacks for IT usage. Business units come at the data centers with a request like this: “I have a server. You install it”. The fastest thing IT can do to stop server sprawl is implement charge-backs to business units. This one behavioral change is more effective than any short-term technical solution.  That silly phrase means that if you won't pay for services, you don't get any! 

Good asset management in your data center means you know who the big spenders are. By implementing chargeback models, you have a new conversation with your teams: one that focuses on cost first, and energy second, and learning takes place on both sides. At AOL, 60 % of facilities’ costs are energy related.

 3. Virtualize: In one instance, regarding virtualization, consumption growth of images from virtualization is outgrowing the consolidation effort. The pace of the images is accelerating faster year after year. Useless virtualization servers are creating the same situations that existed in the real world previously.

 4. Treat efficiency as one of your performance metrics: Here is a common problem: A plant manager knows exactly what goes into his plant; but in IT, the facilities manager and the IT manager “share” that knowledge – it isn’t embodied in a single person. No one owns the master plan upfront for a new data center. It is handed off from IT to facilities managers, CIO doesn’t own it, the Facilities VP doesn’t own it. Pass this paragraph on to your organizational development managers: here is an opportunity for change management.

One panelist said 70% of machines in the data center are running a single application and 70% of the applications are running on the remaining 30 machines. When you drive down the cost and time to deploy new virtual servers, demand goes up. Disciplined management of new virtual machines is NOT the norm.

Have a green day!

 

 

 

Green IT accelerates in Orlando Florida

Tonight is the opening keynote for the Uptime Institute and the crowd is filling the chairs this Sunday evening at Disney’s Swan Hotel. Many faces here are familiar to me from the Charrette last autumn in Santa Fe. But there is a larger contingent of international attendees and there seem to be multiple people from the same organization this evening. It is as if many participants from autumn were able to evangelize Green IT in the interim, and bring along some key decision makers to this event. The 2006 Symposium focus was on high density computing and the 2007 tagline was “the meltdown of Moores Law”. This Symposium is called: 2008 Symposium: Green Enterprise  Computing.                                                                     

 Mike Manos, General Manager, Data Center Services at Microsoft Corporation gave the evening’s keynote.  He works in “the Services Foundation” for the company, supporting 500 Million unique users visiting Microsoft sites each month, with over 150 sites and services to support. And data center operating costs are greater than capital costs. As an industry segment, data centers are the fastest growing energy segment in the US. “To be really good at services, you have to be really good at operations. Infrastructure becomes the strategic advantage for any company”, he said.

 
Microsoft data centers gain a net 20,000 new systems every month. Over the next five years, servers and power will grow 15x, network bandwidth will increase 9x, and data centers (with facilities that are hundreds of megawatts per facility), will (only, just) triple. This is no incremental growth because a product launch by Microsoft could result 30 million subscribers within the first six months – a real provisioning challenge for infrastructure and facilities managers.

New data center costs (in hundreds of millions of dollars) can be computed like this: 2% land, core and shell costs are 9%, architectural matters comprise 7%, and mechanical/electrical infrastructure runs 82%. Construction costs rise 10% every year, in his experience: it has been a constant average for Manos.  

 When describing new data centers, 500,000 sq feet buildings are not uncommon for those built today by Microsoft. They aren’t thinking about cheap and available power in a vacuum, of course. Site selection today includes the local internet population, how close the internet network is, mobile users, construction costs and tax climate, IT labor availability (for building and operating data centers), and your corporate citizenship strategy.

 
For online services, Manos wants disposable data centers – to be able to ramp up overnight for new products.  Microsoft does have a containerized version of bringing in a truckload filled with 1000 – 2000 servers and sliding the container in at 45 degrees – so 200 containers on the first floor with up to 2000 servers can be plugged, cooled, and live in a few hours.

50% of all outages worldwide are caused by human error. Microsoft has mostly Tier 2 data centers, (not designed for critical missions the way a Tier 4 data center is, for example). They have had 100% uptime over the last several years because of strong maintenance programs. Qualified crews, procedure based operations, inhouse electrical and mechanical engineers reviewing all designs or repair and replacements are all part of that program. He cited an example, an old building, with no additional technology, went from a PUE of 2.2 to 1.8 over a 15 month period just using the above good maintenance programs.

 
“I don’t care whether you measure PUE or DCIE,” Manos said. “Just measure something!” Roughly half the audience raised their hands when asked whether they are measuring power usage, up from 10% at a recent ASHRAE conference audience. Microsoft prefers PUE measurements because it is easier to understand in the context of overhead. The Microsoft annual average PUE target is to get to 1.1 annual average by 2012, every 24 months halving the distance to 1. There is a seasonality to data centers based on heat outside, load balance, etc. so you can’t take your “best” readings which probably occur at night in winter.

 With regard to IT carbon emissions: who has the numbers for your company? Manos says start collecting it because regulation on IT load is coming over the next 3-5 years. It is going to come through SOX-style commandments.

 He concluded his talk with a list of hazards which included innovation hoarding. So look forward to more specifics and transparency from Mike Manos and Microsoft. Have a green day!

Along with the data center, focus on the supply chain

Earlier this week I had a wonderful time attending the 2nd annual conference on Socially and Environmentally Responsible Supply Chains at Stanford University.  Last year's conference was so inspiring that I really looked forward to this event, and I was not disappointed. The speakers were very forthcoming and the participants were open and sharing about their progress in "greening" their supply chain.

My overall impression is that many industries are moving rapidly toward a very good understanding of their carbon footprint. Optimizing supply chains starts with understanding the source of emissions and choosing low impact methods of moving and disposing of materials. 

With that understanding comes shared responsibility for improving the quality of those relationships.  A speaker from GAP said that renaming their global compliance department "Global Partnerships" was much deeper than a name change: it was the recognition that the buyer cannot change all the links in the chain, only some of them. The overall change requires a partnering attitude.  This was echoed in another panel, featuring Nike, who said that when they partnered on good sustainable practices with their manufacturing partners throughout Asia, all the competitive name brands benefited - none at the expense of another, but a more cognizant and environment - appreciative  way of doing work for all the players. 

In one example, I spoke with a vendor who had studied a client's supply chain. Both vendor and client had supposed that transportation (trucking, rail, shipping) would be their largest culprit. Both were surprised to learn that their refrigeration footprint was larger than that from transportating goods.

A panel moderator from HP reported that the EPA has calculated it takes twice as much energy to manufacture a computer as it will use in its lifetime. Ouch! Yet most of still desire a new "improved, more efficient" laptop every 30 months or so?  Not good news for us. 

New topic: On Thursday I attended the O'Reilly Web 2.0 Conference in San Francisco.  With 6000 participants, the "cloud" crowd, showing off their web offerings and Platform as a Service (that is PaaS, not SaaS), the energy was positive and vibrant. Lots and lots of data sharing ideas for videos, pictures, text, music, etc.

I keep asking myself who is adding up all the numbers on the energy being used? When you think of all these different kinds of data centers (enterprise plus gamers plus Web 2.0 providers) is that described in the business/commercial category? Or is that in the personal/residential category? 

What I am asking may best be exemplified by these rather random statistics:
(1) 11% of all online minutes in the USA are spent on Myspace
(2) Myspace has 117 million users, about as much as the population of Japan
(3) 70% of all 18 - 20 your olds might be watching TV and on their laptop simultaneously
for six hours of every day.  They may be online to their enterprise data center, or they may be on a social networking site, and I think that with federated ID or Single Signon, one can be in a corporate social network and in a Facebook application at the same time.

So is that personal or professional work you are doing? And how do you visually display "drowning in data" in a non-cliche way?  Will humans become extinct because their greed for digital storage did them in?  Think twice before you buy another terrabyte of storage! Have a green day!

Green IT: A Perspective from KC Mares

This is a long post because my colleague KC Mares, formerly with Yahoo!, provided some very insightful responses to questions I wanted to ask him when we had lunch a few weeks ago.  Thanks KC!  Here goes!

 

1. KC you've seen the evolution of data center energy efficiency from a unique perspective.

In my adventures doing site selection, development and negotiations of data centers around the globe for Yahoo, we started to consider and add in the carbon costs for every location in the world. Initiatives like these drive efficiency improvements and lower-carbon energy supply.

 Years ago we talked about reliability, utility capacity, then density, but it is so great to see energy costs and consequently energy efficiency take center stage now. And even more importantly, that the teams that run the facilities and computing equipment are working together to design, select and operate a much more efficient data center. One excellent outcome of this is the actual efficiency and performance metrics being used to know and improve upon the overall efficiency. This teamwork will advance data center efficiency greater, as what is measured can be improved and competition among companies emerges.

2. In your travels last year around the world looking at data center sites, what surprised you most?

First of all, demand outstrips supply of good quality data centers throughout  Asia.  Data centers are very low quality compared to what is common in US and Europe. This will improve as the market matures, but it is ripe for opportunity for some entrepreneurs!

Real estate is of course very expensive compared to outlying areas and especially compared to rural areas. Locations away from cities have availability of fiber, greater security and geographic diversity and are often strategically located near Internet users. Locations away from cities, but still close to users, should be attractive to cost conscious companies. What I call the “server hugger” mentality is costing many companies 2-6x what they could be spending on data centers along with many other disadvantages (such as capacity issues).

Second, the insistence of market segments of situating their data centers in large cities such as London and New York.    

3. In addition, you have a strong background in renewables. What is the most promising renewable for data centers? Solar?  Battery? What appeals to you when you are thinking out a decade?

 Ah, great question. I’ve always envisioned data centers completely supplied by renewable energy. We achieved it at Yahoo with sites near hydropower with total, electricity 99% carbon free. Many rural sites are located in excellent wind conditions, making the possibility of a wind power plant to provide most or all of the electricity quite achievable. I’ve worked on several sites where the wind provides nearly half of the annual electricity at rates below the national average cost.

Although photovoltaics require many acres to provide all of the electricity for a sizable data centers, energy can be supplied by leveraging parking lots, equipment areas and sides of buildings, providing the side benefit of protection from sun, rain and snow for these areas, in addition to rooftops. It’s like getting the benefit of a building at a low cost with energy to flatten peak loads, reducing demand charges. PVs are expensive, but the benefits are often not fully captured or measured.

 The other often-overlooked renewable is solar thermal, which can be used for absorption chilling. It’s like a co-generation system without the fossil fuels. I worked on a project where the solar thermal output temperature was 400-700 degrees F, which makes for very efficient cooling options. And the roof space was adequately sized to support the cooling load. Open your eyes—the sun is a lot hotter than those servers.

Of course, there are those rare examples of data centers using seawater for cooling and wave or tidal energy, but the likely growth area is geothermal heat and ground water for cooling. In places like Canada, Iceland, and Scandinavia we'll see those options tested.  Overall, it’s an exciting marriage of dense power, low carbon, high efficiency and low cost to support our data centers to tomorrow.

4. Would you comment on the fact that in only very few areas in the world (London, Tokyo, Manhattan) real estate does and will continue to cost more than electricity, and yet density is the number one topic of conversation in the industry, even though less-expensive and more practical solutions are available for all other geographic locations?

Wow, after building data centers in areas below $.02 per kWh, I can’t imagine why anyone would want a data center with energy above $.10 per kWh. Afterall, electricity is the largest cost of any data center over time. I also don’t understand why one needs to be in the areas most susceptible to human-generated disasters, and cities with the highest cost.

 Even many of the undersea cable landing stations and Internet Exchanges are not right in the heart of large cities,so proximity can be near a large city, but the cost advantages of being in a lower energy cost area are tremendous.

 My suggestion is to take a look at what network latency and location is acceptable to the users or connected systems, and find a low-cost option within that area. Consider geographic diversity, as two lower reliability data centers each with full load carrying servers at N that are geographically diverse to each other are almost always more reliable than one highly available data center with 2N servers, and at the same cost or less. With so many ideal areas for data centers with energy costs half to a quarter of the cities but with trivial latency to them, it’s hard to understand why we keep building data centers in the hardest and most expensive places.

5. What is your opinion on “container” data centers, making them thoroughly modular, disposable, and transportable?

The manufacturers of these systems will probably dislike my response, but I believe they are only ideal for a very quick, fairly small deployment. Preparing a large parking lot or warehouse for them takes little time but the UPS, generator and cooling systems to support them requires the same length of time as a built-up data center. It is possible to rent these support systems in trailers, which works well for a quick need solution-- at a very high cost. While building-out a large data center takes a little more time, it costs much less per unit. I see these containerized data centers advantageous in quick need, disaster recovery, or quick to-move deployments with traditional data center buildings for the lion’s share of computing support systems.

6. Anything else you'd like to share with readers?

YES!I’ve seen data center Power Usage Effectiveness  (PUE) ranging from 1.3-3.0 and worked on a project designed to 1.05. Even though every data center is used differently, and some are located in attractive climates, I’d like to challenge the industry to achieve 1.05-1.7.

I feel this range is reasonably achievable in every climate with a well-designed and operated data center. Let’s set our milestones and work toward that. It can be achieved with little to no additional cost and an efficient data center not only serves at a lower cost, but also improves operations for people and servers. HAVE A GREEN DAY!

IDC has lots to say about greening the data center

Don't forget to register and use GROVE1595 for your code to attend the TUI Symposium at a discount. See my blog of February 23, 2008 for full explanation and the link.

I had the opportunity to hear Matthew Eastwood and Michelle Bailey of IDc speak at their Enterprise Data center Forum event in San Francisco on April 7th. IDC, you may know, is generally credited with maintaining the best industry-installed base and projections numbers among the analysts, and I like to hear their insights obtained from "following the numbers".  They did not disappoint.

To begin with, here is Matthew Eastwood’s list of attributes required when designing products to be economically, socially and ecologically sustainable. He gave examples from IT products

1. Durable (longer service life)
2. Energy efficient
3. Designed for reuse and/or recycling
4. Manufactured with local or renewable resources

He had a 5th attribute but I could not understand it in context and I forgot to ask him what it meant, so I will not include it here. Anyway, this is the measure by which we should judge appliances in the data center to be "green". 

Having said that, Eastman moved on to data centers.
Today, there are 365,000 full racks of servers representing US$15B in data center assets. And it takes $30 billion to cool those racks! Thirty million servers sit in those racks today, and that will grow to 45 million servers by 2011. IDC's research indicates that 90% of those racks are filled with X86 servers, and on average, where they are not virtualized, they are 90% underutilized. The good news is that 10% are virtualized, and within an organization that has started a virtualization initiative, 22% are virtualized.

Virtualization is ubiquitous in 100,000-plus employee companies up from 72% of all installed servers to 90% from 2006 – 07.  In medium companies  with 500 - 1000 employees, virtualized servers went from 38% – 71%.

Bailey added more numbers:  Today America has 7000 enterprise class data centers  of which 50% or more are In metropolitan/suburban areas.  They have 50,000 sq ft on average and support more than 1/3 of all systems within their respective companies.  Their average age is 12 years (built in 1996) and building updated facilities will cost an average of $25 – $50 million, with costs of construction exceeding $1000 per sq foot.  Today's "norm is designing with 125kws per sq foot.

Bailey provided this "solution" investment prioritization indicating how enterprises are dividing up their budgets for data center retrofits between today and when those new data centers can be occupied:

1. Room level changes: 30% hot cold aisles, reduced cabling, plenum clean up
2. System level changes: 29% low voltage chips, memory and power supplies
3. Rack level changes 24% blanking panels, in-row cooling, network sensors
4. Consulting services 17% design services & utility rebates, and thermal assessments

As a team, their conclusions were quite clear: Are we green yet? No, not close.  Have a Green Day!

Note: I am writing these notes from memory, not from the slides, so I may have blended the separate presentations by Eastman and Bailey, but I feel pretty sure that the notes are accurate...

 

 

 

 

Free and excellent resource:PG&E's Design Guidelines Sourcebook

I was just rereading this how-to manual published in January 2006 by Pacific Gas & Electric of Northern California. Gosh it is good.  With minimal editing it is still on-target today - and considering how rapidly the "state of the art" is changing, it is a compliment that it is still on the edge.  If rewritten today, authors would perhaps place added emphasis on a wider range of semi-approved temperatures for data center operations, and describe in more detail the fragility of systems within turbulent temperature changes (rapid heat-up and rapid cool-down). 

If you are not a mechanical engineer, it is not easy reading, and I still need some explanations on the terms used.  But an iterative approach, reading some and then returning to it 3-4 months later, helps to bring the concepts into focus and enable you to ask simple questions from the engineers around you. Have a Green Day!

Green IT: Strategies, planning, and risk scenarios

Don't postpone:  Click here and use the GROVE1595 discount to register with an $800 discount for the Uptime Institute's Symposium in Orlando, Florida beginning April 27, 2008.   You won't see the discount box until  the 4th page of the registration but it is there and does activate before you enter your credit card number, so if it isn't immediately obvious, continue until prompted to enter the code.

Day One will focus on why the industry is not taking advantage of simple, energy-saving tactics/measures that would result in 50 percent energy efficiency improvements in less than 36 months.

Day Two is dedicated to the future – what technology and systems developmental strategies, roadmaps and trends will re-shape energy consumption and carbon-footprint contribution in the next generation of data centers?

Day Three is dedicated to global corporate policy, governance and goals regarding the greening of enterprise IT. A joint McKinsey & Company/Institute analytical report on transforming data center energy efficiency will be released to the public and a McKinsey CIO Roundtable will discuss the findings of this industry benchmark report. An open executive session in the afternoon will focus on reports from national and global policy efforts now underway.

In addition:

If you'd like to read two papers that I recently wrote for the prestigious Cutter IT Journal, click here   You'll need to profile yourself first. Parts 1 and 2 together describe a risk scenario wherein a data center manager might have to cut back energy usage within weeks because of socio-political chaos. You realize that is polite talk for other risks that could take place.  Most  data center managers would be stunned, and waste a lot of time deciding what to do first, second and third.  With the assistance of my colleagues, I've summarized a 1-2-3 step process and a rapid response plan.  Then, I argue, you would have started your Green IT strategy (a little too late, regrettably) so why not mitigate your risk and customize your Green IT strategy now in the absence of a real emergency situation, so you can think more clearly and make fewer mistakes. 

I welcome your thoughts and comments on these papers. Have a Green Day!

Green IT: Eat Heat Christian!

Don't forget to click here for the discount to TUI's Symposium in late April. Use code: GROVE1595 when prompted.

The northern California Critical Facilities Roundtable kindly invited me to observe their meeting earlier this week. Thank you Bruce Myatt   It was a four hour session on identifying the most important and still unclear issues that facilities managers need to address for their datacenter peers.

This dire statement sums up the problem:  At 100 w per square foot, your temperature could go up 70 degrees (Centigrade!) in one hour. If you design datacenters for 400 w per square foot, you would not be able to cool fast enough using today's accepted practices. And yet, two vendors are experimenting with one megawatts feeding just 1000 square feet of datacenter: that is equal to 1000 watts per sq feet!

 I learned a new term: ridethrough. The definition is how much time you have before you lose control of your data  when you lose your power. Previous solutions have included ice and water storage, and in the near future, we will need to ensure a return to uptime in seconds, not minutes.   To determine the ridethrough, a data center operator needs to know at what temperatures are the damage threshold, latency, and data loss for different components and different applications in servers, storage systems, networking appliances, etc.

When you have an enclosed place it is efficient, unless power is shut off. But if the airflow is cut by 50%, then the exhaust air is twice as high as earlier. It only takes 30 seconds until the generator starts, but chillers take 3 – 15 minutes to normalize pressure between suction and discharge, and wait for pressure stabilization.  Catastrophic cooling changes (bringing the room's temperature up to quickly) create problems when starting up, not losing cool air on the way down. Hard drives lose the grease in their gears. Servers turn themselves off at a temperature threshold. Some servers throttle down as the heat goes up. Generally speaking, it is the rate of thermal changes that does the damage to the servers rather than the heat.  Sleepy drives spin down and up when in use; performance is a function of speed.  Two drives per blade in a chasse is a normal procedure today but separating them out is an option in the future. A series of sags of low current, rather than long term outages is most frequent.  To mitigate heat damage during the sage, you always need a relief band to circulate air around the equipment.

Something else I had never considered or even understand is that with no air circulating (fans aren't working, remember) you can create an air vacuum. To which, one expert was quoted as saying, "Hot air is better than no air" because both options are bad.  It drew a lot of laughs.

At the end of the day three subcommittees were appointed to design pilots on:

    IT load redesign (reducing heatload at power supplies)

    AC Power loss (efficient UPS support fans and pumps and backup cooling options)

    Cooling capacity improvements: air flow control to improve capacity & thermal energy storage

Next step: change the paradigm. Do something radical with the heat that is generated.  That is what I meant by "Eat Heat", and Christian Belady , now of Microsoft and formerly of HP,  originated the quote  that hot air is better than no air in a previous meeting.  See a related  article I found that Belady co-authored with Magnus Herrlin.  I haven't a clue how, but I am throwing out the challenge to stop thinking about heat dissipation in a conventional way and discover microbes to live in the racks and eat the heat. Stranger things have happened.    Have a green day!