{"id":30214,"date":"2022-12-13T00:38:32","date_gmt":"2022-12-13T01:38:32","guid":{"rendered":"https:\/\/peymantaeidi.net\/stem-cell\/?p=30214"},"modified":"2022-12-13T02:34:57","modified_gmt":"2022-12-13T02:34:57","slug":"burning-up-mitigation-should-sustain-datacentre-operations-during-extreme-heat","status":"publish","type":"post","link":"https:\/\/peymantaeidi.net\/stem-cell\/2022\/12\/13\/burning-up-mitigation-should-sustain-datacentre-operations-during-extreme-heat\/","title":{"rendered":"Burning up: Mitigation should sustain datacentre operations during extreme heat"},"content":{"rendered":"
\"image\"<\/div>\n

Tuesday 19 July 2022 was the hottest day ever recorded in the UK<\/a>, peaking at an unprecedented 40.3\u00baC in Coningsby, Lincolnshire. Reports of datacentre cooling system failures at Google and Oracle<\/a> swiftly followed.<\/p>\n

But while hot weather exacerbates operational challenges, datacentres worldwide have been designed for locations that regularly experience much more extreme conditions, as Simon Brady, Europe, Middle East and Africa (EMEA) services channel business development head at Vertiv and a datacentre optimisation specialist, points out.<\/p>\n

\u201cThink about Asia, Australia, the Middle East, even places in Eastern Europe where you get -20\u00baC in winter and 35\u00baC to 40\u00baC in summer,\u201d says Brady. \u201cSites that fail were either designed or maintained poorly.\u201d<\/p>\n

Even outside a specified temperature window, datacentres \u201ctend not to fail\u201d as such, although they may break their service level agreements (SLAs) and operate outside them for that period, he explains.<\/p>\n

\n

<\/i>Managing expectations around availability<\/h3>\n

SLAs should better account for the increased chance of outlier temperatures, yet operators have been wary of broaching this topic with their customers, often when it comes to agreeing on maintenance levels and related yet necessary downtime, he says.<\/p>\n

\u201cAlthough, with adiabatic or evaporative cooling<\/a>, we can then have a conversation about the lack of water<\/a>,\u201d adds Brady. \u201cA project I did two years ago in Saudi Arabia planned for 55\u00baC external temperatures. You have to put a bit more engineering around it, but it\u2019s no problem whatsoever.\u201d<\/p>\n

If the UK starts hitting those temperatures, \u201cwe\u2019ve got bigger issues than whether datacentres can cope\u201d, he adds.<\/p>\n

\n

\n \u201cMore workloads could be moved to more sustainable locations. Metropolitan datacentres, such as near London, could focus on supporting latency-sensitive applications\u201d
\n <\/figure>
\n Tate Cantrell, Verne Global<\/strong>
\n <\/figcaption><\/i>\n <\/p>\n<\/blockquote>\n

Simon Bennett, EMEA chief technology officer (CTO) at Rackspace, emphasises that failures are typically down to multiple factors. It follows that unexpected heat spikes should be manageable \u2013 unless the datacentre operation in question is already running out of headroom on capacity<\/a> that would allow for outlier events.<\/p>\n

It is likely to be more about making incremental changes on current sites than pouring new concrete, which few operators are doing today anyway, he says. Monitor the temperatures, tidy the cabling, close rack doors to maximise airflow, and so on.<\/p>\n

\u201cThere are really simple things that people don\u2019t do in poorly run datacentres,\u201d says Bennett. \u201cUltimately, you need to make sure you know what sort of fault tolerance you\u2019ve got with hotter days. Can you cope with the loss of a main air-conditioning unit, for example?\u201d<\/p>\n

A bigger issue is managing heat generated internally by the datacentre itself, which means attention to monitoring and improving airflows<\/a> and other basics, rather than implementing a new solution.<\/p>\n

\u201cYou\u2019re generating a lot more heat in a smaller footprint,\u201d Bennett notes. \u201cYou have to offset that our electricity costs have gone through the flippin\u2019 roof. People have to review their air-con capability properly and revise it up to 40\u00baC anyway.\u201d<\/p>\n

When densities are high, liquid cooling technologies<\/a> can make sense if they help reduce electricity costs too. Modelling and digital twins<\/a> can help at scale, he adds.<\/p>\n

Moving workloads somewhere with lower temperatures, like Scotland, and which might offer \u201cconsistent cost of electricity\u201d, like Iceland, might also make sense, especially to procurement departments looking to nail down a few variables for longer-term plans, Bennett suggests.<\/p>\n

Tate Cantrell, CTO at Iceland operator Verne Global<\/a>, points out that cooler-climate datacentres<\/a> can use less energy cooling higher-density workloads. The Icelandic summer average is just 13\u00baC and datacentres can use 100% hydro and geothermal power.<\/p>\n

\u201cMore workloads could be moved to more sustainable locations,\u201d says Cantrell. \u201cMetropolitan datacentres, such as near London, could focus on supporting latency-sensitive applications.\u201d<\/p>\n<\/section>\n

\n

<\/i>Pinpoint remaining inefficiencies for elimination<\/h3>\n

Raymond Ma, general manager for Europe, Australia and New Zealand at Alibaba Cloud Intelligence, maintains that conventional, older datacentres can be \u201cincredibly inefficient\u201d. Standard air-conditioning can eat up 40% of the total energy bill, resulting in a huge environmental impact.<\/p>\n

Innovation remains important, including looking at advanced immersion or water cooling technologies, with redesigns of conventional centres requiring planning around reconstruction of the infrastructure, preparation of specially designed IT, coolant selection, and mechanisation of monitoring and maintenance systems.<\/p>\n

\u201cThis can deliver costless cooling for 90% of a datacentre\u2019s operating time, driving down energy consumption by more than 80% compared to mechanical cooling,\u201d says Ma. \u201cAdopt industry best practices, such as using intelligent algorithms to increase energy efficiency, enhancing renewable electricity use and boosting the recycling of energy such as waste heat generated by servers.\u201d<\/p>\n

Richard Clifford, solutions head at UK-based Keysource, warns there is no silver bullet for better management of soaring temperatures. ldquo;The key is the design and strategy for dealing with higher ambient temperatures, focusing on cooling and IT load, which means reducing the load in line with reduced cooling performance,\u201d he says. \u201cCooler climates have several benefits, but climate change is global.\u201d<\/p>\n

He says a \u201cvast majority\u201d of UK datacentres already operate below 80% of design capacity. They should be able to manage these rising temperatures currently, with appropriate attention paid to proper maintenance, resilience, airflow and capacity planning around resources, including power and, increasingly, water.<\/p>\n

James Petter, vice-president of international sales at Pure Storage, prescribes solutions that can flex up and down, based on data-driven insights. He broadly agrees that capacity planning will likely benefit from more innovation and sustainable<\/a> tech purchasing, because every item uses energy, even if it doesn\u2019t produce any heat while operating.<\/p>\n

For instance, solar may power more datacentres in future, with variable power draws backed up by an uninterruptible power supply (UPS). This could mean greater ability to cope with volatile weather patterns<\/a> exacerbated by an overall hotter climate.<\/p>\n

Coupled with data reduction and compression, racks of the future should consume less power and require less cooling. \u201cI think it\u2019s going to go to nanotechnology,\u201d says Petter.<\/p>\n

Steve Wright, chief operating officer at UK-based 4D Data Centres, reiterates that the outages and facility impacts seen this summer have been a tiny subset of the UK\u2019s 400 or so commercial datacentres \u2013 and at least one outage appears to have involved an accidental rerouting.<\/p>\n

\u201cThose impacts probably shouldn\u2019t have happened. However, the reality is that people need to learn that cloud isn\u2019t this magical thing that is just always there 24\/7,\u201d he says. \u201cYou need a strategy for how you deal with not being able to access your internal IT asset or online service.\u201d<\/p>\n

For that reason, he looks forward to deeper investigation of what happened with the latest hot weather issues. Datacentre operators should keep asking what can be learnt as a separate, independent entity targeting greater stability and resilience \u2013 with skillsets also relevant<\/a>.<\/p>\n

Wright underlines the sense of trying to learn from and benefit from the experiences of datacentre operators already working in much hotter<\/a> or humid<\/a> environments. Improvements around maintenance are still needed, and a lot of retrofitting and mitigation is still possible.<\/p>\n

\u201cMaybe 10 to 15 years ago, you could fit a few chillers in between the buildings and get away with the cooling infrastructure being in the car park,\u201d says Wright.<\/p>\n

It can be as simple as just putting less load in the area that needs to run cooler, he adds.<\/p>\n

\u201cProbably the biggest thing is the elements of the ambient air design condition. I suspect we\u2019ll start working to 43\u00baC to 44\u00baC, maybe 45\u00baC, in future,\u201d he says.<\/p>\n

\u201cThe ASHRAE<\/a> [American Society of Heating, Refrigerating and Air-Conditioning Engineers] operating window for server equipment has got wide over 15 years or so, but we\u2019re still working on pushing that even higher. Actually, we can cool down to 25\u00baC (rather than 16-18\u00baC), which still feels quite warm in the UK but you can still push that higher.\u201d<\/p>\n<\/section>\n

\n

<\/i>Plan, mitigate and maintain<\/h3>\n

Daniel Bizo, research director at Uptime Institute Intelligence, confirms that datacentre operators should start with looking at overall capacity, restrictions and other limitations, where mitigation is possible, aligned as usual with the business case.<\/p>\n

\u201cEveryone\u2019s situation is different \u2013 the vast majority of datacentre sites are unique, and you also need to be aware of localised risks,\u201d he notes.<\/p>\n

\n

\n \u201cWe cannot predict exactly. The problem with climate change is that we know it is getting worse and weather events are becoming more extreme, but we don\u2019t know where exactly and by how much\u201d
\n <\/figure>
\n Daniel Bizo, Uptime Institute Intelligence<\/strong>
\n <\/figcaption><\/i>\n <\/p>\n<\/blockquote>\n

Those caught out may trace issues to \u201clurking equipment failures<\/a>\u201d rather than high temperatures per se. Theoretically, a datacentre design could be virtually bullet-proof against climate change, including atmospheric risk and extreme weather, from storms and wind to hail, rain and floods.<\/p>\n

\u201cMaybe you can move north<\/a>, or close to a big body of water \u2013 the sea or a big river or lake,\u201d says Bizo. \u201cThat said, there can be flow issues and restrictions, as we\u2019ve seen in France. Some are now struggling with nuclear reactor cooling… because the rivers are getting so hot that it\u2019s affecting aquatic life.\u201d<\/p>\n

With legacy systems, however, it might take 10-15 years before optimum architectures can be implemented. After all, the industry comes with at least 20 years of \u201cbaggage\u201d to deal with.<\/p>\n

The rule of thumb for refurbishments has been to consider peaks over the past 20 years, but with increased volatility, operators should perhaps look back further or at least build in more of a buffer to account for never-seen-before temperatures.<\/p>\n

Either way, operators probably need to start thinking about their climate resilience strategies in a \u201cmore creative\u201d manner.<\/p>\n

\u201cWe cannot predict exactly,\u201d notes Bizo. \u201cThe problem with climate change is that we know it is getting worse and weather events are becoming more extreme, but we don\u2019t know where exactly and by how much. Even if we had perfect data and infinite compute, it wouldn\u2019t be possible to pull that off.\u201d<\/p>\n<\/section>\n","protected":false},"excerpt":{"rendered":"

Tuesday 19 July 2022 was the hottest day ever recorded in the UK, peaking at<\/p>\n","protected":false},"author":1,"featured_media":30216,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"_links":{"self":[{"href":"https:\/\/peymantaeidi.net\/stem-cell\/wp-json\/wp\/v2\/posts\/30214"}],"collection":[{"href":"https:\/\/peymantaeidi.net\/stem-cell\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/peymantaeidi.net\/stem-cell\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/peymantaeidi.net\/stem-cell\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/peymantaeidi.net\/stem-cell\/wp-json\/wp\/v2\/comments?post=30214"}],"version-history":[{"count":2,"href":"https:\/\/peymantaeidi.net\/stem-cell\/wp-json\/wp\/v2\/posts\/30214\/revisions"}],"predecessor-version":[{"id":30217,"href":"https:\/\/peymantaeidi.net\/stem-cell\/wp-json\/wp\/v2\/posts\/30214\/revisions\/30217"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/peymantaeidi.net\/stem-cell\/wp-json\/wp\/v2\/media\/30216"}],"wp:attachment":[{"href":"https:\/\/peymantaeidi.net\/stem-cell\/wp-json\/wp\/v2\/media?parent=30214"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/peymantaeidi.net\/stem-cell\/wp-json\/wp\/v2\/categories?post=30214"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/peymantaeidi.net\/stem-cell\/wp-json\/wp\/v2\/tags?post=30214"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}