Explore AI infrastructure challenges: energy efficiency, data management, and skills development are crucial for successful AI implementation.
It’s been just over a year and a half since ChatGPT launched, yet generative AI is now on the lips of most business execs. GenAI has been what Forrester CEO George Colony called “the most important technology thunderstorm of the last 40 years,” and tech leaders across the EMEA region seem to agree. AI is now a priority for 90% of IT, DevOps, and platform engineering decision-makers, according to the Vanson Bourne and Nutanix State of Enterprise AI Report. However, as with any rapid technological advance, questions have started to be asked about the existing IT infrastructure and its ability to cope. Are current systems really up to the demands of an AI gold rush?
The short answer is no. To begin with, we have an energy efficiency problem. Gartner warned about this back in December 2022, saying that by 2025, “AI will consume more energy than the human workforce, significantly offsetting carbon-zero gains.” Also, a study called the Growing Energy Footprint of AI suggested that the AI industry could consume as much energy as a country the size of the Netherlands by 2027.
The State of Enterprise AI Report highlighted the need to address ESG reporting considerations. Today, most AI/ML model inferencing and training are conducted on high-performance GPUs supported by equally high-performance memory and storage. These solutions consume significant electricity and require additional power to cool actively within a private or public data center.
Also Read: The New AI Battleground: Why Your Fancy AI Models Won’t Save You
The report also identified additional skills challenges. Gaps in AI capabilities and ESG are big concerns. Over 40% of EMEA respondents in the report claim they lack GenAI and prompt engineering skills and desperately need data scientists. This will inevitably impact organizations’ ability to meet their expectations with AI projects.
This becomes even more pronounced when considering other major factors, such as managing data and scalability in back-office functionality. Skills shortages and a need to modernize systems do not go well together, so addressing skills is important in meeting the ongoing demands of infrastructure change.
This is the modern world
AI will only add pressure to existing systems, so there is also growing recognition of the need to address the managing and support of running AI workloads at scale. EMEA respondents ranked this as the first challenge over the next two years. In addition, respondents cited security, reliability, and disaster recovery as important considerations for their AI strategy.
With infrastructure modernization and data security outranking cost (the third-lowest consideration for EMEA organizations running or planning to run AI workloads), there is a clear indication within the region that organizations recognize that to benefit from AI, they must get their infrastructure house in order.
The report further illustrates this, with over 90% of EMEA respondents agreeing that AI applications will increase their IT costs and cloud spending. In short, EMEA organizations are willing to spend money to support their AI initiatives. The challenge is where and how to spend it wisely.
It will come down to prioritization. While identifying and remediating skills shortages are a constant issue, especially regarding emerging technologies, infrastructure modernization is key. AI applications and services have a symbiotic relationship with their underlying datasets, models, and infrastructure. The report shows that enterprises are acutely aware of this. Hence, the challenge is developing data security and quality strategies to make their AI technology as reliable and resilient as possible.
Also Read: How Unstructured Data Can Help Companies Thrive in the AI Era
Inevitably, the gold rush nature of GenAI adoption will lead to short-term overspending to plug skills gaps and deliver infrastructure capabilities. However, a longer-term modernization plan is needed to benefit from the technology and ensure scalability and intelligent workloads, optimizing costs and energy use. This will mean effective implementation and management of data across multiple environments – data center, cloud, and edge – as each will play a critical role in supporting an end-to-end AI workflow.
This data management should also consider security, quality, and protection. Given data sovereignty requirements, especially in the EMEA region, this should be a core tenet of any AI strategy. Of course, this is all a work in progress. Organizations are still trying to work out how best to use GenAI but will use it. Inevitably, there will be early adopters, accelerating adoption and making mistakes along the way, but for the majority, there are some fundamentals here. Existing infrastructures are not enough. They will creak and fail under the strain of AI, if not physically, then almost certainly in terms of capability and governance. Thankfully, on that score, AI will be a marathon, not a sprint, for most organizations.
