HPC as a service: High-performance computing when you need it
The power of supercomputers was once largely reserved for government and medical researchers, academics, and innovative moviemakers. After all, not everyone wants to simulate nuclear tests, land people on the moon, map the human genome, cure COVID-19, or resurrect dinosaurs in films.
Today, though, the rise of data-intensive technologies such as artificial intelligence and machine learning, which require what IDC calls massively parallel compute (MPC) capabilities to operate well, has a wider range of organizations examining high-performance computing (HPC) solutions.
But not every company has the time, money, and expertise needed to deploy them.
Industry experts say many can, however, if they abandon the notion they have to spend hundreds of millions of dollars on massive machines that they would have to host, manage, and control. Instead, companies can employ cloud-based HPC-as-a-service (HPCaaS) solutions offering supercomputing-like capabilities—with a lot less pain.
"[In] the last five or six years, there has been a steady shift of HPC to the cloud," says Srini Chari, managing partner of Cabot Partners Group, an IT analyst firm in Connecticut. "It's the fastest-growing part of the HPC market. What's happening here is that a lot of companies are realizing it's quite a headache for them to actually manage infrastructure because of the rate and pace of change in technology and the skills needed to operate on-premises HPC. So, instead of buying technology, they are looking to use it as a cloud service."
With HPC delivered as a service, organizations can work with a vendor to configure the infrastructure they need for their specific computing needs. Instead of purchasing or leasing hardware or software, however, they subscribe to it through a pay-as-you-go consumption model. All the hardware and software either reside on premises or in a colocation facility. Control, meantime, remains with customers, who commit to a certain amount of HPC usage but have the option to expand or shrink their capacity, as needed.
Leveling the HPC playing field
This may sound remarkably similar to what's happening in the $44.5 billion global infrastructure-as-a-service market. And in a way, it is. The key difference is that at least one HPC-as-a-service vendor is adding intelligence software to provide more command and customization options for customers. As a result, HPC as a service is becoming a complete platform where companies can choose from a menu of options to achieve their specific goals. A "single pane of glass" view of all their consumption services that allows them to modify their usage as needed gives customers greater control over their IT load capacity. It also adds the ability to not only expand or contract existing service needs but add new services as well.
Dan Olds, a partner at OrionX, an IT consultancy, says HPC as a service should be particularly attractive for smaller organizations that need to run HPC but don't have the budget for it. Startups, companies that have fresh ideas about how to use AI or ML and need to do heavy processing, or those involved in numbers crunching and quantitative analysis that need a little technical boost are obvious candidates for HPC as a service. And oftentimes, he says, early investors actually encourage and fund the use of HPC as a service by their startups.
"Whether it's designing new products, services, or whatever, it's inescapable today that many companies will require a lot of computing power," Olds says. "And where do you get that if you're a small startup? Your venture capitalist probably isn't going to be happy to give you $10 million to build out a data center, hire people for it, then come back to them three months later to ask for more. What they are willing to do is hook you up with a cloud provider that can give you the cycles you need to prove your point or introduce your product."
The HPC-as-a-service model is also attractive for midsize and large organizations looking for additional capacity that they won't have to manage or use 24 hours a day, Olds says. Retailers, for instance, could use the capacity during peak sales seasons and scale it back during lull times. Oil and gas companies could also tee up capacity to support seismic studies and trim back after the project ends, Olds notes. But the utility of HPC delivered as a service is not limited to just these examples.
Financial and technological advantages
These types of scenarios and the potential returns offered are what's driving interest in HPC broadly. According to a Hyperion Research report looking at 763 successful HPC projects, companies using the technology saw an average return on investment of $507 in revenues and $47 in profits for every dollar spent. The largest average revenue returns, Hyperion found, occurred in transportation ($1,804), financial services ($641), and oil and gas ($416). The largest profits were in insurance ($280).
Uli Plechschmidt, worldwide product marketing lead for the HPC storage unit at Cray, a Hewlett Packard Enterprise company, says such results are increasingly common because of the rapid innovation, modeling, and analysis HPC enables for companies of all sizes.
"The fact these types of capabilities are now available to a broader range of companies through HPC as a service essentially levels the playing field for all sorts of companies to innovate, compete, and deliver differentiated products to market," says Plechschmidt. "There was a time when organizations had to have some exotic department staffed with dozens of data scientists with doctorates. That's no longer the case. Now, you can outsource as much of that as you like to trusted and capable third parties."
For example, HPC is enabling manufacturers to accelerate design and testing to bring products to market much more quickly, he says. It's also paving the way for material science innovation to make those new products with greater environmental sustainability. It is helping to advance Internet of Things innovation, especially with self-driving vehicles. And perhaps most important, it's central to efforts by pharmaceutical companies to deliver new drugs and vaccines for treating what ails us.
In fact, some say efforts to identify COVID-19 therapeutics could give HPC as a service a huge boost.
"Because of COVID-19 and the increased confidence people have in cloud-based computing, we're seeing more willingness to move workloads," says Karl Freund, senior analyst at Moor Insights & Strategy. "COVID-19 has definitely accelerated the adoption of cloud computing for HPC."
"For companies to have the best of both worlds—meaning they get scalable HPC behind the firewall but don't have to pay CapEx for the hardware and software—that's really a great solution," says Peter Rutten, an IDC research director focused on supercomputing, MPC, and AI.
No need to cede
Even so, Rutten and other analysts note HPC as a service is in its relatively early stages and likely to face resistance from risk-averse organizations and individuals concerned about parking any of their proprietary, sensitive, or private data in the cloud. Many of these groups will naturally continue buying and running their own supercomputers behind their own firewalls and with their own people.
But analysts agree this doesn't have to be the case. Many organizations are likely to do both—putting the information that is absolutely top secret or sensitive on their own systems while outsourcing less concerning data and operations to third parties. That hybrid model allows them to still reap some of the benefits of the cloud without sacrificing too much privacy or security.
Chari of Cabot Partners predicts most companies utilizing HPC will likely end up with a hybrid approach at some point anyway. Smaller companies with limited capital budgets would start with a pure cloud model and may begin adding their own on-premises HPC capabilities as they grow and utilization levels increase. Similarly, larger firms would understand the financial and operational advantages in outsourcing at least part of their HPC operations and may go that way as well.
"In either case, the hybrid HPC model is going to grow," Chari predicts. "And it will grow both ways."
"COVID-19 has definitely accelerated the adoption of cloud computing for HPC."
This article/content was written by the individual writer identified and does not necessarily reflect the view of Hewlett Packard Enterprise Company.