Pittsburgh Supercomputing Center democratizes HPC as a service

Upping the ante on artificial intelligence

Artificial intelligence until now has been constrained by the massive amounts of data and computational power needed to run algorithms. The Bridges supercomputer from the Pittsburgh Supercomputing Center brings advances enabling researchers to tackle big real-world problems.

A casino in Pittsburgh. Four of the world’s top poker players. An AI program named Libratus. Who won? Bet on Libratus.

The artificial intelligence (AI) program—written by researchers from Carnegie Mellon University (CMU)—calculated a strategy. And every night of the 20-day tournament, Libratus analyzed what had happened that day so it could adapt to improvements the humans were making in their strategies. This AI has now surpassed the best human poker players. While the promise of AI has beckoned since the 1960s, there has been tremendous progress in AI over the last few years, driven by new AI algorithms.

Behind Libratus was a new type of supercomputer called Bridges. Funded by the National Science Foundation (NSF), Bridges was designed by the Pittsburgh Supercomputing Center (PSC) to converge AI with HPC and Big Data—and bring it within reach of researchers nationwide. Not only is Bridges super-powerful, it’s designed to be accessible through software interfaces that don’t require specialized programming skills. PSC makes Bridges available at no charge for open research in a wide range of fields, from genomics to social sciences, and by arrangement, to industry.

The stakes are high. While heads-up no-limit Texas Hold’em poker has become the leading benchmark in evaluating progress in algorithms for strategic reasoning under imperfect information, the algorithms used to solve it are application independent and apply to a host of important strategic reasoning settings. The real payoff will be for people and the planet. Bridges democratizes supercomputing by making it available and user-friendly. Only the strongest proposals are chosen, through a highly selective peer-reviewed process. Researchers are using Bridges to study how to make the energy grid more efficient, how species respond to environmental change, the causes of lung disease—and many other high-impact problems that once were too big to solve.

Supercomputing for solving super problems

The Pittsburgh Supercomputing Center provides researchers with access to supercomputers for solving the most challenging problems in science and engineering—including treating complex diseases and protecting the environment.

PSC provides university, government, and industrial researchers nationwide with access to powerful and otherwise prohibitively expensive systems for HPC, data analytics and management, and communications.

PSC empowers researchers to solve the most challenging problems in science and engineering—including treating complex diseases, protecting the environment, and other critical issues facing humanity.

The center is a leading partner in Extreme Science and Engineering Discovery Environment (XSEDE), the National Science Foundation cyberinfrastructure program. PSC is a joint effort of Carnegie Mellon University and the University of Pittsburgh, with support from federal agencies, the Commonwealth of Pennsylvania, and private industry.

Bringing advanced supercomputing to non-programmers

PSC wanted to make HPC resources more accessible to a broader swath of the research community. That meant building a supercomputer for people whose core skillset is science and research, not programming.

Detecting cyberattacks. Curing breast cancer. Predicting severe storms. A common thread among these vexing challenges is their computational complexity. No ordinary computer can crunch calculations on a large enough scale to help researchers understand them. And the alternative—that class of high-powered systems known as supercomputers—tends to be expensive and complex to build and manage.

That’s where PSC comes into the picture. Supported by the National Science Foundation and other funding, PSC provides HPC resources at no charge to researchers studying everything from neuroscience and the causes of cancer to meteorology and the national economy.

PSC’s traditional user base used to include engineers, chemists, and physicists—people who typically brought some specialized HPC skills. Now, PSC is bringing its resources to researchers who are experts in their fields but might never have programmed a computer. To reach these new communities, PSC delivers HPC muscle through the types of software interfaces researchers are accustomed to using on their laptop PCs.

Delivering HPC as a service

In creating Bridges, PSC had to bring together breakthrough levels of supercomputing storage, memory, and processing power—and make it all available through desktop interfaces already familiar to most researchers.

As it planned the new supercomputer it calls Bridges, PSC faced two types of technical challenges.

One was to bring together breakthrough levels of supercomputing storage, memory, and processing power. Bridges needs massive memory, for example to accelerate assembly of large genome sequences from small DNA fragments from days to hours. It must perform rapid calculations on problems split into small components, such as computing the energy use of hundreds of buildings over many time cycles. It needs AI capabilities, such as those that Libratus used to play poker, applied to problems of biology, chemistry, and materials science.

The other challenge was to make all this available through the applications, languages, and research paradigms already familiar to researchers. These include software tools such as Python, R, MATLAB, and browser-based gateways used by tens of thousands of researchers. This barrier-free access allows researchers to amp up their current projects without having to learn specialized HPC skills.

Unprecedented power with desktop convenience: PSC set out to create HPC as a service.

Supercomputing muscle with laptop ease

Bridges is a uniquely capable, easy-to-use resource for empowering diverse communities by bringing together HPC and Big Data.

PSC’s flagship HPC system, Bridges, runs applications that accelerate discovery in the physical sciences, biology, economics, business and policy, and even the humanities.

Bridges is a uniquely capable petascale resource for empowering diverse communities by bringing together HPC, AI, and Big Data. Its richly-connected set of interacting nodes and active storage systems offers exceptional flexibility for data analytics, simulation, workflows and gateways, leveraging interactivity, parallel computing, the Spark processing engine, and the Hadoop programming framework. NVIDIA graphics processing units (GPUs) enable deep learning and accelerate simulations. Bridges is a heterogeneous system: Thanks to software written at PSC and Intel’s new OmniPath Architecture, scientists can apply different parts of the supercomputer to different portions of their computational problems, allowing productive reuse of existing applications and accelerating results.

What’s more, Bridges is designed to support familiar, convenient software and environments for both traditional and nontraditional HPC users. It’s supercomputing as easy to use as your laptop.

The jackpot winner: All of us

Diverse projects yield important new insights.

Since 1986, more than 36,000 users, 6,800 distinct principal scientists and engineers, and over 11,000 distinct grants at 1,525 affiliations and research centers in 53 states and territories have used PSC computing resources. One generation after another, it's been an evolution of more memory and faster processing to better data storage and accessibility. Bridges is the best yet.

Four times a year, researchers may apply for time on Bridges. Diverse projects have launched this way. One team is exploring the factors that cause cancer, lung disease, and brain function, drawing on genomic, imaging, and other Big Data. Another is searching hundreds of thousands of historical documents for clues to the history and life experiences of Black women in the United States from the 1700s onward. Yet another is studying neutrino behavior for new insights into basic laws of physics and the origin of the Universe.

It's not about poker, it’s about life. The jackpot winner is mankind.