HPE's Dr. Goh on harnessing the power of swarm learning
Data is everywhere today, coming from edge devices, data centers, the cloud—you name it. And that massive volume of data is a good thing for training AI and machine learning, technologies that are bringing greater intelligence and automation to scientific research, business and manufacturing processes, consumer applications, and much more.
The problem is, not all data can be shared. There needs to be a way to share insights from protected data without actually exposing it—and there is: swarm learning.
"What swarm learning does is to try to avoid that sharing of data, or totally prevent the sharing of data, [under a model] where you only share the insights, you share the learnings," explains Dr. Eng Lim Goh, senior vice president and chief technologist for AI at Hewlett Packard Enterprise. "And that's why it is fundamentally more secure."
More input, better results
Goh, who joins MIT Technology Review's Laurel Ruma in this Business Lab discussion, uses the example of a hospital that wants to build a machine learning model that analyzes patient data to predict outcomes. Because the hospital has access to only its own data, the model will evolve based solely on the demographic of patients the hospital typically sees, which creates a bias in the learnings, he explains.
Using swarm learning, the hospital can combine its data with that of hospitals serving different demographics in other regions and then use a private blockchain to learn from a global average, or parameter, of results—without sharing actual patient information.
Please see: What's all the buzz about swarm learning?
Under this model, "each hospital is able to predict, with accuracy and with reduced bias, as though [it has] collected all the patient data globally in one place and learned from it," Goh says.
The decentralization of data
Swarm learning is just one example of how organizations are harnessing the power of the edge, where intelligence and interconnectedness are key.
As Goh points out, with tens of billions of connected devices generating data at the edge—and that volume growing exponentially—it will soon become impossible to backhaul all of it to data centers or cloud platforms, even with next-generation network capabilities such as 5G. That means the processing of data, and how we share learnings from it, will become increasingly decentralized.
Please read the report: Your edge. Your future.
"The center of gravity will shift from the data center being the main location generating data to one where the center of gravity will be at the edge," Goh says.
"That's the reason why swarm learning will become more and more important as this progresses," he says, noting we'll need to do "more peer-to-peer communications, more peer-to-peer collaboration, more peer-to-peer learning."
This article/content was written by the individual writer identified and does not necessarily reflect the view of Hewlett Packard Enterprise Company.