Skip to main content
Exploring what’s next in tech – Insights, information, and ideas for today’s IT and business leaders

Closing the data science talent gap

Industry and universities are collaborating on programs to create a steady stream of data science and AI/ML professionals in light of the ongoing skills crunch.

Every day, a team of students rolls into the Synchrony Emerging Technology Center at the University of Illinois Research Park and gets to work tackling real-world data science and AI problems. There are projects exploring the application of machine learning in credit risk modeling, automation initiatives, and visualization and analytics work on transactional fraud.

As part of their efforts, the students immerse themselves in data science staples like Python and R languages to create data models and build scripts. They practice agile software development, partaking in regular stand-up meetings with Synchrony data scientists as well as daily scrums. The internship experience gives students an opportunity to embrace the data science role and apply theoretical principles to solving real-world business problems. For Synchrony, the partnership primes a fresh pipeline of data science talent to help close the gap on one of the most important and expanding technology areas empowering its business.

Please read: The best ways to manage analytics in the data-driven enterprise

"The market for data science talent is competitive," notes Carol Juel, chief technology and operating officer at Synchrony. "You have to create different ways to attract and build a pipeline of talent, and we've done historically well with strong university partnerships."

The scramble for data science skills

Given corporate America's obsession with using data to drive insights, it's no wonder data science roles are in hot demand. Organizations of all sizes are increasingly trying to leverage data and advanced technologies like artificial intelligence and machine learning to better serve customers, reduce costs, optimize operations, and even recalibrate logistics in light of pandemic-era supply chain disruptions. LinkedIn ranked data science specialists as one of its top jobs of 2021, noting that hiring for roles like data scientist, data visualization expert, and data management analyst have grown nearly 46 percent since 2019. Similarly, hiring of machine learning engineers and AI specialists has surged 32 percent between 2019 and 2020.

Please read: What is a data analyst, what is a data scientist—and how can you get started in these lucrative careers?

Demand for data science professionals is strong, but supply—not so much. Based on its analysis of various job listing sites, QuantHub estimates a shortage of 250,000 data scientists in 2020. Data science and analytics skills were at the top of the hiring list for 31 percent of the CIOs responding to the 2021 State of the CIO survey, but 20 percent expected to have trouble sourcing AI and machine learning experts and 19 percent anticipated challenges finding and hiring analytics and data science talent.

Part of the problem is education-focused: Fewer than one-third of the U.S. News & World Report's Top 100 Global Universities offer degrees in data science and most programs are still taught at a graduate or PhD level, according to research conducted by the University of California, Riverside, as part of the pitch for its own graduate-level program.

"The number of open [data science] roles compared to the number of graduates produced across the ecosystem of universities, bootcamps, and self-based learning platforms like Coursera are a fraction of what the U.S. economy needs," says Shamsudeen Mustafa, co-founder and co-CEO of Correlation One, a technology company aiming to create equal access to the data-driven jobs of the future, including advancement through public-private partnerships. "That confluence of factors has led to an absolute war for talent in the last two years, and I don't see the supply imbalance correcting any time soon."

Feeding the data science pipeline

While freshly minted undergraduate and graduate students aren't a panacea for filling more senior-level data science positions, they are a perfect feeder for entry-level posts and for cultivating and growing employees into more advanced data and analytics roles. Industry and university partnerships are critical to priming this pipeline of fresh talent for a number of reasons.

On the one hand, corporate enterprises gain a hand in influencing data science curriculum, which is still relatively new and evolving, to ensure it emphasizes what's important to business as opposed to pure theoretical book learning. For their part, universities get exposure to real-world business problems that provide practical context and experience for students. Partnerships also open up access to modern software tools and development practices, mentorships, and potential internships—all essential elements for career advancement and increasing students' job marketability.

Please read: How to find the right data scientists

Dozens of companies are experimenting with industry-university partnerships to bridge much-needed skills gaps, especially in data science-related areas and cybersecurity. Microsoft, for example, is partnering with higher education institutions like Purdue University Global, the University of London, and Bellevue College to offer blended and flex-learning opportunities based on Microsoft technical skills courses in such competencies as AI, data science, and big data. As part of its Digital Readiness Program, Intel is offering more than 225 hours of AI-related content to community colleges in an effort to help them develop AI certificates or launch full AI associate programs.

In a slightly different twist, Infosys, a global consulting leader, is tapping North Carolina State University for a three-year collaboration designed to advance its internal employees' education and skills development in foundational data science areas such as statistics, data visualization, machine learning, and Python programming. The company is aiming to send at least 150 new Infosys employees through the six week, full-time program.

Correlation One is pushing data science skills through similar programs. On one hand, it partners with companies like J.P. Morgan, EY, Johnson & Johnson, and Experian to help develop data science competencies through a variety of methods, including assessing data literacy, developing organization-wide customized training programs to upskill employees, and creating pathways to train and hire talent from underrepresented communities, according to Mustafa. The employer-embedded training options are offered free to employees, and courses embed real-world datasets, which helps determine who has an aptitude for this kind of work. "This eliminates bias in resumes and all sorts of inefficiencies in the recruiting process," he explains. "It helps cut cycle time for finding new talent as well as onboarding time."

Getting everyone onboard

As part of its Data Science for All (DS4A)/Empowerment initiative, Correlation One is pursuing public-private partnerships with industry and universities to create more qualified data science candidates from underrepresented groups. Employers that partake in the program underwrite data science training that Correlation One offers free to qualified candidates (students or working professionals), with those representing Black, Latinx, and LGBTQ+ communities being prioritized. Last year, Correlation One received 75,000 applications for 2,500 seats, and 95 percent of those accepted were Black or Hispanic, Mustafa says.

Please read: The best post-pandemic opportunities for IT workers

In one of its more recent DS4A initiatives, Correlation One entered into a partnership with the city of Miami, backed by SoftBank Group International. Over the next three years, the partners aim to provide data science training to at least 10,000 people from underrepresented communities in Greater Miami. The program is also open to Miami Dade College students, who, if accepted, participate in 14 weeks of data and analytics training working on real-world problems and case studies developed by SoftBank's portfolio of companies. In return, they receive up to 16 college credits toward a bachelor's degree in data analytics at MDC.

Correlation One doesn't evaluate candidates on their data science background, but rather considers foundational capabilities like an aptitude for logical and quantitative reasoning and their interest in pursuing a data science-related career. Graduates of the DS4A programs are adequately prepared to take on basic analytics work and can be groomed for more advanced roles. More important, this stream of potential job candidates brings much needed diversity to data science roles.

"People building algorithms are not diverse, which leads to bias in datasets," Mustafa says. "As a society, we're at a critical juncture unless we create a new supply of diverse talent."

Please read: How job seekers can tell whether a company is serious about diversity

Senthil Gandhi, principal scientist for AI and data at HPE Pointnext Services, is all for leveraging partnerships and pursuing creative hiring and training alternatives to address what he says is a severe data science pipeline problem. Given that data scientist as a formal role hasn't been around that long, it's next to impossible to source a candidate with decades-plus years of experience. "The term data scientist started seven or eight years ago, so you can't expect to find someone with 15 years of experience in the field," he says. "Therefore, you have to pick people who have a propensity to move into these roles and coach them into it." Employees with strong math backgrounds and familiarity with programming languages like Python are the best candidates, he says.

At the same time, Gandhi stresses the need to address the pipeline problem—and shore up basic data science-related competencies—well before the university level. "Anyone who's enrolled in a data science master's degree or engaged in the subject in a university has already demonstrated a propensity and is on the right path," he says. "We need to assist them more, but the pipeline is broken well before university starts. We have to start at the high school level."

Start the process early in the educational path

Synchrony is looking at addressing the pipeline problem at both the university and high school levels. As part of its Synchrony Skills Academy High School Program, the company is planning a skills center designed to give eligible high school students real-world experience through training programs as well as college and technology career assistance. The initiative, a private-public partnership between Synchrony and nonprofits such as the University of Connecticut's Engineering Ambassadors, District Arts and Education, and Future 5 will help eligible high school juniors and seniors hone critical skills in web development, user experience design, and data analytics.

At the university level, interns working at the Synchrony Emerging Technology Center are knee-deep in real data science work, including developing robotic process automation bots to improve back-office operations, building interactive reporting dashboards to analyze and build credit models, and examining fraud analytics data to uncover trends and optimize the statistical approach used for transactional fraud. To reach a wider audience at the university, Synchrony is hosting data-thons where hundreds of students work as part of a team to develop AI and analytics applications that solve a particular business problem.

Please read: Investing in people is key to successful transformation

The university-level programs have created a steady stream of data science job candidates in a short time—about 50 have been hired by Synchrony so far—but the initiatives take work. Juel says companies need to be willing to make the appropriate investments, including putting the right people on the ground and getting top leaders on board. Beyond any individual corporate efforts, Juel contends private-public collaboration at scale is necessary for building a pipeline of talent that is flush enough to serve future data science needs.

"University and industry partnerships are a start," she says, "but we need government leaders, nonprofits, and other businesses working together to develop the talent needed to close the gap."

This article/content was written by the individual writer identified and does not necessarily reflect the view of Hewlett Packard Enterprise Company.