Kubernetes PaaS for Splunk

Delivered by HPE GreenLake on Intel Architecture

+ show more
Solution brief

Exponential data growth from increased infrastructure, application, and network traffic have outpaced most security solutions’ ability to effectively leverage data to detect, alert, and prevent security events.

As data rates exceed an organization’s ability to index, store, and analyze the data, it is estimated that more than half* of the security-related digital exhaust isn’t even being utilized in security analytics because the data simply can’t be loaded and processed in time. This means there are huge blind spots in IT security and operations.

The problem is that legacy infrastructure not optimized for Splunk combined with lack of experience with Kubernetes has resulted in the underutilization of oversized systems. Even in optimal solutions, indexing is measured in the 100 GB to 300 GB per day per host range and still operate at relatively low CPU utilization requiring massive overdeployments of infrastructure to keep pace with data growth. This excessive infrastructure bloat means that data centers are nearing maximum capacity as IT support teams struggle to scale out ingestion, processing, storage, and analysis of the data. The increased infrastructure coupled with the growing backlog of data and security insights, is forcing organizations to find ways to optimize their delivery and consumption of Splunk analytics.

  • Innovation from HPE, Intel®, and Splunk

    Solving the blind spot problem is paramount for security, but it requires application, architecture, and consumption model modernization. Individually these changes can be risky but combined are a daunting task. This is why Hewlett Packard Enterprise, Intel, and Splunk collaborated to solve the ingest problem with a unique platform-as-a-service (PaaS) solution that allows for independently scaling indexers and search heads; up, down, and out. It allows organizations to leverage the critical data to get a full view of the IT security landscape and get more value from existing Splunk investments with efficient, right-sized deployments.

    Workload-optimized infrastructure to help eliminate bottlenecks at scale

    It helps eliminating the ingest bottleneck required while helping optimize HPE ProLiant DL380 Gen10 servers for concurrent indexing. HPE and Intel found the ideal Intel® Xeon® Scalable processors and RAM configuration and paired it with local Intel NVMe NAND SSDs as cache, expandable up to 122 TB per host, to deliver massive ingestion performance. The solution is complemented with an object store using HPE Scalable Object Storage with Scality RING on the HPE Apollo 4500 Gen10 storage servers and the Splunk SmartStore to safely scale retention independently from hot cache with > 14 9s of data durability.  1

    Containerized software to scale up

    HPE and Splunk partnered to leverage the power of the HPE Ezmeral Container Platform and open-source Kubernetes to bring agility and scale to the new containerized Splunk operator.  2 This has two immediate benefits: the solution deploys new indexer and search heads in a matter of minutes, and independently scales them up within a host to fully saturate the infrastructure and scale out across the entire available information estate.

    HPE GreenLake cloud services

    With HPE GreenLake complemented by HPE Pointnext Services, the solution is provided aaS on-premises or in a colocated solution, managed by HPE up through the container and storage layer. With the HPE GreenLake model, pay for what you use with no up-front outlay, scaling ability up or down quickly. There’s no patching, performance tuning, or maintenance required, and you don’t need hard to find Kubernetes skills; HPE takes care of it! Additionally, this configuration offers burst options, and the containerized solution is multi/hybrid cloud ready when the time is right to expand from edge to cloud.

  • Key solution results
    The HPE GreenLake service enables a modern delivery of Splunk in containers that can take full advantage of the optimized Intel Xeon Scalable processors and Intel NVMe NAND to drive utilization and throughput. The HPE, Splunk, and Intel team tested concurrent searches and data model acceleration, using Splunk Enterprise Security with Intel IT production data (real world, high cardinality data; i.e., non-synthetic data) from seven different data sources. While running these searches the solution was able to independently scale from 1 to 6, and up to 12 indexers per host delivering an astounding 17X indexer throughput improvement keeping CPU saturation below 70%.  3

Figure 1. Indexer daily throughput performance and CPU utilization per host

  • What this means for you

    The tight collaboration between HPE, Intel, and Splunk brings dark data to light by making it simple to collect, analyze, and act upon the untapped value of the Big Data generated by your technology infrastructure, security systems, and business applications—giving you the insights to drive operational performance and business results. The enhanced delivery of Splunk creates a single datastore of all machine data that leverages open-source Kubernetes and S3, and is available as a fully managed aaS solution from HPE.

    • Helps eliminate the data blind spot with up to 17X higher data ingest per host

    • Shrinks the infrastructure footprint and significantly lowers TCO with a loosely coupled architecture that independently scales search heads, indexers, and storage

    • Adds new use cases deploying new indexers and search heads in minutes

    • Leverages Splunk SmartStore to efficiently balance hot cache and S3 object storage for exabyte scale cold data

    • Allows you to pay for what you use using the flexible consumption model, with rapid scalability and no up-front outlay

    • Enables organizations to focus on their business with fully managed PaaS solution

Download the PDF

Hewlett Packard Enterprise shall not be liable for technical or editorial errors or omissions contained herein.

Intel, Intel Xeon, and the Intel logo are trademarks of Intel Corporation in the U.S. and other countries. All third-party marks are property of their respective owners.

Kubernetes operator for Splunk is currently only available in pre-release. 
Based on Splunk, Intel, and HPE testing with up to 12 indexers per host (Results may vary). August 2020 

© Copyright 2020 Hewlett Packard Enterprise Development LP. The information contained herein is subject to change without notice. The only warranties for Hewlett Packard Enterprise products and services are set forth in the express warranty statements accompanying such products and services. Nothing herein should be construed as constituting an additional warranty.