Networking Considerations

Note: Please read Networks and Subnets to understand networking in HPE Ezmeral Container Platform.

When setting up a Kubernetes cluster, you will provide a CIDR IP range that is typically in a private (non-routable) IP range. Both private (non-routable) and public (routable) container networks are supported. Here are some areas of consideration when deciding whether to use a private or public container network.

Benefits Considerations
Private, non-routable container network
  • No dependency on network team, such as for getting any additional subnets beyond the base set of hosts that each have an IP address and FQDN.
  • Create and use an arbitrarily large private IP range with no need for planning. A single /16 private subnet will support ~16000 containers. It is not normally possible for networking teams to provide such a large single range; they will probably provide multiple different subnets that will add administrative overhead for both the Platform Administrator and the network team.
  • Private IP ranges make security compliance significantly easier because it simplifies controlling the specific container service endpoints that need to be exposed to the end user network.
  • Many organizations treat any virtual machine or container with a routable IP address similarly to a physical machine. Deleting these hosts requires approval and signoff. This goes against the BDaaS user experience where users want to shrink and delete clusters at will.
  • Hosts can span multiple subnets, as described in Multiple Subnets.
  • Each Gateway host (or common FQDN) can support up to 40,000 service endpoints. You may reserve one or more port range(s) from 10,000 to 50,000. Ports reserved for HPE Ezmeral Container Platform may not be used for any other purpose.
  • Only control traffic (e.g. Console Hue, SSH, scp, etc.) goes through the Gateway host. Data I/O does not go through the Gateway host.
  • If you add new services to an existing virtual cluster, then you need to follow the procedure described in Adding a Cluster Service to expose the newly-added services to users.
  • If the port number changes for a service endpoint on an existing cluster, then the existing HTTP link to that service will no longer work. You will need to manually enter the new port number in the URL address.
Public, routable container network All service endpoints are directly accessible using the IP address of the virtual node.
  • Make sure that the pool of reserved IP addresses is large enough to support both current and projected future needs.
  • All hosts must be in the same subnet.

The cluster networking uses a VXLAN overlay network. The underlying physical network must be implemented with this in mind. Hewlett Packard Enterprise recommends the following best practices:

In a large complex scaled distributed environment, factors such as pod scheduling characteristics across these physical hosts and application design can impact your network operation, as described here (see the use cases; link opens an external website in a new browser tab/window).