Kubernetes Metadata

Note: This page is intended for Kubernetes administrators and other advanced Kubernetes users.

HPE Ezmeral Container Platform includes two sets of custom resources and two operators running in the hpecp namespace that manage those resources:

The kubedirector operator manages the kubedirectorcluster, kubedirectorapp, and kubedirectorconfig resources.
- kubedirectorconfig is a singleton resource that does not usually require modification.
- The kubedirectorapp (app definition) and especially kubedirectorcluster (app instantiation) resources are more likely to be actively created, edited, etc. KubeDirector is an open-source project documented on GitHub, but some aspects of its behavior are documented in this article (link opens an external website in a new browser tab/window).
The hpecp-agent operator manages the hpecptenant, hpecpfsmount, and hpecpconfig resources.
- hpecpconfig is a singleton resource that does not usually require modification.
- hpecptenant (located in the hpecp namespace) models a tenant.
- hpecpfsmount (located in a tenant namespace) models an FS Mount that was likely created in a tenant.

These custom resources can have labels and annotations on them that communicate useful information about their properties or context. Users who manually/explicitly create Kubernetes pod and service resources can also choose to add certain labels or other properties to those objects to trigger additional feature behaviors in the hpecp-agent operator.

Labels That Can be Used to Trigger Features

Pod label usable to trigger FS mount: hpecp.hpe.com/fsmount: <FS mount namespace>

(Can be auto-generated by HPECP Agent for KubeDirector pods; see below.)
Pod label usable to trigger DataTap setup: hpecp.hpe.com/dtap

(The value is not important, just the label key existence.)
Service label usable to control gateway mapping (NodePort only): hpecp.hpe.com/hpecp-internal-gateway: <"true" or "false">

(Can be auto-generated by HPECP Agent; see below.)
Service label usable to force port name (single-port service only): hpecp.hpe.com/portname-override: <desired port name>

(This label is useful for tools like kubectlexpose that don't allow direct specification of port names.)

Other Feature Controls

If the port name within a Kubernetes service object starts with the prefix http- or https-, then this can affect its exposure through the Gateway host and the web interface:

Only endpoints with such port name prefixes will get clickable links in the Kubernetes Service Endpoints screen. See Kubernetes Service Endpoints Tab.
If https- prefixed, then that UI link will correctly be an https link regardless of the SSL configuration (or lack thereof) for the Gateway hosts.
If http- prefixed, and if the Gateway does not support SSL termination, then the service will be exposed as normal http through the Gateway and the interface links.
If http- prefixed, and if the gateway supports SSL termination, then this service will get SSL termination at the gateway, and the interface link will be https.

Services and port names generated by KubeDirector will always have a port name prefix that comes from the urlScheme for that endpoint, as defined by the KubeDirector app. Manual explicit port naming is therefore usually only of interest when you are creating http/https services outside of KubeDirector.

Labels Generated by KubeDirector

The labels generated by KubeDirector on any statefulset, pod, or service (either per-member or headless) are:

kubedirector.hpe.com/kdcluster: <kdcluster resource name>
kubedirector.hpe.com/kdapp: <kdapp resource name>
kubedirector.hpe.com/appCatalog: <either local or system>

Labels generated by KubeDirector on any statefulset, pod, or per-member service created by KubeDirector :

kubedirector.hpe.com/role: <kdapp role ID>

Labels generated by KubeDirector on any statefulset or pod created by KubeDirector :

kubedirector.hpe.com/headless: <name of headless cluster service>

Labels generated by HPECP Agent on any statefulset pod created by KubeDirector :

hpecp.hpe.com/fsmount: <pod namespace> (only created by HPECP Agent if label does not already exist in the statefulset pod template)

Labels generated by HPECP Agent on any NodePort service:

hpecp.hpe.com/hpecp-internal-gateway: <true or false> (only created if label does not already exist; if in a tenant namespace, the value is driven by the tenant setting; otherwise false.)

Labels generated by HPECP Agent on any namespace associated with an HPE Ezmeral Container Platform Tenant:

hpecp.hpe.com/hpecptenant: <hpecptenant resource name>

User-Requested Labels through KubeDirector

The optional podLabels array in a role in a KubeDirector cluster can be used to specify additional labels to put on its generated statefulset pods, and/or to override the labels that would normally be generated for those pods. For example, this is used on cluster admin webterms to mount all FS mount namespaces.
The optional serviceLabels array in a role in a KubeDirector cluster can be used to specify additional labels to put on its generated member services, and/or override the labels that would normally be generated for those services. E.g. our platform uses this on webterms to enable gateway mapping (setting hpecp.hpe.com/hpecp-internal-gateway to true) even though the webterm is not in a tenant namespace.

Annotations Generated by KDKubeDirector

Annotations generated by KDKubeDirector on any statefulset, pod, or service created by KubeDirector :

kubedirector.hpe.com/kdapp-prettyName: <KD app label name>

Annotations generated by HPECP Agent on any service where gateway mapping is enabled:

hpecp-internal-gateway/<pod port>: <gateway hostname>:<gateway port>

Other Conventions

HPE Ezmeral Container Platform gives the following annotations to a Kubernetes tenant resource. These annotations are not required by the tenant CRD, but they are useful as FYIs for anyone examining the tenant object.

hpecp-tenant: <HPECP tenant path, e.g. "/api/v2/tenant/4">
hpecp-tenant-name: <HPECP tenant label name>

The tenant Kubernetes resource name also always follows the convention hpecp-tenant-<tenant ID>. For example, if the tenant in the API is /api/v2/tenant/44, then the Kubernetes resource will be named hpecp-tenant-44. Some functionality around reporting existing tenant/namespace associations relies on this convention.