Skip to content

Expose and manage Kubernetes Custom Resource Definitions (Operators) in a Kubernetes Cluster

OpenSlice is capable of exposing Kubernetes Resources and Definitions as Service Specifications

Intended Audience: Service Designers

Kubernetes is an orchestration system for automating software deployment, scaling, and management. One can interact though the Kubernetes API and it has a set of objects ready for use out of the box. Custom Resource Definitions (CRDs) is a way that allows to manage things other than Kubernetes itself and allows to create our own objects The use of CRDs makes the possibilities of Kubernetes management almost limitless. You can extend the base Kubernetes API with any object you like using CRDs.

By allowing the design and lifecycle management of services/resources that expose CRDs/CRs in a Kubernetes cluster via the TMF APIs, OSL can be used in many complex scenarios now involing resources from multiple domains.

  1. OpenSlice is capable to:

    • Create and manage Custom Resources (CRs) using installed CRDs on a target Kubernetes cluster.
    • Facilitate complex orchestration scenarios by wrapping Kubernetes APIs as TMF APIs and models.
    • Handles connectivity to a Kubernetes cluster and manages the lifecycle of CRDs
    • Wraps the Kubernetes API, Receives and provides resources towards other OpenSlice services via the service bus

  2. Enabling Loose Coupling and Orchestration

    • Language Flexibility: Developers can write CRDs in any language and expose them via the Kubernetes APIs. OSL will reuse these CRDs, enhancing flexibility and integration capabilities.
    • Familiar Deployment: Developers can create and deploy applications using familiar tools such as Helm charts, simplifying the process and reducing the learning curve.

  3. Ecosystem Reusability

    • OpenSlice capitalizes on the extensive Kubernetes ecosystem, particularly focusing on operators (CRDs).
    • Key repositories and hubs such as artifacthub.io and Operatorhub.io can be utilized for finding and deploying operators.

  4. Service Catalog Exposure and Deployment

    • OpenSlice can expose CRs in service catalogs, facilitating their deployment in complex scenarios.
    • These scenarios may include service bundles that involve multiple systems, such as RAN controllers or other Kubernetes clusters, providing a robust and versatile deployment framework.

Approach

OpenSlice in general is responible for exposing service specifications which are ready to be ordered and orchestrated, through tmforum Open APIs as defined in the OSL Service Spec Catalog. Usually for a service specification a corresponding (one or more) resource specification (resourceSpecificationReference) is registered in the OSL Resource Spec Catalog.

The following image illustrates the approach.

img01.png

  1. A CRD in a cluster will be mapped in TMF model as a Resource specification and therefore can be exposed as a service specification in a catalog
  2. Service Orders can be created for this service specification.
  3. OSOM creates a Resource in OSL Resource inventory and requests new Custom Resource (CR) in the target cluster
    • The resource is created in a specific namespace (for example the UUID of the Service Order)
    • A CR in a cluster will be mapped in TMF model as a Resource in the resource Inventory
    • Other related resources created by the CRD Controller within the namespace are automatically created in OSL Resource Inventory under the same Service Order

Awareness for CRDs and CRs in cluster

CRDs and CRs can appear (disappear) or change status at any time in a cluster. OpenSlice Resource Inventory need to be aware of these events.

When installing OpenSlice you can configure at least one management cluster. OpenSlice connects via a provided kubeconf

  • On Start up OSL tries to register this cluster and context to OSL catalogs.
  • After the registration of this cluster as a Resource in OSL OSL is always aware of all CRDs and their CRs in the cluster, even if a CRD or CR is added/updated/deleted in the K8S cluster outside of OSL
  • Resources created by OpenSlice have labels, e.g. (org.etsi.osl.*)

Expose CRDs as Service Specifications in OpenSlice catalogs

A CRD by default is exposed as a Resource Specification

To ensure unique names across the clusters that OpenSlice can manage, the name of a CRD is constructed as follows:

Kind @ ApiGroup/version @ ContextCluster @ masterURL

For example you might see resource Specifications like:

- ```Application@argoproj.io/v1alpha1@kubernetes@https://10.10.10.144:6443/```
- ```IPAddressPool@metallb.io/v1beta1@kubernetes@https://10.10.10.144:6443/```
- ```Provider@pkg.crossplane.io/v1@kubernetes@https://10.10.10.144:6443/```

All attributes of the CRD are translated into characteristics

The following specific characteristics are added:

    - _CR_SPEC: Used for providing the json Custom Resource description to apply
    - _CR_CHECK_FIELD: Used for providing the field that need to be checked for the resource status
    - _CR_CHECKVAL_STANDBY: Used for providing the equivalent value from resource to signal the standby status
    - _CR_CHECKVAL_ALARM: Used for providing the equivalent value from resource to signal the alarm status
    - _CR_CHECKVAL_AVAILABLE: Used for providing the equivalent value from resource to signal the available status
    - _CR_CHECKVAL_RESERVED: Used for providing the equivalent value from resource to signal the reserved status
    - _CR_CHECKVAL_UNKNOWN: Used for providing the equivalent value from resource to signal the unknown status
    - _CR_CHECKVAL_SUSPENDED: Used for providing the equivalent value from resource to signal the suspended status

  1. Create a new Service Specification and use this Resource Specification in Resource Specification Relationships

    • Then the Service Specification is saved as ResourceFacingServiceSpecification

    1.1. You can give at this stage values to the characteristics:

    - _CR_SPEC, 
- _CR_CHECK_FIELD
- _CR_CHECKVAL_STANDBY
- _CR_CHECKVAL_ALARM
- _CR_CHECKVAL_AVAILABLE
- _CR_CHECKVAL_RESERVED
- _CR_CHECKVAL_UNKNOWN
- _CR_CHECKVAL_SUSPENDED

    1.2. You can now create LCM rules if you wish

  2. Create a new Service Specification and use the Resource Facing Service Specification in Service Specification Relationships

    • Then the Service Specification is saved as CustomerFacingServiceSpecification

    2.1. You can give at this stage values to the characteristics: 

    - _CR_SPEC, 
- _CR_CHECK_FIELD
- _CR_CHECKVAL_STANDBY
- _CR_CHECKVAL_ALARM
- _CR_CHECKVAL_AVAILABLE
- _CR_CHECKVAL_RESERVED
- _CR_CHECKVAL_UNKNOWN
- _CR_CHECKVAL_SUSPENDED

    2.2. You We can create LCM rules for this new Service Specification

    2.3. You Expose configurable values for users to configure during service order

img06.png

Service Orchestration and CRDs/CRs

OSOM - OpenSlice Service Orchestrator, checks the presence of attribute _CR_SPEC at the RFS to make a request for a CR deployment

  • _CR_SPEC is a JSON or YAML string that is used for the request
    • It is similar to what one will do with e.g. a kubectl apply
    • There are tools to translate a yaml file to a json

LCM rules can be used to change attributes of this yaml/json file, before sending this for orchestration

Mapping the CR lifecycle that is defined in the CRD with the OpenSLice (TMF-based) resource Lifecycle

OpenSlice adds automatically as we see the following characteristics: 

    - _CR_CHECK_FIELD
    - _CR_CHECKVAL_STANDBY
    - _CR_CHECKVAL_ALARM
    - _CR_CHECKVAL_AVAILABLE
    - _CR_CHECKVAL_RESERVED
    - _CR_CHECKVAL_UNKNOWN
    - _CR_CHECKVAL_SUSPENDED

These characteristics instrument OpenSlice services to manage and reflect the lifecycle of a kubernetes resource to OpenSlice's (TMF based) lifecycle

  • _CR_CHECK_FIELD: The name of the field that is needed to be monitored in order to monitor the status of the service and translate it to TMF resource statys (RESERVED AVAILABLE, etc)
  • _CR_CHECKVAL_STANDBY: The CR specific value (of the CheckFieldName) that needs to me mapped to the TMF resource state STANDBY (see org.etsi.osl.tmf.ri639.model.ResourceStatusType)
  • _CR_CHECKVAL_ALARM: The CR specific value (of the CheckFieldName) that needs to me mapped to the TMF resource state ALARMS (see org.etsi.osl.tmf.ri639.model.ResourceStatusType)
  • _CR_CHECKVAL_AVAILABLE: The CR specific value (of the CheckFieldName) that needs to me mapped to the TMF resource state AVAILABLE (see org.etsi.osl.tmf.ri639.model.ResourceStatusType)
  • _CR_CHECKVAL_RESERVED: The CR specific value (of the CheckFieldName) that needs to me mapped to the TMF resource state RESERVED (see org.etsi.osl.tmf.ri639.model.ResourceStatusType)
  • _CR_CHECKVAL_UNKNOWN: The CR specific value (of the CheckFieldName) that needs to me mapped to the TMF resource state UNKNOWN (see org.etsi.osl.tmf.ri639.model.ResourceStatusType)
  • _CR_CHECKVAL_SUSPENDED: The CR specific value (of the CheckFieldName) that needs to me mapped to the TMF resource state SUSPENDED (see org.etsi.osl.tmf.ri639.model.ResourceStatusType)

Probe further