By apipark —

How to Read a Custom Resource Using Cynamic Client Golang

How to Read a Custom Resource Using Cynamic Client Golang
read a custom resource using cynamic client golang
XRoute.AI
XPack.AI

Introduction: Navigating the Extended Kubernetes Landscape with Go

Kubernetes has evolved far beyond a mere container orchestrator; it is now a powerful, extensible platform capable of managing virtually any workload or infrastructure component. A cornerstone of this extensibility is the Custom Resource Definition (CRD), which allows users to define their own resource types, thereby extending the Kubernetes API. These Custom Resources (CRs) empower developers to model domain-specific objects directly within the Kubernetes ecosystem, enabling the creation of powerful operators and controllers that automate complex application lifecycle management.

While client-go, the official Go client library for Kubernetes, provides generated typed clients for all built-in Kubernetes resources and for CRDs when their Go types are known at compile time, there are scenarios where this approach falls short. What if you need to interact with a CRD whose Go type hasn't been generated, or worse, isn't even known at compile time? What if you're building a generic tool or an API gateway that needs to work with arbitrary custom resources without being recompiled every time a new CRD is introduced? This is precisely where the dynamic client in client-go becomes indispensable.

The dynamic client offers a powerful, flexible mechanism to interact with any Kubernetes resource, whether built-in or custom, without requiring specific Go types. It operates on Unstructured objects, representing resources as generic map[string]interface{} data structures. This capability is critical for building highly adaptable applications, generic controllers, or even open platform tools that need to introspect and manipulate the Kubernetes api in a schema-agnostic manner.

This comprehensive guide will delve deep into the mechanics of using the dynamic client in Golang to read Custom Resources. We will explore the underlying concepts, practical implementation details, and best practices to equip you with the knowledge to confidently leverage this powerful feature. By the end of this article, you will understand how to configure the client, discover resources, retrieve specific CRs, and extract meaningful data from their unstructured representations, paving the way for more sophisticated Kubernetes interactions and the creation of highly extensible solutions.

Understanding Custom Resources and Their Role in Kubernetes

Before we dive into the dynamic client, it's crucial to have a solid understanding of Custom Resources and why they are so fundamental to modern Kubernetes operations. Custom Resources are extensions of the Kubernetes api that allow you to define your own resource types. They enable you to tailor Kubernetes to your specific needs, making it a truly adaptable open platform.

The Need for Custom Resources

Historically, Kubernetes provided a rich set of built-in resources like Pods, Deployments, Services, and Ingresses. These cover many common container orchestration patterns. However, as organizations started running more complex, domain-specific applications on Kubernetes, the need to manage application-specific constructs within the Kubernetes control plane became evident.

Imagine you're developing a data processing platform. You might have concepts like "DataPipeline," "StreamProcessor," or "FeatureStore." Instead of managing these as external configurations or through separate tools, CRDs allow you to define them as first-class citizens within Kubernetes. This brings several advantages:

  1. Unified Control Plane: All your application's components, whether standard Kubernetes objects or domain-specific ones, are managed through the same Kubernetes api, using familiar tools like kubectl.
  2. Declarative Management: You can define the desired state of your custom resources in YAML, just like Deployments or Services. Kubernetes then works to reconcile the actual state with the desired state.
  3. Operator Pattern: CRDs are the bedrock of the Operator pattern. An Operator is a software extension to Kubernetes that uses custom resources to manage applications and their components. It watches for changes to your custom resources and takes specific actions to achieve the desired state, effectively encoding human operational knowledge into software.
  4. Abstraction: CRDs provide a powerful layer of abstraction. Complex underlying infrastructure or application logic can be encapsulated within a custom resource, exposing a simpler, higher-level api to users.

Anatomy of a Custom Resource Definition (CRD)

A Custom Resource Definition (CRD) is a Kubernetes resource that defines a new kind of resource. When you create a CRD, you tell Kubernetes about your new type, including its name, scope, and schema.

Here's a breakdown of key CRD components:

  • apiVersion and kind: Standard Kubernetes api identifiers. For CRDs, apiVersion is typically apiextensions.k8s.io/v1 and kind is CustomResourceDefinition.
  • metadata: Standard Kubernetes metadata like name. The name of a CRD follows the format <plural>.<group>.
  • spec: This is where the core definition resides:
    • group: The API group for your custom resource (e.g., example.com). This helps organize and avoid naming conflicts.
    • names: Defines the various names for your resource:
      • plural: The plural name used in kubectl commands (e.g., applications).
      • singular: The singular name (e.g., application).
      • kind: The camel-cased Kind name (e.g., Application). This is what you put in the kind field of your custom resource YAML.
      • shortNames: Optional, shorter aliases for kubectl (e.g., app).
    • scope: Can be Namespaced or Cluster. Most custom resources are Namespaced, meaning instances of the resource exist within a specific namespace. Cluster-scoped resources are global to the cluster.
    • versions: A list of api versions supported by your CRD. Each version defines:
      • name: The version string (e.g., v1alpha1, v1).
      • served: Boolean, whether this version is served by the api server.
      • storage: Boolean, whether this version is the primary storage version.
      • schema: An OpenAPI v3 schema that validates instances of your custom resource. This is crucial for ensuring data integrity and providing api documentation.
      • subresources: Optional definitions for /status or /scale subresources.

Example CRD: Application

Let's consider a simple CRD for managing a hypothetical "Application" resource.

apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
  name: applications.example.com
spec:
  group: example.com
  versions:
    - name: v1
      served: true
      storage: true
      schema:
        openAPIV3Schema:
          type: object
          properties:
            apiVersion: {type: string}
            kind: {type: string}
            metadata: {type: object}
            spec:
              type: object
              properties:
                image:
                  type: string
                  description: The container image to deploy.
                replicas:
                  type: integer
                  minimum: 1
                  description: The number of desired replicas.
                ports:
                  type: array
                  items:
                    type: object
                    properties:
                      name: {type: string}
                      port: {type: integer}
                      protocol: {type: string}
                    required: ["name", "port"]
              required: ["image", "replicas"]
  scope: Namespaced
  names:
    plural: applications
    singular: application
    kind: Application
    shortNames:
      - app

Once this CRD is applied to a Kubernetes cluster (kubectl apply -f application-crd.yaml), the Kubernetes api server will begin serving a new api endpoint for applications.example.com/v1. You can then create instances of this custom resource:

apiVersion: example.com/v1
kind: Application
metadata:
  name: my-webapp
  namespace: default
spec:
  image: "nginx:latest"
  replicas: 3
  ports:
    - name: http
      port: 80
      protocol: TCP

This my-webapp Custom Resource is now a first-class object in your Kubernetes cluster, and an Operator could watch for Application resources and create corresponding Deployments, Services, and other standard Kubernetes objects to realize this desired state. This demonstrates the power of extending Kubernetes as an open platform for managing diverse workloads.

The Golang Ecosystem for Kubernetes Interaction

Golang plays a pivotal role in the Kubernetes ecosystem. Kubernetes itself is written in Go, and client-go, the official Go client library, is the standard for interacting with the Kubernetes api server from Go applications.

client-go: The Official Go Client

client-go provides a robust and comprehensive set of tools for developing Kubernetes controllers, operators, and other applications that interact with the cluster. It abstracts away the complexities of HTTP requests, api versions, and object serialization/deserialization.

Key components of client-go include:

  • Clientsets: Type-safe clients for built-in Kubernetes resources (e.g., corev1.Pods(), appsv1.Deployments()). These are generated based on the Kubernetes api definitions.
  • Informers: Mechanisms for efficiently watching and caching Kubernetes resources, reducing api server load and simplifying event-driven programming.
  • Listers: Index-based caches for fast, local access to resources managed by informers.
  • Scheme: A registry for Go types and their Kubernetes api group, version, and kind (GVK) information, used for serialization and deserialization.
  • Dynamic Client: The focus of this article, providing a generic way to interact with any resource, including custom ones, without requiring generated types.

Typed Clients vs. Dynamic Client: A Crucial Distinction

When you know the Go type definition for a Kubernetes resource at compile time (e.g., v1.Pod for a Pod, or a custom struct generated from a CRD), you typically use a typed client. Typed clients offer strong type safety: you work directly with Go structs that accurately reflect the resource's schema. This provides compile-time checks and IDE auto-completion, which are incredibly beneficial for developer productivity and error prevention.

For example, to get a Pod using a typed client:

// Assuming clientset is already configured
pod, err := clientset.CoreV1().Pods("default").Get(ctx, "my-pod", metav1.GetOptions{})
if err != nil {
    // handle error
}
fmt.Printf("Pod name: %s, Image: %s\n", pod.Name, pod.Spec.Containers[0].Image)

However, the world of Kubernetes extensibility often presents scenarios where typed clients are not feasible:

  1. Unknown CRDs: You might be developing a generic tool that needs to interact with CRDs that don't even exist yet or whose definitions are only known at runtime. Generating Go types for every possible CRD is impractical.
  2. Generic Controllers/Gateways: An API gateway or a generic controller might need to process various custom resources without specific compile-time knowledge of their types. It needs to be flexible enough to handle any resource defined by a CRD.
  3. Ad-hoc api Interactions: For scripts or utilities that perform one-off api calls to CRDs without the overhead of generating and maintaining Go types.

In these situations, the dynamic client steps in. It operates on Unstructured objects, which are essentially map[string]interface{}. This provides immense flexibility but comes at the cost of compile-time type safety. You'll need to use type assertions and careful introspection at runtime to access specific fields within the Unstructured data. This trade-off between flexibility and type safety is a core consideration when deciding which client to use for your specific needs.

Demystifying the Dynamic Client in client-go

The dynamic client is a cornerstone for building highly adaptable Kubernetes tools. It allows you to interact with Kubernetes resources without compile-time knowledge of their Go types, making it ideal for generic controllers, API gateway implementations, or any application that needs to operate on arbitrary Custom Resources.

What is the Dynamic Client?

At its core, the dynamic client provides an api that operates on the fundamental map[string]interface{} representation of Kubernetes objects. In client-go, this is encapsulated by the Unstructured struct. When you fetch a resource using the dynamic client, it returns an Unstructured object, which you then need to inspect and parse dynamically.

The dynamic client doesn't care about the specific Go struct that represents your CRD. Instead, it only needs to know the GroupVersionResource (GVR) of the resource you want to interact with.

GroupVersionResource (GVR) vs. GroupVersionKind (GVK)

Understanding the distinction between GVR and GVK is crucial for working with the dynamic client:

  • GroupVersionKind (GVK): This identifies a specific type of Kubernetes object.
    • Group: The api group (e.g., apps, example.com).
    • Version: The api version within that group (e.g., v1, v1alpha1).
    • Kind: The specific Kind of the resource (e.g., Deployment, Application).
    • Example: {Group: "apps", Version: "v1", Kind: "Deployment"}
  • GroupVersionResource (GVR): This identifies a specific resource endpoint on the Kubernetes api server. It's the path segment used in the api URL.
    • Group: The api group.
    • Version: The api version.
    • Resource: The plural name of the resource (e.g., deployments, applications). This is derived from the plural field in the CRD's names section.
    • Example: {Group: "apps", Version: "v1", Resource: "deployments"}

The Kubernetes api server works with GVRs for its REST endpoints. The dynamic client, being a low-level client, also operates directly with GVRs. When you want to interact with a CRD using the dynamic client, you must provide its GVR. If you only know the GVK (which is common from a custom resource's YAML), you'll need a mechanism to map the GVK to its corresponding GVR. This mapping is typically handled by the RESTMapper component, which we will discuss shortly.

Key Interfaces and Structures

The client-go/dynamic package provides the core interfaces:

  • dynamic.Interface: The main interface for interacting with the dynamic client. You get an instance of this interface by calling dynamic.NewForConfig.
  • ResourceInterface: An interface returned by dynamic.Interface.Resource(gvr), which allows you to perform operations (Get, List, Create, Update, Delete) on resources of a specific GVR. For namespaced resources, you'll chain Namespace("your-namespace") after Resource(gvr).
  • unstructured.Unstructured: The Go struct that represents any Kubernetes object in a generic map[string]interface{} format. It contains helper methods for accessing apiVersion, kind, metadata, and spec fields.
  • unstructured.UnstructuredList: A list of Unstructured objects.
APIPark is a high-performance AI gateway that allows you to securely access the most comprehensive LLM APIs globally on the APIPark platform, including OpenAI, Anthropic, Mistral, Llama2, Google Gemini, and more.Try APIPark now! 👇👇👇
Install APIPark – it’s free

Prerequisites for Working with the Dynamic Client

Before diving into the code, ensure you have the necessary environment set up.

1. Go Environment

  • Install Go (version 1.16 or higher is recommended) on your development machine.
  • Set up your GOPATH.

2. Kubernetes Cluster Access

You need access to a Kubernetes cluster (local like minikube/kind, or a remote cloud cluster). The dynamic client will connect to this cluster's api server.

  • Ensure your kubeconfig file is correctly configured, typically located at ~/.kube/config. This file contains the cluster api endpoint, user credentials, and context information.
  • Verify connectivity using kubectl get nodes.

3. client-go Library

You need to include client-go in your Go project.

go mod init your-module-name
go get k8s.io/client-go@latest

This command will fetch the latest version of client-go and add it to your go.mod file.

4. Sample Custom Resource Definition (CRD) and Custom Resource (CR)

For our examples, we will use the Application CRD and a sample my-webapp Custom Resource introduced earlier.

  1. Save the CRD: application-crd.yaml yaml apiVersion: apiextensions.k8s.io/v1 kind: CustomResourceDefinition metadata: name: applications.example.com spec: group: example.com versions: - name: v1 served: true storage: true schema: openAPIV3Schema: type: object properties: apiVersion: {type: string} kind: {type: string} metadata: {type: object} spec: type: object properties: image: type: string description: The container image to deploy. replicas: type: integer minimum: 1 description: The number of desired replicas. ports: type: array items: type: object properties: name: {type: string} port: {type: integer} protocol: {type: string} required: ["name", "port"] required: ["image", "replicas"] scope: Namespaced names: plural: applications singular: application kind: Application shortNames: - app
  2. Apply the CRD: bash kubectl apply -f application-crd.yaml
  3. Save the Custom Resource: my-webapp.yaml yaml apiVersion: example.com/v1 kind: Application metadata: name: my-webapp namespace: default spec: image: "nginx:latest" replicas: 3 ports: - name: http port: 80 protocol: TCP
  4. Apply the Custom Resource: bash kubectl apply -f my-webapp.yaml

Now you have a CRD and an instance of a Custom Resource in your cluster, ready to be read by our Go program using the dynamic client.

Building the Dynamic Client: Step-by-Step

Let's walk through the process of setting up and using the dynamic client to read our Application Custom Resource.

Step 1: Loading Kubernetes Configuration

The first step for any client-go application is to load the Kubernetes configuration, which tells your application how to connect to the cluster api server.

package main

import (
    "context"
    "flag"
    "fmt"
    "os"
    "path/filepath"
    "time"

    metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
    "k8s.io/client-go/dynamic"
    "k8s.io/client-go/tools/clientcmd"
    "k8s.io/client-go/util/homedir"
)

func main() {
    var kubeconfig *string
    if home := homedir.HomeDir(); home != "" {
        kubeconfig = flag.String("kubeconfig", filepath.Join(home, ".kube", "config"), "(optional) absolute path to the kubeconfig file")
    } else {
        kubeconfig = flag.String("kubeconfig", "", "absolute path to the kubeconfig file")
    }
    flag.Parse()

    // Use the current context in kubeconfig
    config, err := clientcmd.BuildConfigFromFlags("", *kubeconfig)
    if err != nil {
        fmt.Printf("Error building kubeconfig: %s\n", err.Error())
        os.Exit(1)
    }

    fmt.Println("Kubernetes configuration loaded successfully.")

    // The rest of our code will go here
}

Explanation: * We use flag to allow specifying the kubeconfig path, defaulting to ~/.kube/config. * clientcmd.BuildConfigFromFlags constructs a rest.Config object, which contains all the necessary information for connecting to the cluster (host, authentication, TLS details). This configuration is fundamental for creating any client-go client.

Step 2: Creating the Dynamic Client

With the rest.Config object, we can now create an instance of the dynamic client.

// ... inside main after config is loaded ...

    dynamicClient, err := dynamic.NewForConfig(config)
    if err != nil {
        fmt.Printf("Error creating dynamic client: %s\n", err.Error())
        os.Exit(1)
    }

    fmt.Println("Dynamic client created successfully.")

// ...

Explanation: * dynamic.NewForConfig(config) takes our rest.Config and returns a dynamic.Interface, which is our entry point for interacting with custom resources.

Step 3: Identifying the Custom Resource - GVR

This is a critical step. The dynamic client requires the GVR (GroupVersionResource) of the custom resource you want to interact with. For our Application CRD:

  • Group: example.com
  • Version: v1
  • Resource: applications (the plural name)

So, our GVR will be:

// ... inside main after dynamic client is created ...

    applicationGVR := schema.GroupVersionResource{
        Group:    "example.com",
        Version:  "v1",
        Resource: "applications",
    }

    fmt.Printf("Targeting Custom Resource with GVR: %s\n", applicationGVR.String())

// ...

Explanation: * We import k8s.io/apimachinery/pkg/runtime/schema for the GroupVersionResource struct. * We explicitly define the GVR based on our CRD's spec.group, spec.versions[].name, and spec.names.plural.

Using RESTMapper for GVK to GVR Conversion (More Generic Approach)

In many real-world scenarios, you might only know the GVK (GroupVersionKind) from a user input or a resource definition, and you need to dynamically resolve the GVR. This is where the RESTMapper comes in handy. The RESTMapper is part of the discovery client and helps map between GVKs and GVRs, and vice versa, by querying the api server's discovery information.

// ... inside main ...
    // You need a discovery client for RESTMapper
    dc, err := discovery.NewDiscoveryClientForConfig(config)
    if err != nil {
        fmt.Printf("Error creating discovery client: %s\n", err.Error())
        os.Exit(1)
    }

    mapper := restmapper.NewDeferredDiscoveryRESTMapper(cacheddiscovery.NewMemCacheClient(dc))

    // Define the GVK of your custom resource
    applicationGVK := schema.GroupVersionKind{
        Group:   "example.com",
        Version: "v1",
        Kind:    "Application",
    }

    // Resolve the GVK to a GVR using the RESTMapper
    mapping, err := mapper.RESTMapping(applicationGVK, applicationGVK.Version)
    if err != nil {
        fmt.Printf("Error getting RESTMapping for GVK %s: %s\n", applicationGVK.String(), err.Error())
        os.Exit(1)
    }

    applicationGVR := mapping.Resource
    fmt.Printf("Resolved GVK %s to GVR %s using RESTMapper.\n", applicationGVK.String(), applicationGVR.String())

// ...

Explanation: * We create a discovery.NewDiscoveryClientForConfig to get the discovery client. * We then instantiate restmapper.NewDeferredDiscoveryRESTMapper, which internally uses a cached discovery client (cacheddiscovery.NewMemCacheClient) to efficiently query the api server's discovery information. * mapper.RESTMapping(gvk, version) attempts to find the corresponding GVR for the given GVK and version. The mapping.Resource field holds the GVR.

Using RESTMapper is more robust, especially if you're building a generic tool that might deal with different versions or custom resources whose plural names aren't immediately obvious. For a fixed, known CRD like our Application, directly defining the GVR is simpler, but understanding RESTMapper is crucial for truly flexible solutions.

Step 4: Reading a Specific Custom Resource (Get)

Now we can use the dynamic client to fetch our my-webapp Custom Resource.

// ... inside main after applicationGVR is defined ...

    ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
    defer cancel()

    namespace := "default" // Assuming our CR is in the 'default' namespace
    crName := "my-webapp"

    fmt.Printf("Attempting to get Custom Resource '%s/%s'...\n", namespace, crName)

    unstructuredObj, err := dynamicClient.Resource(applicationGVR).Namespace(namespace).Get(ctx, crName, metav1.GetOptions{})
    if err != nil {
        fmt.Printf("Error getting Custom Resource '%s/%s': %s\n", namespace, crName, err.Error())
        os.Exit(1)
    }

    fmt.Printf("Successfully retrieved Custom Resource: %s\n", unstructuredObj.GetName())

    // Accessing data from the Unstructured object
    fmt.Printf("API Version: %s\n", unstructuredObj.GetAPIVersion())
    fmt.Printf("Kind: %s\n", unstructuredObj.GetKind())
    fmt.Printf("Namespace: %s\n", unstructuredObj.GetNamespace())

    // Accessing spec fields
    spec, found, err := unstructured.NestedMap(unstructuredObj.Object, "spec")
    if err != nil {
        fmt.Printf("Error accessing spec field: %s\n", err.Error())
    } else if found {
        image, found, err := unstructured.NestedString(spec, "image")
        if err != nil {
            fmt.Printf("Error accessing spec.image: %s\n", err.Error())
        } else if found {
            fmt.Printf("Image: %s\n", image)
        }

        replicas, found, err := unstructured.NestedInt64(spec, "replicas")
        if err != nil {
            fmt.Printf("Error accessing spec.replicas: %s\n", err.Error())
        } else if found {
            fmt.Printf("Replicas: %d\n", replicas)
        }

        ports, found, err := unstructured.NestedSlice(spec, "ports")
        if err != nil {
            fmt.Printf("Error accessing spec.ports: %s\n", err.Error())
        } else if found {
            fmt.Println("Ports:")
            for i, p := range ports {
                if portMap, ok := p.(map[string]interface{}); ok {
                    name, _, _ := unstructured.NestedString(portMap, "name")
                    port, _, _ := unstructured.NestedInt64(portMap, "port")
                    protocol, _, _ := unstructured.NestedString(portMap, "protocol")
                    fmt.Printf("  - Port %d: Name=%s, Port=%d, Protocol=%s\n", i+1, name, port, protocol)
                }
            }
        }
    }

// ...

Explanation: * dynamicClient.Resource(applicationGVR) returns a ResourceInterface for the specified CRD. * .Namespace(namespace) scopes the operation to a particular namespace. If the CRD is cluster-scoped, you would omit this call. * .Get(ctx, crName, metav1.GetOptions{}) performs the GET request to the api server. * The result is an *unstructured.Unstructured object. * We use unstructuredObj.GetName(), GetAPIVersion(), GetKind(), GetNamespace() for common metadata. * To access fields within spec, status, or other nested fields, we use helper functions from k8s.io/apimachinery/pkg/apis/meta/v1/unstructured. These functions (e.g., NestedMap, NestedString, NestedInt64, NestedSlice) provide safe ways to extract data from the underlying map[string]interface{} structure, handling found booleans and errors gracefully. This is where the "dynamic" aspect comes into play, as you're navigating a generic data structure at runtime.

Step 5: Listing Custom Resources (List)

To retrieve all instances of our Application Custom Resource within a namespace or across the cluster, we use the List method.

// ... inside main after reading a specific CR ...

    fmt.Printf("\nAttempting to list all Custom Resources of kind 'Application' in namespace '%s'...\n", namespace)

    unstructuredList, err := dynamicClient.Resource(applicationGVR).Namespace(namespace).List(ctx, metav1.ListOptions{})
    if err != nil {
        fmt.Printf("Error listing Custom Resources: %s\n", err.Error())
        os.Exit(1)
    }

    fmt.Printf("Found %d Custom Resources:\n", len(unstructuredList.Items))
    for _, cr := range unstructuredList.Items {
        fmt.Printf("  - Name: %s, Namespace: %s, APIVersion: %s, Kind: %s\n",
            cr.GetName(), cr.GetNamespace(), cr.GetAPIVersion(), cr.GetKind())

        // Access some spec details from each listed CR
        if spec, found, _ := unstructured.NestedMap(cr.Object, "spec"); found {
            if image, found, _ := unstructured.NestedString(spec, "image"); found {
                fmt.Printf("    Image: %s\n", image)
            }
            if replicas, found, _ := unstructured.NestedInt64(spec, "replicas"); found {
                fmt.Printf("    Replicas: %d\n", replicas)
            }
        }
    }

// ...

Explanation: * .List(ctx, metav1.ListOptions{}) fetches a list of resources. metav1.ListOptions can be used to filter results (e.g., with labels selectors). * The result is an *unstructured.UnstructuredList, which contains a slice of unstructured.Unstructured objects in its Items field. * We iterate through the list and print details for each found Custom Resource, demonstrating how to access common fields and nested spec data.

Full Example Program

Here's the complete Go program combining all the steps:

package main

import (
    "context"
    "flag"
    "fmt"
    "os"
    "path/filepath"
    "time"

    metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
    "k8s.io/apimachinery/pkg/apis/meta/v1/unstructured"
    "k8s.io/apimachinery/pkg/runtime/schema"
    "k8s.io/client-go/discovery"
    "k8s.io/client-go/dynamic"
    "k8s.io/client-go/rest"
    "k8s.io/client-go/restmapper"
    "k8s.io/client-go/tools/clientcmd"
    "k8s.io/client-go/util/homedir"
    cacheddiscovery "k8s.io/client-go/discovery/cached/memory"
)

func main() {
    var kubeconfig *string
    if home := homedir.HomeDir(); home != "" {
        kubeconfig = flag.String("kubeconfig", filepath.Join(home, ".kube", "config"), "(optional) absolute path to the kubeconfig file")
    } else {
        kubeconfig = flag.String("kubeconfig", "", "absolute path to the kubeconfig file")
    }
    flag.Parse()

    // 1. Load Kubernetes Configuration
    config, err := clientcmd.BuildConfigFromFlags("", *kubeconfig)
    if err != nil {
        fmt.Printf("Error building kubeconfig: %s\n", err.Error())
        os.Exit(1)
    }
    fmt.Println("Kubernetes configuration loaded successfully.")

    // 2. Create the Dynamic Client
    dynamicClient, err := dynamic.NewForConfig(config)
    if err != nil {
        fmt.Printf("Error creating dynamic client: %s\n", err.Error())
        os.Exit(1)
    }
    fmt.Println("Dynamic client created successfully.")

    // 3. Identify the Custom Resource - GVK to GVR using RESTMapper
    // For a more generic approach, we'll use RESTMapper to resolve GVK to GVR
    dc, err := discovery.NewDiscoveryClientForConfig(config)
    if err != nil {
        fmt.Printf("Error creating discovery client: %s\n", err.Error())
        os.Exit(1)
    }
    mapper := restmapper.NewDeferredDiscoveryRESTMapper(cacheddiscovery.NewMemCacheClient(dc))

    applicationGVK := schema.GroupVersionKind{
        Group:   "example.com",
        Version: "v1",
        Kind:    "Application",
    }

    mapping, err := mapper.RESTMapping(applicationGVK, applicationGVK.Version)
    if err != nil {
        fmt.Printf("Error getting RESTMapping for GVK %s: %s\n", applicationGVK.String(), err.Error())
        os.Exit(1)
    }

    applicationGVR := mapping.Resource
    fmt.Printf("Resolved GVK %s to GVR %s using RESTMapper.\n", applicationGVK.String(), applicationGVR.String())

    ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
    defer cancel()

    namespace := "default" // Assuming our CR is in the 'default' namespace
    crName := "my-webapp"

    // 4. Reading a Specific Custom Resource (Get)
    fmt.Printf("\n--- Reading Specific Custom Resource ---\n")
    fmt.Printf("Attempting to get Custom Resource '%s/%s'...\n", namespace, crName)

    unstructuredObj, err := dynamicClient.Resource(applicationGVR).Namespace(namespace).Get(ctx, crName, metav1.GetOptions{})
    if err != nil {
        fmt.Printf("Error getting Custom Resource '%s/%s': %s\n", namespace, crName, err.Error())
        os.Exit(1)
    }

    fmt.Printf("Successfully retrieved Custom Resource: %s\n", unstructuredObj.GetName())

    // Accessing data from the Unstructured object
    fmt.Printf("  API Version: %s\n", unstructuredObj.GetAPIVersion())
    fmt.Printf("  Kind: %s\n", unstructuredObj.GetKind())
    fmt.Printf("  Namespace: %s\n", unstructuredObj.GetNamespace())

    // Accessing spec fields
    spec, found, err := unstructured.NestedMap(unstructuredObj.Object, "spec")
    if err != nil {
        fmt.Printf("  Error accessing spec field: %s\n", err.Error())
    } else if found {
        image, found, err := unstructured.NestedString(spec, "image")
        if err != nil {
            fmt.Printf("  Error accessing spec.image: %s\n", err.Error())
        } else if found {
            fmt.Printf("  Image: %s\n", image)
        }

        replicas, found, err := unstructured.NestedInt64(spec, "replicas")
        if err != nil {
            fmt.Printf("  Error accessing spec.replicas: %s\n", err.Error())
        } else if found {
            fmt.Printf("  Replicas: %d\n", replicas)
        }

        ports, found, err := unstructured.NestedSlice(spec, "ports")
        if err != nil {
            fmt.Printf("  Error accessing spec.ports: %s\n", err.Error())
        } else if found {
            fmt.Println("  Ports:")
            for i, p := range ports {
                if portMap, ok := p.(map[string]interface{}); ok {
                    name, _, _ := unstructured.NestedString(portMap, "name")
                    port, _, _ := unstructured.NestedInt64(portMap, "port")
                    protocol, _, _ := unstructured.NestedString(portMap, "protocol")
                    fmt.Printf("    - Port %d: Name=%s, Port=%d, Protocol=%s\n", i+1, name, port, protocol)
                }
            }
        }
    }

    // 5. Listing Custom Resources (List)
    fmt.Printf("\n--- Listing All Custom Resources ---\n")
    fmt.Printf("Attempting to list all Custom Resources of kind 'Application' in namespace '%s'...\n", namespace)

    unstructuredList, err := dynamicClient.Resource(applicationGVR).Namespace(namespace).List(ctx, metav1.ListOptions{})
    if err != nil {
        fmt.Printf("Error listing Custom Resources: %s\n", err.Error())
        os.Exit(1)
    }

    fmt.Printf("Found %d Custom Resources:\n", len(unstructuredList.Items))
    if len(unstructuredList.Items) > 0 {
        for i, cr := range unstructuredList.Items {
            fmt.Printf("  CR #%d: Name: %s, Namespace: %s, APIVersion: %s, Kind: %s\n",
                i+1, cr.GetName(), cr.GetNamespace(), cr.GetAPIVersion(), cr.GetKind())

            // Access some spec details from each listed CR for demonstration
            if spec, found, _ := unstructured.NestedMap(cr.Object, "spec"); found {
                if image, found, _ := unstructured.NestedString(spec, "image"); found {
                    fmt.Printf("    Image: %s\n", image)
                }
                if replicas, found, _ := unstructured.NestedInt64(spec, "replicas"); found {
                    fmt.Printf("    Replicas: %d\n", replicas)
                }
            }
        }
    } else {
        fmt.Println("  No Application Custom Resources found.")
    }
}

To run this program:

  1. Save the code as main.go.
  2. Run go mod tidy to ensure all dependencies are resolved.
  3. Execute: go run main.go

You should see output similar to this, detailing the configuration, dynamic client creation, and the successful retrieval and listing of your my-webapp Custom Resource.

Kubernetes configuration loaded successfully.
Dynamic client created successfully.
Resolved GVK {example.com v1 Application} to GVR {example.com v1 applications} using RESTMapper.

--- Reading Specific Custom Resource ---
Attempting to get Custom Resource 'default/my-webapp'...
Successfully retrieved Custom Resource: my-webapp
  API Version: example.com/v1
  Kind: Application
  Namespace: default
  Image: nginx:latest
  Replicas: 3
  Ports:
    - Port 1: Name=http, Port=80, Protocol=TCP

--- Listing All Custom Resources ---
Attempting to list all Custom Resources of kind 'Application' in namespace 'default'...
Found 1 Custom Resources:
  CR #1: Name: my-webapp, Namespace: default, APIVersion: example.com/v1, Kind: Application
    Image: nginx:latest
    Replicas: 3

This output confirms that our dynamic client is successfully interacting with the custom resources, demonstrating its capability to read and interpret custom objects without explicit Go type definitions.

Advanced Considerations and Best Practices

While reading custom resources with the dynamic client is straightforward, several advanced considerations and best practices can enhance the robustness, performance, and maintainability of your applications.

Error Handling Strategies

Robust error handling is paramount in production-grade Kubernetes applications. When interacting with the api server, various errors can occur: network issues, api server unavailability, authorization failures, resource not found, or invalid requests.

  • Check err at every step: Never assume an api call will succeed. Always check the returned error.
  • Context for Timeouts/Cancellations: Use context.Context (as shown in the examples) to manage timeouts and cancellations for api requests, preventing your application from hanging indefinitely.
  • Distinguish Error Types: client-go returns Kubernetes api errors that can be inspected. For example, apierrors.IsNotFound(err) can check if a resource simply doesn't exist, allowing for different handling than a permission error.
  • Retry Logic: For transient errors (e.g., network glitches, api server rate limiting), consider implementing exponential backoff and retry logic. The k8s.io/client-go/util/retry package provides helper functions for this.

Type Safety vs. Flexibility: When to Choose

The dynamic client offers unparalleled flexibility, but it comes at the cost of compile-time type safety. Here's a table to help you decide:

Feature Typed Client Dynamic Client
Type Safety High (Go structs, compile-time checks) Low (Generic map[string]interface{}, runtime assertions)
Flexibility Low (Requires generated types for each resource) High (Works with any resource, known or unknown at compile time)
Development Faster for known types (IDE, auto-completion) Slower for specific field access (manual key checks, type assertions)
Maintenance Generated types can become outdated with CRD changes More resilient to minor CRD schema changes, but logic needs to be robust
Use Cases Custom Operators for specific CRDs, well-defined applications Generic tools, API gateway components, generic controllers, introspection tools, open platform utilities
Performance Slightly better (direct struct access) Slightly worse (reflection, map lookups)
Boilerplate Requires code generation for CRDs No code generation, but more manual data extraction

Recommendation: * Use typed clients whenever you are developing a specific controller or operator for a known, stable CRD where the Go types can be generated. * Use the dynamic client when you need to build generic tools, an api gateway, or open platform solutions that must interact with a wide array of resources, including those whose schemas are not known or stable at compile time.

Performance Implications

While typically not a major bottleneck for most applications, it's worth noting that the dynamic client, by its nature, involves more runtime overhead compared to typed clients.

  • Reflection/Map Lookups: Accessing fields in Unstructured objects involves map lookups and type assertions at runtime, which are inherently slower than direct struct field access.
  • JSON Serialization/Deserialization: api responses are typically JSON. The dynamic client deserializes this into map[string]interface{}, which is flexible but might involve more memory allocations than direct deserialization into a predefined struct.

For most operations, these performance differences are negligible. However, if you are building an extremely high-throughput system that performs millions of api calls per second and processes vast amounts of custom resource data, these factors might warrant consideration. In such extreme cases, careful benchmarking and profiling are advised.

Watch and Informers with Dynamic Client

The dynamic client can also be used with Kubernetes informers to efficiently watch for changes to custom resources. Instead of repeatedly polling the api server (which is inefficient and generates unnecessary load), informers establish a watch and maintain a local cache of resources.

The client-go/dynamic/informer package provides NewFilteredDynamicSharedInformerFactory. You can create an informer for a specific GVR, allowing your application to react to Add, Update, and Delete events for custom resources. This is how most production-grade Kubernetes operators are built, as it provides a robust, scalable, and api-friendly way to manage resource state.

While a full implementation of a dynamic informer is beyond the scope of this article (which focuses on reading), understanding its existence is crucial for building reactive systems that manage custom resources.

Managing Custom Resources at Scale: The Broader API Ecosystem

Custom Resources often represent complex application configurations or services that are integral to an organization's internal api landscape. As the number of CRDs and custom resources grows, the challenge shifts from simply reading them to managing their entire lifecycle, exposure, and consumption as part of a coherent api strategy.

Consider a scenario where your Custom Resources define the endpoints, authentication, and backend services for various internal microservices. These services, while managed via Kubernetes CRDs, ultimately need to be consumed by other applications or developers. Managing a multitude of such custom api definitions, along with their access control, traffic management, rate limiting, and analytics, can quickly become an overwhelming task. This is especially true for an open platform that encourages diverse api development.

This is where a robust API gateway and API management platform becomes indispensable. Such platforms complement Kubernetes' extensibility by providing the crucial layer of governance and discoverability for all your APIs, regardless of whether they originate from standard services or complex Custom Resources.

For organizations looking to streamline this process, especially in an environment rich with AI services or complex microservices defined by CRDs, solutions like APIPark offer a comprehensive approach. APIPark is an open-source AI gateway and API management platform designed to help developers and enterprises manage, integrate, and deploy AI and REST services with ease. It can standardize api formats, encapsulate prompts into REST APIs, and provide end-to-end API lifecycle management, even for services whose configurations might originate from Custom Resources within Kubernetes. Its ability to handle unified API formats, provide detailed call logging, and manage access permissions aligns perfectly with the need to professionalize the exposure and consumption of services, whether they are native Kubernetes resources or custom extensions. By centralizing api governance, platforms like APIPark ensure that custom resources, once defined and deployed, can be seamlessly integrated into a broader, managed api ecosystem, enhancing security, performance, and developer experience.

Conclusion: Empowering Flexible Kubernetes Development

The dynamic client in client-go is an incredibly powerful and flexible tool for interacting with the Kubernetes api, especially when dealing with Custom Resources whose Go types are not known at compile time. It liberates developers from the constraints of generated code, enabling the creation of generic controllers, versatile command-line tools, and adaptable API gateway solutions that can effortlessly adapt to an evolving Kubernetes landscape.

By understanding how to load configurations, resolve GVKs to GVRs using the RESTMapper, create the dynamic client, and leverage Unstructured objects for data extraction, you gain a deep capability to programmatically interact with any resource in your cluster. This level of extensibility is a hallmark of Kubernetes as an open platform, allowing it to be molded to virtually any operational need.

As the complexity of cloud-native applications continues to grow, and as organizations increasingly rely on domain-specific operators and Custom Resources, the dynamic client will remain a vital component in the toolkit of any Go developer working within the Kubernetes ecosystem. Whether you're building a sophisticated API management platform or a simple diagnostic utility, mastering the dynamic client empowers you to build more resilient, adaptable, and future-proof Kubernetes applications. Remember to balance its flexibility with robust error handling and thoughtful design, especially when extracting data from Unstructured objects, to create solutions that are both powerful and maintainable. This mastery is a significant step towards unlocking the full potential of Kubernetes as a truly extensible and programmable infrastructure.

Frequently Asked Questions (FAQs)

1. What is the primary advantage of using the dynamic client over typed clients in client-go?

The primary advantage of the dynamic client is its flexibility. It allows you to interact with any Kubernetes resource, including Custom Resources, without requiring their specific Go type definitions at compile time. This is invaluable for building generic tools, controllers, or an API gateway that needs to adapt to unknown or evolving CRDs, avoiding the need for code generation and recompilation every time a new custom resource type is introduced. Typed clients, while offering strong compile-time type safety, are limited to predefined Go structs.

2. When should I choose the dynamic client instead of a typed client for my Kubernetes Go application?

You should choose the dynamic client in scenarios where: * You need to interact with Custom Resources whose Go types are not known or generated at compile time. * You are building a generic tool or an open platform component that needs to work with arbitrary Kubernetes resources without specific schema knowledge. * You are developing a component (like a generic API gateway) that must be highly adaptable to new api extensions or evolving resource definitions. * The CRD definition is unstable or frequently changes, making continuous code generation for typed clients cumbersome.

For stable, well-defined CRDs where you control the schema and have generated Go types, typed clients are generally preferred for their type safety and development efficiency.

3. What is the difference between GroupVersionKind (GVK) and GroupVersionResource (GVR), and why is it important for the dynamic client?

  • GroupVersionKind (GVK) identifies a specific type of Kubernetes object (e.g., Deployment, Application). It consists of an API Group, Version, and Kind.
  • GroupVersionResource (GVR) identifies a specific REST endpoint on the Kubernetes api server for a collection of resources (e.g., /apis/apps/v1/deployments, /apis/example.com/v1/applications). It consists of an API Group, Version, and the plural Resource name.

The dynamic client interacts directly with the Kubernetes api server's REST endpoints, which are addressed by GVRs. Therefore, to use the dynamic client, you must provide the GVR of the resource you want to query. If you only have the GVK, you'll need to use a RESTMapper (as shown in the article) to resolve the GVK to its corresponding GVR.

4. How do I access specific fields within an Unstructured object retrieved by the dynamic client?

An Unstructured object is essentially a map[string]interface{}. To access its fields, you use helper functions provided by k8s.io/apimachinery/pkg/apis/meta/v1/unstructured, such as NestedString, NestedInt64, NestedBool, NestedMap, and NestedSlice. These functions provide a safe way to traverse the nested map structure, checking for existence (found boolean) and handling potential type assertion errors. For example, unstructured.NestedString(unstructuredObj.Object, "spec", "image") would retrieve the image field from the spec.

5. Can the dynamic client be used with Kubernetes informers for watching custom resources?

Yes, absolutely. The dynamic client is fully compatible with Kubernetes informers. The client-go/dynamic/informer package provides NewFilteredDynamicSharedInformerFactory which allows you to create informers for specific GVRs. These dynamic informers enable your application to watch for Add, Update, and Delete events on custom resources, maintaining a local cache and significantly improving performance and responsiveness compared to polling. This is the foundation for building robust, event-driven controllers and operators that manage custom resources.

🚀You can securely and efficiently call the OpenAI API on APIPark in just two steps:

Step 1: Deploy the APIPark AI gateway in 5 minutes.

APIPark is developed based on Golang, offering strong product performance and low development and maintenance costs. You can deploy APIPark with a single command line.

curl -sSO https://download.apipark.com/install/quick-start.sh; bash quick-start.sh
APIPark Command Installation Process

In my experience, you can see the successful deployment interface within 5 to 10 minutes. Then, you can log in to APIPark using your account.

APIPark System Interface 01

Step 2: Call the OpenAI API.

APIPark System Interface 02
Install APIPark – it’s free
Previous

How Much Is HQ Cloud Services? Ultimate Pricing Guide.

 Next

Mosaic AI Gateway: Simplify AI Integration & Boost Performance