📦 Create Your First Custom Resource Definition 1
kubectl apply -f https://raw.githubusercontent.com/kubernetes/website/main/content/en/examples/customresourcedefinition/shirt-resource-definition.yaml
By default, the Kubernetes Controller Manager caches every resource to support background operations such as garbage collection and internal indexing.
To reduce memory usage and improve performance in high-volume environments, you can label specific resources with skip-controller-manager-metadata-caching=true to exclude them from being cached. That means Kubernetes can’t clean-up resources by ownerReference. Select a database which automatic GC is supported, or you have to do garbage collection on your own.
💡 Ensure Label with Mutation Admission Webhook MutatingAdmissionPolicies allow you to modify (or “mutate”) incoming requests to the Kubernetes API.
However, if you only need a declarative policy to ensure a label on each resources, the MutatingAdmissionPolicy is a simpler and more effective choice. We’ve provided a simple example below, but for complete and detailed information, please refer to the following the link: Mutating Admission Policy .
⚠️ To use the feature, enable the MutatingAdmissionPolicy feature gate (which is off by default) and set --runtime-config=admissionregistration.k8s.io/v1beta1=true on the kube-apiserver.
Ensure skip-controller-manager-metadata-caching Label on resources without owners
This version adds label to resources without OwnerReferences. In this way you should have the best of both worlds, not caching standalone resources, but managing resources have owner references.
skip-controller-manager-metadata-caching.yaml
copy
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
apiVersion : admissionregistration.k8s.io/v1beta1
kind : MutatingAdmissionPolicy
metadata :
name : "skip-controller-manager-metadata-caching"
spec :
matchConstraints :
resourceRules :
- apiGroups : [ "*" ]
apiVersions : [ "*" ]
operations : [ "CREATE" ]
resources : [ "*" ]
matchConditions :
- name : owner-references-empty
expression : >
!has(object.metadata.ownerReferences) ||
size(object.metadata.ownerReferences) == 0
failurePolicy : Fail
reinvocationPolicy : IfNeeded
mutations :
- patchType : JSONPatch
jsonPatch :
expression : >
has(object.metadata.labels)
? [
JSONPatch{
op: "add",
path: "/metadata/labels/skip-controller-manager-metadata-caching",
value: "true"
}
]
: [
JSONPatch{
op: "add",
path: "/metadata/labels",
value: {}
},
JSONPatch{
op: "add",
path: "/metadata/labels/skip-controller-manager-metadata-caching",
value: "true"
}
]
---
apiVersion : admissionregistration.k8s.io/v1beta1
kind : MutatingAdmissionPolicyBinding
metadata :
name : "skip-controller-manager-metadata-caching"
spec :
policyName : "skip-controller-manager-metadata-caching"
matchResources :
resourceRules :
- apiGroups : [ "*" ]
apiVersions : [ "*" ]
operations : [ "CREATE" ]
resources : [ "*" ]
Ensure `skip-controller-manager-metadata-caching` Label on Shirts
skip-controller-manager-metadata-caching.yaml
copy
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
apiVersion : admissionregistration.k8s.io/v1beta1
kind : MutatingAdmissionPolicy
metadata :
name : "skip-controller-manager-metadata-caching"
spec :
matchConstraints :
resourceRules :
- apiGroups : [ "stable.example.com" ]
apiVersions : [ "v1" ]
operations : [ "CREATE" ]
resources : [ "shirts" ]
matchConditions :
- name : label-does-not-exist
expression : >
!has(object.metadata.labels) ||
!('skip-controller-manager-metadata-caching' in object.metadata.labels)
failurePolicy : Fail
reinvocationPolicy : IfNeeded
mutations :
- patchType : JSONPatch
jsonPatch :
expression : >
has(object.metadata.labels)
? [
JSONPatch{
op: "add",
path: "/metadata/labels/skip-controller-manager-metadata-caching",
value: "true"
}
]
: [
JSONPatch{
op: "add",
path: "/metadata/labels",
value: {}
},
JSONPatch{
op: "add",
path: "/metadata/labels/skip-controller-manager-metadata-caching",
value: "true"
}
]
---
apiVersion : admissionregistration.k8s.io/v1beta1
kind : MutatingAdmissionPolicyBinding
metadata :
name : "skip-controller-manager-metadata-caching"
spec :
policyName : "skip-controller-manager-metadata-caching"
matchResources :
resourceRules :
- apiGroups : [ "stable.example.com" ]
apiVersions : [ "v1" ]
operations : [ "CREATE" ]
resources : [ "shirts" ]
Ensure `skip-controller-manager-metadata-caching` Label on Everything - vCluster
‼️ This MutatingAdmissionPolicy disables all Controller Manager features to the resources. Be sure storage-side garbage collection is supported by all the selected databases, and the feature is enabled on the middleware.
skip-controller-manager-metadata-caching.yaml
copy
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
apiVersion : admissionregistration.k8s.io/v1beta1
kind : MutatingAdmissionPolicy
metadata :
name : "skip-controller-manager-metadata-caching"
spec :
matchConstraints :
resourceRules :
- apiGroups : [ "*" ]
apiVersions : [ "*" ]
operations : [ "CREATE" ]
resources : [ "*" ]
matchConditions :
- name : label-does-not-exist
expression : >
!has(object.metadata.labels) ||
!('skip-controller-manager-metadata-caching' in object.metadata.labels)
failurePolicy : Fail
reinvocationPolicy : IfNeeded
mutations :
- patchType : JSONPatch
jsonPatch :
expression : >
has(object.metadata.labels)
? [
JSONPatch{
op: "add",
path: "/metadata/labels/skip-controller-manager-metadata-caching",
value: "true"
}
]
: [
JSONPatch{
op: "add",
path: "/metadata/labels",
value: {}
},
JSONPatch{
op: "add",
path: "/metadata/labels/skip-controller-manager-metadata-caching",
value: "true"
}
]
---
apiVersion : admissionregistration.k8s.io/v1beta1
kind : MutatingAdmissionPolicyBinding
metadata :
name : "skip-controller-manager-metadata-caching"
spec :
policyName : "skip-controller-manager-metadata-caching"
matchResources :
resourceRules :
- apiGroups : [ "*" ]
apiVersions : [ "*" ]
operations : [ "CREATE" ]
resources : [ "*" ]
👕 Create Your First Custom Resources 1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
cat | kubectl apply -f - <<EOF
---
apiVersion: stable.example.com/v1
kind: Shirt
metadata:
name: example1
spec:
color: blue
size: S
---
apiVersion: stable.example.com/v1
kind: Shirt
metadata:
name: example2
spec:
color: blue
size: M
---
apiVersion: stable.example.com/v1
kind: Shirt
metadata:
name: example3
spec:
color: green
size: M
EOF
Verify the resources are exists.
NAME COLOR SIZE
example1 blue S
example2 blue M
example3 green M