There are multiple ways to update topology config on the go. You can store configurations in a file and update it’s content, or you can use a webserver as a source of the configurations. An other option is using Kubernetes secret as a source of truth. This documentation is focusing on using HariKube operator to merge multiple configuration sources into the final tree of topology configs.
🔧 Configure Kubernetes with HariKube
Start by installing HariKube and configuring its middleware in your Kubernetes cluster. This will enable the platform to route database requests to different databases, based on configuration.
For installation instruction, please follow the documentation.
For TOPOLOGY_CONFIG environment variable, please use secret://<namespace>/<name> format, and ensure the secret is empty during the initial cluster start.
🛢️ Setup Database Operators
HariKube works alongside existing database operators (e.g., MySQL, PostgreSQL) to provision and manage actual database instances. Install your favorite operators in your cluster for example:
Once installation has finished, create your database instances themselves. Please follow the operator’s manual, how to do it properly.
🔌 Deploy HariKube Operator
Next step is configuring and deploying HariKube Operator. The operator is a standard Kubernetes operator, so if you are familiar with operators, you are good to go.
apiVersion:apiextensions.k8s.io/v1kind:CustomResourceDefinitionmetadata:annotations:controller-gen.kubebuilder.io/version:v0.18.0name:topologyconfigs.harikube.infospec:group:harikube.infonames:kind:TopologyConfiglistKind:TopologyConfigListplural:topologyconfigssingular:topologyconfigscope:Namespacedversions:- additionalPrinterColumns:- jsonPath:.status.deployStatusname:DeployStatustype:stringname:v1schema:openAPIV3Schema:description:TopologyConfig is the Schema for the topologyconfigs API.properties:apiVersion:description:|- APIVersion defines the versioned schema of this representation of an object.
Servers should convert recognized schemas to the latest internal value, and
may reject unrecognized values.
More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resourcestype:stringkind:description:|- Kind is a string value representing the REST resource this object represents.
Servers may infer this from the endpoint the client submits requests to.
Cannot be updated.
In CamelCase.
More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kindstype:stringmetadata:type:objectspec:description:TopologyConfigSpec defines the desired state of TopologyConfig.properties:backends:description:Backends represents the group of topology configs.items:properties:backendTLS:description:BackendTLS defines TLS configuration for connection.properties:caFile:description:CAFile is the CA file for the secure connection.type:stringcertFile:description:CertFile is the certificate file for the secureconnection.type:stringkeyFile:description:KeyFile is the key file for the secure connection.type:stringskipVerify:description:SkipVerify skips TLS verfication.type:booleantype:objectconnectionPool:description:ConnectionPool defines connection pool configurationfor connection.properties:maxIdle:minimum:0type:integermaxIdleLifeTime:allOf:- format:int64- format:durationdescription:|- A Duration represents the elapsed time between two instants
as an int64 nanosecond count. The representation limits the
largest representable duration to approximately 290 years.type:integermaxLifeTime:allOf:- format:int64- format:durationdescription:|- A Duration represents the elapsed time between two instants
as an int64 nanosecond count. The representation limits the
largest representable duration to approximately 290 years.type:integermaxOpen:minimum:0type:integertype:objectcustomresource:description:CustomResource defines custom resource based routing.properties:group:description:Group is the API group to match.type:stringkind:description:Kind is the API kind to match.type:stringtype:objectendpoint:description:Endpoint is database specific connection string.type:stringname:description:Name is the global unique name of the config.type:stringnamespace:description:Namespace defines namespace routing.properties:kinds:description:Kinds is list of kind to match.items:type:stringtype:arraynamespace:description:Namespace is the name of namespace to match.type:stringtype:objectprefix:description:Prefix defines prefix based routing.properties:prefix:description:Prefix is the ETCD key prefix to match.type:stringtype:objectregexp:description:RegExp defines regexp based routing.properties:key:description:Key is the regexp to match if individual operationneeds different match.type:stringprefix:description:Prefix is the regexp to match.type:stringtype:objectrequired:- endpoint- nametype:objectminItems:1type:arrayendpointDecryptSecret:description:|- EndpointDecryptSecret specifies the secret containing the decryption key for sensitive BackendConfig data.
AES-GCM encryption is supported, and multiple keys can be defined for key rotation or compatibility.
The encrypted endpoint has to be encoded with Base64.type:stringposition:description:Position represents the position fo the config in themerged config.minimum:1type:integertargetSecret:default:harikube/topology-configdescription:TargetSecret defines the namespaced name of the generatedtopology config.type:stringrequired:- backends- position- targetSecrettype:objectstatus:description:TopologyConfigStatus defines the observed state of TopologyConfig.properties:deployStatus:default:Pendingdescription:DeployStatus defines the in cluster state of config.type:stringdeployStatusReason:description:DeployStatus stores the the reason of DeployStatus.type:stringtype:objecttype:objectserved:truestorage:truesubresources:status:{}---apiVersion:v1kind:ServiceAccountmetadata:labels:app.kubernetes.io/managed-by:kustomizeapp.kubernetes.io/name:operatorname:controller-managernamespace:system---apiVersion:rbac.authorization.k8s.io/v1kind:Rolemetadata:labels:app.kubernetes.io/managed-by:kustomizeapp.kubernetes.io/name:operatorname:leader-election-rolerules:- apiGroups:- ""resources:- configmapsverbs:- get- list- watch- create- update- patch- delete- apiGroups:- coordination.k8s.ioresources:- leasesverbs:- get- list- watch- create- update- patch- delete- apiGroups:- ""resources:- eventsverbs:- create- patch---apiVersion:rbac.authorization.k8s.io/v1kind:ClusterRolemetadata:name:manager-rolerules:- apiGroups:- harikube.inforesources:- topologyconfigsverbs:- create- delete- get- list- patch- update- watch- apiGroups:- harikube.inforesources:- topologyconfigs/finalizersverbs:- update- apiGroups:- harikube.inforesources:- topologyconfigs/statusverbs:- get- patch- update- apiGroups:- v1resources:- secretsverbs:- create- delete- get- list- patch- update- watch- apiGroups:- v1resources:- secrets/finalizersverbs:- update---apiVersion:rbac.authorization.k8s.io/v1kind:ClusterRolemetadata:name:metrics-auth-rolerules:- apiGroups:- authentication.k8s.ioresources:- tokenreviewsverbs:- create- apiGroups:- authorization.k8s.ioresources:- subjectaccessreviewsverbs:- create---apiVersion:rbac.authorization.k8s.io/v1kind:ClusterRolemetadata:name:metrics-readerrules:- nonResourceURLs:- /metricsverbs:- get---apiVersion:rbac.authorization.k8s.io/v1kind:ClusterRolemetadata:labels:app.kubernetes.io/managed-by:kustomizeapp.kubernetes.io/name:operatorname:topologyconfig-admin-rolerules:- apiGroups:- harikube.inforesources:- topologyconfigsverbs:- '*'- apiGroups:- harikube.inforesources:- topologyconfigs/statusverbs:- get---apiVersion:rbac.authorization.k8s.io/v1kind:ClusterRolemetadata:labels:app.kubernetes.io/managed-by:kustomizeapp.kubernetes.io/name:operatorname:topologyconfig-editor-rolerules:- apiGroups:- harikube.inforesources:- topologyconfigsverbs:- create- delete- get- list- patch- update- watch- apiGroups:- harikube.inforesources:- topologyconfigs/statusverbs:- get---apiVersion:rbac.authorization.k8s.io/v1kind:ClusterRolemetadata:labels:app.kubernetes.io/managed-by:kustomizeapp.kubernetes.io/name:operatorname:topologyconfig-viewer-rolerules:- apiGroups:- harikube.inforesources:- topologyconfigsverbs:- get- list- watch- apiGroups:- harikube.inforesources:- topologyconfigs/statusverbs:- get---apiVersion:rbac.authorization.k8s.io/v1kind:RoleBindingmetadata:labels:app.kubernetes.io/managed-by:kustomizeapp.kubernetes.io/name:operatorname:leader-election-rolebindingroleRef:apiGroup:rbac.authorization.k8s.iokind:Rolename:leader-election-rolesubjects:- kind:ServiceAccountname:controller-managernamespace:system---apiVersion:rbac.authorization.k8s.io/v1kind:ClusterRoleBindingmetadata:labels:app.kubernetes.io/managed-by:kustomizeapp.kubernetes.io/name:operatorname:manager-rolebindingroleRef:apiGroup:rbac.authorization.k8s.iokind:ClusterRolename:manager-rolesubjects:- kind:ServiceAccountname:controller-managernamespace:system---apiVersion:rbac.authorization.k8s.io/v1kind:ClusterRoleBindingmetadata:name:metrics-auth-rolebindingroleRef:apiGroup:rbac.authorization.k8s.iokind:ClusterRolename:metrics-auth-rolesubjects:- kind:ServiceAccountname:controller-managernamespace:system---apiVersion:v1kind:Namespacemetadata:labels:app.kubernetes.io/managed-by:kustomizeapp.kubernetes.io/name:operatorcontrol-plane:controller-managername:system---apiVersion:apps/v1kind:Deploymentmetadata:labels:app.kubernetes.io/managed-by:kustomizeapp.kubernetes.io/name:operatorcontrol-plane:controller-managername:controller-managernamespace:systemspec:replicas:1selector:matchLabels:app.kubernetes.io/name:operatorcontrol-plane:controller-managertemplate:metadata:annotations:kubectl.kubernetes.io/default-container:managerlabels:app.kubernetes.io/name:operatorcontrol-plane:controller-managerspec:containers:- args:- --leader-elect- --health-probe-bind-address=:8081command:- /managerimage:registry.harikube.info/harikube/operator:alpha-v1.0.0livenessProbe:httpGet:path:/healthzport:8081initialDelaySeconds:15periodSeconds:20name:managerports:[]readinessProbe:httpGet:path:/readyzport:8081initialDelaySeconds:5periodSeconds:10resources:limits:cpu:500mmemory:128Mirequests:cpu:10mmemory:64MisecurityContext:allowPrivilegeEscalation:falsecapabilities:drop:- ALLvolumeMounts:[]securityContext:runAsNonRoot:trueseccompProfile:type:RuntimeDefaultserviceAccountName:controller-managerterminationGracePeriodSeconds:10volumes:[]
🗂️ Create your first routing configuration
The operator provides a custom resource type for routing configurations called topologyconfigs.harikube.info. You can bundle TopologyConfig custom resource with your microservice manifests, the operator would fetch all the configs and merge them into a final config for the middleware. Each microservice can declare how its data should be partitioned, what database backend to use, and whether to reuse or isolate instances.
For simplicity this example contains plaintext database endpoint configuration. The operator supports AES-GCM encryption for endpoints. Store your decryption key in a secret, reference it’s namespaced name in spec.endpointDecryptSecret, and use Base64 encoded endpoint secret in the configuration.
!!! Custom resource reconciliation, topology config validation and secret update takes some time. Please make sure the status.deployStatus of the custom resource is Active before you create any resource affected by the policy. Error reason is stored at status.deployStatusReason. Creating resources before the policy has been applied will lead to “data loss” (more specifically they would stuck in the main database, and the middleware never fetch them again).
By automating these steps, HariKube transforms Kubernetes into a true Platform-as-a-Service solution, because HariKube provides full control of data isolation and distributes database requests based on the customer’s needs.