Skip to content

feat: new multi kube support

Marco Wettstein requested to merge f/new-multi-kube into main

@gug @stb

i implemented namespace based deployment role, but could not test it fully yet, as my own account lacks certain roles.

But first of all here is how i implemented it (that's just based on my understanding how it should work). This happens on "setup-time", that means this runs under a certain user to connect gitlab with clusters using catladder/cli. It will run for each namespace:

  1. get the $KUBE_URL (can be done using kubectl cluster-info)
  2. create the namespace
  3. create a service-account "cl-deploy" in the namespace
  4. create a role "cl-deploy", which has (hopefully) the right amount of rights to CRUD resources in the namespace <-- this currently fails on skynet-swiss with this error: roles.rbac.authorization.k8s.io is forbidden: User "maw@panter.ch" cannot create resource "roles" in API group "rbac.authorization.k8s.io" in the namespace "pan-ci-monorepotest-dev": requires one of ["container.roles.create"] permission(s).
  5. create a role binding "cl-deploy" to connect the service account with the role
  6. get the service accounts token (KUBE_TOKEN) and ca.cert (KUBE_CA_PEM) (this is created automatically as it seems)
  7. push these three vars as secret to gitlab (KUBE_CA_PEM, KUBE_TOKEN, KUBE_URL)
  8. deploy will then use these vars to setup a kube context <-- this already works, i tested it with the default token in the old cluster, which seems to have cluster-admin rights)/

apart from small details and the credentials-problem with my account i think i am on the right track. This command can be invoked by devs with admin rights on the cluster (or at least the rights to create namespaces and roles) and limits the scope of what gitlab can do to the namespace.

So where i need your help is giving me the right roles ("container.roles.create") and check whether this makes sense. My account further has no right to list iam etc, so i can't elevate myself i guess.

Edited by Marco Wettstein

Merge request reports