Add override to allow custom CA location #4

Open
opened 3 months ago by liamsorsby · 7 comments

Hi @hjacobs,

We've recently been looking into using this kube-downscaler but run into issues as we run a large on prem cluster that uses it's own certificate authority. This means that we need to provide the CA path as an environment variable to allow us to successfuly comminicate with the Kubernetes API.

I've attempted to create a PR to allow the ability to use the environment variable and also add a test to ensure it does as we expect it to.

I'm not actually familiar with Python so I hope the PR isn't too terrible!

Thank you in advance!

Hi @hjacobs, We've recently been looking into using this kube-downscaler but run into issues as we run a large on prem cluster that uses it's own certificate authority. This means that we need to provide the CA path as an environment variable to allow us to successfuly comminicate with the Kubernetes API. I've attempted to create a PR to allow the ability to use the environment variable and also add a test to ensure it does as we expect it to. I'm not actually familiar with Python so I hope the PR isn't too terrible! Thank you in advance!
Owner

@liamsorsby can you elaborate on why the certificate-authority key in the Kubeconfig file does not work for you?

@liamsorsby can you elaborate on why the `certificate-authority` key in the Kubeconfig file does not work for you?
Poster

Hi @hjacobs the root CA that is in the Kubeconfig file is the root CA that was used for our kubernetes cluster. However, the API uses a different internal certificate authority so when it attempts to use the certificate-authority from the kubeconfig file it returns an exception. Our normal process is to mount the CA into the pod and then provide an environment variable with the path to the certificate-authority.

We've tried the above PR by bundling it into the kube-downscaler, building a new docker image and trying it on our kubernetes cluster which works!

Hi @hjacobs the root CA that is in the Kubeconfig file is the root CA that was used for our kubernetes cluster. However, the API uses a different internal certificate authority so when it attempts to use the certificate-authority from the kubeconfig file it returns an exception. Our normal process is to mount the CA into the pod and then provide an environment variable with the path to the certificate-authority. We've tried the above PR by bundling it into the kube-downscaler, building a new docker image and trying it on our kubernetes cluster which works!
Owner

@liamsorsby how do you interact with the Kubernetes API when you are not using pykbe-ng with this patch? How do you provide the CA when using a different client?

@liamsorsby how do you interact with the Kubernetes API when you are not using `pykbe-ng` with this patch? How do you provide the CA when using a different client?
Poster

@hjacobs We aren't using pykube-ng yet. We're wanting this PR to be accepted so that we can start using the kube-downscaler project that you started :) The only issue we're having is that the API request gets rejected. Below is the stack trace:

raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: https://kubernetes.default.svc.cluster.local/api/v1/pods
@hjacobs We aren't using pykube-ng yet. We're wanting this PR to be accepted so that we can start using the kube-downscaler project that you started :) The only issue we're having is that the API request gets rejected. Below is the stack trace: ``` raise HTTPError(http_error_msg, response=self) requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: https://kubernetes.default.svc.cluster.local/api/v1/pods ```
Owner

@liamsorsby are you using any other Kubernetes clients other than pykube-ng? How do these other clients connect to the Kubernetes API in your cluster?

@liamsorsby are you using any other Kubernetes clients _other_ than `pykube-ng`? How do these other clients connect to the Kubernetes API in your cluster?
Poster

@hjacobs I've not used any personally in our cluster. I can get the above to work okay locally just not inside the cluster without mounting the root ca.
I've just had a look at some of our custom operators which are written in go. These have an issuer helper passed in which takes an os.file

Unfortunately, I'm not at all involved in the management of the kubernetes cluster / infrastructure so I'm not sure why they are different.

@hjacobs I've not used any personally in our cluster. I can get the above to work okay locally just not inside the cluster without mounting the root ca. I've just had a look at some of our custom operators which are written in go. These have an issuer helper passed in which takes an os.file Unfortunately, I'm not at all involved in the management of the kubernetes cluster / infrastructure so I'm not sure why they are different.
Poster

@hjacobs I've just asked the question. It looks like the cert that's populated into the kube config is the intermediate cert but nodejs / python clients don't seem to like using the intermediate and only seem to work using the root CA.

@hjacobs I've just asked the question. It looks like the cert that's populated into the kube config is the intermediate cert but nodejs / python clients don't seem to like using the intermediate and only seem to work using the root CA.
Sign in to join this conversation.
No Label
No Milestone
No Assignees
2 Participants
Notifications
Due Date

No due date set.

Dependencies

This issue currently doesn't have any dependencies.

Loading…
There is no content yet.