Clean up (delete) Kubernetes resources after a configured TTL (time to live)
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
Henning Jacobs d99063e0b6 v19.9.0 2 weeks ago
deploy v19.9.0 2 weeks ago
kube_janitor Skip non-listable API groups (#42) 2 weeks ago
tests Skip non-listable API groups (#42) 2 weeks ago
.gitignore [Feature] Expire notification (#26) 6 months ago
.travis.yml initial commit 7 months ago
Dockerfile update deps 1 month ago
LICENSE initial commit 7 months ago
Makefile switching to CalVer 1 month ago
Pipfile use pykube-ng instead of pykube (#23) 6 months ago
Pipfile.lock update deps 2 weeks ago
README.rst grammar 1 month ago initial commit 7 months ago
tox.ini initial commit 7 months ago


Kubernetes Janitor

.. image::
:alt: Travis CI Build Status

.. image::;_=1
:alt: Code Coverage

.. image::
:alt: Docker pulls

.. image::
:alt: CalVer versioning scheme

Kubernetes Janitor cleans up (deletes) Kubernetes resources on (1) a configured TTL (time to live) or (2) a configured expiry date (absolute timestamp).

It processes all namespaces and all namespaced resources including custom resource definitions (CRDs) and will delete them (1) if the ``janitor/ttl`` annotation or a TTL rule indicates the resource as expired, or (2) if the ``janitor/expires`` annotation marks the resource as expired.

Example Use Cases

* Deploy the janitor to a test (non-prod) cluster and use namespaces with a TTL of 7 days (``janitor/ttl: 7d`` on the namespace object) for prototyping
* Annotate your temporary manual test nginx deployment with ``kubectl annotate deploy nginx janitor/ttl=24h`` to automatically delete it after 24 hours
* Automatically set ``janitor/ttl`` on resources created by your CI/CD pipeline for pull requests (so PR tests can run and resources are cleaned up later)
* Define a rule to automatically delete resources after 4 days if required labels were not set (see Rules File below)

Expiry date
* Deploy the janitor to a test (non-prod) cluster and use namespaces with an expiry date (``janitor/expires: 2020-01-17T15:14:38Z`` on the namespace object)
* Annotate your temporary manual test nginx deployment with ``kubectl annotate deploy nginx janitor/expires=2020-01-01`` to automatically delete it at midnight (UTC) of 1st of January 2020.


Deploy the janitor into your cluster via (also works with Minikube_):

.. code-block:: bash

$ kubectl apply -f deploy/common/
$ kubectl apply -f deploy/deployment/

The example configuration uses the ``--dry-run`` as a safety flag to prevent any deletion --- remove it to enable the janitor, e.g. by editing the deployment:

.. code-block:: bash

$ kubectl edit deploy kube-janitor

To see the janitor in action, deploy a simple nginx and annotate it accordingly:

.. code-block:: bash

$ kubectl run temp-nginx --image=nginx
$ kubectl annotate deploy temp-nginx janitor/ttl=5m

You should see the ``temp-nginx`` deployment being deleted after 5 minutes.

Edit the example rules file via ``kubectl edit configmap kube-janitor`` to try out generic TTL rules (needs a pod restart to reload rules).


The janitor is configured via command line args, environment variables, Kubernetes annotations, and an optional YAML rules file.

Supported Kubernetes annotations:

Maximum time to live (TTL) for the annotated resource. Annotation value must be a string composed of a integer value and a unit suffix (one of ``s``, ``m``, ``h``, ``d``, or ``w``), e.g. ``120s`` (120 seconds), ``5m`` (5 minutes), ``8h`` (8 hours), ``7d`` (7 days), or ``2w`` (2 weeks).
In the case that the resource should not be deleted by Janitor, the special value ``forever`` can be specified as TTL.
Note that the actual time of deletion depends on the Janitor's clean up interval. The resource will be deleted if its age (delta between now and the resource creation time) is greater than the specified TTL.
Absolute timestamp in the format ``YYYY-MM-DDTHH:MM:SSZ``, ``YYYY-MM-DDTHH:MM`` or ``YYYY-MM-DD`` to mark the resource for deletion after the specified date/time.
The date format ``YYYY-MM-DD`` is short for ``YYYY-MM-DDT00:00:00Z``, i.e. the resource will expire at midnight UTC of the specified date.
Example annotation values: ``2019-02-28T20:40:00Z``, ``2019-02-28T20:40``, ``2019-02-28``.

Available command line options:

Dry run mode: do not change anything, just print what would be done
Debug mode: print more information
Run only once and exit. This is useful if you run the Kubernetes Janitor as a ``CronJob``.
Loop interval (default: 30s). This option only makes sense when the ``--once`` flag is not set.
Include resources for clean up (default: all resources), can also be configured via environment variable ``INCLUDE_RESOURCES``. This option can be used if you want to clean up only certain resource types, e.g. only ``deployments``.
Exclude resources from clean up (default: events,controllerrevisions), can also be configured via environment variable ``EXCLUDE_RESOURCES``.
This option takes precedence over ``--include-resources``, i.e. ``--exclude-resources=foos`` in combination with ``--include-resources=foos,bars`` would make ``kube-janitor`` only process ``bars`` resources.
Include namespaces for clean up (default: all namespaces), can also be configured via environment variable ``INCLUDE_NAMESPACES``
Exclude namespaces from clean up (default: kube-system), can also be configured via environment variable ``EXCLUDE_NAMESPACES``.
This option takes precedence over ``--include-namespaces``, i.e. ``--exclude-namespaces=ns1`` in combination with ``--include-namespaces=ns1,ns2`` would only process resources in the ``ns2`` namespace.
Optional: filename pointing to a YAML file with a list of rules to apply TTL values to arbitrary Kubernetes objects, e.g. to delete all deployments without a certain label automatically after N days. See Rules File configuration section below.

Example flags:







Rules File

When using the ``--rules-file`` option, the path needs to point to a valid YAML file with the following format:

.. code-block:: yaml

# remove deployments and statefulsets without a label "application"
- id: require-application-label
- deployments
- statefulsets
jmespath: "!(spec.template.metadata.labels.application)"
ttl: 4d
# delete all deployments with a name starting with "pr-*"
- id: temporary-pr-deployments
- deployments
jmespath: "starts_with(, 'pr-')"
ttl: 4h
# delete all resources within the "temp" namespace after 3 days
- id: temp-namespace-cleanup
- "*"
jmespath: "metadata.namespace == 'temp'"
ttl: 3d
# require the "foo" pod label for all new deployments starting April 2019
- id: require-foo-label-april-2019
- deployments
- statefulsets
jmespath: "!( && metadata.creationTimestamp > '2019-04-01'"
ttl: 7d

The first matching rule will define the TTL (``ttl`` field). Kubernetes objects with a ``janitor/ttl`` annotation will not be matched against any rule.

A rule matches for a given Kubernetes object if all of the following criteria is true:

* the object has no ``janitor/ttl`` annotation (otherwise the TTL value from the annotation is applied)
* the object's type is included in the ``resources`` list of the rule or the special value ``*`` is part of the ``resources`` list (similar to Kubernetes RBAC)
* the JMESPath_ evaluates to a truth-like value (boolean ``true``, non-empty list, non-empty object, or non-empty string)

The first matching rule will define the TTL for the object (as if the object would have a ``janitor/ttl`` annotation with the same value).

Each rule has the following attributes:

Some string identifying the rule (e.g. for log output), must be lowercase and match the regex ``^[a-z][a-z0-9-]*$``. The ID has no special meaning and is only used to refer to the rule in log output/statistics.
List of resources (e.g. ``deployments``, ``namespaces``, ..) this rule should be applied to. The special value ``*`` will match all resource types.
JMESPath_ expression to evaluate on the resource object. The rule will only match if the expression evaluates to true. The expression will get the Kubernetes object as input.
The expression ```` would evaluate to true if the object has the label ``foo`` and it has a non-empty string as value.
TTL value (e.g. ``15m``) to apply to the object if the rule matches.


Easiest way to contribute is to provide feedback! We would love to hear what you like and what you think is missing.
Create an issue or `ping try_except_ on Twitter`_.

PRs are welcome. Please also have a look at `issues labeled with "help wanted"`_.

Local Development

You can run Kubernetes Janitor against your current kubeconfig context, e.g. local Minikube_:

.. code-block:: bash

$ pipenv install --dev
$ pipenv shell
$ python3 -m kube_janitor --dry-run --debug --once

To run PEP8 (flake8) checks and unit tests including coverage report:

.. code-block:: bash

$ make test


This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
GNU General Public License for more details.

You should have received a copy of the GNU General Public License
along with this program. If not, see

.. _Minikube:
.. _ping try_except_ on Twitter:
.. _issues labeled with "help wanted":
.. _JMESPath: