Quarkus offers the ability to automatically generate Kubernetes resources based on sane defaults and user-supplied configuration using dekorate. It currently supports generating resources for vanilla Kubernetes, OpenShift and Knative. Furthermore, Quarkus can deploy the application to a target Kubernetes cluster by applying the generated manifests to the target cluster’s API Server. Finally, when either one of container image extensions is present (see the container image guide for more details), Quarkus has the ability to create a container image and push it to a registry before deploying the application to the target platform.
Prerequisites
To complete this guide, you need:
-
Roughly 15 minutes
-
An IDE
-
JDK 11+ installed with
JAVA_HOME
configured appropriately -
Apache Maven 3.8.4
-
Optionally the Quarkus CLI if you want to use it
-
Access to a Kubernetes cluster (Minikube is a viable option)
Kubernetes
Let’s create a new project that contains both the Kubernetes and Jib extensions:
This added the following dependencies to the build file:
<dependency>
<groupId>io.quarkus</groupId>
<artifactId>quarkus-resteasy</artifactId>
</dependency>
<dependency>
<groupId>io.quarkus</groupId>
<artifactId>quarkus-kubernetes</artifactId>
</dependency>
<dependency>
<groupId>io.quarkus</groupId>
<artifactId>quarkus-container-image-jib</artifactId>
</dependency>
implementation("io.quarkus:quarkus-resteasy")
implementation("io.quarkus:quarkus-kubernetes")
implementation("io.quarkus:quarkus-container-image-jib")
By adding these dependencies, we enable the generation of Kubernetes manifests each time we perform a build while also enabling the build of a container image using Jib. For example, following the execution of:
quarkus build
./mvnw clean package
./gradlew build
you will notice amongst the other files that are created, two files named
kubernetes.json
and kubernetes.yml
in the target/kubernetes/
directory.
If you look at either file you will see that it contains both a Kubernetes Deployment
and a Service
.
The full source of the kubernetes.json
file looks something like this:
{
{
"apiVersion" : "apps/v1",
"kind" : "Deployment",
"metadata" : {
"annotations": {
"app.quarkus.io/vcs-url" : "<some url>",
"app.quarkus.io/commit-id" : "<some git SHA>",
},
"labels" : {
"app.kubernetes.io/name" : "test-quarkus-app",
"app.kubernetes.io/version" : "1.0.0-SNAPSHOT",
},
"name" : "test-quarkus-app"
},
"spec" : {
"replicas" : 1,
"selector" : {
"matchLabels" : {
"app.kubernetes.io/name" : "test-quarkus-app",
"app.kubernetes.io/version" : "1.0.0-SNAPSHOT",
}
},
"template" : {
"metadata" : {
"labels" : {
"app.kubernetes.io/name" : "test-quarkus-app",
"app.kubernetes.io/version" : "1.0.0-SNAPSHOT"
}
},
"spec" : {
"containers" : [ {
"env" : [ {
"name" : "KUBERNETES_NAMESPACE",
"valueFrom" : {
"fieldRef" : {
"fieldPath" : "metadata.namespace"
}
}
} ],
"image" : "yourDockerUsername/test-quarkus-app:1.0.0-SNAPSHOT",
"imagePullPolicy" : "Always",
"name" : "test-quarkus-app"
} ]
}
}
}
},
{
"apiVersion" : "v1",
"kind" : "Service",
"metadata" : {
"annotations": {
"app.quarkus.io/vcs-url" : "<some url>",
"app.quarkus.io/commit-id" : "<some git SHA>",
},
"labels" : {
"app.kubernetes.io/name" : "test-quarkus-app",
"app.kubernetes.io/version" : "1.0.0-SNAPSHOT",
},
"name" : "test-quarkus-app"
},
"spec" : {
"ports" : [ {
"name" : "http",
"port" : 8080,
"targetPort" : 8080
} ],
"selector" : {
"app.kubernetes.io/name" : "test-quarkus-app",
"app.kubernetes.io/version" : "1.0.0-SNAPSHOT"
},
"type" : "ClusterIP"
}
}
}
Beside generating a Deployment
resource, you can also choose to get a StatefulSet
instead via application.properties
:
quarkus.kubernetes.deployment-kind=StatefulSet
The generated manifest can be applied to the cluster from the project root using kubectl
:
kubectl apply -f target/kubernetes/kubernetes.json
An important thing to note about the Deployment
(or StatefulSet
) is that is uses yourDockerUsername/test-quarkus-app:1.0.0-SNAPSHOT
as the container image of the Pod
.
The name of the image is controlled by the Jib extension and can be customized using the usual application.properties
.
For example with a configuration like:
quarkus.container-image.group=quarkus #optional, default to the system user name
quarkus.container-image.name=demo-app #optional, defaults to the application name
quarkus.container-image.tag=1.0 #optional, defaults to the application version
The image that will be used in the generated manifests will be quarkus/demo-app:1.0
Namespace
By default Quarkus omits the namespace in the generated manifests, rather than enforce the default
namespace. That means that you can apply the manifest to your chosen namespace when using kubctl
, which in the example below is test
:
kubectl apply -f target/kubernetes/kubernetes.json -n=test
To specify the namespace in your manifest customize with the following property in your application.properties
:
quarkus.kubernetes.namespace=mynamespace
Defining a Docker registry
The Docker registry can be specified with the following property:
quarkus.container-image.registry=my.docker-registry.net
By adding this property along with the rest of the container image properties of the previous section, the generated manifests will use the image my.docker-registry.net/quarkus/demo-app:1.0
.
The image is not the only thing that can be customized in the generated manifests, as will become evident in the following sections.
Labels and Annotations
Labels
The generated manifests use the Kubernetes recommended labels.
These labels can be customized using quarkus.kubernetes.name
, quarkus.kubernetes.version
and quarkus.kubernetes.part-of
.
For example by adding the following configuration to your application.properties
:
quarkus.kubernetes.part-of=todo-app
quarkus.kubernetes.name=todo-rest
quarkus.kubernetes.version=1.0-rc.1
As is described in detail in the OpenShift section, customizing OpenShift (or Knative) properties is done in the same way, but replacing
|
The labels in generated resources will look like:
"labels" : {
"app.kubernetes.io/part-of" : "todo-app",
"app.kubernetes.io/name" : "todo-rest",
"app.kubernetes.io/version" : "1.0-rc.1"
}
You can also remove the
|
Custom Labels
To add additional custom labels, for example foo=bar
just apply the following configuration:
quarkus.kubernetes.labels.foo=bar
When using the quarkus-container-image-jib extension to build a container image, then any label added via the aforementioned property will also be added to the generated container image.
|
Annotations
Out of the box, the generated resources will be annotated with version control related information that can be used either by tooling, or by the user for troubleshooting purposes.
"annotations": {
"app.quarkus.io/vcs-url" : "<some url>",
"app.quarkus.io/commit-id" : "<some git SHA>",
}
Custom Annotations
Custom annotations can be added in a way similar to labels.For example to add the annotation foo=bar
and app.quarkus/id=42
just apply the following configuration:
quarkus.kubernetes.annotations.foo=bar
quarkus.kubernetes.annotations."app.quarkus/id"=42
Environment variables
Kubernetes provides multiple ways of defining environment variables:
-
key/value pairs
-
import all values from a Secret or ConfigMap
-
interpolate a single value identified by a given field in a Secret or ConfigMap
-
interpolate a value from a field within the same resource
Environment variables from key/value pairs
To add a key/value pair as an environment variable in the generated resources:
quarkus.kubernetes.env.vars.my-env-var=foobar
The command above will add MY_ENV_VAR=foobar
as an environment variable.
Please note that the key my-env-var
will be converted to uppercase and dashes will be replaced by underscores resulting in MY_ENV_VAR
.
Environment variables from Secret
To add all key/value pairs of Secret
as environment variables just apply the following configuration, separating each Secret
to be used as source by a comma (,
):
quarkus.kubernetes.env.secrets=my-secret,my-other-secret
which would generate the following in the container definition:
envFrom:
- secretRef:
name: my-secret
optional: false
- secretRef:
name: my-other-secret
optional: false
The following extracts a value identified by the keyName
field from the my-secret
Secret into a foo
environment variable:
quarkus.kubernetes.env.mapping.foo.from-secret=my-secret
quarkus.kubernetes.env.mapping.foo.with-key=keyName
This would generate the following in the env
section of your container:
- env:
- name: FOO
valueFrom:
secretKeyRef:
key: keyName
name: my-secret
optional: false
Environment variables from ConfigMap
To add all key/value pairs from ConfigMap
as environment variables just apply the following configuration, separating each
ConfigMap
to be used as source by a comma (,
):
quarkus.kubernetes.env.configmaps=my-config-map,another-config-map
which would generate the following in the container definition:
envFrom:
- configMapRef:
name: my-config-map
optional: false
- configMapRef:
name: another-config-map
optional: false
The following extracts a value identified by the keyName
field from the my-config-map
ConfigMap into a foo
environment variable:
quarkus.kubernetes.env.mapping.foo.from-configmap=my-configmap
quarkus.kubernetes.env.mapping.foo.with-key=keyName
This would generate the following in the env
section of your container:
- env:
- name: FOO
valueFrom:
configMapRefKey:
key: keyName
name: my-configmap
optional: false
Environment variables from fields
It’s also possible to use the value from another field to add a new environment variable by specifying the path of the field to be used as a source, as follows:
quarkus.kubernetes.env.fields.foo=metadata.name
As is described in detail in the OpenShift section, customizing OpenShift properties is done in the same way, but replacing
|
Validation
A conflict between two definitions, e.g. mistakenly assigning both a value and specifying that a variable is derived from a field, will result in an error being thrown at build time so that you get the opportunity to fix the issue before you deploy your application to your cluster where it might be more difficult to diagnose the source of the issue.
Similarly, two redundant definitions, e.g. defining an injection from the same secret twice, will not cause an issue but will indeed report a warning to let you know that you might not have intended to duplicate that definition.
Backwards compatibility
Previous versions of the Kubernetes extension supported a different syntax to add environment variables.The older syntax is still supported but is deprecated and it’s advised that you migrate to the new syntax.
Old |
New |
||
Plain variable |
|
|
|
From field |
|
|
|
All from |
|
|
|
All from |
|
|
|
From one |
|
|
|
|
|
||
From one |
|
|
|
|
|
If you redefine the same variable using the new syntax while keeping the old syntax, ONLY the new version will be kept
and a warning will be issued to alert you of the problem.For example, if you define both
quarkus.kubernetes.env-vars.my-env-var.value=foobar and quarkus.kubernetes.env.vars.my-env-var=newValue , the extension will
only generate an environment variable MY_ENV_VAR=newValue and issue a warning.
|
Mounting volumes
The Kubernetes extension allows the user to configure both volumes and mounts for the application. Any volume can be mounted with a simple configuration:
quarkus.kubernetes.mounts.my-volume.path=/where/to/mount
This will add a mount to the pod for volume my-volume
to path /where/to/mount
.
The volumes themselves can be configured as shown in the sections below.
Passing application configuration
Quarkus supports passing configuration from external locations (via Smallrye Config). This usually requires setting an additional environment variable or system propertiy. When you need to use a secret or a config map for the purpose of application configuration, you need to:
-
define a volume
-
mount the volume
-
create an environment variable for
SMALLRYE_CONFIG_LOCATIONS
To simplify things, quarkus provides single step alternative:
quarkus.kubernetes.app-secret=<name of the secret containing the configuration>
or
quarkus.kubernetes.app-config-map=<name of the config map containing the configuration>
When these properties are used, the generated manifests will contain everything required.
The application config volumes will be created using path: /mnt/app-secret
and /mnt/app-config-map
for secrets and configmaps respectively.
Note: Users may use both properties at the same time.
Changing the number of replicas:
To change the number of replicas from 1 to 3:
quarkus.kubernetes.replicas=3
Add readiness and liveness probes
By default, the Kubernetes resources do not contain readiness and liveness probes in the generated Deployment
. Adding them however is just a matter of adding the SmallRye Health extension like so:
<dependency>
<groupId>io.quarkus</groupId>
<artifactId>quarkus-smallrye-health</artifactId>
</dependency>
implementation("io.quarkus:quarkus-smallrye-health")
The values of the generated probes will be determined by the configured health properties: quarkus.smallrye-health.root-path
, quarkus.smallrye-health.liveness-path
and quarkus.smallrye-health.readiness-path
.
More information about the health extension can be found in the relevant guide.
Customizing the readiness probe:
To set the initial delay of the probe to 20 seconds and the period to 45:
quarkus.kubernetes.readiness-probe.initial-delay=20s
quarkus.kubernetes.readiness-probe.period=45s
Add hostAliases
To add entries to a Pod’s /etc/hosts
file (more information can be found in Kubernetes documentation), just apply the following configuration:
quarkus.kubernetes.hostaliases."10.0.0.0".hostnames=foo.com,bar.org
This would generate the following hostAliases
section in the deployment
definition:
kind: Deployment
spec:
template:
spec:
hostAliases:
- hostnames:
- foo.com
- bar.org
ip: 10.0.0.0
Container Resources Management
CPU & Memory limits and requests can be applied to a Container
(more info in Kubernetes documentation) using the following configuration:
quarkus.kubernetes.resources.requests.memory=64Mi
quarkus.kubernetes.resources.requests.cpu=250m
quarkus.kubernetes.resources.limits.memory=512Mi
quarkus.kubernetes.resources.limits.cpu=1000m
This would generate the following entry in the container
section:
containers:
resources:
limits:
cpu: 1000m
memory: 512Mi
requests:
cpu: 250m
memory: 64Mi
Using the Kubernetes client
Applications that are deployed to Kubernetes and need to access the API server will usually make use of the kubernetes-client
extension:
<dependency>
<groupId>io.quarkus</groupId>
<artifactId>quarkus-kubernetes-client</artifactId>
</dependency>
implementation("io.quarkus:quarkus-kubernetes-client")
To access the API server from within a Kubernetes cluster, some RBAC related resources are required (e.g. a ServiceAccount, a RoleBinding etc.).
So, when the kubernetes-client
extension is present, the kubernetes
extension is going to create those resources automatically, so that application will be granted the view
role.
If more roles are required, they will have to be added manually.
Deploying to Minikube
Minikube is quite popular when a Kubernetes cluster is needed for development purposes. To make the deployment to Minikube
experience as frictionless as possible, Quarkus provides the quarkus-minikube
extension. This extension can be added to a project like so:
<dependency>
<groupId>io.quarkus</groupId>
<artifactId>quarkus-minikube</artifactId>
</dependency>
implementation("io.quarkus:quarkus-minikube")
The purpose of this extension is to generate Kubernetes manifests (minikube.yaml
and minikube.json
) that are tailored to Minikube.
This extension assumes a couple things:
-
Users won’t be using an image registry and will instead make their container image accessible to the Kubernetes cluster by building it directly into Minikube’s Docker daemon. To use Minikube’s Docker daemon you must first execute:
eval $(minikube -p minikube docker-env)
-
Applications deployed to Kubernetes won’t be accessed via a Kubernetes
Ingress
, but rather as aNodePort
Service
. The advantage of doing this is that the URL of an application can be retrieved trivially by executing:minikube service list
To control the nodePort that is used in this case, users can set quarkus.kubernetes.node-port
.
Note however that this configuration is entirely optional because Quarkus will automatically use a proper (and non-changing) value if none is set.
It is highly discouraged to use the manifests generated by the Minikube extension when deploying to production as these manifests are intended for development purposes only. When deploying to production, consider using the vanilla Kubernetes manifests (or the OpenShift ones when targeting OpenShift). |
If the assumptions the Minikube extension makes don’t fit your workflow, nothing prevents you from using the regular Kubernetes extension to generate Kubernetes manifests and apply those to your Minikube cluster. |
Tuning the generated resources using application.properties
The Kubernetes extension allows tuning the generated manifest, using the application.properties
file.
Here are some examples:
Configuration options
The table below describe all the available configuration options.
Property |
Type |
Description |
Default Value |
quarkus.kubernetes.name |
String |
${quarkus.container-image.name} |
|
quarkus.kubernetes.version |
String |
${quarkus.container-image.tag} |
|
quarkus.kubernetes.deployment-kind |
String |
Deployment |
|
quarkus.kubernetes.part-of |
String |
||
quarkus.kubernetes.init-containers |
Map<String, Container> |
||
quarkus.kubernetes.namespace |
String |
||
quarkus.kubernetes.labels |
Map |
||
quarkus.kubernetes.annotations |
Map |
||
quarkus.kubernetes.app-secret |
String |
||
quarkus.kubernetes.app-config-map |
String |
||
quarkus.kubernetes.env-vars |
Map<String, Env> |
||
quarkus.kubernetes.working-dir |
String |
||
quarkus.kubernetes.command |
String[] |
||
quarkus.kubernetes.arguments |
String[] |
||
quarkus.kubernetes.replicas |
int |
1 |
|
quarkus.kubernetes.service-account |
String |
||
quarkus.kubernetes.ports |
Map<String, Port> |
||
quarkus.kubernetes.service-type |
ServiceType |
ClusterIP |
|
quarkus.kubernetes.pvc-volumes |
Map<String, PersistentVolumeClaimVolume> |
||
quarkus.kubernetes.secret-volumes |
Map<String, SecretVolume> |
||
quarkus.kubernetes.config-map-volumes |
Map<String, ConfigMapVolume> |
||
quarkus.kubernetes.git-repo-volumes |
Map<String, GitRepoVolume> |
||
quarkus.kubernetes.aws-elastic-block-store-volumes |
Map<String, AwsElasticBlockStoreVolume> |
||
quarkus.kubernetes.azure-disk-volumes |
Map<String, AzureDiskVolume> |
||
quarkus.kubernetes.azure-file-volumes |
Map<String, AzureFileVolume> |
||
quarkus.kubernetes.mounts |
Map<String, Mount> |
||
quarkus.kubernetes.image-pull-policy |
ImagePullPolicy |
Always |
|
quarkus.kubernetes.image-pull-secrets |
String[] |
||
quarkus.kubernetes.liveness-probe |
Probe |
( see Probe ) |
|
quarkus.kubernetes.readiness-probe |
Probe |
( see Probe ) |
|
quarkus.kubernetes.sidecars |
Map<String, Container> |
||
quarkus.kubernetes.ingress.expose |
boolean |
false |
|
quarkus.kubernetes.ingress.host |
String |
||
quarkus.kubernetes.ingress.annotations |
Map<String, String> |
||
quarkus.kubernetes.headless |
boolean |
false |
|
quarkus.kubernetes.hostaliases |
Map<String, HostAlias> |
||
quarkus.kubernetes.resources.requests.cpu |
String |
||
quarkus.kubernetes.resources.requests.memory |
String |
||
quarkus.kubernetes.resources.limits.cpu |
String |
||
quarkus.kubernetes.resources.limits.memory |
String |
Properties that use non-standard types, can be referenced by expanding the property.
For example to define a kubernetes-readiness-probe
which is of type Probe
:
quarkus.kubernetes.readiness-probe.initial-delay=20s
quarkus.kubernetes.readiness-probe.period=45s
In this example initial-delay
and period
are fields of the type Probe
.
Below you will find tables describing all available types.
Client Connection Configuration
You may need to configure the connection to your Kubernetes cluster.
By default, it automatically uses the active context used by kubectl
.
For instance, if your cluster API endpoint uses a self-signed SSL Certificate you need to explicitly configure the client to trust it. You can achieve this by defining the following property:
quarkus.kubernetes-client.trust-certs=true
The full list of the Kuberneters client configuration properties is provided below.
Basic Types
Allowed values: cluster-ip
, node-port
, load-balancer
, external-name
Property |
Type |
Description |
Default Value |
value |
String |
||
secret |
String |
||
configmap |
String |
||
field |
String |
Property |
Type |
Description |
Default Value |
http-action-path |
String |
||
exec-action |
String |
||
tcp-socket-action |
String |
||
initial-delay |
Duration |
0 |
|
period |
Duration |
30s |
|
timeout |
Duration |
10s |
Property |
Type |
Description |
Default Value |
container-port |
int |
||
host-port |
int |
0 |
|
path |
String |
/ |
|
protocol |
Protocol |
TCP |
Property |
Type |
Description |
Default Value |
image |
String |
||
env-vars |
Env[] |
||
working-dir |
String |
||
command |
String[] |
||
arguments |
String[] |
||
ports |
Port[] |
||
mounts |
Mount[] |
||
image-pull-policy |
ImagePullPolicy |
Always |
|
liveness-probe |
Probe |
||
readiness-probe |
Probe |
Property |
Type |
Description |
Default Value |
hostnames |
String[] |
list of hostnames |
Mounts and Volumes
Property |
Type |
Description |
Default Value |
path |
String |
||
sub-path |
String |
||
read-only |
boolean |
false |
Property |
Type |
Description |
Default Value |
config-map-name |
String |
||
default-mode |
int |
0600 |
|
optional |
boolean |
false |
Property |
Type |
Description |
Default Value |
secret-name |
String |
||
default-mode |
int |
0600 |
|
optional |
boolean |
false |
Property |
Type |
Description |
Default Value |
disk-name |
String |
||
disk-uri |
String |
||
kind |
String |
Managed |
|
caching-mode |
String |
ReadWrite |
|
fs-type |
String |
ext4 |
|
read-only |
boolean |
false |
Property |
Type |
Description |
Default Value |
volume-id |
String |
||
partition |
int |
||
fs-type |
String |
ext4 |
|
read-only |
boolean |
false |
Property |
Type |
Description |
Default Value |
repository |
String |
||
directory |
String |
||
revision |
String |
Property |
Type |
Description |
Default Value |
claim-name |
String |
||
read-only |
boolean |
false |
Property |
Type |
Description |
Default Value |
share-name |
String |
||
secret-name |
String |
||
read-only |
boolean |
false |
OpenShift
One way to deploy an application to OpenShift is to use s2i (source to image) to create an image stream from the source and then deploy the image stream:
quarkus extension remove kubernetes,jib
quarkus extension add openshift
oc new-project quarkus-project
quarkus build -Dquarkus.container-image.build=true
oc new-app --name=greeting quarkus-project/kubernetes-quickstart:1.0.0-SNAPSHOT
oc expose svc/greeting
oc get route
curl <route>/greeting
./mvnw quarkus:remove-extension -Dextensions="kubernetes, jib"
./mvnw quarkus:add-extension -Dextensions="openshift"
oc new-project quarkus-project
./mvnw clean package -Dquarkus.container-image.build=true
oc new-app --name=greeting quarkus-project/kubernetes-quickstart:1.0.0-SNAPSHOT
oc expose svc/greeting
oc get route
curl <route>/greeting
./gradlew removeExtension --extensions="kubernetes, jib"
./gradlew addExtension --extensions="openshift"
oc new-project quarkus-project
./gradlew build -Dquarkus.container-image.build=true
oc new-app --name=greeting quarkus-project/kubernetes-quickstart:1.0.0-SNAPSHOT
oc expose svc/greeting
oc get route
curl <route>/greeting
See further information in Deploying to OpenShift.
A description of OpenShift resources and customisable properties is given below alongside Kubernetes resources to show similarities where applicable. This includes an alternative to oc new-app …
above, i.e. oc apply -f target/kubernetes/openshift.json
.
To enable the generation of OpenShift resources, you need to include OpenShift in the target platforms:
quarkus.kubernetes.deployment-target=openshift
If you need to generate resources for both platforms (vanilla Kubernetes and OpenShift), then you need to include both (comma separated).
quarkus.kubernetes.deployment-target=kubernetes,openshift
Following the execution of ./mvnw package -Dquarkus.container-image.build=true
you will notice amongst the other files that are created, two files named
openshift.json
and openshift.yml
in the target/kubernetes/
directory.
These manifests can be deployed as is to a running cluster, using kubectl
:
kubectl apply -f target/kubernetes/openshift.json
OpenShift users might want to use oc
rather than kubectl
:
oc apply -f target/kubernetes/openshift.json
Quarkus also provides the OpenShift extension. This extension is basically a wrapper around the Kubernetes extension and
relieves OpenShift users of the necessity of setting the deployment-target property to openshift
|
The OpenShift resources can be customized in a similar approach with Kubernetes.
Property |
Type |
Description |
Default Value |
quarkus.openshift.name |
String |
${quarkus.container-image.name} |
|
quarkus.openshift.version |
String |
${quarkus.container-image.tag} |
|
quarkus.openshift.deployment-kind |
String |
DeploymentConfig |
|
quarkus.openshift.part-of |
String |
||
quarkus.openshift.init-containers |
Map<String, Container> |
||
quarkus.openshift.labels |
Map |
||
quarkus.openshift.annotations |
Map |
||
quarkus.openshift.app-secret |
String |
||
quarkus.openshift.app-config-map |
String |
||
quarkus.openshift.env-vars |
Map<String, Env> |
||
quarkus.openshift.working-dir |
String |
||
quarkus.openshift.command |
String[] |
||
quarkus.openshift.arguments |
String[] |
||
quarkus.openshift.jvm-arguments |
String[] |
The JVM arguments to pass to the JVM when starting the application |
-Dquarkus.http.host=0.0.0.0,-Djava.util.logging.manager=org.jboss.logmanager.LogManager |
quarkus.openshift.jvm-additional-arguments |
String[] |
Additional JVM arguments to pass to the JVM when starting the application |
|
quarkus.openshift.replicas |
int |
1 |
|
quarkus.openshift.service-account |
String |
||
quarkus.openshift.ports |
Map<String, Port> |
||
quarkus.openshift.service-type |
ServiceType |
ClusterIP |
|
quarkus.openshift.pvc-volumes |
Map<String, PersistentVolumeClaimVolume> |
||
quarkus.openshift.secret-volumes |
Map<String, SecretVolume> |
||
quarkus.openshift.config-map-volumes |
Map<String, ConfigMapVolume> |
||
quarkus.openshift.git-repo-volumes |
Map<String, GitRepoVolume> |
||
quarkus.openshift.aws-elastic-block-store-volumes |
Map<String, AwsElasticBlockStoreVolume> |
||
quarkus.openshift.azure-disk-volumes |
Map<String, AzureDiskVolume> |
||
quarkus.openshift.azure-file-volumes |
Map<String, AzureFileVolume> |
||
quarkus.openshift.mounts |
Map<String, Mount> |
||
quarkus.openshift.image-pull-policy |
ImagePullPolicy |
Always |
|
quarkus.openshift.image-pull-secrets |
String[] |
||
quarkus.openshift.liveness-probe |
Probe |
( see Probe ) |
|
quarkus.openshift.readiness-probe |
Probe |
( see Probe ) |
|
quarkus.openshift.sidecars |
Map<String, Container> |
||
quarkus.openshift.route.expose |
boolean |
false |
|
quarkus.openshift.route.host |
String |
||
quarkus.openshift.route.annotations |
Map<String, String> |
||
quarkus.openshift.headless |
boolean |
false |
Knative
To enable the generation of Knative resources, you need to include Knative in the target platforms:
quarkus.kubernetes.deployment-target=knative
Following the execution of ./mvnw package
you will notice amongst the other files that are created, two files named
knative.json
and knative.yml
in the target/kubernetes/
directory.
If you look at either file you will see that it contains a Knative Service
.
The full source of the knative.json
file looks something like this:
{
{
"apiVersion" : "serving.quarkus.knative.dev/v1alpha1",
"kind" : "Service",
"metadata" : {
"annotations": {
"app.quarkus.io/vcs-url" : "<some url>",
"app.quarkus.io/commit-id" : "<some git SHA>"
},
"labels" : {
"app.kubernetes.io/name" : "test-quarkus-app",
"app.kubernetes.io/version" : "1.0.0-SNAPSHOT"
},
"name" : "knative"
},
"spec" : {
"runLatest" : {
"configuration" : {
"revisionTemplate" : {
"spec" : {
"container" : {
"image" : "dev.local/yourDockerUsername/test-quarkus-app:1.0.0-SNAPSHOT",
"imagePullPolicy" : "Always"
}
}
}
}
}
}
}
}
The generated manifest can be deployed as is to a running cluster, using kubectl
:
kubectl apply -f target/kubernetes/knative.json
The generated service can be customized using the following properties:
Property |
Type |
Description |
Default Value |
quarkus.knative.name |
String |
${quarkus.container-image.name} |
|
quarkus.knative.version |
String |
${quarkus.container-image.tag} |
|
quarkus.knative.part-of |
String |
||
quarkus.knative.init-containers |
Map<String, Container> |
||
quarkus.knative.labels |
Map |
||
quarkus.knative.annotations |
Map |
||
quarkus.knative.app-secret |
String |
||
quarkus.knative.app-config-map |
String |
||
quarkus.knative.env-vars |
Map<String, Env> |
||
quarkus.knative.working-dir |
String |
||
quarkus.knative.command |
String[] |
||
quarkus.knative.arguments |
String[] |
||
quarkus.knative.replicas |
int |
1 |
|
quarkus.knative.service-account |
String |
||
quarkus.knative.host |
String |
||
quarkus.knative.ports |
Map<String, Port> |
||
quarkus.knative.service-type |
ServiceType |
ClusterIP |
|
quarkus.knative.pvc-volumes |
Map<String, PersistentVolumeClaimVolume> |
||
quarkus.knative.secret-volumes |
Map<String, SecretVolume> |
||
quarkus.knative.config-map-volumes |
Map<String, ConfigMapVolume> |
||
quarkus.knative.git-repo-volumes |
Map<String, GitRepoVolume> |
||
quarkus.knative.aws-elastic-block-store-volumes |
Map<String, AwsElasticBlockStoreVolume> |
||
quarkus.knative.azure-disk-volumes |
Map<String, AzureDiskVolume> |
||
quarkus.knative.azure-file-volumes |
Map<String, AzureFileVolume> |
||
quarkus.knative.mounts |
Map<String, Mount> |
||
quarkus.knative.image-pull-policy |
ImagePullPolicy |
Always |
|
quarkus.knative.image-pull-secrets |
String[] |
||
quarkus.knative.liveness-probe |
Probe |
( see Probe ) |
|
quarkus.knative.readiness-probe |
Probe |
( see Probe ) |
|
quarkus.knative.sidecars |
Map<String, Container> |
||
quarkus.knative.revision-name |
String |
||
quarkus.knative.traffic |
Traffic[] |
( see Traffic ) |
|
quarkus.knative.min-scale |
int |
See link |
|
quarkus.knative.max-scale |
int |
See link |
|
quarkus.knative.scale-to-zero-enabled |
boolean |
See link |
true |
quarkus.knative.revision-auto-scaling |
AutoScalingConfig |
( see AutoScalingConfig ) |
|
quarkus.knative.global-auto-scaling |
GlobalAutoScalingConfig |
( see GlobalAutoScalingConfig ) |
Property |
Type |
Description |
Default Value |
revision-name |
String |
A specific revision to which to send this portion of traffic |
|
tag |
String |
Expose a dedicated url for referncing this target |
|
latest-revision |
Boolean |
Optionally provided to indicate that the latest revision should be used for this traffic target |
false |
percent |
Logn |
Indicates the percent of traffic that is be routed to this revision |
100 |
Property |
Type |
Description |
Default Value |
auto-scaler-class |
String |
The auto-scaler class. Possible values: |
kpa |
metric |
String |
The autoscaling metric to use. Possible values (concurency, rps, cpu) |
|
target |
int |
This value specifies the autoscaling target |
|
container-concurrency |
int |
The exact amount of requests allowed to the replica at a time |
|
target-utilization-percentage |
int |
This value specifies a percentage of the target to actually be targeted by the autoscaler |
Property |
Type |
Description |
Default Value |
auto-scaler-class |
String |
The auto-scaler class. Possible values: |
kpa |
container-concurrency |
int |
The exact amount of requests allowed to the replica at a time |
|
target-utilization-percentage |
int |
This value specifies a percentage of the target to actually be targeted by the autoscaler |
|
requests-per-second |
Logn |
The requests per second per replica |
Deployment targets
Mentioned in the previous sections was the concept of deployment-target
. This concept allows users to control which Kubernetes manifests will be generated
and deployed to a cluster (if quarkus.kubernetes.deploy
has been set to true
).
By default, when no deployment-target
is set, then only vanilla Kubernetes resources are generated and deployed. When multiple values are set (for example
quarkus.kubernetes.deployment-target=kubernetes,openshift
) then the resources for all targets are generated, but only the resources
that correspond to the first target are applied to the cluster (if deployment is enabled).
In the case of wrapper extensions like OpenShift and Minikube, when these extensions have been explicitly added to the project, the default deployment-target
is set by those extensions. For example if quarkus-minikube
has been added to a project, then minikube
becomes the default deployment target and its
resources will be applied to the Kubernetes cluster when deployment via quarkus.kubernetes.deploy
has been set.
Users can still override the deployment-targets manually using quarkus.kubernetes.deployment-target
.
Deprecated configuration
The following categories of configuration properties have been deprecated.
Properties without the quarkus prefix
In earlier versions of the extension, the quarkus.
was missing from those properties. These properties are now deprecated.
Docker and S2i properties
The properties for configuring docker
and s2i
are also deprecated in favor of the new container-image extensions.
Config group arrays
Properties referring to config group arrays (e.g. kubernetes.labels[0]
, kubernetes.env-vars[0]
etc) have been converted to maps, to align with the rest of the Quarkus ecosystem.
The code below demonstrates the change in labels
config:
# Old labels config:
kubernetes.labels[0].name=foo
kubernetes.labels[0].value=bar
# New labels
quarkus.kubernetes.labels.foo=bar
The code below demonstrates the change in env-vars
config:
# Old env-vars config:
kubernetes.env-vars[0].name=foo
kubernetes.env-vars[0].configmap=my-configmap
# New env-vars
quarkus.kubernetes.env-vars.foo.configmap=myconfigmap
env-vars
properties
quarkus.kubernetes.env-vars
are deprecated (though still currently supported as of this writing) and the new declaration style should be used instead.
See Environment variables and more specifically Backwards compatibility for more details.
Deployment
To trigger building and deploying a container image you need to enable the quarkus.kubernetes.deploy
flag (the flag is disabled by default - furthermore it has no effect during test runs or dev mode).
This can be easily done with the command line:
./mvnw clean package -Dquarkus.kubernetes.deploy=true
Building a container image
Building a container image is possible, using any of the 3 available container-image
extensions:
Each time deployment is requested, a container image build will be implicitly triggered (no additional properties are required when the Kubernetes deployment has been enabled).
Deploying
When deployment is enabled, the Kubernetes extension will select the resources specified by quarkus.kubernetes.deployment-target
and deploy them.
This assumes that a .kube/config
is available in your user directory that points to the target Kubernetes cluster.
In other words the extension will use whatever cluster kubectl
uses. The same applies to credentials.
At the moment no additional options are provided for further customization.
Using existing resources
Sometimes it’s desirable to either provide additional resources (e.g. a ConfigMap, a Secret, a Deployment for a database etc) or provide custom ones that will be used as a base
for the generation process.
Those resources can be added under src/main/kubernetes
directory and can be named after the target environment (e.g. kubernetes.json, openshift.json, knative.json, or the yml equivalents). The correlation between provided and generated files is done by file name.
So, a kubernetes.json
/kubernetes.yml
file added in src/main/kubernetes
will only affect the generated kubernetes.json
/kubernetes.yml
. An openshift.json
/openshift.yml
file added in src/main/kubernetes
will only affect the generated openshift.json
/openshift.yml
.
A knative.json
/knative.yml
file added in src/main/kubernetes
will only affect the generated knative.json
/knative.yml
and so on. The provided file may be either in json or yaml format and may contain one or more resources. These resources will end up in both generated formats (json and yaml). For example, a secret added in src/main/kubernetes/kubernetes.yml
will be added to both the generated kubernetes.yml
and kubernetes.json
.
Note: At the time of writing there is no mechanism in place that allows a one to many relationship between provided and generated files. Minikube is not an exception to the rule above, so if you want to customize the generated minikube manifests, the file placed under src/main/kubernetes
will have to be named minikube.json
or minikube.yml
(naming it kubernetes.yml
or kubernetes.json
will result in having only the generated kubernetes.yml
and kubernetes.json
affected).
Any resource found will be added in the generated manifests. Global modifications (e.g. labels, annotations etc) will also be applied to those resources. If one of the provided resources has the same name as one of the generated ones, then the generated resource will be created on top of the provided resource, respecting existing content when possible (e.g. existing labels, annotations, environment variables, mounts, replicas etc).
The name of the resource is determined by the application name and may be overridden by quarkus.kubernetes.name
, quarkus.openshift.name
and quarkus.knative.name
.
For example, in the kubernetes-quickstart
application, we can add a kubernetes.yml
file in the src/main/kubernetes
that looks like:
apiVersion: apps/v1
kind: Deployment
metadata:
name: kubernetes-quickstart
labels:
app: quickstart
spec:
replicas: 3
selector:
matchLabels:
app: quickstart
template:
metadata:
labels:
app: quickstart
spec:
containers:
- name: kubernetes-quickstart
image: someimage:latest
ports:
- containerPort: 80
env:
- name: FOO
value: BAR
The generated kubernetes.yml
will look like:
apiVersion: "apps/v1"
kind: "Deployment"
metadata:
annotations:
app.quarkus.io/build-timestamp: "2020-04-10 - 12:54:37 +0000"
labels:
app: "quickstart"
name: "kubernetes-quickstart"
spec:
replicas: 3 (1)
selector:
matchLabels:
app.kubernetes.io/name: "kubernetes-quickstart"
app.kubernetes.io/version: "1.0.0-SNAPSHOT"
template:
metadata:
annotations:
app.quarkus.io/build-timestamp: "2020-04-10 - 12:54:37 +0000"
labels:
app: "quickstart" (2)
spec:
containers:
- env:
- name: "FOO" (3)
value: "BAR"
image: "<<yourDockerUsername>>/kubernetes-quickstart:1.0.0-SNAPSHOT" (4)
imagePullPolicy: "Always"
name: "kubernetes-quickstart"
ports:
- containerPort: 8080 (5)
name: "http"
protocol: "TCP"
serviceAccount: "kubernetes-quickstart"
The provided replicas <1>, labels <2> and environment variables <3> were retained. However, the image <4> and container port <5> were modified. Moreover, the default annotations have been added.
|
Service Binding
Quarkus supports the Service Binding Specification for Kubernetes to bind services to applications.
Specifically, Quarkus implements the Workload Projection part of the specification, therefore allowing applications to bind to services, such as a Database or a Broker, without the need for user configuration.
To enable Service Binding for supported extensions, add the quarkus-kubernetes-service-binding
extension to the application dependencies.
-
The following extensions can be used with Service Binding and are supported for Workload Projection:
-
quarkus-jdbc-mariadb
-
quarkus-jdbc-mssql
-
quarkus-jdbc-mysql
-
quarkus-jdbc-postgresql
-
quarkus-mongo-client
-
quarkus-kafka-client
-
quarkus-smallrye-reactive-messaging-kafka
-
Workload Projection
Workload Projection is a process of obtaining the configuration for services from the Kubernetes cluster. This configuration takes the form of directory structures that follow certain conventions and is attached to an application or to a service as a mounted volume. The kubernetes-service-binding
extension uses this directory structure to create configuration sources, which allows you to configure additional modules, such as databases or message brokers.
During application development, users can use workload projection to connect their application to a development database, or other locally-run services, without changing the actual application code or configuration.
For an example of a workload projection where the directory structure is included in the test resources and passed to integration test, see the Kubernetes Service Binding datasource GitHub repository.
|
Introduction to the Service Binding Operator
The Service Binding Operator is an Operator that implements Service Binding Specification for Kubernetes and is meant to simplify the binding of services to an application. Containerized applications that support Workload Projection obtain service binding information in the form of volume mounts. The Service Binding Operator reads binding service information and mounts it to the application containers that need it.
The correlation between application and bound services is expressed through the ServiceBinding
resources, which declares the intent of what services are meant to be bound to what application.
The Service Binding Operator watches for ServiceBinding
resources, which inform the Operator what applications are meant to be bound with what services. When a listed application is deployed, the Service Binding Operator collects all the binding information that must be passed to the application, then upgrades the application container by attaching a volume mount with the binding information.
The Service Binding Operator completes the following actions:
-
Observes
ServiceBinding
resources for workloads intended to be bound to a particular service -
Applies the binding information to the workload using volume mounts
The following chapter describes the automatic and semi-automatic service binding approaches and their use cases. With either approach, the kubernetes-service-binding
extension generates a ServiceBinding
resource. With the semi-automatic approach, users must provide a configuration for target services manually. With the automatic approach, for a limited set of services generating the ServiceBinding
resource, no additional configuration is needed.
Semi-automatic service binding
A service binding process starts with a user specification of required services that will be bound to a certain application. This expression is summarized in the ServiceBinding
resource that is generated by the kubernetes-service-binding
extension. The use of the kubernetes-service-binding
extensions helps users to generate ServiceBinding
resources with minimal configuration, therefore simplifying the process overall.
The Service Binding Operator responsible for the binding process then reads the information from the ServiceBinding
resource and mounts the required files to a container accordingly.
-
An example of the
ServiceBinding
resource:apiVersion: binding.operators.coreos.com/v1beta1 kind: ServiceBinding metadata: name: binding-request namespace: service-binding-demo spec: application: name: java-app group: apps version: v1 resource: deployments services: - group: postgres-operator.crunchydata.com version: v1beta1 kind: Database name: db-demo id: postgresDB
-
The
quarkus-kubernetes-service-binding
extension provides a more compact way of expressing the same information. For example:quarkus.kubernetes-service-binding.services.db-demo.api-version=postgres-operator.crunchydata.com/v1beta1 quarkus.kubernetes-service-binding.services.db-demo.kind=Database
-
After adding the earlier configuration properties inside your application.properties
, the quarkus-kubernetes
, in combination with the quarkus-kubernetes-service-binding
extension, automatically generates the ServiceBinding
resource.
The earlier mentioned db-demo
property-configuration identifier now has a double role and also completes the following actions:
-
Correlates and groups
api-version
andkind
properties together -
Defines the
name
property for the custom resource with a possibility for a later edit. For example:quarkus.kubernetes-service-binding.services.db-demo.api-version=postgres-operator.crunchydata.com/v1beta1 quarkus.kubernetes-service-binding.services.db-demo.kind=Database quarkus.kubernetes-service-binding.services.db-demo.name=my-db
-
For a semi-automatic service binding demonstration, see How to use Quarkus with the Service Binding Operator
-
List of List of bindable Operators
Automatic service binding
The quarkus-kubernetes-service-binding
extension can generate the ServiceBinding
resource automatically after detecting that an application requires access to the external services that are provided by available bindable Operators.
Automatic service binding can be generated for a limited number of service types. To be consistent with established terminology for Kubernetes and Quarkus services, this chapter refers to these service types as kinds. |
Operator |
API Version |
Kind |
|
|
postgres-operator.crunchydata.com/v1beta1 |
PostgresCluster |
|
|
pxc.percona.com/v1-9-0 |
PerconaXtraDBCluster |
|
|
psmdb.percona.com/v1-9-0 |
PerconaServerMongoDB |
Automatic datasource binding
For traditional databases, automatic binding is initiated whenever a datasource is configured as follows:
quarkus.datasource.db-kind=postgresql
The previous configuration, combined with the presence of quarkus-datasource
, quarkus-jdbc-postgresql
, quarkus-kubernetes
, and quarkus-kubernetes-service-binding
properties in the application, results in the generation of the ServiceBinding
resource for the the postgresql
database type.
By using the apiVersion
and kind
properties of the Operator resource, which matches the used postgresql
Operator, the generated ServiceBinding
resource binds the service or resource to the application.
When you do not specify a name for your database service, the the value of the db-kind
property is used as the default name.
services:
- apiVersion: postgres-operator.crunchydata.com/v1beta1
kind: PostgresCluster
name: postgresql
Specified the name of the datasource as follows:
quarkus.datasource.fruits-db.db-kind=postgresql
The service
in the generated ServiceBinding
then displays as follows:
services:
- apiVersion: postgres-operator.crunchydata.com/v1beta1
kind: PostgresCluster
name: fruits-db
Similarly, if you use mysql
, the name of the datasource can be specified as follows:
quarkus.datasource.fruits-db.db-kind=mysql
The generated service
contains the following:
services:
- apiVersion: pxc.percona.com/v1-9-0
kind: PerconaXtraDBCluster
name: fruits-db
Customizing Automatic Service Binding
Even though automatic binding was developed to eliminate as much manual configuration as possible, there are cases where modifying the generated ServiceBinding
resource might still be needed. The generation process exclusively relies on information extracted from the application and the knowledge of the supported Operators, which may not reflect what is deployed in the cluster. The generated resource is based purely on the knowledge of the supported bindable Operators for popular service kinds and a set of conventions that were developed to prevent possible mismatches, such as:
-
The target resource name does not match the datasource name
-
A specific Operator needs to be used rather than the default Operator for that service kind
-
Version conflicts that occur when a user needs to use any other version than default or latest
-
The target resource coordinates are determined based on the type of Operator and the kind of service.
-
The target resource name is set by default to match the service kind, such as
postgresql
,mysql
,mongo
. -
For named datasources, the name of the datasource is used.
-
For named
mongo
clients, the name of the client is used.
For cases in which you need to modify the generated ServiceBinding
to fix a name mismatch, use the quarkus.kubernetes-service-binding.services
properties and specify the service’s name as the service key.
The service key
is usually the name of the service, for example the name of the datasource, or the name of the mongo
client. When this value is not available, the datasource type, such as postgresql
, mysql
, mongo
, is used instead.
To avoid naming conflicts between different types of services, prefix the service key
with a specific datasource type, such as postgresql-<person>
.
The following example shows how to customize the apiVersion
property of the PostgresCluster
resource:
quarkus.datasource.db-kind=postgresql
quarkus.kubernetes-service-binding.services.postgresql.api-version=postgres-operator.crunchydata.com/v1beta2
In Example 1, the db-kind
(postgresql
) was used as a service key. In this example, because the datasource is named, according to convention, the datasource name (fruits-db
) is used instead.
The following example shows that for a named datasource, the datasource name is used as the name of the target resource:
quarkus.datasource.fruits-db.db-kind=postgresql
This has the same effect as the following configuration:
quarkus.kubernetes-service-binding.services.fruits-db.api-version=postgres-operator.crunchydata.com/v1beta1
quarkus.kubernetes-service-binding.services.fruits-db.kind=PostgresCluster
quarkus.kubernetes-service-binding.services.fruits-db.name=fruits-db
-
For more details about the available properties and how do they work, see the Workload Projection part of the Service Binding specification.