Skip to content

Commit 3b1d1f6

Browse files
authored
Add volumes and volumeMounts values to the Helm chart for file-based credentials (#62)
1 parent 0e89db0 commit 3b1d1f6

6 files changed

Lines changed: 395 additions & 48 deletions

File tree

.gitignore

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,7 @@
11
### PostgreSQL Operator ###
22
config/
3+
# Written by the fabric8 Kubernetes client when the tests run against the Dev Service
4+
operator/.kube/
35

46
### STS ###
57
.apt_generated

docs/cluster-connection.md

Lines changed: 128 additions & 38 deletions
Original file line numberDiff line numberDiff line change
@@ -7,14 +7,14 @@ Other Custom Resources (like `Database`, `Role`, `Schema`, `Grant`, `DefaultPriv
77

88
## Spec
99

10-
| Field | Type | Description | Required | Mutable |
11-
|----------------------|----------------------|-----------------------------------------------------------------------|----------|---------|
12-
| `host` | `string` | The hostname of the PostgreSQL instance. | Yes | Yes |
13-
| `port` | `integer` | The port of the PostgreSQL instance (1-65535). | Yes | Yes |
14-
| `database` | `string` | The database to connect to (usually `postgres` for admin operations). | Yes | Yes |
15-
| `adminSecretRef` | `ResourceRef` | Reference to the Kubernetes Secret containing the admin credentials. | No | Yes |
16-
| `adminSecretFileRef` | `FileRef` | Reference to a file containing the admin credentials. | No | Yes |
17-
| `parameters` | `map[string]string` | Additional connection parameters. | No | Yes |
10+
| Field | Type | Description | Required | Mutable |
11+
|----------------------|---------------------|-----------------------------------------------------------------------|----------|---------|
12+
| `host` | `string` | The hostname of the PostgreSQL instance. | Yes | Yes |
13+
| `port` | `integer` | The port of the PostgreSQL instance (1-65535). | Yes | Yes |
14+
| `database` | `string` | The database to connect to (usually `postgres` for admin operations). | Yes | Yes |
15+
| `adminSecretRef` | `ResourceRef` | Reference to the Kubernetes Secret containing the admin credentials. | No | Yes |
16+
| `adminSecretFileRef` | `FileRef` | Reference to a file containing the admin credentials. | No | Yes |
17+
| `parameters` | `map[string]string` | Additional connection parameters. | No | Yes |
1818

1919
> **Note:** Exactly one of `adminSecretRef` or `adminSecretFileRef` must be provided.
2020
@@ -29,11 +29,12 @@ The referenced secret must be of type `kubernetes.io/basic-auth` and contain the
2929

3030
### FileRef (`adminSecretFileRef`)
3131

32-
| Field | Type | Description | Required |
33-
|--------|----------|----------------------------------------------------------------|----------|
34-
| `path` | `string` | The path to the file containing the admin credentials. | Yes |
32+
Use this option when the credentials should be mounted as a file inside the operator Pod instead of reading a Kubernetes Secret directly.
33+
34+
| Field | Type | Description | Required |
35+
|--------|----------|-----------------------------------------------------------------------------------------|----------|
36+
| `path` | `string` | The absolute path inside the operator Pod to the file containing the admin credentials. | Yes |
3537

36-
Use this option when the credentials are mounted as a file instead of a Kubernetes Secret.
3738

3839
#### File format
3940

@@ -51,35 +52,23 @@ The file must contain JSON with the following fields:
5152

5253
#### Mount the credentials file
5354

54-
The file must be accessible inside the operator pod at the path specified in `adminSecretFileRef.path`. Mount it using a Volume and VolumeMount on the operator Deployment:
55+
The file must be accessible inside the operator Pod at the path in `adminSecretFileRef.path`.
5556

56-
```yaml
57-
apiVersion: apps/v1
58-
kind: Deployment
59-
metadata:
60-
name: postgresql-operator
61-
spec:
62-
template:
63-
spec:
64-
containers:
65-
- name: postgresql-operator
66-
volumeMounts:
67-
- name: db-credentials
68-
mountPath: /mnt/secrets
69-
readOnly: true
70-
volumes:
71-
- name: db-credentials
72-
secret:
73-
secretName: db-credentials-secret
74-
```
57+
The Helm chart exposes the `app.volumes` and `app.volumeMounts` values for this.
58+
Both take the raw Kubernetes syntax, so any volume source that provides a file works.
7559

76-
> **Note:** The volume source can be any type that provides a file.
60+
The value of `adminSecretFileRef.path` is the `mountPath` plus the name of the file. The volume source decides the file name:
7761

78-
> **Note:** The Helm chart does not support extra volumes yet.
62+
| Volume source | The file name comes from |
63+
|----------------------------------|---------------------------------|
64+
| `secret` | the key of the Secret |
65+
| `csi` (Secrets Store CSI driver) | the `objectAlias` of the object |
7966

80-
### Examples
67+
See [Using a file reference](#using-a-file-reference-adminsecretfileref) in the examples for a complete setup with each volume source.
8168

82-
#### Using a Kubernetes Secret (`adminSecretRef`)
69+
## Examples
70+
71+
### Using a Kubernetes Secret (`adminSecretRef`)
8372

8473
```yaml
8574
apiVersion: v1
@@ -110,18 +99,119 @@ spec:
11099
#connectTimeout: "10" # Timeout in seconds for connection attempts
111100
```
112101

113-
#### Using a file reference (`adminSecretFileRef`)
102+
### Using a file reference (`adminSecretFileRef`)
114103

115104
```yaml
116105
apiVersion: postgresql.aboutbits.it/v1
117106
kind: ClusterConnection
118107
metadata:
119-
name: quarkus-postgres-connection
108+
name: my-postgres-connection
120109
spec:
121110
adminSecretFileRef:
122111
path: "/mnt/secrets/db-credentials.json"
123112
host: localhost
124113
port: 5432
125114
database: postgres
115+
# Example parameters
116+
parameters:
117+
ApplicationName: "k8s-operator" # Helps identify this connection in Postgres logs
118+
#sslmode: "require" # Enforce SSL encryption
119+
#connectTimeout: "10" # Timeout in seconds for connection attempts
126120
```
127121

122+
The mount that creates `/mnt/secrets/db-credentials.json` depends on the volume source.
123+
124+
#### From a Secret volume
125+
126+
Create the Secret. Its key becomes the file name:
127+
128+
```yaml
129+
apiVersion: v1
130+
kind: Secret
131+
metadata:
132+
name: db-credentials-secret
133+
stringData:
134+
db-credentials.json: |
135+
{
136+
"username": "root",
137+
"password": "password"
138+
}
139+
```
140+
141+
Then mount it through the chart values:
142+
143+
```yaml
144+
app:
145+
volumes:
146+
- name: db-credentials
147+
secret:
148+
secretName: db-credentials-secret
149+
volumeMounts:
150+
- name: db-credentials
151+
mountPath: /mnt/secrets
152+
readOnly: true
153+
```
154+
155+
#### From the Secrets Store CSI driver
156+
157+
Use this option to read the credentials from an external secret store, for example AWS Secrets Manager.
158+
159+
> **Note:** Install the [Secrets Store CSI driver](https://secrets-store-csi-driver.sigs.k8s.io/getting-started/installation) and the [provider](https://secrets-store-csi-driver.sigs.k8s.io/providers) for your secret store first. Neither the operator nor the chart installs them. Without the driver, the operator Pod stays in `ContainerCreating` and reports a failed mount.
160+
161+
The chart does not create the `SecretProviderClass`, so you have to apply it yourself. Its `objectAlias` becomes the file name:
162+
163+
```yaml
164+
apiVersion: secrets-store.csi.x-k8s.io/v1
165+
kind: SecretProviderClass
166+
metadata:
167+
name: db-credentials
168+
spec:
169+
provider: aws
170+
parameters:
171+
objects: |
172+
- objectName: "my/db/credentials"
173+
objectAlias: "db-credentials.json"
174+
```
175+
176+
> **Note:** The `SecretProviderClass` must live in the namespace of the operator.
177+
178+
Then mount it through the chart values:
179+
180+
```yaml
181+
app:
182+
volumes:
183+
- name: db-credentials
184+
csi:
185+
driver: secrets-store.csi.k8s.io
186+
readOnly: true
187+
volumeAttributes:
188+
secretProviderClass: db-credentials
189+
volumeMounts:
190+
- name: db-credentials
191+
mountPath: /mnt/secrets
192+
readOnly: true
193+
```
194+
195+
#### Without the Helm chart
196+
197+
If you deploy the operator directly from the OCI image, set the same `volumes` and `volumeMounts` fields on the Deployment:
198+
199+
```yaml
200+
apiVersion: apps/v1
201+
kind: Deployment
202+
metadata:
203+
name: postgresql-operator
204+
spec:
205+
template:
206+
spec:
207+
containers:
208+
- name: postgresql-operator
209+
volumeMounts:
210+
- name: db-credentials
211+
mountPath: /mnt/secrets
212+
readOnly: true
213+
volumes:
214+
- name: db-credentials
215+
secret:
216+
secretName: db-credentials-secret
217+
```

operator/src/main/helm/values.yaml

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
---
2+
# This file overrides the default values that the quarkus-helm extension generates.
3+
#
4+
# Why it exists: a list field of the operator Deployment becomes a Helm value only if the key
5+
# already exists in `src/main/kubernetes/kubernetes.yml`. An empty list `[]` does not survive
6+
# there. The fabric8 model marks `PodSpec.imagePullSecrets`, `PodSpec.volumes` and
7+
# `Container.volumeMounts` with `@JsonInclude(NON_EMPTY)`. A list with one null element does
8+
# survive, but the generated default is then unusable, so this file replaces it with a real
9+
# empty list.
10+
#
11+
# The unusable default takes one of two shapes, and the path of the value decides which:
12+
# - A plain path, such as `spec.template.spec.volumes`, produces `- {}`, a list that holds
13+
# one empty object. A user who copies that default and appends an entry gets invalid YAML.
14+
# - A container-filtered path, such as
15+
# `spec.template.spec.containers.(name == postgresql-operator).volumeMounts`, produces
16+
# `{}`, an object. That shape also contradicts the `type: array` of `values.schema.json`.
17+
#
18+
# See https://github.com/quarkiverse/quarkus-helm/issues/453
19+
app:
20+
imagePullSecrets: []
21+
volumes: []
22+
volumeMounts: []
Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,19 @@
11
---
2+
# See https://quarkus.io/guides/deploying-to-kubernetes#using-existing-resources
23
apiVersion: apps/v1
34
kind: Deployment
45
metadata:
6+
# The name must match `quarkus.kubernetes.name`, otherwise Dekorate adds a second Deployment.
7+
# `HelmTest` has a test that makes sure this never drifts apart.
58
name: postgresql-operator
69
spec:
710
template:
811
spec:
912
affinity: {}
13+
# The `[~]` placeholders are required, see operator/src/main/helm/values.yaml for the reason.
1014
imagePullSecrets: [~]
15+
volumes: [~]
16+
containers:
17+
# The name must match `quarkus.kubernetes.name`, otherwise Dekorate adds a second container.
18+
- name: postgresql-operator
19+
volumeMounts: [~]

operator/src/main/resources/application.yml

Lines changed: 28 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -86,12 +86,9 @@ quarkus:
8686
- (kind == Deployment).spec.template.spec.containers.(name == ${quarkus.kubernetes.name}).imagePullPolicy
8787
image-pull-secrets:
8888
property: imagePullSecrets
89-
value:
90-
- null
9189
paths:
9290
- (kind == Deployment).spec.template.spec.imagePullSecrets
93-
expression: "{{- if eq (toYaml .Values.app.imagePullSecrets | trim) \"- {}\" }} null{{- else }}{{- toYaml .Values.app.imagePullSecrets | nindent 8 }}{{- end }}"
94-
description: Kubernetes image pull secrets to use if the OCI image is hosted on a private registry
91+
expression: "{{- toYaml (.Values.app.imagePullSecrets | default list) | nindent 8 }}"
9592
resource-requests-cpu:
9693
property: resources.requests.cpu
9794
value: ${quarkus.kubernetes.resources.requests.cpu}
@@ -113,6 +110,16 @@ quarkus:
113110
paths:
114111
- (kind == Deployment).spec.template.spec.affinity
115112
description: Kubernetes affinity configuration for Pod scheduling
113+
volumes:
114+
property: volumes
115+
paths:
116+
- (kind == Deployment).spec.template.spec.volumes
117+
expression: "{{- toYaml (.Values.app.volumes | default list) | nindent 8 }}"
118+
volume-mounts:
119+
property: volumeMounts
120+
paths:
121+
- (kind == Deployment).spec.template.spec.containers.(name == ${quarkus.kubernetes.name}).volumeMounts
122+
expression: "{{- toYaml (.Values.app.volumeMounts | default list) | nindent 12 }}"
116123
console-color:
117124
property: envs.QUARKUS_CONSOLE_COLOR
118125
value-as-bool: ${quarkus.console.color}
@@ -127,9 +134,26 @@ quarkus:
127134
description: Specify the format of the produced JSON. Supported values are "DEFAULT", "ECS", and "GCP".
128135
values-schema:
129136
properties:
137+
# The type must be set explicitly for every non-scalar value, because the generated
138+
# schema otherwise falls back to `string`.
139+
#
140+
# A value that `src/main/helm/values.yaml` provides also loses the `description` of its
141+
# `quarkus.helm.values` entry, so the description belongs here instead.
130142
"affinity":
131143
name: app.affinity
132144
type: object
145+
"imagePullSecrets":
146+
name: app.imagePullSecrets
147+
type: array
148+
description: Kubernetes image pull secrets to use if the OCI image is hosted on a private registry
149+
"volumes":
150+
name: app.volumes
151+
type: array
152+
description: Additional volumes for the operator Pod, for example a Secret volume or a Secrets Store CSI volume
153+
"volumeMounts":
154+
name: app.volumeMounts
155+
type: array
156+
description: Additional volume mounts for the operator container
133157
expressions:
134158
release-name-labels:
135159
expression: "{{ .Release.Name }}"

0 commit comments

Comments
 (0)