apache/pulsar-helm-chart
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Support mechanism to provide external zookeeper-server list to build global/configuration zookeeper (#269)

Browse Source 
* Support mechanism to provide external zookeeper-server list to build global/configuration zookeeper

* Add external zk example

* add external zk list into values.yaml

Fixes #268

### Motivation
Right now, [chart dynamically](https://github.com/apache/pulsar-helm-chart/blob/master/charts/pulsar/templates/zookeeper-statefulset.yaml#L140) creates zk cluster with zk pods initialized in the same namespace. However, for global/configuration zookeeper, user requires to build zk clusters with pods deployed in different namespaces. Therefore, user needs a mechanism to pass an external list of zk-servers to the chart and build zk-cluster with pods across different namespaces.

### Modification
- Chart should be considering zk-value's configuration for external zookeeper and generate zk-configuration file with appropriate zk-server list and unique id of that zookeeper.

This PR sets `ZOOKEEPER_SERVERS` value provided by user and also sets override-value flag which will be used by [generate-zookeeper-config.sh](https://github.com/apache/pulsar/blob/master/docker/pulsar/scripts/generate-zookeeper-config.sh) to override external zk list in config file and assign appropriate id to the host.

https://github.com/apache/pulsar/pull/15987 fixes [generate-zookeeper-config.sh](https://github.com/apache/pulsar/blob/master/docker/pulsar/scripts/generate-zookeeper-config.sh) changes.


### Result
- User can add `ZOOKEEPER_SERVERS` string into `zookeeper.configData` in [Values.yaml](https://github.com/apache/pulsar-helm-chart/blob/master/charts/pulsar/values.yaml#L385) file to override external zk-server list.
This commit is contained in:
Rajan Dhabalia 2022-10-18 15:41:43 -07:00 committed by GitHub
parent 1bcf255e12
commit 89f28bca9c
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
3 changed files with 18 additions and 4 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

17
charts/pulsar/templates/zookeeper-statefulset.yaml
View File

@ -138,10 +138,19 @@ spec:
containerPort: {{ .Values.zookeeper.ports.clientTls }}
{{- end }}
env:
- name: ZOOKEEPER_SERVERS
value:
{{- $global := . }}
{{ range $i, $e := until (.Values.zookeeper.replicaCount | int) }}{{ if ne $i 0 }},{{ end }}{{ template "pulsar.fullname" $global }}-{{ $global.Values.zookeeper.component }}-{{ printf "%d" $i }}{{ end }}
- name: ZOOKEEPER_SERVERS
{{- if .Values.zookeeper.configData.ZOOKEEPER_SERVERS }}
value: {{ .Values.zookeeper.configData.ZOOKEEPER_SERVERS }}
{{- else }}
{{- $global := . }}
value: {{ range $i, $e := until (.Values.zookeeper.replicaCount | int) }}{{ if ne $i 0 }},{{ end }}{{ template "pulsar.fullname" $global }}-{{ $global.Values.zookeeper.component }}-{{ printf "%d" $i }}{{ end }}
{{- end }}
- name: EXTERNAL_PROVIDED_SERVERS
{{- if .Values.zookeeper.configData.ZOOKEEPER_SERVERS }}
value: "true"
{{- else }}
value: "false"
{{- end }}
envFrom:
- configMapRef:
name: "{{ template "pulsar.fullname" . }}-{{ .Values.zookeeper.component }}"

2
charts/pulsar/values.yaml
View File

@ -397,6 +397,8 @@ zookeeper:
## templates/zookeeper-configmap.yaml
##
configData:
# External zookeeper server list in case of global-zk list to create zk cluster across zk deployed on different clusters/namespaces
# ZOOKEEPER_SERVERS: "us-east1-pulsar-zookeeper-0.us-east1-pulsar-zookeeper.us-east1.svc.cluster.local:2888:3888,us-east1-pulsar-zookeeper-1.us-east1-pulsar-zookeeper.us-east1.svc.cluster.local:2888:3888,us-east1-pulsar-zookeeper-2.us-east1-pulsar-zookeeper.us-east1.svc.cluster.local:2888:3888,us-west1-pulsar-zookeeper-0.us-west1-pulsar-zookeeper.us-west1.svc.cluster.local:2888:3888,us-west1-pulsar-zookeeper-1.us-west1-pulsar-zookeeper.us-west1.svc.cluster.local:2888:3888,us-west1-pulsar-zookeeper-2.us-west1-pulsar-zookeeper.us-west1.svc.cluster.local:2888:3888"
PULSAR_MEM: >
-Xms64m -Xmx128m
PULSAR_GC: >

3
examples/values-zookeeper-aws.yaml
View File

@ -45,6 +45,9 @@ monitoring:
node_exporter: false
zookeeper:
configData:
# External zookeeper server list in case of global-zk list to create zk cluster across zk deployed on different clusters/namespaces
# ZOOKEEPER_SERVERS: "us-east1-pulsar-zookeeper-0.us-east1-pulsar-zookeeper.us-east1.svc.cluster.local:2888:3888,us-east1-pulsar-zookeeper-1.us-east1-pulsar-zookeeper.us-east1.svc.cluster.local:2888:3888,us-east1-pulsar-zookeeper-2.us-east1-pulsar-zookeeper.us-east1.svc.cluster.local:2888:3888,us-west1-pulsar-zookeeper-0.us-west1-pulsar-zookeeper.us-west1.svc.cluster.local:2888:3888,us-west1-pulsar-zookeeper-1.us-west1-pulsar-zookeeper.us-west1.svc.cluster.local:2888:3888,us-west1-pulsar-zookeeper-2.us-west1-pulsar-zookeeper.us-west1.svc.cluster.local:2888:3888"
volumes:
# use a persistent volume or emptyDir
persistence: true