185 Commits

Author SHA1 Message Date
Lari Hotari
fdf9dd7757
Add -XX:+ExitOnOutOfMemoryError to Zookeeper's PULSAR_GC parameters in default values.yaml (#211) 2022-01-26 15:34:07 +02:00
Lari Hotari
22f4b9b3bd
Wrap Zookeeper probe script with timeout command (#214)
so that the probe doesn't continue running indefinitely

- resolves the issue with Kubernetes <1.20
  "Before Kubernetes 1.20, the field timeoutSeconds was not respected for exec probes:
    probes continued running indefinitely, even past their configured deadline,
    until a result was returned."
    in https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/#configure-probes

- #179 already fixed the issue for Kubernetes 1.20+
2022-01-26 15:17:15 +02:00
Lari Hotari
475a4b0b39
Remove references to tag: 2.6.0 in examples (#210)
### Motivation

It's better to not maintain out-dated examples referencing the 2.6.0 tag version.

### Modifications

- remove out-dated examples
2022-01-25 23:30:46 -06:00
Lari Hotari
fa9c22d895
Upgrade default images for Grafana & Pulsar Manager (#206)
- Grafana Dashboard image from v0.0.10 to v0.0.16
  - changes:
    https://github.com/streamnative/apache-pulsar-grafana-dashboard/compare/d50e2758...v0.0.16

- Pulsar Manager from v0.1.0 to v0.2.0
  - changes:
    https://github.com/apache/pulsar-manager/compare/v0.1.0...v0.2.0
2022-01-25 10:11:33 +02:00
Shen Liu
1b3e875ba2
Fix ci error caused by wrong block of if clause. (#208)
Co-authored-by: druidliu <druidliu@tencent.com>
2022-01-25 07:44:08 +02:00
Lari Hotari
40a4d50923
Upgrade cert-manager to 1.5.4 (#204) 2022-01-23 09:24:44 +02:00
Shen Liu
91f8b6f6b1
Add multi volume support in bookkeeper. (#113)
* Add multi volume support in bookkeeper. (#112)

* Add multi volumes support in bookkeeper configmap.

Co-authored-by: druidliu <druidliu@tencent.com>

Fixes #112 

### Motivation

*Add option for user to choose whether using multi volume in bookeeper, especially while using `local-storage`.*

### Modifications

Add `useMultiVolumes` option under `.Values.bookkeeper.volumes.journal` and `.Values.bookkeeper.volumes.ledgers`.
User can choose how many volumes could be used for bookkeeper jounal or ledgers.

### Verifying this change

- [x] Make sure that the change passes the CI checks.
2022-01-22 23:08:07 -06:00
Lari Hotari
90933d508d
[CI] Show status and logs for metadata job in CI logs (#207) 2022-01-20 16:03:28 +02:00
Lari Hotari
63cbdfe687
Increase default initialDelaySeconds for Zookeeper probes to workaround ZOOKEEPER-3988 (#202)
- When TLS is enabled for Zookeeper, NettyServerCnxnFactory will be used.
  It contains the issue https://github.com/apache/pulsar/issues/11070 /
  https://issues.apache.org/jira/browse/ZOOKEEPER-3988
  - as a workaround, increase the initialDelaySeconds from 10 to 20 to
    reduce the likely hood of ZOOKEEPER-3988
2022-01-18 18:38:29 +02:00
Lari Hotari
a27ec0aebf
Change default podManagementPolicy to Parallel for Zookeeper (#203) 2022-01-18 18:38:22 +02:00
Hang Chen
aea6a4f367
useV2WireProtocol for bookkeeper autorecovery (#165) 2022-01-18 09:06:26 +02:00
cogito-kyle
adbc6b7fcf
Add custom labels to all k8s objects in chart (#201) pulsar-2.7.10 2022-01-18 08:47:49 +02:00
Lari Hotari
7219a37065
Fix chart releaser command which uses docker image (#199) pulsar-2.7.9 2022-01-12 12:45:58 +02:00
csthomas1
ccf78f1c9d
Added -Dlog4j2.formatMsgNoLookups=true to PULSAR_MANAGER_OPTS (#198)
* Added -Dlog4j2.formatMsgNoLookups=true to PULSAR_MANAGER_OPTS

* Bump the chart version to release changes

Co-authored-by: Lari Hotari <lhotari@apache.org>
2022-01-12 10:42:43 +02:00
Lari Hotari
46689ab30d
[CI] Upgrade k8s to 1.18 and also upgrade helm, kind & chart releaser versions (#192) 2022-01-10 22:57:58 +02:00
Aaron Johnson
cee3b5c5e6
added additionalCommand parameter (#150)
Co-authored-by: Aaron Johnson <aaron.johnson@crowdstrike.com>
2022-01-05 10:26:55 -06:00
Frank Kelly
a919f309c6
Add ability to run extra commands in the initialization jobs e.g. to quit istio sidecars (#181) pulsar-2.7.8 2022-01-05 16:24:19 +02:00
shaoyue
41dd2f5034
Fix #175 cluster initialize blocked when fail (#176) 2022-01-05 16:10:09 +02:00
Valeriano Manassero
25e997a425
Automate initialize (#138)
- no need to do "--set initialize=true" anymore
2022-01-05 16:08:11 +02:00
matejhasul
706c8c292b
Workaround kustomize bug in pulsar cluster init (#166)
Replace folding block with multiline string to workaround https://github.com/kubernetes-sigs/kustomize/issues/4201

There are also other places where this bug is hit, but extra generated newline is not significant.

Co-authored-by: Lari Hotari <lhotari@users.noreply.github.com>
2022-01-04 11:08:52 -06:00
Lari Hotari
d3589b1c82
[CI] Improve logging in CI scripts and add timeouts (#195)
- show events every 15 seconds
- dump logs every 5 minutes
- dump logs when timeouting
2022-01-04 17:24:28 +02:00
Lari Hotari
9b672379dd
[CI] Improve change detection by reusing solution from apache/pulsar (#194)
* [CI] Improve change detection by reusing solution from apache/pulsar

* Fix verify release
2022-01-04 11:40:40 +02:00
Shu.Wang
83bb8bd60f
Conditionally update ingress api version based on k8s version (#183)
* Update ingress api version, extension/v1beta1 will not be supported in new k8s version, this change keep backward compatibility for lower kubernetes version

* Update deprecated util Capabilities.KubeVersion.GitVersion to Capabilities.KubeVersion.Version
2022-01-04 00:53:21 -06:00
Shu.Wang
0a82ab0f9a
Fixes #177 Fix indentation of component, as it should be under the label tag (#182) 2022-01-03 21:57:45 +02:00
Lari Hotari
d74d08a89d
Use NIOServerCnxnFactory for Zookeeper to fix NPE issues with Pulsar 2.8.x+ (#180)
- follow recommendation in https://github.com/apache/pulsar/issues/11070#issuecomment-936539979
2022-01-03 11:59:58 +01:00
Lari Hotari
567bd7c5b3
[CI] Add job 45min timeout and cancel duplicate jobs (#193) 2022-01-03 10:17:21 +01:00
Jiwei Guo
0f6dea8022
Bump to Pulsar 2.7.4 (#189)
* Bump to Pulsar 2.7.4

* update

* update
pulsar-2.7.7
2021-12-30 08:55:57 +02:00
Lari Hotari
b4b2fa7b80
[Security] Workaround for CVE-2021-44228 Log4J RCE when Log4J >= 2.10.0 (#186)
* [Security] Workaround for CVE-2021-44228 Log4J RCE when Log4J >= 2.10.0

- prevents the exploit by disabling message pattern lookups

* Bump the chart version
pulsar-2.7.6
2021-12-10 18:30:01 +02:00
Lari Hotari
a16c6bbf19
Make k8s probe timeoutSeconds configurable and set default to 5s for k8s 1.20+ compatibility (#179)
- set to 5 seconds by default

- address compatibility with Kubernetes 1.20+. This impacts "bin/pulsar-zookeeper-ruok.sh" exec probe used in ZK.
  "Before Kubernetes 1.20, the field timeoutSeconds was not respected for exec probes: probes continued running indefinitely, even past their configured deadline, until a result was returned."
   https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/#configure-probes
2021-11-25 08:46:42 +01:00
Frank Kelly
1956a870ff
Fixes #173 Support both Role Binding and Cluster Role Binding dependi… (#174)
* Fixes #173 Support both Role Binding and Cluster Role Binding depending on rbac.limit_to_namespace

* Rev version

* Get Role/Cluster the right way around
pulsar-2.7.5
2021-11-12 07:56:35 -08:00
Frank Kelly
617308147d
Missing fix for #152. Bookie Service also needs the prefix on the port name (#172)
Fixes #158 (This is the second PR - see also https://github.com/apache/pulsar-helm-chart/pull/162)

### Motivation

* All non-standard port-names need a proper protocol prefix to support Istio
 https://istio.io/latest/docs/ops/configuration/traffic-management/protocol-selection/#explicit-protocol-selection
 
### Modifications

Add the prefix value before `bookie`
pulsar-2.7.4
2021-11-09 09:18:26 -08:00
Frank Kelly
5b10f48f5b
Fix #152 Add Helm chart support for Istio port naming (attempt 2) (#162)
Fixes #152 

### Motivation

Support prefix in front of port names to abide by Istio protocol rules
https://istio.io/latest/docs/ops/configuration/traffic-management/protocol-selection/#explicit-protocol-selection

### Modifications

Support adding a prefix
- pulsar -> tcp-pulsar
- pulsarssl --> tls-pulsarssl etc
pulsar-2.7.3
2021-09-10 08:56:16 +08:00
Peter Tinti
f307cc32af
updates pulsar ca name generation to use suffix making cert swappable (#141)
Updates CA name generation to be configurable allowing the swapping in of a CA.

### Motivation

We recently swapped out cert issuers and found that with the current helm chart we were unable to do a hot swap without downtime (via helm) because the CA cert name is not configurable. Being able to change the name of the CA allows us to create a new CA first -> Validate -> then swap over in follow up apply/release.

### Modifications

Adds the ability to specify the suffix used to generate the CA name (not the whole name in order to preserve back compatibility regardless of the release name.)
2021-08-25 23:14:03 -07:00
Frank Kelly
65dc68654b
ZooKeeper HTTP port should be exposed by service so we can use prometheus (#143)
Fixes #142 

### Motivation

Expose HTTP Port on ZooKeeper service so we can use Prometheus

### Modifications

Bug fix to expose HTTP port on ZooKeeper service
2021-08-25 23:13:47 -07:00
Aaron Johnson
c45813ffe5
added extraVolumes and extraVolumeMounts (#149)
Fixes #147

### Motivation
This gives the helm chart user the ability to specify a secret or other type of volume to be mounted into any of the statefulset pods

### Modifications
* Added conditionals to `bookkeeper`, `broker`, `proxy`, `toolset`, and `zookeeper` statefulsets which allow the chart user to specify extraVolumes and extraVolumeMounts for deployed pods.
* Added `extraVolumes` and `extraVolumeMounts` parameters to values.yaml
2021-08-25 23:13:27 -07:00
Thomas O'Neill
19d6ce6488
Add Support for imagePullSecrets (#140)
Fixes #125

### Motivation

The default images in the values.yaml are in docker hub. This PR allows us to provide image pull secrets for the containers which will allow us to get around Docker Hub's rate limiting if the nodes are not logged into Docker Hub.

### Modifications

Added a new template to generate `imagePullSecrets`, and included them in the deployments and statefulsets. This will only add them if they are specified under `images.imagePullSecrets`

### Verifying this change

- [] Make sure that the change passes the CI checks.
2021-08-20 17:22:50 -07:00
Lari Hotari
c3e4ea272b
Fix deprecation warning about rbac.authorization.k8s.io/v1beta1 (#135) 2021-07-03 10:56:58 +03:00
Florian Knip
f00c66a14a
Feature/fix typo in readme (#129)
Fixes #128 

### Motivation

Improve flow when reading the docs

### Modifications

fixed a typo in the docs
2021-06-23 21:13:41 -07:00
TC-robV
75169707fb
add enableAdminApi for prometheus (#121)
Fixes #<xyz>

### Motivation

would be nice to have this option here so people can run admin commands against the prometheus. 

### Modifications

added a new value and modified the deployment, taken from the official prom helm.

### Verifying this change

- [ ] Make sure that the change passes the CI checks.
2021-06-23 21:12:20 -07:00
MMeent
11a1d578dd
Fix indentation issue on checksum/config (#117)
Fixes #116

### Motivation

Theres indentation issues for the `checksum/config` annotation in these templates, which would either fail linting or not apply at all in some situations.

### Modifications

I've added indentation at the specified places such that this isn't an issue anymore.

### Verifying this change

- [ ] Make sure that the change passes the CI checks.
2021-06-23 21:11:38 -07:00
Peter Tinti
d6d240a123
Updates internal issuer cert to include duration and renew configs (#131)
### Motivation
* While component certs can be configured with a custom duration the CA cert for self-signed configuration uses default values. It can be convenient to have this certificate expire more than a month out.

### Modifications
* Updates the internal issuer `{{ .Release.Name }}-ca-tls` certificate to make `duration` and `renewBefore` configurable. Does not use `common` so that the CA can be configured to last much longer than individual components certs if desired.

### Verifying this change
- [x] Make sure that the change passes the CI checks.
2021-06-23 21:00:17 -07:00
Enrico Olivelli
6d0db35216
Update to Pulsar 2.7.2 (#119)
Co-authored-by: Enrico Olivelli <eolivelli@datastax.com>
pulsar-2.7.2
2021-06-03 11:31:47 +03:00
Yong Zhang
0816ac2dfd
Reduce the TLS common name length (#115)
---

*Motivation*

Reduce the TLS command name to avoid getting a too long name
that could not generate a certificate.
2021-04-23 12:43:44 +08:00
Jean Helou
ba356e5df7
makes cert-manager apiVersion configurable (#107)
This commit let's users override the apiVersion referenced in this
chart so that the chart can be used with newer cert-manager releases.
(script/cert-manager/install-cert-manager.sh installs 0.13.0 when
current version is 1.2.0...)

Fixes #68

### Motivation

cert-manager apiVersion changed after cert-manager 1.0.0 was released, which prevents the chart from provisionning certificates with newer cert-manager installation because of an incompatible apiVersion.

I have a cluster with cert-manager >1.0.0 installed, making `apiVersion` overridable makes it easy for me to install pulsar on that cluster

### Modifications

I introduced the value `certs.internal_issuer.apiVersion`, which by default uses the apiVersion that was previously hardcoded (`cert-manager.io/v1alpha2`) 
I replaced all occurrences of that apiVersion by a reference to the value so that users can override it to `cert-manager.io/v1` if they have a newer version of cert-manager installed.

### Verifying this change

- [x] Make sure that the change passes the CI checks.
2021-03-16 00:44:38 -07:00
Yong Zhang
e0903c633c
Bump pulsar version to 2.7.1 (#109)
### Motivation

Release with pulsar 2.7.1

### Modification

- update pulsar version from 2.7.0 to 2.7.1
- add a script for updating the pulsar version
pulsar-2.7.1
2021-03-16 00:43:30 -07:00
Miecio
c059ea25d8
Feat: Dynamic superusers configuration (#104)
Adds dynamic superusers configuration

### Motivation

Allow dynamic superusers management. Adding new superuser entry to `.Values.auth.superUsers` will results in adding concatenated list to config

### Modifications

Change static list to dynamic one
2021-02-09 00:59:54 -08:00
wuYin
67818a48cb
Support common volume for journal and ledgers (#93)
### Motivation

In some case, my k8s node only have 1 large capacity ssd, for deploying 1 bookie, I need:

- Partition the ssd into 2 disks, and make 2 pv over it.
- Just make 1 pv over it, but journal & ledgers under same mount path (this PR did)

Both can't isolate IO for journal & ledgers, so I prefer the second one for reusability.


### Modifications

values.yaml
  - add `useSingleCommonVolume` option, default false

bookkeeper-statefulset.yaml
   - mount the only PV to path `/pulsar/data/bookkeeper`
   - use configured common storageClassName

bookkeeper-storageclass.yaml
  - use configured provisioner for the common storageClass 

### Others
This may not be an issue for everyone, if it's not necessary to merge, I'll just use it locally

### Verifying this change

- [x] Make sure that the change passes the CI checks.
2021-01-30 09:28:45 -08:00
wangyufan
d73361eb1e
fix broker configmap forbidden (#95)
Fixes #94

### Motivation

fix `io.kubernetes.client.openapi.ApiException: Forbidden`

### Modifications

fix typo

### Verifying this change

- [x] Make sure that the change passes the CI checks.
2021-01-30 09:28:00 -08:00
Miecio
b24ba1adf5
Fix namespace handling and missing dnsNames (#99)
Fixes for wrong namespace handling in some RBAC and missing dnsNames for TLS

### Motivation

Fixes old unused handling of namespace name in RBAC for autorecovery and bookkeeper.
Fixes Helm exception of missing key when not defining TLS dnsNames

### Modifications

Use namespace template in RBAC definitions for bookkeeper and autorecovery. Add if around every `toYaml .Values.tls.bookie.dnsNames` clause in TLS certs definitions.

### Verifying this change

- [x] Make sure that the change passes the CI checks.
2021-01-30 09:27:18 -08:00
Miecio
025b263206
Extend podmonitor and add relabels (#100)
### Motivation

As I wanted to use [streamnative/apache-pulsar-grafana-dashboard](https://github.com/streamnative/apache-pulsar-grafana-dashboard) with this helm chart and own cluster wide Prometheus stack I decided that use of PodMonitor CRD is a good way. Unfortunately prometheus config has some metrics relabelings that are required by grafana dashboard. I decied to port them directly to PodMonitor definition

### Modifications

* Added missing PodMonitor for autorecovery
* Port relabelings from `prometheus-configmap.yaml` to each PodMonitor

### Verifying this change

- [x] Make sure that the change passes the CI checks.
2021-01-30 09:24:21 -08:00