292 Commits

Author SHA1 Message Date
Frank Kelly
9613ee0292
Make PodSecurityPolicy name unique in k8s cluster when rbac.limit_to_namespace is true (#224)
- allows having multiple Pulsar clusters in different K8S namespaces but having the same helm release name
  - PodSecurityPolicy is a cluster-level-resource and name would collide without this change
pulsar-2.7.11
2022-02-04 10:41:10 +02:00
Lari Hotari
dd0e6d827d
Increase Zookeeper probe timeouts (#220)
- 5 seconds seems to be a too short probe timeout on a system with low resources such as in CI
2022-01-31 19:24:19 +02:00
Lari Hotari
dc97bd4ac6
[CI] Tolerate errors when collecting k8s logs in CI (#217)
- The log collection failed after a command failed.
- tolerate errors
2022-01-26 14:50:48 -06:00
Lari Hotari
d3e7a7e6c9
[CI] Fix issue with k8s log collection (#216)
- slash needs to be replaced with underscore
2022-01-26 20:49:06 +02:00
Lari Hotari
0093f91410
[CI] Collect and upload k8s logs on failure (#215) 2022-01-26 19:43:49 +02:00
MMeent
c0a8c1b97f
Use the 'pulsar.matchLabels' template for matching components of this chart. (#118)
This also limits the scope of the PodMonitors to the resources of only this install, instead of all installs that share `component:` label values.

Co-authored-by: Matthias van de Meent <matthias.vandemeent@cofano.nl>
2022-01-26 15:38:52 +02:00
Lari Hotari
41ff20ec5e
Don't enable pulsar manager by default (#213)
- because of security reasons
  - it increases the attack surface
- it's an unnecessary feature for most users
  - wasted resource consumption
2022-01-26 15:34:30 +02:00
Lari Hotari
fdf9dd7757
Add -XX:+ExitOnOutOfMemoryError to Zookeeper's PULSAR_GC parameters in default values.yaml (#211) 2022-01-26 15:34:07 +02:00
Lari Hotari
22f4b9b3bd
Wrap Zookeeper probe script with timeout command (#214)
so that the probe doesn't continue running indefinitely

- resolves the issue with Kubernetes <1.20
  "Before Kubernetes 1.20, the field timeoutSeconds was not respected for exec probes:
    probes continued running indefinitely, even past their configured deadline,
    until a result was returned."
    in https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/#configure-probes

- #179 already fixed the issue for Kubernetes 1.20+
2022-01-26 15:17:15 +02:00
Lari Hotari
475a4b0b39
Remove references to tag: 2.6.0 in examples (#210)
### Motivation

It's better to not maintain out-dated examples referencing the 2.6.0 tag version.

### Modifications

- remove out-dated examples
2022-01-25 23:30:46 -06:00
Lari Hotari
fa9c22d895
Upgrade default images for Grafana & Pulsar Manager (#206)
- Grafana Dashboard image from v0.0.10 to v0.0.16
  - changes:
    https://github.com/streamnative/apache-pulsar-grafana-dashboard/compare/d50e2758...v0.0.16

- Pulsar Manager from v0.1.0 to v0.2.0
  - changes:
    https://github.com/apache/pulsar-manager/compare/v0.1.0...v0.2.0
2022-01-25 10:11:33 +02:00
Shen Liu
1b3e875ba2
Fix ci error caused by wrong block of if clause. (#208)
Co-authored-by: druidliu <druidliu@tencent.com>
2022-01-25 07:44:08 +02:00
Lari Hotari
40a4d50923
Upgrade cert-manager to 1.5.4 (#204) 2022-01-23 09:24:44 +02:00
Shen Liu
91f8b6f6b1
Add multi volume support in bookkeeper. (#113)
* Add multi volume support in bookkeeper. (#112)

* Add multi volumes support in bookkeeper configmap.

Co-authored-by: druidliu <druidliu@tencent.com>

Fixes #112 

### Motivation

*Add option for user to choose whether using multi volume in bookeeper, especially while using `local-storage`.*

### Modifications

Add `useMultiVolumes` option under `.Values.bookkeeper.volumes.journal` and `.Values.bookkeeper.volumes.ledgers`.
User can choose how many volumes could be used for bookkeeper jounal or ledgers.

### Verifying this change

- [x] Make sure that the change passes the CI checks.
2022-01-22 23:08:07 -06:00
Lari Hotari
90933d508d
[CI] Show status and logs for metadata job in CI logs (#207) 2022-01-20 16:03:28 +02:00
Lari Hotari
63cbdfe687
Increase default initialDelaySeconds for Zookeeper probes to workaround ZOOKEEPER-3988 (#202)
- When TLS is enabled for Zookeeper, NettyServerCnxnFactory will be used.
  It contains the issue https://github.com/apache/pulsar/issues/11070 /
  https://issues.apache.org/jira/browse/ZOOKEEPER-3988
  - as a workaround, increase the initialDelaySeconds from 10 to 20 to
    reduce the likely hood of ZOOKEEPER-3988
2022-01-18 18:38:29 +02:00
Lari Hotari
a27ec0aebf
Change default podManagementPolicy to Parallel for Zookeeper (#203) 2022-01-18 18:38:22 +02:00
Hang Chen
aea6a4f367
useV2WireProtocol for bookkeeper autorecovery (#165) 2022-01-18 09:06:26 +02:00
cogito-kyle
adbc6b7fcf
Add custom labels to all k8s objects in chart (#201) pulsar-2.7.10 2022-01-18 08:47:49 +02:00
Lari Hotari
7219a37065
Fix chart releaser command which uses docker image (#199) pulsar-2.7.9 2022-01-12 12:45:58 +02:00
csthomas1
ccf78f1c9d
Added -Dlog4j2.formatMsgNoLookups=true to PULSAR_MANAGER_OPTS (#198)
* Added -Dlog4j2.formatMsgNoLookups=true to PULSAR_MANAGER_OPTS

* Bump the chart version to release changes

Co-authored-by: Lari Hotari <lhotari@apache.org>
2022-01-12 10:42:43 +02:00
Lari Hotari
46689ab30d
[CI] Upgrade k8s to 1.18 and also upgrade helm, kind & chart releaser versions (#192) 2022-01-10 22:57:58 +02:00
Aaron Johnson
cee3b5c5e6
added additionalCommand parameter (#150)
Co-authored-by: Aaron Johnson <aaron.johnson@crowdstrike.com>
2022-01-05 10:26:55 -06:00
Frank Kelly
a919f309c6
Add ability to run extra commands in the initialization jobs e.g. to quit istio sidecars (#181) pulsar-2.7.8 2022-01-05 16:24:19 +02:00
shaoyue
41dd2f5034
Fix #175 cluster initialize blocked when fail (#176) 2022-01-05 16:10:09 +02:00
Valeriano Manassero
25e997a425
Automate initialize (#138)
- no need to do "--set initialize=true" anymore
2022-01-05 16:08:11 +02:00
matejhasul
706c8c292b
Workaround kustomize bug in pulsar cluster init (#166)
Replace folding block with multiline string to workaround https://github.com/kubernetes-sigs/kustomize/issues/4201

There are also other places where this bug is hit, but extra generated newline is not significant.

Co-authored-by: Lari Hotari <lhotari@users.noreply.github.com>
2022-01-04 11:08:52 -06:00
Lari Hotari
d3589b1c82
[CI] Improve logging in CI scripts and add timeouts (#195)
- show events every 15 seconds
- dump logs every 5 minutes
- dump logs when timeouting
2022-01-04 17:24:28 +02:00
Lari Hotari
9b672379dd
[CI] Improve change detection by reusing solution from apache/pulsar (#194)
* [CI] Improve change detection by reusing solution from apache/pulsar

* Fix verify release
2022-01-04 11:40:40 +02:00
Shu.Wang
83bb8bd60f
Conditionally update ingress api version based on k8s version (#183)
* Update ingress api version, extension/v1beta1 will not be supported in new k8s version, this change keep backward compatibility for lower kubernetes version

* Update deprecated util Capabilities.KubeVersion.GitVersion to Capabilities.KubeVersion.Version
2022-01-04 00:53:21 -06:00
Shu.Wang
0a82ab0f9a
Fixes #177 Fix indentation of component, as it should be under the label tag (#182) 2022-01-03 21:57:45 +02:00
Lari Hotari
d74d08a89d
Use NIOServerCnxnFactory for Zookeeper to fix NPE issues with Pulsar 2.8.x+ (#180)
- follow recommendation in https://github.com/apache/pulsar/issues/11070#issuecomment-936539979
2022-01-03 11:59:58 +01:00
Lari Hotari
567bd7c5b3
[CI] Add job 45min timeout and cancel duplicate jobs (#193) 2022-01-03 10:17:21 +01:00
Jiwei Guo
0f6dea8022
Bump to Pulsar 2.7.4 (#189)
* Bump to Pulsar 2.7.4

* update

* update
pulsar-2.7.7
2021-12-30 08:55:57 +02:00
Lari Hotari
b4b2fa7b80
[Security] Workaround for CVE-2021-44228 Log4J RCE when Log4J >= 2.10.0 (#186)
* [Security] Workaround for CVE-2021-44228 Log4J RCE when Log4J >= 2.10.0

- prevents the exploit by disabling message pattern lookups

* Bump the chart version
pulsar-2.7.6
2021-12-10 18:30:01 +02:00
Lari Hotari
a16c6bbf19
Make k8s probe timeoutSeconds configurable and set default to 5s for k8s 1.20+ compatibility (#179)
- set to 5 seconds by default

- address compatibility with Kubernetes 1.20+. This impacts "bin/pulsar-zookeeper-ruok.sh" exec probe used in ZK.
  "Before Kubernetes 1.20, the field timeoutSeconds was not respected for exec probes: probes continued running indefinitely, even past their configured deadline, until a result was returned."
   https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/#configure-probes
2021-11-25 08:46:42 +01:00
Frank Kelly
1956a870ff
Fixes #173 Support both Role Binding and Cluster Role Binding dependi… (#174)
* Fixes #173 Support both Role Binding and Cluster Role Binding depending on rbac.limit_to_namespace

* Rev version

* Get Role/Cluster the right way around
pulsar-2.7.5
2021-11-12 07:56:35 -08:00
Frank Kelly
617308147d
Missing fix for #152. Bookie Service also needs the prefix on the port name (#172)
Fixes #158 (This is the second PR - see also https://github.com/apache/pulsar-helm-chart/pull/162)

### Motivation

* All non-standard port-names need a proper protocol prefix to support Istio
 https://istio.io/latest/docs/ops/configuration/traffic-management/protocol-selection/#explicit-protocol-selection
 
### Modifications

Add the prefix value before `bookie`
pulsar-2.7.4
2021-11-09 09:18:26 -08:00
Frank Kelly
5b10f48f5b
Fix #152 Add Helm chart support for Istio port naming (attempt 2) (#162)
Fixes #152 

### Motivation

Support prefix in front of port names to abide by Istio protocol rules
https://istio.io/latest/docs/ops/configuration/traffic-management/protocol-selection/#explicit-protocol-selection

### Modifications

Support adding a prefix
- pulsar -> tcp-pulsar
- pulsarssl --> tls-pulsarssl etc
pulsar-2.7.3
2021-09-10 08:56:16 +08:00
Peter Tinti
f307cc32af
updates pulsar ca name generation to use suffix making cert swappable (#141)
Updates CA name generation to be configurable allowing the swapping in of a CA.

### Motivation

We recently swapped out cert issuers and found that with the current helm chart we were unable to do a hot swap without downtime (via helm) because the CA cert name is not configurable. Being able to change the name of the CA allows us to create a new CA first -> Validate -> then swap over in follow up apply/release.

### Modifications

Adds the ability to specify the suffix used to generate the CA name (not the whole name in order to preserve back compatibility regardless of the release name.)
2021-08-25 23:14:03 -07:00
Frank Kelly
65dc68654b
ZooKeeper HTTP port should be exposed by service so we can use prometheus (#143)
Fixes #142 

### Motivation

Expose HTTP Port on ZooKeeper service so we can use Prometheus

### Modifications

Bug fix to expose HTTP port on ZooKeeper service
2021-08-25 23:13:47 -07:00
Aaron Johnson
c45813ffe5
added extraVolumes and extraVolumeMounts (#149)
Fixes #147

### Motivation
This gives the helm chart user the ability to specify a secret or other type of volume to be mounted into any of the statefulset pods

### Modifications
* Added conditionals to `bookkeeper`, `broker`, `proxy`, `toolset`, and `zookeeper` statefulsets which allow the chart user to specify extraVolumes and extraVolumeMounts for deployed pods.
* Added `extraVolumes` and `extraVolumeMounts` parameters to values.yaml
2021-08-25 23:13:27 -07:00
Thomas O'Neill
19d6ce6488
Add Support for imagePullSecrets (#140)
Fixes #125

### Motivation

The default images in the values.yaml are in docker hub. This PR allows us to provide image pull secrets for the containers which will allow us to get around Docker Hub's rate limiting if the nodes are not logged into Docker Hub.

### Modifications

Added a new template to generate `imagePullSecrets`, and included them in the deployments and statefulsets. This will only add them if they are specified under `images.imagePullSecrets`

### Verifying this change

- [] Make sure that the change passes the CI checks.
2021-08-20 17:22:50 -07:00
Lari Hotari
c3e4ea272b
Fix deprecation warning about rbac.authorization.k8s.io/v1beta1 (#135) 2021-07-03 10:56:58 +03:00
Florian Knip
f00c66a14a
Feature/fix typo in readme (#129)
Fixes #128 

### Motivation

Improve flow when reading the docs

### Modifications

fixed a typo in the docs
2021-06-23 21:13:41 -07:00
TC-robV
75169707fb
add enableAdminApi for prometheus (#121)
Fixes #<xyz>

### Motivation

would be nice to have this option here so people can run admin commands against the prometheus. 

### Modifications

added a new value and modified the deployment, taken from the official prom helm.

### Verifying this change

- [ ] Make sure that the change passes the CI checks.
2021-06-23 21:12:20 -07:00
MMeent
11a1d578dd
Fix indentation issue on checksum/config (#117)
Fixes #116

### Motivation

Theres indentation issues for the `checksum/config` annotation in these templates, which would either fail linting or not apply at all in some situations.

### Modifications

I've added indentation at the specified places such that this isn't an issue anymore.

### Verifying this change

- [ ] Make sure that the change passes the CI checks.
2021-06-23 21:11:38 -07:00
Peter Tinti
d6d240a123
Updates internal issuer cert to include duration and renew configs (#131)
### Motivation
* While component certs can be configured with a custom duration the CA cert for self-signed configuration uses default values. It can be convenient to have this certificate expire more than a month out.

### Modifications
* Updates the internal issuer `{{ .Release.Name }}-ca-tls` certificate to make `duration` and `renewBefore` configurable. Does not use `common` so that the CA can be configured to last much longer than individual components certs if desired.

### Verifying this change
- [x] Make sure that the change passes the CI checks.
2021-06-23 21:00:17 -07:00
Enrico Olivelli
6d0db35216
Update to Pulsar 2.7.2 (#119)
Co-authored-by: Enrico Olivelli <eolivelli@datastax.com>
pulsar-2.7.2
2021-06-03 11:31:47 +03:00
Yong Zhang
0816ac2dfd
Reduce the TLS common name length (#115)
---

*Motivation*

Reduce the TLS command name to avoid getting a too long name
that could not generate a certificate.
2021-04-23 12:43:44 +08:00