### Motivation
In some case, my k8s node only have 1 large capacity ssd, for deploying 1 bookie, I need:
- Partition the ssd into 2 disks, and make 2 pv over it.
- Just make 1 pv over it, but journal & ledgers under same mount path (this PR did)
Both can't isolate IO for journal & ledgers, so I prefer the second one for reusability.
### Modifications
values.yaml
- add `useSingleCommonVolume` option, default false
bookkeeper-statefulset.yaml
- mount the only PV to path `/pulsar/data/bookkeeper`
- use configured common storageClassName
bookkeeper-storageclass.yaml
- use configured provisioner for the common storageClass
### Others
This may not be an issue for everyone, if it's not necessary to merge, I'll just use it locally
### Verifying this change
- [x] Make sure that the change passes the CI checks.
Fixes#94
### Motivation
fix `io.kubernetes.client.openapi.ApiException: Forbidden`
### Modifications
fix typo
### Verifying this change
- [x] Make sure that the change passes the CI checks.
Fixes for wrong namespace handling in some RBAC and missing dnsNames for TLS
### Motivation
Fixes old unused handling of namespace name in RBAC for autorecovery and bookkeeper.
Fixes Helm exception of missing key when not defining TLS dnsNames
### Modifications
Use namespace template in RBAC definitions for bookkeeper and autorecovery. Add if around every `toYaml .Values.tls.bookie.dnsNames` clause in TLS certs definitions.
### Verifying this change
- [x] Make sure that the change passes the CI checks.
### Motivation
As I wanted to use [streamnative/apache-pulsar-grafana-dashboard](https://github.com/streamnative/apache-pulsar-grafana-dashboard) with this helm chart and own cluster wide Prometheus stack I decided that use of PodMonitor CRD is a good way. Unfortunately prometheus config has some metrics relabelings that are required by grafana dashboard. I decied to port them directly to PodMonitor definition
### Modifications
* Added missing PodMonitor for autorecovery
* Port relabelings from `prometheus-configmap.yaml` to each PodMonitor
### Verifying this change
- [x] Make sure that the change passes the CI checks.
### Motivation
When using standard bookkeeper installation on PSP cluster initialization fails because has to be started as root
### Modifications
Add same ServiceAccount and SecurityContext for bookkeeper-cluster-initialize as in bookkeeper specyfication.
UPDATE: Seems that when using in cluster TLS encryption other components also require RW access to root FS, I added PSP for proxy, zookeepe, broker and toolset
### Verifying this change
- [x] Make sure that the change passes the CI checks.
Fixes#71
### Motivation
Pods are not restarting when config maps are changed after changing values.yaml file, so they need to be restarted manually in order to pick up new values from config map.
### Modifications
As I mentioned `restartPodsOnConfigMapChange` flag for each component is added in values.yaml file whether to restart pods on configmap change or not, default is `false`.
In statefulset templates for each component is added part which is adding annotation that contains hash of corresponding configmap if `restartPodsOnConfigMapChange` is `true`, which will cause pods to restart if corresponding configmap has been changed (https://helm.sh/docs/howto/charts_tips_and_tricks/#automatically-roll-deployments).
### Verifying this change
- [ ] Make sure that the change passes the CI checks.
Add PSP and add/modify RBAC. I'm open for all discussion.
### Motivation
On clusters which use PSP and restrictive default policy pulsar cannot be installed, because it uses root user and requires writable container root directory. Additionally default RBAC for broker are too permissive (usage of ClusterRoleBinding) in my opinion.
### Modifications
Add PSP and RBAC for bookkeeper and autorecovery to add
exception to allow startup even in secure environment
where containers cannot access RW on root by default.
Add option for limiting broker ClusterRoleBinding
to single namespace by replacing to RoleBinding
### Verifying this change
- [x] Make sure that the change passes the CI checks.
Co-authored-by: Sijie Guo <sijie@apache.org>
Fixes inability to validate self-signed certs from external clients
### Motivation
Currently self-signed certificates can only be used inside of the same cluster as they are labeled with internal dns names without the possibility of appending additional values. Some use-cases require the connection of external clients. This PR aims to allow users add additional dnsNames (IP or domain) to the self-signed certificates.
### Modifications
* Adds the ability to add `dnsNames` to self-signed certificates to any component like so:
```yaml
tls:
enabled: true
proxy:
enabled: true
dnsNames:
- test.example.com
```
### Verifying this change
- [x] Make sure that the change passes the CI checks.
It remains possible to override the current release namespace by setting
the `namespace` value though this may lead to having the helm metadata
and the pulsar components in different namespaces
Fixes#66
### Motivation
Trying to deploy the chart in a namespace using the usual helm pattern fails for example
```
kubectl create ns pulsartest
helm upgrade --install pulsar -n pulsartest apache/pulsar
Error: namespaces "pulsar" not found
```
fixing that while keeping the helm metadata and the deployed objects in the same namespace requires declaring the namespace twice
```
kubectl create ns pulsartest
helm upgrade --install pulsar -n pulsartest apache/pulsar --set namespace=pulsartest
Error: namespaces "pulsar" not found
```
This is needlessly confusing for newcomers who follow the helm documentation and is contrary to helm best practices.
### Modifications
I changed the chart to use the context namespace `.Release.Namespace` by default while preserving the ability to override that by explicitly providing a namespace on the commande line, with the this modification both examples behave as expected
### Verifying this change
- [x] Make sure that the change passes the CI checks.
Signed-off-by: xiaolong.ran <rxl@apache.org>
### Motivation
Bump the image version to 2.6.2
### Verifying this change
- [x] Make sure that the change passes the CI checks.
### Motivation
* ```publishNotReadyAddresses``` is a service spec and not a service annotation. This is mentioned in the K8s API docs at https://v1-17.docs.kubernetes.io/docs/reference/generated/kubernetes-api/v1.15/#servicespec-v1-core
### Modifications
* Modified ```publishNotReadyAddresses``` from annotation to service spec
### Verifying this change
- [x] Make sure that the change passes the CI checks.
### Motivation
* It's not recommended to run a production zookkeeper cluster with forceSync as "no". This is also mentioned in the forceSync section in https://pulsar.apache.org/docs/en/next/reference-configuration/#zookeeper
### Modifications
* Removed ```-Dzookeeper.forceSync=no``` from ```values.yaml``` as default ```forceSync``` is ```yes```.
Fixes#50
### Motivation
The host option is not required to setup an ingress, so I made it an optional value
### Modifications
*Describe the modifications you've done.*
Made setting the host optional.
Co-authored-by: Elad Dolev <elad@firebolt.io>
### Motivation
Give the ability to deploy multi-cluster instance on K8s clusters with non-default `clusterDomain`, and connect to external configuration-store
### Modifications
- give the ability to change cluster's name
- give the ability to change `clusterDomain`
- fix external configuration store functionality
- use broker ports variables
- use label templates, and add `component` label in several places
### Verifying this change
- [x] Make sure that the change passes the CI checks.
Fixes#47
### Motivation
Only create the initialize job on install.
### Modifications
- Added an initialize value that can be set to true on install, matching the documentation in the README.md
Fixes#39
### Motivation
The match expression for the "app" label was incorrect breaking the antiaffinity since they would never match. Fixing this makes the podAntiAffinity work, but now requires at least N nodes to be in the cluster where N = largest replica set with affinity. Added the option to set the affinity type to preferredDuringSchedulingIgnoredDuringExecution where it will try to follow the affinity, but will still deploy a pod if it needs to break it.
### Modifications
- Fixed app matchExpression
- Added option to set the affinity type
- bumped chart version
### Verifying this change
- [X] Make sure that the change passes the CI checks.
Fixes#46
### Motivation
There were some templates that relied on extra values that are deprecated.
### Modifications
Modified the checks to check for non deprecated values or deprecated values.
### Verifying this change
- [X] Make sure that the change passes the CI checks.
### Motivation
Allow Grafana to be served from a sub path.
### Modifications
- Added a config map to add extra environment variables to the grafana deployment. As the grafana image adds new features that require environment variables, this can be used to set them.
- Bumped the grafana image to allow a reverse proxy
- removed ingress annotations as they are specific to nginx, and to match all the other ingresses
- bumped the chart version as per the README
Example values:
```
grafana:
configData:
GRAFANA_ROOT_URL: /pulsar/grafana
GRAFANA_SERVE_FROM_SUB_PATH: "true"
ingress:
enabled: true
port: 3000
path: "/pulsar/grafana/?(.*)"
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /$1
```
* Add 'http' port specification to zookeeper statefulset
This makes the zookeeper spec inline with the other statefulset specs
in this chart and it provides a port target for custom podMonitors
* Added PodMonitors for bookie, broker, proxy, and zookeeper
New PodMonitors are needed for prometheus-operator to pickup scrape
targets.
Defaults to disabled so users need to opt in to deploy
* Added Apache license info to podmonitor yamls
### Motivation
PR #37 updated the location of the ports in the default values yaml. This causes a null pointer exception when rendering this helm chart.
### Modifications
Fix variable reference
## Motivation
### Case
I have a physical zk cluster and want configure bookkeeper & broker & proxy to use it.
So I set components.zookeeper as false, and only found pulsar.zookeeper.connect to set my physical zk address.
But deploy stage was stucked in bookkeeper wait-zookeeper-ready container.
### Issue
The wait-zookeeper-ready initContainer in bookkeeper-cluster-initialize Job used spliced zk Service hosts to detect zk ready or not, other component init Job initContainer do the same thing. Actually, zk service are unreachable because I disabled zk component.
## Modifications
- Add optional pulsar_metadata.userProvidedZookeepers config for this case, and make component's init Job use user zk to detect liveness, instead of spliced Service hosts.
- Delete redundant image reference in bookkeeper init Job.
*Motivation*
based on [helm documentation](https://helm.sh/docs/topics/charts/),
the `appVersion` is the version of the app that this contains. Since the repo
is using 2.6.0 image, update `appVersion` to 2.6.0
### Motivation
We need to be able to change annotation to inject AWS IAM role (EKS based deployment).
https://docs.aws.amazon.com/eks/latest/userguide/specify-service-account-role.html
With 2.6.0 and this annotation change we were able to use Tiered Storage with S3 and EKS/IAM(OIDC).
e.g :
```
annotations:
eks.amazonaws.com/role-arn: arn:aws:iam::66666:role/my-iam-role-with-s3-access
```
values.yaml
```
broker:
service_account:
annotations:
eks.amazonaws.com/role-arn: arn:aws:iam::66666:role/my-iam-role-with-s3-access
```
### Modifications
Added a value to allow to change annotations fro broker service account.
I've tried following style from other part of the code.
### Verifying this change
- [ ] Make sure that the change passes the CI checks.
Fixes#6
### Motivation
As suggested here: https://pulsar.apache.org/docs/en/helm-deploy/#prepare-the-helm-release. The ```prepare_helm_release.sh``` script provided with this Helm chart can create a secret credentials resource and
> The username and password are used for logging into Grafana dashboard and Pulsar Manager.
However, I haven't been able to make use of such a feature for a number of reasons:
1. This secret doesn't seem to affect the ```pulsar-manager-deployment.yaml``` definition. Instead, the ```./templates/pulsar-manager-admin-secret.yaml``` seems to be the one providing the credentials for the pulsar manager (UI) (with the added possibility to overwrite via values.yaml at ```pulsar_manager.admin.user/password```).
2. Using the Pulsar chart as a dependency for an umbrella chart (this is currently my use case), will bring extra hassle that will make it very hard to have all resources follow the same naming structure, thus causing some resources to never be deployed successfully e.g.: ```./templates/grafana-deployment.yaml``` will complain that it couldn't find the secret created by the bash script. Attempting to fix this issue via the ```-k``` flag passed to the script will cause the JWT secret tokens to have a name that's unexpected by the broker, etc.
### Modifications
Decouple grafana credentials from pulsar manager via a new secret resource named ```./charts/pulsar/templates/grafana-admin-secret.yaml```.
Add credentials overriding via values.yaml in the same way as pulsar_manager (grafana.admin.user/password) & delete secret resource manipulation from bash scripts (cleaup_helm_release.sh & prepare_helm_release.sh)
### Verifying this change
- [x] Make sure that the change passes the CI checks.
### Motivation
As seen below, there is a fix for one of the Grafana dashboards that are currently broken in this project (available since version 0.0.5):
- [The Pulsar-topics metrics can't load in Grafana](https://github.com/streamnative/charts/issues/49)
Additionally, upgrading Prometheus to the latest version improves performance as seen here: https://prometheus.io/blog/2017/11/08/announcing-prometheus-2-0
### Modifications
Bring Docker images to their most up-to-date version (streamnative/apache-pulsar-grafana-dashboard-k8s:0.0.6, prom/prometheus:v2.17.2) to fix the following issues:
- https://github.com/streamnative/charts/issues/49 <- fixes Pulsar-topics metrics failure to load
- https://github.com/prometheus/prometheus/pull/2859 <- prevent escalation vulnerabilities by defaulting to the ```nobody``` user
**Note**: upgrading to the latest version of Prometheus (currently v2.17.2) caused the pod to fail with the following error: ```open /prometheus/queries.active: permission denied```. In order to fix this issue I followed the instructions from these 2 comments:
- [Permission denied UID/GID solution](https://github.com/prometheus/prometheus/issues/5976#issuecomment-532942295)
- [Unable to create mmap-ed active query log securityContext fix](https://github.com/aws/eks-charts/issues/21#issuecomment-607031756)
### Verifying this change
- [x] Make sure that the change passes the CI checks.
