* Refactor GitHub Actions CI to a single workflow
* Handle case where "ct lint" fails because of no chart changes
* Re-order scenarios
* Remove excessive default GC logging
* Bump cert-manager version to v1.12.2
* Use compatible cert-manager version
* Install debugging tools (k9s) for ssh access
* Only apply for interactive shells
* Fix JWT symmetric test
* Fix part that was missing from #356
* Install k9s on the fly when k9s is used
- set KUBECONFIG on the fly for kubectl too
* Copy release process doc from Apache Airflow
Source: fb741fd872/dev/README_RELEASE_HELM_CHART.md
* Adapt to Apache Pulsar
* Remove old release process notes
* Fix typo
* Apply suggestions from code review
Co-authored-by: tison <wander4096@gmail.com>
* Add sign.sh script for release artifacts
Script is copied from 395ad7110e/dev/sign.sh
* Add some updates (more to might follow)
* Add some more updates to the rest of the release plan
* Fix rat check command
Co-authored-by: tison <wander4096@gmail.com>
Relates to #290
### Motivation
Make the Apache Pulsar Helm Chart release follow ASF rules for voting, and make the helm binary available via dist.apache.org. By following the information in https://issues.apache.org/jira/browse/LEGAL-573 and in the Apache Airflow project https://github.com/apache/airflow/blob/main/dev/README_RELEASE_HELM_CHART.md, I built this new release process. It will likely need some iterative improvement.
### Modifications
* Add a release process that is based on the Apache Airflow release process
### Verifying this change
- [ ] Make sure that the change passes the CI checks.
* Upgrade to Cert Manager 1.10.0
* Fail fast when installing cert manager
* Upgrade to 1.7.3
Here is the relevant documentation for k8s compatibility:
https://cert-manager.io/docs/installation/supported-releases/
### Motivation
The current version is out of date.
### Modifications
* Upgrade from 1.5.4 to 1.7.3
### Verifying this change
Once #306 is merged, the test suite will verify this PR.
* Use cert-manager to generate certs for tests
* Install Cert-Manager in test env
### Motivation
Currently, we use hard coded certificates for the tests. Instead, we can use Cert Manager to generate the certificates. The primary benefit of this change is that it ensure we're testing the cert manager integration.
### Modifications
* Remove `.ci/tls` directory since we no longer need these certs.
* Remove `scripts/pulsar/clean_tls.sh` (it wasn't used)
* Remove `scripts/pulsar/upload_tls.sh` since we are not uploading any certs
* Update the `helm.sh` test script
* Update the `.ci/clusters` configurations to generate the relevant cert manager manifests
### Verifying this change
- [ ] Make sure that the change passes the CI checks.
Signed-off-by: Paul Gier <paul.gier@datastax.com>
This is just a minor improvement to the error handling of one of the bash scripts
### Motivation
Currently if you run `./scripts/pulsar/prepare_helm_release.sh` and the pulsar namespace does not currently exist, you get several error messages that make it not that clear what still needs to be done next.
```
generate the token keys for the pulsar cluster
The private key and public key are generated to /var/folders/cn/r5tb0zln1bgbfzz_7x72tgzm0000gn/T/tmp.ITrq1a4C and /var/folders/cn/r5tb0zln1bgbfzz_7x72tgzm0000gn/T/tmp.qi0dl2WO successfully.
error: failed to create secret namespaces "pulsar" not found
generate the tokens for the super-users: proxy-admin,broker-admin,admin
generate the token for proxy-admin
pulsar-dev-token-asymmetric-key
kubectl get -n pulsar secrets pulsar-dev-token-asymmetric-key -o jsonpath={.data.PRIVATEKEY} | base64 --decode > /var/folders/cn/r5tb0zln1bgbfzz_7x72tgzm0000gn/T/tmp.CikEhIxe
Error from server (NotFound): namespaces "pulsar" not found
generate the token for broker-admin
pulsar-dev-token-asymmetric-key
kubectl get -n pulsar secrets pulsar-dev-token-asymmetric-key -o jsonpath={.data.PRIVATEKEY} | base64 --decode > /var/folders/cn/r5tb0zln1bgbfzz_7x72tgzm0000gn/T/tmp.G1PU9MMj
Error from server (NotFound): namespaces "pulsar" not found
generate the token for admin
pulsar-dev-token-asymmetric-key
kubectl get -n pulsar secrets pulsar-dev-token-asymmetric-key -o jsonpath={.data.PRIVATEKEY} | base64 --decode > /var/folders/cn/r5tb0zln1bgbfzz_7x72tgzm0000gn/T/tmp.HddlCq8e
Error from server (NotFound): namespaces "pulsar" not found
-------------------------------------
The jwt token secret keys are generated under:
- 'pulsar-dev-token-asymmetric-key'
The jwt tokens for superusers are generated and stored as below:
- 'proxy-admin':secret('pulsar-dev-token-proxy-admin')
- 'broker-admin':secret('pulsar-dev-token-broker-admin')
- 'admin':secret('pulsar-dev-token-admin')
```
### Modifications
I added a check for the existence of the namespace which fails immediately instead of continuing, and added an error message that describes what the problem is and how to resolve it.
```
error: failed to get namespace 'pulsar'
please check that this namespace exists, or use the '-c' option to create it
```
### Verifying this change
- [X] Make sure that the change passes the CI checks.
Updates CA name generation to be configurable allowing the swapping in of a CA.
### Motivation
We recently swapped out cert issuers and found that with the current helm chart we were unable to do a hot swap without downtime (via helm) because the CA cert name is not configurable. Being able to change the name of the CA allows us to create a new CA first -> Validate -> then swap over in follow up apply/release.
### Modifications
Adds the ability to specify the suffix used to generate the CA name (not the whole name in order to preserve back compatibility regardless of the release name.)
It remains possible to override the current release namespace by setting
the `namespace` value though this may lead to having the helm metadata
and the pulsar components in different namespaces
Fixes#66
### Motivation
Trying to deploy the chart in a namespace using the usual helm pattern fails for example
```
kubectl create ns pulsartest
helm upgrade --install pulsar -n pulsartest apache/pulsar
Error: namespaces "pulsar" not found
```
fixing that while keeping the helm metadata and the deployed objects in the same namespace requires declaring the namespace twice
```
kubectl create ns pulsartest
helm upgrade --install pulsar -n pulsartest apache/pulsar --set namespace=pulsartest
Error: namespaces "pulsar" not found
```
This is needlessly confusing for newcomers who follow the helm documentation and is contrary to helm best practices.
### Modifications
I changed the chart to use the context namespace `.Release.Namespace` by default while preserving the ability to override that by explicitly providing a namespace on the commande line, with the this modification both examples behave as expected
### Verifying this change
- [x] Make sure that the change passes the CI checks.
This allows operation in environemnts where direct installation of objects into
kubernetes cluster is not desired or possible. For example when using sealedsecrets
or SOPS, where the secrets are firs encrypted and then commited into repository
and deployed latter by some other deployment system.
Co-authored-by: Jiří Pinkava <jiri.pinkava@rossum.ai>
### Motivation
The secret resources generation was appending a newline at the end of the JWT token strings (```\n```). From my understanding, this is not an issue inside Pulsar likely because it trims the contents of the JWT programmatically. However, when setting pulsar as a sink destination for [Vector](https://vector.dev/) (vector produces messages into Pulsar), I noticed the token was always invalid due to this extra newline.
### Modifications
Remove newline from secret tokens generation by using the utility command tr. Granted, this is not the nicest way to go about it but given that the contents are JWT strings, it appears to do the job just fine while keeping everything else working (e.g.: producing/consuming as well as other components like Prometheus). Please advise if you have any concerns or suggestions.
Fixes#6
### Motivation
As suggested here: https://pulsar.apache.org/docs/en/helm-deploy/#prepare-the-helm-release. The ```prepare_helm_release.sh``` script provided with this Helm chart can create a secret credentials resource and
> The username and password are used for logging into Grafana dashboard and Pulsar Manager.
However, I haven't been able to make use of such a feature for a number of reasons:
1. This secret doesn't seem to affect the ```pulsar-manager-deployment.yaml``` definition. Instead, the ```./templates/pulsar-manager-admin-secret.yaml``` seems to be the one providing the credentials for the pulsar manager (UI) (with the added possibility to overwrite via values.yaml at ```pulsar_manager.admin.user/password```).
2. Using the Pulsar chart as a dependency for an umbrella chart (this is currently my use case), will bring extra hassle that will make it very hard to have all resources follow the same naming structure, thus causing some resources to never be deployed successfully e.g.: ```./templates/grafana-deployment.yaml``` will complain that it couldn't find the secret created by the bash script. Attempting to fix this issue via the ```-k``` flag passed to the script will cause the JWT secret tokens to have a name that's unexpected by the broker, etc.
### Modifications
Decouple grafana credentials from pulsar manager via a new secret resource named ```./charts/pulsar/templates/grafana-admin-secret.yaml```.
Add credentials overriding via values.yaml in the same way as pulsar_manager (grafana.admin.user/password) & delete secret resource manipulation from bash scripts (cleaup_helm_release.sh & prepare_helm_release.sh)
### Verifying this change
- [x] Make sure that the change passes the CI checks.
### Motivation
While making use of the scripts provided in this repo to prepare helm releases, I noticed that providing the ```-d``` flag (delete namespace) for the ```./scripts/pulsar/cleanup_helm_release.sh``` would always fail claiming that the **namespace already exists**. Upon closer examination, I noticed that the kubectl command to delete the provided namespace is actually attempting to create it instead.
### Modifications
I've gone ahead and made the corresponding modification on the script to delete the namespace (went from ```kubectl create namespace ${namespace}``` to ```kubectl delete namespace ${namespace}```).
### Verifying this change
I'm not sure what possible verifications I can provide for this PR. Please advise.
