330 Commits

Author SHA1 Message Date
Lari Hotari
2757074763 Update default versions in ct.sh 2023-12-11 14:47:26 +02:00
Lari Hotari
91178e8300 Revisit release process 2023-12-11 14:46:00 +02:00
Lari Hotari
7f84d6e966 Revisit release process 2023-12-11 14:13:11 +02:00
Lari Hotari
a7247b6edb Bump chart version to 3.1.0 pulsar-3.1.0 pulsar-3.1.0-candidate-1 2023-12-11 13:32:54 +02:00
Lari Hotari
08bac44c37 Remove RELEASE_NOTES.md
Release notes will be maintained at https://github.com/apache/pulsar-helm-chart/releases
2023-12-11 13:29:29 +02:00
Lari Hotari
8167d22bda
Run CI tests with k8s 1.21 and 1.27 and bump tool versions (#414)
* Run CI tests with k8s v1.21 and v1.27

* Bump versions used in testing
2023-12-11 03:00:27 -08:00
Jasper Vaneessen
89da2b2890
Zookeeper replicas: adhere to own recommendations (#336)
The comments say ZK replicacount should be >=3 but defaults to 1, should adhere to own recommendations by default
2023-12-11 02:36:25 -08:00
Lari Hotari
608b51e872
Bump appVersion to 3.0.2 (#413) 2023-12-11 01:37:34 -08:00
Đình Tuyên
a8d667a391
Correct Keytool env generation (#375) 2023-12-11 01:23:43 -08:00
Ragnar Rova
e349bc54c0
Update common_auth.sh (#243)
Co-authored-by: Lari Hotari <lhotari@users.noreply.github.com>
2023-12-11 01:07:50 -08:00
Michał Koziorowski
fba51bd87e
Lowered BOOKIE_MEM and PULSAR_MEM in init containers (#386)
* Lowered BOOKIE_MEM and PULSAR_MEM in init containers. Default BOOKIE_MEM and PULSAR_MEM settings from conf/pulsar_env.sh and conf/bkenv.sh (-Xms2g -Xmx2g -XX:MaxDirectMemorySize=4g) are too high for low-memory systems.
2023-12-11 00:50:08 -08:00
Julien Jakubowski
d1729e8e0a
fix issue https://github.com/apache/pulsar-manager/issues/505 (#385) 2023-12-11 00:31:56 -08:00
Emre Aydın
9542c7b226
Use container ports for proxy stateful set probes (#410)
Using service ports cause probes to fail.

Co-authored-by: emre <emre.aydin@zapata.ai>
2023-12-11 00:20:07 -08:00
Frank Kelly
8cb3c18377
Allow Proxy and Broker HPA to specify scaling policies on scaleUp or scaleDown. (#391) 2023-09-15 14:12:12 -05:00
Michał Koziorowski
ea5404c421
Fixed bookkeeper volume mounts indentation (#384) 2023-08-24 09:32:58 +08:00
tison
e8a1c08cc2
docs: Update README to reflect unmaintained status (#378) 2023-08-20 23:37:17 +08:00
Ethan-Merrill
73fe688a43
Add support for stateful functions using the bookie as state storage (#171)
### Motivation

Enables support for using the Pulsar bookies as persistent state storage for functions.

### Modifications

- Added an option to enable/disable using bookies as state storage
- Adds extra server components options to the bookkeeper to enable necessary features for bookies to be used as state storage
- Adds stateStorageServiceUrl to the broker configmap
2023-07-18 21:37:03 -05:00
Tomasz Jegorow
042fd5b6d4
Configure custom topologyKey for podAntiAffinity (#351) 2023-07-12 18:19:49 +03:00
Atkins
79ec5ba333
Fix pod annotations when restartPodsOnConfigMapChange is true (#353)
Signed-off-by: Atkins Chang <atkinschang@gmail.com>
2023-07-12 18:18:48 +03:00
Atkins
b30eb6fff8
Improve HPA (#354)
* Use `autoscaling/v2` if Kubernetes version >= 1.23

Signed-off-by: Atkins Chang <atkinschang@gmail.com>

* Disable replicas when autoscaling enabled

Signed-off-by: Atkins Chang <atkinschang@gmail.com>

---------

Signed-off-by: Atkins Chang <atkinschang@gmail.com>
2023-07-12 18:18:19 +03:00
Brad Shelton
f8ad65066e
To address the function role vs clusterrole issue (#236)
* To address the function role vs clusterrole issue

* making backwards compatable

* updated value.yaml to include limit functions to namespace

* Added documentation to clarify the new attribute

* moved limit_to_namespace under functions.rbac
2023-07-12 18:11:36 +03:00
Lari Hotari
49f4acdf5a
Refactor GitHub Actions CI to a single workflow (#371)
* Refactor GitHub Actions CI to a single workflow

* Handle case where "ct lint" fails because of no chart changes

* Re-order scenarios

* Remove excessive default GC logging

* Bump cert-manager version to v1.12.2

* Use compatible cert-manager version

* Install debugging tools (k9s) for ssh access

* Only apply for interactive shells

* Fix JWT symmetric test

* Fix part that was missing from #356

* Install k9s on the fly when k9s is used

- set KUBECONFIG on the fly for kubectl too
2023-07-11 15:55:35 +03:00
huis
2d646f4efe
Fix PVC selector scope error when bookkeeper uses multiple data volumes (#342)
Fix PVC selector scope error when bookkeeper uses multiple data volumes
2023-07-11 10:03:38 +03:00
Robert Moucha
71450334cf
Fix zookeeper annotations (#348)
Fixed the case when no ZK annotations are set and `zookeeper.restartPodsOnConfigMapChange: true`
helm can not render template
2023-07-11 10:01:54 +03:00
mfuxi
786e182de4
add ingressClassName (#360) 2023-07-11 10:00:45 +03:00
Chris Johnson
90a26b2dc8
fix: proxy should not use priviledged port numbers (#356)
* fix: proxy should not use priviledged port numbers

This fixes issue #335

* fix: making the change backward compatible
2023-07-11 10:00:17 +03:00
Lari Hotari
f43c6f6d9e
Fix GitHub Actions based CI checks which have been failing (#370)
* Upgrade upgrade kind, chart releaser and helm versions

* Disable podMonitory for values-broker-tls.yaml file

- was missing from #317

* Use k8s 1.18.20

* Use ubuntu-20.04 runtime

- k8s < 1.19 doesn't support cgroup v2

* Upgrade to k8s 1.19 as baseline

* Baseline to k8s 1.20

* Set ip family to ipv4

* Add more logging to kind cluster creation

* Simplify duplicate job deletion

* use verbosity flag

* Upgrade to k8s 1.24

* Replace removed tolerate-unready-endpoints annotation with publishNotReadyAddresses

(cherry picked from commit e90926053a2b01bb95529fbaddc8d2ce2cdeec63)

* Use k8s 1.21 as baseline

* Run on ubuntu-22.04

* Use Pulsar 2.10.4
2023-07-10 12:45:37 -07:00
Yuwei Sung
b3c19fd7fc
added arm64 in common_auth.sh (#191) 2023-03-14 15:06:34 +08:00
Michael Marshall
8ad7cf6b65
Remove unnecessary version bump guidance 2022-11-03 23:22:32 -05:00
Michael Marshall
9436d03656
Use GitHub release notes until the pulsar.a.o website hosts them 2022-11-03 23:21:03 -05:00
Michael Marshall
bcfe47ced4
Releases are available at downloads.apache.org 2022-11-03 23:03:39 -05:00
Michael Marshall
e49bd36422
Minor changes to release process 2022-11-03 23:02:04 -05:00
Michael Marshall
4dfb96fb36
Add RELEASE_NOTES.md for 3.0.0 Release (#322)
* Add RELEASE_NOTES.txt

* Add section for upgrade considerations

* Minor fixes

* Add license header

# Motivation

As part of the 3.0.0 release, I've prepared release notes. Please review them and suggest improvements.
2022-11-03 16:22:46 -05:00
Michael Marshall
7bcf1af6e5
Revert "Remove unused license test file"
This reverts commit bcbc674092bf2cf5ad93db181a9bbeb75406fc49.
2022-10-28 14:30:42 -05:00
Michael Marshall
34678b8e1b
Clarify Apache Pulsar 2.10.0 upgrade section name 2022-10-28 14:25:56 -05:00
Michael Marshall
c70a7877eb
Minor wording improvements to release process 2022-10-28 14:19:59 -05:00
Michael Marshall
bcbc674092
Remove unused license test file 2022-10-28 14:18:18 -05:00
Michael Marshall
c8f5e7f9ce
New Release Process (#301)
* Copy release process doc from Apache Airflow

Source: fb741fd872/dev/README_RELEASE_HELM_CHART.md

* Adapt to Apache Pulsar

* Remove old release process notes

* Fix typo

* Apply suggestions from code review

Co-authored-by: tison <wander4096@gmail.com>

* Add sign.sh script for release artifacts

Script is copied from 395ad7110e/dev/sign.sh

* Add some updates (more to might follow)

* Add some more updates to the rest of the release plan

* Fix rat check command

Co-authored-by: tison <wander4096@gmail.com>

Relates to #290

### Motivation

Make the Apache Pulsar Helm Chart release follow ASF rules for voting, and make the helm binary available via dist.apache.org. By following the information in https://issues.apache.org/jira/browse/LEGAL-573 and in the Apache Airflow project https://github.com/apache/airflow/blob/main/dev/README_RELEASE_HELM_CHART.md, I built this new release process. It will likely need some iterative improvement.

### Modifications

* Add a release process that is based on the Apache Airflow release process

### Verifying this change

- [ ] Make sure that the change passes the CI checks.
pulsar-3.0.0 pulsar-3.0.0-candidate-1
2022-10-21 00:33:08 -05:00
Michael Marshall
687060aa27
Chart: Bump version to 3.0.0 2022-10-21 00:26:42 -05:00
Michael Marshall
8072c38ecc
Document Grafana Dashboards to prepare for 3.0.0 release (#321) 2022-10-21 08:24:30 +03:00
Michael Marshall
f4cb7a2c95
Simplify deployment instructions in README (#324)
Fixes #287

### Motivation

The current steps to install the Apache Pulsar Helm Chart include an unnecessary script `scripts/pulsar/prepare_helm_release.sh`. It relies on tooling that has not been maintained and is not a part of the Apache Pulsar project. As such, I propose we remove these references.

Note that one of the reasons we used these scripts historically is to simplify deployment. Without these scripts, we should document what is necessary. I am tracking that work here https://github.com/apache/pulsar-helm-chart/issues/323.
2022-10-21 00:23:32 -05:00
edward.zeng
95c218b218
Fix PodMonitor name conflicts for multiple releases in same namespace (#258)
* Fix PodMonitor name conflicts for multiple releases in same namespace

Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>

* Use pulsar.fullname for PodMonitor name prefix

Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
Co-authored-by: Michael Marshall <mmarshall@apache.org>

Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>

Fixes #257

### Motivation

Fix PodMonitor name conflicts for multiple releases in same namespace

### Modifications

Use release name instead of hardcode `pulsar.name` for pod monitor name.

### Verifying this change

- [x] Make sure that the change passes the CI checks.
2022-10-20 21:15:16 -05:00
Michael Marshall
d9769a9519
Add missing license headers and .rat-excludes (#319)
* Add missing license headers and .rat-excludes

* Fix .rat-excludes files

### Motivation

As part of our updated release process, we need to make sure that all relevant files have license headers.

### Modifications

* Add license headers formatted appropriately for each file type

### Verifying this change

The follow script shows that the solution is complete:

```shell
$ java -jar ../apache-rat-0.15/apache-rat-0.15.jar . -E .rat-excludes 
Ignored 18 lines in your exclusion files as comments or empty lines.


*****************************************************
Summary
-------
Generated at: 2022-10-20T17:54:42-05:00

Notes: 4
Binaries: 1
Archives: 0
Standards: 92

Apache Licensed: 92
Generated Documents: 0

JavaDocs are generated, thus a license header is optional.
Generated files do not require license headers.

0 Unknown Licenses

*****************************************************
  Files with Apache License headers will be marked AL
  Binary files (which do not require any license headers) will be marked B
  Compressed archives will be marked A
  Notices, licenses etc. will be marked N
  AL    ./.asf.yaml
  AL    ./.rat-excludes
  N     ./LICENSE
  N     ./NOTICE
  AL    ./README.md
  AL    ./Vagrantfile
  AL    ./license_test.go
  AL    ./charts/pulsar/.helmignore
  AL    ./charts/pulsar/Chart.yaml
  N     ./charts/pulsar/LICENSE
  N     ./charts/pulsar/NOTICE
  AL    ./charts/pulsar/values.yaml
  B     ./charts/pulsar/charts/kube-prometheus-stack-41.5.1.tgz
  AL    ./charts/pulsar/templates/_autorecovery.tpl
  AL    ./charts/pulsar/templates/_bookkeeper.tpl
  AL    ./charts/pulsar/templates/_broker.tpl
  AL    ./charts/pulsar/templates/_configurationstore.tpl
  AL    ./charts/pulsar/templates/_helpers.tpl
  AL    ./charts/pulsar/templates/_toolset.tpl
  AL    ./charts/pulsar/templates/_zookeeper.tpl
  AL    ./charts/pulsar/templates/autorecovery-configmap.yaml
  AL    ./charts/pulsar/templates/autorecovery-podmonitor.yaml
  AL    ./charts/pulsar/templates/autorecovery-rbac.yaml
  AL    ./charts/pulsar/templates/autorecovery-service.yaml
  AL    ./charts/pulsar/templates/autorecovery-statefulset.yaml
  AL    ./charts/pulsar/templates/bookkeeper-cluster-initialize.yaml
  AL    ./charts/pulsar/templates/bookkeeper-configmap.yaml
  AL    ./charts/pulsar/templates/bookkeeper-pdb.yaml
  AL    ./charts/pulsar/templates/bookkeeper-podmonitor.yaml
  AL    ./charts/pulsar/templates/bookkeeper-rbac.yaml
  AL    ./charts/pulsar/templates/bookkeeper-service.yaml
  AL    ./charts/pulsar/templates/bookkeeper-statefulset.yaml
  AL    ./charts/pulsar/templates/bookkeeper-storageclass.yaml
  AL    ./charts/pulsar/templates/broker-cluster-role-binding.yaml
  AL    ./charts/pulsar/templates/broker-configmap.yaml
  AL    ./charts/pulsar/templates/broker-hpa.yaml
  AL    ./charts/pulsar/templates/broker-pdb.yaml
  AL    ./charts/pulsar/templates/broker-podmonitor.yaml
  AL    ./charts/pulsar/templates/broker-rbac.yaml
  AL    ./charts/pulsar/templates/broker-service-account.yaml
  AL    ./charts/pulsar/templates/broker-service.yaml
  AL    ./charts/pulsar/templates/broker-statefulset.yaml
  AL    ./charts/pulsar/templates/dashboard-deployment.yaml
  AL    ./charts/pulsar/templates/dashboard-ingress.yaml
  AL    ./charts/pulsar/templates/dashboard-service.yaml
  AL    ./charts/pulsar/templates/function-worker-configmap.yaml
  AL    ./charts/pulsar/templates/keytool.yaml
  AL    ./charts/pulsar/templates/namespace.yaml
  AL    ./charts/pulsar/templates/proxy-configmap.yaml
  AL    ./charts/pulsar/templates/proxy-hpa.yaml
  AL    ./charts/pulsar/templates/proxy-ingress.yaml
  AL    ./charts/pulsar/templates/proxy-pdb.yaml
  AL    ./charts/pulsar/templates/proxy-podmonitor.yaml
  AL    ./charts/pulsar/templates/proxy-rbac.yaml
  AL    ./charts/pulsar/templates/proxy-service.yaml
  AL    ./charts/pulsar/templates/proxy-statefulset.yaml
  AL    ./charts/pulsar/templates/pulsar-cluster-initialize.yaml
  AL    ./charts/pulsar/templates/pulsar-manager-admin-secret.yaml
  AL    ./charts/pulsar/templates/pulsar-manager-configmap.yaml
  AL    ./charts/pulsar/templates/pulsar-manager-deployment.yaml
  AL    ./charts/pulsar/templates/pulsar-manager-ingress.yaml
  AL    ./charts/pulsar/templates/pulsar-manager-service.yaml
  AL    ./charts/pulsar/templates/tls-cert-internal-issuer.yaml
  AL    ./charts/pulsar/templates/tls-certs-internal.yaml
  AL    ./charts/pulsar/templates/toolset-configmap.yaml
  AL    ./charts/pulsar/templates/toolset-rbac.yaml
  AL    ./charts/pulsar/templates/toolset-service.yaml
  AL    ./charts/pulsar/templates/toolset-statefulset.yaml
  AL    ./charts/pulsar/templates/zookeeper-configmap.yaml
  AL    ./charts/pulsar/templates/zookeeper-pdb.yaml
  AL    ./charts/pulsar/templates/zookeeper-podmonitor.yaml
  AL    ./charts/pulsar/templates/zookeeper-rbac.yaml
  AL    ./charts/pulsar/templates/zookeeper-service.yaml
  AL    ./charts/pulsar/templates/zookeeper-statefulset.yaml
  AL    ./charts/pulsar/templates/zookeeper-storageclass.yaml
  AL    ./examples/values-bookkeeper-aws.yaml
  AL    ./examples/values-cs.yaml
  AL    ./examples/values-jwt-asymmetric.yaml
  AL    ./examples/values-jwt-symmetric.yaml
  AL    ./examples/values-local-cluster.yaml
  AL    ./examples/values-local-pv.yaml
  AL    ./examples/values-minikube.yaml
  AL    ./examples/values-no-persistence.yaml
  AL    ./examples/values-one-node.yaml
  AL    ./examples/values-tls.yaml
  AL    ./examples/values-zookeeper-aws.yaml
  AL    ./hack/common.sh
  AL    ./hack/kind-cluster-build.sh
  AL    ./scripts/set-pulsar-version.sh
  AL    ./scripts/cert-manager/install-cert-manager.sh
  AL    ./scripts/pulsar/cleanup_helm_release.sh
  AL    ./scripts/pulsar/common.sh
  AL    ./scripts/pulsar/common_auth.sh
  AL    ./scripts/pulsar/generate_token.sh
  AL    ./scripts/pulsar/generate_token_secret_key.sh
  AL    ./scripts/pulsar/get_token.sh
  AL    ./scripts/pulsar/prepare_helm_release.sh
 
*****************************************************

```
2022-10-20 20:29:09 -05:00
Michael Marshall
a963e736fc
Improve documentation and testing for PodMonitors (#317)
* Imrpove documentation and testing for PodMonitors

* Fix missed references and a typo

### Motivation

Before upgrading to 3.0.0, we want to make sure the kube-prometheus-stack is well documented. 

### Modifications

* Update tests and examples to fully disable `PodMonitors` and the installation of the kube-prometheus-stack CRDs.

### Verifying this change

The current tests will cover these changes.
2022-10-20 17:40:47 -05:00
Michael Marshall
35090ec822
Include LICENSE and NOTICE in distribution 2022-10-20 15:48:07 -05:00
Michael Marshall
f509bc80ea
Add NOTICE file 2022-10-20 15:42:33 -05:00
Michael Marshall
49d345b6a3
Add LICENSE file before distributing source (#318)
### Motivation

In order to update the release process, we need to have a LICENSE file in the root directory. See https://www.apache.org/legal/release-policy.html#license-file and https://www.apache.org/legal/release-policy.html#full-copy-for-each-source-file.

### Modifications

* Copied the LICENSE file from the apache/pulsar repo
2022-10-20 15:39:20 -05:00
Michael Marshall
9324a9a270
Fix bookkeeper metadata init when specifying metadataPrefix (#316)
Fixes #309

### Motivation

Fix the metadataPrefix initialization.

### Modifications

* Fix the script by adding `&& echo`

### Verifying this change

I manually verified that this change works and correctly puts the metadata in the prefixed location.
2022-10-20 15:24:20 -05:00
Claudio Vellage
343ce0527d
Allow to use selectors with volumeClaimTemplates (#286)
* Allow to use selectors with volumeClaimTemplates

* Fixed naming inconsistency, added null value

Co-authored-by: Claudio Vellage <claudio.vellage@pm.me>
Co-authored-by: Michael Marshall <mmarshall@apache.org>

### Motivation

Currently it's not possible to use selectors with volumeClaimTemplates which makes it hard/impossible to bind statically provisioned PVs.

### Modifications

Added (optional) selectors to `volumeClaimTemplates` and documented in values file.

### Verifying this change

- [ ] Make sure that the change passes the CI checks.
2022-10-20 13:46:23 -05:00
Michael Marshall
1e8491aebd
Fix CI by modifying Chart.yaml and updating ct lint command (#315)
### Motivation

Fix the CI lint step by modifying the Chart.yaml and by removing the maintainers validation step.
2022-10-20 13:17:51 -05:00