172 Commits

Author SHA1 Message Date
gulecroc
6e824f0c4e
Fix bookkeeper.extraVolumes (#596) 2025-04-15 01:04:10 -07:00
Lari Hotari
b703761a52
Upgrade Oxia to 0.11.15 (#600) 2025-04-15 00:50:32 -07:00
Lari Hotari
637cf11d1a
Fix Grafana dashboards for Broker with honorLabels, remove unnecessary *_created metrics and improve docs (#593)
* Drop _created metrics for broker and proxy

* Enable all metrics by default for broker

* change default dashboard

* Remove messy dashboards

* Enable default dashboards in Grafana

* Add testing values with more aggressive disk cleanup

* Add VictoriaMetrics debugging instructions

* Set honorLabels to true

* Document disabling monitoring

* Set password in testing values

* Fix linting issue detected by kubeconform
2025-03-10 16:46:28 -07:00
Lari Hotari
e6f05809bd
Migrate from kube-prometheus-metrics to victoria-metrics-k8s-stack (#592) 2025-03-08 16:36:41 -08:00
Lari Hotari
302db43e91
Remove PSP support (#591) 2025-03-08 12:00:35 -08:00
Lari Hotari
75119dd6d7
Remove Prometheus scrape annotations when podmonitors are enabled (#590) 2025-03-07 09:51:06 -08:00
Lari Hotari
6fe37a373f
Use bookkeeperMetadataServiceUri in broker and make PulsarMetadataClientDriver configurable (#589) 2025-03-07 09:24:03 -08:00
Lari Hotari
dd1325216f
Change Pulsar Proxy service load balancer type to ClusterIP (#588) 2025-03-06 05:03:42 -08:00
Lari Hotari
80999ff1d8
Use BookKeeper BP-29 metadataServiceUri to configure bookie metadata store, also when using Zookeeper (#585) 2025-03-05 23:24:07 -08:00
Lari Hotari
9f61859d19
Use PIP-45 metadata store config to replace deprecated ZK config and make PulsarMetadataBookieDriver configurable in BK (#576) 2025-03-04 20:23:35 +02:00
Lari Hotari
43f8dfa04e
Revisit solution to configure Bookkeeper RocksDB settings - default to individual config files (#583) 2025-03-04 04:04:38 -08:00
Lari Hotari
f98ee7d69c
Replace ">" with "|" to avoid Go Yaml issue go-yaml/yaml#789 (#582) 2025-03-04 02:21:39 -08:00
Lari Hotari
5c1b7a9288
Restore support for dbStorage_rocksDB_* settings defined in bookkeeper.configData (#580) 2025-03-03 22:05:59 -08:00
Lari Hotari
4de387e726
Workaround issue with Prometheus 3.0 and metrics (#577)
* Add "fallbackScrapeProtocol: PrometheusText0.0.4" to all pod monitors
2025-03-03 06:26:04 -08:00
Lari Hotari
4386eacba8
[fix] Fix broker service annotations issue and other annotations issues (#574)
* Fix broker services annotations issues

* Add annotations support to autorecovery.service

* Consistently use similar way to handle annotations

* Add autorecovery service annotations key to values.yaml
2025-02-28 09:17:54 -08:00
Lari Hotari
f928380124
Fix pulsar-cluster-initialize / pulsar-init rendering with kustomize (#572)
* Fix pulsar-cluster-initialize / pulsar-init rendering with kustomize

- reapply #166 changes that were reverted by #544 changes

* Add validation for kustomize output in CI
2025-02-19 00:46:24 -08:00
Alejandro Ramirez
0b6b03002c
Fix OOM issue on broker wait-zookeeper-ready initContainer (#568) 2025-02-18 22:26:39 -08:00
Eric Shen
b5ff00b16b
feat(tls): support ca type issuer and v1alpha* version cert-manager api (#561) 2024-12-18 07:11:54 -08:00
Raúl Sánchez
df9284dc97
Fix helm chart to allow configurable ingress pathType (#558) 2024-12-11 07:21:03 -08:00
Lari Hotari
0eeb7830a9
Revert "Wrap Zookeeper probe script with timeout command (#214)" (#556)
This reverts commit 22f4b9b3bd18a16c477003338464dfe5a689e074.
2024-12-02 01:35:22 -08:00
Lari Hotari
07689860f6
Fix Oxia config so that it includes a list of all pods in the statefulset (#553)
* Fix Oxia config so that it includes a list of all pods in the statefulset

* Test Oxia with 3 replicas since some issues only come up with more nodes

* Make internal name not a fqdn

* Fix issue with insufficient cpu requests in CI
2024-11-22 05:54:11 -08:00
Lari Hotari
cc12992d8f
Fix invalid internal server name in Oxia config (#552)
.svc doesn't resolve. it's better to use the fully qualified name
2024-11-22 04:35:54 -08:00
Yuwei Sung
c6ce11a9b7
Add support for using Oxia as the metadata store for Pulsar and BookKeeper (#544)
Co-authored-by: Lari Hotari <lhotari@apache.org>
2024-11-21 16:52:20 -08:00
Liam Gibson
17b739d10a
Add support for admin port on ZooKeeper (#550)
* Add support for admin port on ZooKeeper

* Make ZK admin port conditional
2024-11-20 09:27:44 -08:00
doug-ba
f6b6d88847
Correct pulsar proxy prometheus.io/port annotation (#548) 2024-11-18 21:39:24 -08:00
lenglet-k
ed50c68633
feat: add loadBalancerClass for proxy and pulsar-manager (#546)
* feat: add loadBalancerClass for proxy and pulsar-manager

Co-authored-by: Lari Hotari <lhotari@users.noreply.github.com>
2024-11-08 07:23:45 -08:00
Lari Hotari
d877fc3312
Use Pulsar 4.0.0 image, bump chart version to 3.7.0, kube-prometheus-stack to 65.x (#542)
* Use Pulsar 4.0.0 image, bump chart version to 3.7.0

* Bump kube-prometheus-stack to 65.x.x

* Remove testing with latest and test with previous LTS version

- run kube-prometheus-stack test with previous LTS version since
  the older chart version doesn't support Pulsar 4.0.0 image

* Fix passing "--values" to helm command

* Move ci runner config to a script

* Attempt to fix pulsar-manager-cluster-initialize
2024-10-29 15:29:27 -07:00
ChaoYang
64e67c1a88
update role (#543) 2024-10-29 15:28:47 -07:00
lenglet-k
db20c2bfa6
fix: broker extraEnv variable (#540)
* fix: broker extraEnv variable

* fix: comment extraEnv for broker as default values

* fix(typo): rename extreEnvs to extraEnvs
2024-10-18 00:07:24 -07:00
lenglet-k
346c5cdcd4
feat! add extraVolumes and Mounts for pulsar-manager (#535) 2024-10-08 05:00:00 -07:00
lenglet-k
75c00ebc7a
feat: add imagepullsecrets on pulsar-manager-initialize job (#533) 2024-10-02 17:15:46 -07:00
Lari Hotari
fffdcfc1ad
Fix compatibility with Pulsar 3.3.x+ docker images where /pulsar isn't writable (#531) 2024-09-27 12:17:12 -07:00
Shu.Wang
a45bc4bfe1
Add topologyspreadconstraint to deploy pods in sts cross different az evenly (#526)
Signed-off-by: Wang, Shu <shu.wang@fmr.com>
2024-09-26 21:37:15 -07:00
ludmanl
54401c0b9a
feat: Support to customize broker podManagementPolicy from values.yaml (#525) 2024-09-03 03:47:52 -07:00
Duncan Schulze
0031827761
Support using self generated certificates (#523)
* Support using self generated certificates

* chore: fix linting
2024-08-23 17:49:36 +03:00
Lari Hotari
dc817205a1
Bump minimum k8s version to 1.23.0 (#518) 2024-08-15 00:55:22 -07:00
Starry
093fa273f8
Add initContainers to templates (#516) 2024-08-05 09:40:55 -07:00
Lari Hotari
70f36ffe43
Add timeouts for cluster metadata initialization and for init containers (#218)
- Add timeouts for waiting for zk and bk to become available.
- If the waiting gets stuck for some reason, the Pulsar deployment never
  becomes starts the broker services.
  - timeouts will help failures recover eventually
2024-06-20 10:07:48 -07:00
Lari Hotari
023f902a02
Allow specifying default pull policy and functions pull policy (#507) 2024-06-12 04:16:48 -07:00
Lari Hotari
9db0cccaca
Make zookeeper healthchecks compatible with Alpine's busybox nc (#504)
* Make zookeeper healthchecks compatible with Alpine's busybox nc

* Test Pulsar 3.3.0 image

* Use 127.0.0.1 instead of localhost in zookeeper healthchecks

- Alpine nc fails if "localhost" is used.
  - perhaps it defaults to use IPv6?

* Disable testing with Pulsar 3.3.0 image until 3.3.1 is released

- the image needs "apk add bind-tools" since busybox nslookup isn't compatible with kubernetes
2024-06-08 08:52:06 +03:00
Lari Hotari
47c2ac442a
Add defaultPulsarImageRepository configuration (#503)
- makes it easier to use a custom image
2024-06-05 04:20:16 -07:00
Massimiliano Mirelli
6e84409b48
Support NodePort Proxy service (#500)
* Enables nodeport support for the proxy

* Correct indentation and remove null `nodePort`

Removing null `nodePort` causes k8s to pick up a random port

* Address review comment

https://github.com/apache/pulsar-helm-chart/pull/500/files#r1605762312
2024-06-04 08:46:16 -07:00
Massimiliano Mirelli
cb5c44f8ec
Allow broker's service clusterIP customisation (#498)
* Allow broker's service clusterIP customisation

This customisation is useful to configure headless vs non-headless
broker's service. The default is headless broker service, i.e. a
service for which kubernetes  does not allocate an IP
address (https://kubernetes.io/docs/concepts/services-networking/service/#type-clusterip). A
headless service is a very simple type of service that doesn't seem to work well
when pulsar service is exposed by pulsar-proxy via a nodeport.

Addresses #497.

* Address review comments

https://github.com/apache/pulsar-helm-chart/pull/498/files#r1605762934
and https://github.com/apache/pulsar-helm-chart/pull/498/files#r1605763245

* Move doc to Values.broker.service
2024-06-04 08:45:14 -07:00
MonicaMagoniCom
c4941b32d1
Add namespace to hpa templates (#494) 2024-05-03 11:48:02 -07:00
Lari Hotari
bd8bc633df
Change default statusFilePath to /pulsar/logs/status (#489)
* Change default statusFilePath to /pulsar/logs/status

* Write OK to statusFilePath
2024-04-15 05:41:17 -07:00
Martin
347326e0c3
Fix pulsar-manager persistence (#485)
- only setup environment in pulsar manager if broker is deployed
- fix indent
- enable persistence for manager and move configs around
2024-04-03 21:28:46 -07:00
Lari Hotari
fdd46f9b74
Add basic NOTES.txt (#482) 2024-03-27 04:32:36 -07:00
Lari Hotari
fdec9c69ef
Use podManagementPolicy OrderedReady for Broker sts when Functions are enabled (#474)
* Use podManagementPolicy OrderedReady for Broker sts when Functions are enabled

* Don't change podManagementPolicy when the sts already exists

* Fix template issue

* Fix apiVersion
2024-03-26 10:49:33 -07:00
doug-ba
9929b80b3c
add ability to use separate disk for zookeeper tx log (#476)
* add ability to use separate disk for zookeeper tx log

* Use absolute path

---------

Co-authored-by: Lari Hotari <lhotari@users.noreply.github.com>
2024-03-26 07:51:31 -07:00
Lari Hotari
eb0a878d9c
Make job.ttl.enabled consistent and effective only when k8s >= 1.23 (#481) 2024-03-26 06:23:15 -07:00