Upgrade to kube-prometheus-stack 69.x including prometheus-operator 0.80.0 defaulting to Prometheus 3.x (#578)

* Upgrade to kube-prometheus-stack 67.x
  * Prometheus operator is upgraded to 0.80.0
  * Prometheus is upgraded from 2.55.0 to 3.2.1

* Enable pod monitors to test them

* Run linting with kube-prometheus-stack enabled

* Validate all CI configs

.ci/clusters/values-prometheus-grafana.yaml

@@ -28,3 +28,31 @@ kube-prometheus-stack:
     enabled: false
   prometheus-node-exporter:
     enabled: true
+
+zookeeper:
+  podMonitor:
+    enabled: true
+
+bookkeeper:
+  podMonitor:
+    enabled: true
+
+broker:
+  podMonitor:
+    enabled: true
+    
+autorecovery:
+  podMonitor:
+    enabled: true
+
+proxy:
+  podMonitor:
+    enabled: true
+
+oxia:
+  coordinator:
+    podMonitor:
+      enabled: true
+  server:
+    podMonitor:
+      enabled: true

.github/workflows/pulsar-helm-chart-ci.yaml

@@ -127,7 +127,7 @@ jobs:
             --validate-maintainers=false \
             --target-branch ${{ github.event.repository.default_branch }}
 
-      - name: Run kubeconform check for helm template with every major k8s version 1.23.0-1.30.0
+      - name: Run kubeconform check for helm template with every major k8s version 1.23.0-1.32.0
         if: ${{ steps.check_changes.outputs.docs_only != 'true' }}
         run: |
           PULSAR_CHART_HOME=$(pwd)
@@ -151,11 +151,14 @@ jobs:
               kubeconform -schema-location default -schema-location 'https://raw.githubusercontent.com/datreeio/CRDs-catalog/main/{{.Group}}/{{.ResourceKind}}_{{.ResourceAPIVersion}}.json' -strict -kubernetes-version $kube_version -summary
           }
           set -o pipefail
-          for k8s_version_part in {23..30}; do
+          for k8s_version_part in {23..32}; do
             k8s_version="1.${k8s_version_part}.0"
+            echo "Validating default values with k8s version $k8s_version"
             validate_helm_template_with_k8s_version $k8s_version
-            echo "Validating with Oxia enabled"
-            validate_helm_template_with_k8s_version $k8s_version --set components.zookeeper=false --set components.oxia=true
+            for config in .ci/clusters/*.yaml; do
+              echo "Validating $config with k8s version $k8s_version"
+              validate_helm_template_with_k8s_version $k8s_version --values .ci/values-common.yaml --values $config
+            done
           done
 
       - name: Validate kustomize yaml for extra new lines in pulsar-init commands

README.md

@@ -245,6 +245,20 @@ helm upgrade -f pulsar.yaml \
 
 For more detailed information, see our [Upgrading](http://pulsar.apache.org/docs/helm-upgrade/) guide.
 
+## Upgrading from Helm Chart version 3.x.x to 4.0.0 version and above
+
+The kube-prometheus-stack version has been upgraded to 69.x.x in Pulsar Helm Chart version 4.0.0 .
+Before running "helm upgrade", you should first upgrade the Prometheus Operator CRDs as [instructed
+in kube-prometheus-stack upgrade notes](https://github.com/prometheus-community/helm-charts/tree/main/charts/kube-prometheus-stack#from-68x-to-69x). 
+
+There's a script to run the required commands:
+
+```shell
+./scripts/kube-prometheus-stack/upgrade_prometheus_operator_crds.sh 0.80.0
+```
+
+After, this you can proceed with `helm upgrade`.
+
 ## Upgrading from Helm Chart version 3.0.0-3.6.0 to 3.7.0 version and above
 
 The kube-prometheus-stack version has been upgraded to 65.x.x in Pulsar Helm Chart version 3.7.0 .

charts/pulsar/Chart.yaml

@@ -21,7 +21,7 @@ apiVersion: v2
 appVersion: "4.0.3"
 description: Apache Pulsar Helm chart for Kubernetes
 name: pulsar
-version: 3.9.0
+version: 4.0.0
 kubeVersion: ">=1.23.0-0"
 home: https://pulsar.apache.org
 sources:
@@ -33,6 +33,6 @@ maintainers:
     email: dev@pulsar.apache.org
 dependencies:
   - name: kube-prometheus-stack
-    version: 65.x.x
+    version: 67.x.x
     repository: https://prometheus-community.github.io/helm-charts
     condition: kube-prometheus-stack.enabled

scripts/kube-prometheus-stack/upgrade_prometheus_operator_crds.sh

@@ -21,7 +21,7 @@
 # This script is used to upgrade the Prometheus Operator CRDs before running "helm upgrade"
 # source: https://github.com/prometheus-community/helm-charts/tree/main/charts/kube-prometheus-stack#upgrading-an-existing-release-to-a-new-major-version
 # "Run these commands to update the CRDs before applying the upgrade."
-PROMETHEUS_OPERATOR_VERSION="${1:-"0.77.1"}"
+PROMETHEUS_OPERATOR_VERSION="${1:-"0.80.0"}"
 PREFIX_URL="https://raw.githubusercontent.com/prometheus-operator/prometheus-operator/v${PROMETHEUS_OPERATOR_VERSION}/example/prometheus-operator-crd"
 for crd in alertmanagerconfigs alertmanagers podmonitors probes prometheusagents prometheuses prometheusrules scrapeconfigs servicemonitors thanosrulers; do
   # "--force-conflicts" is required to upgrade the CRDs. Following instructions from https://github.com/prometheus-community/helm-charts/issues/2489