### Motivation
Fixes feature enhancement request #6143:
Currently, there are quite a few undocumented steps that are needed to be performed manually in order to make sure that the functions can be submitted as pods in K8s runtime environment. It would be much better if this process would be automated.
#### Proposed solution:
Automate this process via helm install and update the helm charts with templates.
### Modifications
I've added an additional `functionsAsPods` filed in extra components inside the values file. If the setting is set to `yes`, then it would add `serviceAccount` to the broker deployment. It will also add the rbac policy to give the brokers permissions to deploy functions. The policies can be found in the new `broker-rbac.yaml` template file. Moreover, it will also change the `functions_worker` settings and set the function runtime factory setting that can be found inside `broker-configmap.yaml`.
### Verifying this change
1) Set `functionsAsPods: yes` inside helm values yaml file.
2) Follow the instructions on how deploying helm and run:
`helm install pulsar --values pulsar/values-mini.yaml ./pulsar/`.
3) Wait until all the services are up and running.
4) Set up tenant, namespace.
5) Create a function, sink and source and submit it using the CLI to make sure the pods are running alongside the Pulsar cluster. In addition, set up such a flow where the data is flowing from source to topics, the processed by a function and sink outputs the data
6) Push data into cluster through the source and make sure it comes out of the sink into destination. There shouldn't be any errors in the logs of brokers, bookie, sources, sinks and functions.
#### Modules affected:
The changes in the PR are affecting the deployment using the helm charts. Now the if the flag `functionsAsPods` is set to `yes` inside the `values.yaml. file, the functions would run as pods.
### Documentation
Currently, the documentations explaining the helm chart deployment process is lacking and this should be updated.
Fixes ##6314
### Motivation
Pulsar Manager do not work if Pulsar authentication is enabled.
### Modifications
pulsar-manager-configmap.yaml was created in order to allow configuration of the enviroment properties in values.yaml
### Motivation
Exposing Grafana via soft ingress controller so that it can be exposed through a Load Balancer.
#### Proposed solution:
Create ingress template for Grafana so that it can be automatically picked up if ingress controller instance is running in the cluster. The other solutions are to expose Grafana as NodePort or setting it as a LoadBalancer.
### Modifications
Added `grafana-ingress.yaml` template in the templates and an `ingress` section for Grafana in the values file.
### Verifying this change
1) Set ingress to `true` for Grafana in values file and provide hostname. Currently tested with NGINX, but can use another ingress controller, but will need to change the ingress controller class to another one in the template.
2) Add NGINX Helm repository :
```bash
helm repo add nginx-stable https://helm.nginx.com/stable
helm repo update
```
3) Install with Helm 3:
```bash
helm install nginix-ingress-crl nginx-stable/nginx-ingress
```
4) Follow the instructions on how deploying helm and run:
`helm install pulsar --values pulsar/values-mini.yaml ./pulsar/`.
5) Wait until all the services are up and running.
6) Verify that Grafana is accessible via url.
**Path settings**
Currently, by default the path setting is set to `/grafana`. For that to work, the NGINX configuration file `nginx.conf` should have `grafana` sub path enabled:
```
See https://grafana.com/docs/grafana/latest/installation/behind_proxy/
To avoid having to mess with NGINX configurations files `path` can be changed to `/`, but this path might conflict with other services that are being proxied in the cluster.
#### Modules affected:
The changes in the PR are affecting the deployment using the helm charts. Now the if the flag `functionsAsPods` is set to `yes` inside the `values.yaml. file, the functions would run as pods.
### Documentation
This PR will be adding ingress capability for Grafana and this should be documented.
### Motivation
Fixes#6131 (caused by #5675):
When upgrading an existing 2.4.1 bookie cluster to 2.5.0 on kubernetes, the bookie fails to start with the following exception during initialization: io.netty.util.internal.OutOfDirectMemoryError: failed to allocate 16777216 byte(s) of direct memory (used: 2147483648, max: 2147483648). This is caused by the fact that the bookie environment variables `BOOKIE_MEM` and `BOOKIE_FC` defined in conf/bkenv.sh has no effect, and it is always using the defaults values.
#### Proposed solution:
Set `BOOKIE_MEM` and `BOOKIE_GC` in the helm deployments charts and default to `PULSAR_MEM` if the `BOOKIE` settings are not set and then use the default settings if none of those environment variables are set.
#### Changes made
Helm chart deployment `values.yaml` and `values-mini.yaml` along with the `bkenv.sh` configuration script.
### Documentation
Currently, the documentation explaining the deployment process and how to change settings is lacking and need to be updated.
Motivation:
Fixes#5958:
The following error appears when trying to deploy Pulsar using helm and values-mini.yaml:
```unable to build kubernetes objects from release manifest: error validating "": error validating data: ValidationError(Deployment.spec.template.spec.containers[0]): unknown field "requests" in io.k8s.api.core.v1.Container```
Cause:
Mistake in the `pulsar-manager-deployment.yaml` deployment file:
First line **63** should be:
`{{- if .Values.pulsar_manager.resources }}` and it is currently `{{- if .Values.grafana.resources }}`
There is also a mistake at line **65**:
`{{ toYaml .Values.grafana.resources | indent 10 }}` should be `{{ toYaml .Values.pulsar_manager.resources | indent 12 }}`
Modifications:
Changed values in `values.yaml` and `templates/pulsar-manager-deployment.yaml`
Test:
Deploy the application into a kubernetes local cluster with
`helm install pulsar-cluster --values pulsar/values-mini.yaml pulsar`
AND
`helm install pulsar-cluster --values pulsar/values.yaml pulsar`
Documentation:
Does this pull request introduce a new feature? - **No**
Fixes#5857
### Motivation
With current aproach for specifying storage class in persistent volume claim it's not possible to customize the provisioner parameters. If the property 'storageClass' is declared the chart always create a new storage class with hardcoded parameters.
### Modifications
A property 'storageClassName' was added to support an existent storage class.
### Verifying this change
This change is a trivial rework / code cleanup without any test coverage.
This patch allows tls to be enabled with an empty secretName for ingress controllers might be able to provide a default certificate.
Fixes#5858, provides better defaults for the Ingress object and allows TLS to be enabled with an empty secretName.
### Motivation
The current helm chart can create an Ingress with TLS, but it requires a secretName to be added. This is not an Ingress requirement and, in some cases, the ingress controller can provide a default certificate when the Ingress object does not declare one.
### Modifications
Modifications include `values.yaml` and `dashboard-ingress.yaml` to address the issue.
Signed-off-by: xiaolong.ran <rxl@apache.org>
### Modifications
- Add [pulsar-manager](https://github.com/apache/pulsar-manager) to helm chart
- Replace pulsar-dashboard with pulsar-manager
- Currently, we can deprecate pulsar-dashboard, In later versions, we can use `pulsar-manager` replace `pulsar-dashboard`.
Signed-off-by: xiaolong.ran <rxl@apache.org>
Fixes#5687
### Motivation
When the user wants to add new keys for Env, adding fails if no prefix is added.
Currently, add new keys for Env use the script of [apply-config-from-env.py](https://github.com/apache/pulsar/commits/master/docker/pulsar/scripts/apply-config-from-env.py), to ensure that the env set by the user can take effect, add the prefix(**PULSAR_PREFIX_**) for all keys.
### Modifications
- Add prefix for new keys from Env
Allows to opt-in for an ingress on top of the dashboard service.
This is very important in production-grade deployments where
you want to expose the Pulsar dashboard through an easy to remember URL.
### Motivation
The following errors occurs when running :
helm lint pulsar/
==> Linting pulsar/
[INFO] Chart.yaml: icon is recommended
[ERROR] templates/: render error in "pulsar/templates/zookeeper-metadata.yaml": template: pulsar/templates/zookeeper-metadata.yaml:49:20: executing "pulsar/templates/zookeeper-metadata.yaml" at <.Values.zookeeper_me...>: can't evaluate field resources in type interface {}
### Modifications
Change zookeeper_metadata in deployment/kubernetes/helm/pulsar/templates/zookeeper-metadata.yaml to zookeeperMetadata
### Result
helm lint pulsar/
==> Linting pulsar/
[INFO] Chart.yaml: icon is recommended
1 chart(s) linted, no failures
* [documentation][deploy] Update deployment instructions for deploying to Minikube
* Enable functions workers
* [documentation][deploy] Improve helm deployment script to deploy Pulsar to minikube
### Changes
- update the helm scripts: bookie/autorecovery/broker pods should wait until metadata is initialized
- disable `autoRecovery` on bookies since we start `AutoRecovery` in separate pods
- enable function worker on brokers
- provide a values file for minikube
- update documentation for using helm chart to deploy a cluster to minikube
* move the service type definition to values file
* Helm charts for deployment on GKE
* Repackaginh helm charts under deployment/kubernetes/helm
* Formatting licences
* Removing cloud specific values to enable more generic deployments
