Installation

Install the Firebolt Operator and its CRDs with Helm.

This page explains how to install the Firebolt Operator in a Kubernetes cluster.

Prerequisites

• Kubernetes 1.28 or later. The CRDs use CEL transition rules for field immutability.
• Helm installed on the machine or automation that deploys the Firebolt Operator.
• Access to the OCI Helm chart registry that hosts the Firebolt Operator charts.
• Engine nodes that run Linux kernel 6.1 or later. See Kernel requirement.
• Engine nodes that provide a locked-memory (memlock) limit of at least 8 GiB. See Engine node requirements.

Kernel requirement

The engine uses io_uring for asynchronous I/O, so every engine node must run Linux kernel 6.1 or later. On older kernels, such as 5.10, engines crash-loop at startup.

Check the kernel version on an engine node:

uname -r

Any version 6.1 or later is enough. If a node runs an older kernel, move your engine nodes to a node image that ships kernel 6.1 or later. Most current node images from managed Kubernetes providers already meet this requirement.

Engine node requirements

When an engine starts, it locks memory for io_uring, so it needs a memlock limit of at least 8 GiB. Each engine pod takes that limit from containerd, the container runtime on its node. If the limit is too low, the engine crashes at startup with this error:

io_uring_register_buffers failed: Cannot allocate memory

The Firebolt Operator cannot raise this limit for you, so every engine node must provide it. Before you change anything, check the current limit on one of your engine nodes:

systemctl show containerd --property=LimitMEMLOCK

This prints either infinity or a value in bytes. Anything that is infinity or at least 8 GiB (8589934592 bytes) is enough, and that node needs no further change. Some node images already provide a high enough limit, while others default to around 8 MiB, which is far too low.

If the limit is too low, raise memlock on the container runtime (containerd) with a systemd drop-in:

# /etc/systemd/system/containerd.service.d/memlock.conf
[Service]
LimitMEMLOCK=infinity

infinity is simplest. A bounded value such as LimitMEMLOCK=8G also works, as long as it is at least 8 GiB. Reload systemd, restart containerd, and confirm the limit:

systemctl daemon-reload
systemctl restart containerd
systemctl show containerd --property=LimitMEMLOCK

If you need to set it, bake the change into your node configuration so that upgraded, rotated, and autoscaled nodes get it automatically. How you deliver it depends on your cloud provider.

Amazon EKS

If you are using the Amazon EKS AMI, the memlock limit is already infinity, so you don’t need to change anything on your engine nodes.

If you are using a different OS, you need to verify the setting. One way to configure it on AWS is via an EC2 user-data script.

#!/bin/bash
mkdir -p /etc/systemd/system/containerd.service.d
printf '[Service]\nLimitMEMLOCK=infinity\n' \
  > /etc/systemd/system/containerd.service.d/memlock.conf
systemctl daemon-reload
systemctl restart containerd

Google GKE

GKE node system configuration does not expose memlock, so apply the drop-in with a privileged DaemonSet that writes it to each node and restarts containerd. The DaemonSet reapplies it to new nodes as they join, which covers upgrades and autoscaling:

apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: engine-memlock
  namespace: kube-system
spec:
  selector:
    matchLabels:
      app: engine-memlock
  template:
    metadata:
      labels:
        app: engine-memlock
    spec:
      hostPID: true
      # Restrict this to your engine node pool with a matching nodeSelector.
      containers:
        - name: set-memlock
          image: debian:stable-slim
          securityContext:
            privileged: true
          command: ["nsenter", "--target", "1", "--mount", "--uts", "--ipc", "--net", "--pid", "--", "sh", "-c"]
          args:
            - |
              mkdir -p /etc/systemd/system/containerd.service.d
              printf '[Service]\nLimitMEMLOCK=infinity\n' \
                > /etc/systemd/system/containerd.service.d/memlock.conf
              if ! systemctl show containerd --property=LimitMEMLOCK | grep -q infinity; then
                systemctl daemon-reload && systemctl restart containerd
              fi
              sleep infinity

The guard around the restart matters: restarting containerd also restarts this pod, and the check lets the replacement pod skip a second restart once the limit is already in place.

GKE Autopilot blocks privileged DaemonSets, so you cannot use this approach there directly. Either request a privileged-workload allowlist (--autopilot-privileged-admission) or run your engine node pool on GKE Standard.

Microsoft Azure (AKS)

AKS does not expose LimitMEMLOCK through its supported node configuration, so apply the same privileged DaemonSet shown for Google GKE, or bake the drop-in into a custom node image. Either way, it reapplies automatically to nodes added by node image upgrades and cluster autoscaling, so you do not have to touch nodes by hand.

Install the CRDs

The Firebolt Operator comes with three CustomResourceDefinitions:

• FireboltInstance
• FireboltEngine
• FireboltEngineClass

We provide two ways to install the CRDs. Either via a separate CRD chart or via the Firebolt Operator chart crds/ directory. If you would like to learn more about the pros and cons of each option, please refer to the Helm chart best practices for CRDs.

Option 1 (recommended): Separate CRD chart

Install the CRD chart before installing the Firebolt Operator chart. This is the recommended option as it allows you full control over the CRDs lifecycle.

helm upgrade --install firebolt-crds oci://oci.firebolt.io/firebolt-db/helm-charts/firebolt-operator-crds

Option 2: Firebolt Operator chart crds/ directory

The Firebolt Operator chart bundles CRDs in its crds/ directory. Helm gives this directory special handling: on helm install, CRDs in crds/ are installed before the chart templates are rendered. If a CRD already exists, Helm skips it with a warning. You can opt out of this bundled CRD installation with --skip-crds.

Helm does not upgrade or delete CRDs from a chart’s crds/ directory (this is the trade-off the Helm CRD best practices caution against for ongoing use). Prefer Option 1 for any deployment that will need to upgrade the operator over time.

Install the Firebolt Operator

Install the Firebolt Operator controller:

helm upgrade --skip-crds --install firebolt-operator oci://oci.firebolt.io/firebolt-db/helm-charts/firebolt-operator

Omit --skip-crds if you chose the bundled crds/ directory option above.

The oci.firebolt.io host is Firebolt’s Scarf gateway. It records the requested chart version and platform and redirects to GitHub Container Registry. To bypass the gateway, replace oci://oci.firebolt.io with oci://ghcr.io.

By default, the operator also sends one anonymous aggregate usage event to Scarf at most once per day. The event contains operator and engine versions, Kubernetes minor version, OS and architecture, and bucketed instance, engine, and replica counts. It contains no names, stable identifiers, query data, schemas, connection parameters, secrets, or configuration. As with any network request, the source IP is visible to Scarf; Scarf may use it to infer the company and does not store it.

Set --set telemetry.enabled=false to disable runtime telemetry and switch the default operator and engine image repositories to GHCR. Images explicitly configured in Helm values, FireboltEngine, or FireboltEngineClass resources are preserved. To bypass Scarf for the chart download too, install from oci://ghcr.io/firebolt-db/helm-charts; Helm selects the chart repository before reading chart values. DO_NOT_TRACK=1 and SCARF_NO_ANALYTICS=1 disable runtime events only.

Multi-tenant install: scope the Firebolt Operator to specific namespaces

By default the Firebolt Operator watches every namespace and the chart renders a ClusterRole plus ClusterRoleBinding so it can read and write the resources it manages anywhere in the cluster. If you only want the Firebolt Operator to act in a fixed set of namespaces, list them under watchNamespaces:

helm upgrade --skip-crds --install firebolt-operator \
  oci://oci.firebolt.io/firebolt-db/helm-charts/firebolt-operator \
  --set 'watchNamespaces={tenant-a,tenant-b}'

The chart then renders a Role plus RoleBinding in each listed namespace (carrying the same rule set the ClusterRole would have) and starts the manager with --namespaces=tenant-a,tenant-b so its cache only spans those namespaces. The Firebolt Operator’s blast radius is bounded to that list; CRs created in other namespaces are silently ignored.

To onboard a new tenant namespace, add it to watchNamespaces and re-run helm upgrade. The new Role and RoleBinding land in the new namespace and the manager restart picks up the extended flag.

If you also need the apiserver pod-proxy permission (because you set FireboltInstance.spec.metricScrapeMode=ApiserverProxy on any instance), enable the matching opt-in:

--set rbac.apiserverProxyGrant=true

This renders a dedicated ClusterRole (or per-namespace Role when watchNamespaces is set) that grants only pods/proxy: get. The default metricScrapeMode=PodIP does not need this permission.

Split CR ownership between installs with a label selector

A cluster-wide install can coexist with namespace-scoped installs by ignoring the CRs those installs own. Give the cluster-wide install a selector that excludes the label the scoped installs stamp on their CRs:

helm upgrade --skip-crds --install firebolt-operator \
  oci://oci.firebolt.io/firebolt-db/helm-charts/firebolt-operator \
  --set-string 'watchLabelSelector=!example.com/managed'

Each install then reconciles only the Firebolt CRs matching its selector (see --watch-label-selector in the flag table below for the exact scope). Three rules keep a split-ownership cluster healthy:

• Label whole stacks. CRs that reference each other (an engine and the instance or engine class it points at) must land on the same side of the selector. Never label individual CRs out of a stack.
• One install per namespace. Leader election uses a fixed lease ID, so two operator releases deployed into the same namespace contend for one lease and only one of them ever runs. Give every install its own release namespace.
• The selector is not an isolation boundary. RBAC is governed by watchNamespaces alone, and the chart registers admission webhooks cluster-wide: every install with webhook.enabled=true validates every Firebolt CR in the cluster, whatever its selector says. This is deliberate. Kubernetes could scope the webhook itself (an objectSelector on the webhook configuration, or a label check inside the handler), but the chart does not wire the cache selector into either: selector-scoped installs run webhook-less, so the cluster-wide install’s webhook is the only admission validation their CRs receive, and filtering it by the same selector would silently remove that. Enable webhooks on at most one install, and make it the cluster-wide one (empty watchNamespaces): the validators read live API state in whatever namespace a CR is admitted in, which needs the chart’s ClusterRole. A namespace-scoped install serving webhooks would fail admission for CRs outside its namespaces under the default failurePolicy: Fail.

Firebolt Operator flags

The Firebolt Operator supports these runtime flags. The binary default is what the manager uses when you run it directly. The Helm chart default is what the firebolt-operator chart passes with its default values.yaml.

Flag	Binary default	Helm chart default	Description
--version	false	Not set	Print the version and exit.
--namespaces	""	Derived from watchNamespaces (omitted when empty)	Comma-separated list of namespaces to watch. Empty watches every namespace (cluster-wide install, requires the chart’s ClusterRole). A non-empty list confines the manager cache to those namespaces and pairs with per-namespace Role and RoleBinding from the chart.
--watch-label-selector	""	Derived from watchLabelSelector (omitted when empty)	Label selector restricting which FireboltEngine, FireboltInstance, and FireboltEngineClass objects the operator caches and reconciles. Empty applies no restriction. Applies only to the three Firebolt CRD types; child objects and third-party Secrets are cached unfiltered, so they never need the label. Lets a cluster-wide install ignore CRs owned by namespace-scoped installs, e.g. !example.com/managed. CRs that reference each other (an engine and the instance or engine class it points at) must land on the same side of the selector: label whole stacks, never individual CRs.
--metrics-bind-address	0	:8443	Address for the metrics endpoint. Use 0 to disable metrics.
--metrics-secure	true	true	Serve metrics over HTTPS with Kubernetes authentication and authorization.
--metrics-cert-path	""	Not set	Directory that contains the metrics server certificate.
--metrics-cert-name	tls.crt	Not set	Metrics server certificate file name.
--metrics-cert-key	tls.key	Not set	Metrics server key file name.
--health-probe-bind-address	:8081	:8081	Address for health probes.
--leader-elect	false	true	Enable leader election for HA deployments.
--enable-webhooks	true	false	Enable the admission webhook server.
--webhook-cert-path	""	Not set	Directory that contains the webhook certificate. The chart sets /tmp/k8s-webhook-server/serving-certs when webhook.enabled=true.
--webhook-cert-name	tls.crt	Not set	Webhook certificate file name.
--webhook-cert-key	tls.key	Not set	Webhook key file name.
--enable-http2	false	Not set	Enable HTTP/2 for the metrics and webhook servers.
--engine-max-cpu	""	Not set	Maximum allowed engine-container CPU request and limit (resolved from FireboltEngine.spec.template.spec.containers[engine].resources or the referenced FireboltEngineClass’s container resources). Empty disables the bound.
--engine-max-memory	""	Not set	Maximum allowed engine-container memory request and limit (same resolution as --engine-max-cpu). Empty disables the bound.
--engine-max-ephemeral-storage	""	Not set	Maximum allowed engine-container ephemeral-storage request and limit (same resolution as --engine-max-cpu). Empty disables the bound.
--zap-devel	false	Not set	Enable controller-runtime development logging defaults.
--zap-encoder	json	json	Log encoding. Valid values are json and console.
--zap-log-level	info	info	Minimum log level. Valid values include debug, info, error, and panic.
--zap-stacktrace-level	error	error	Level at and above which stack traces are captured.
--zap-time-encoding	rfc3339	Not set	Timestamp encoding for zap logs.

Next step

After installation, follow the quickstart to create your first FireboltEngine.