docker-podman-installation-with-nginx-manager/README_en.md

# Checkmk in a container – Docker (Debian) & Podman (RHEL) with NGINX Proxy Manager

*[Deutsche Fassung: README.md](README.md)*

A complete, production-ready container installation of **Checkmk** with
**NGINX Proxy Manager** for HTTPS termination (Let's Encrypt *or* self-signed),
including tooling for **backup**, **restore** and the **migration of existing
sites from a bare-metal installation** – without having to re-register the
agents.

> **Note on language:** the scripts and their menus are in German. Literal
> program output and menu screenshots in this document are therefore shown
> unchanged; the surrounding text explains what they mean.

---

## Table of contents

1. [Overview & architecture](#1-overview--architecture)
2. [Directory layout](#2-directory-layout)
3. [Requirements & ports](#3-requirements--ports)
4. [Installation on Debian/Ubuntu (Docker)](#4-installation-on-debianubuntu-docker)
5. [Installation on RHEL/Rocky/Alma/Fedora (Podman)](#5-installation-on-rhelrockyalmafedora-podman)
6. [Starting automatically after a host reboot (Podman)](#6-starting-automatically-after-a-host-reboot-podman)
7. [Configuration (`.env`)](#7-configuration-env)
8. [Obtaining the Checkmk image (Raw, Enterprise & co.)](#8-obtaining-the-checkmk-image-raw-enterprise--co)
9. [First start](#9-first-start)
10. [Setting up HTTPS – Let's Encrypt or self-signed](#10-setting-up-https--lets-encrypt-or-self-signed)
11. [Configuring the Proxy Manager locally (terminal browser)](#11-configuring-the-proxy-manager-locally-terminal-browser)
12. [Moving the admin interface to a different port](#12-moving-the-admin-interface-to-a-different-port)
13. [Firewall](#13-firewall)
14. [Management: `cmk-manage.sh` (CLI and ncurses interface)](#14-management-cmk-managesh)
15. [Backup](#15-backup)
16. [Restore](#16-restore)
17. [Migrating a bare-metal installation (`migrate_creator.sh`)](#17-migrating-a-bare-metal-installation-migrate_creatorsh)
18. [Several sites in one container](#18-several-sites-in-one-container)
19. [Upgrading Checkmk](#19-upgrading-checkmk)
20. [Troubleshooting](#20-troubleshooting)
21. [Security notes](#21-security-notes)
22. [Uninstalling](#22-uninstalling)

---

## 1. Overview & architecture

```
                    Internet / LAN
                          |
                     :80  :443  :81
                          |
        +-----------------------------------+
        |  NGINX Proxy Manager (container)  |   network_mode: host
        |  - Let's Encrypt OR self-signed   |
        |  - web interface on port 81       |
        +-----------------------------------+
                          |  http://127.0.0.1:5000
                          v
        +-----------------------------------+
        |  Checkmk / OMD (container)        |   network_mode: host
        |  :5000  web GUI (Apache)          |
        |  :8000  agent receiver            |   <-- agents (cmk-agent-ctl)
        |  :6557  Livestatus (optional)     |
        +-----------------------------------+
                          |
             bind mount: ./data/checkmk/sites  ->  /omd/sites
```

**Deliberate design decisions**

| Topic | Implementation | Rationale |
|---|---|---|
| Checkmk on the host network | `network_mode: host`, **no** `ports:` entries | agents, SNMP traps, ICMP and the agent receiver work without NAT; the source IP of the monitored hosts is preserved |
| NGINX Proxy Manager also on the host network | `network_mode: host` | reaches the Checkmk backend directly via `127.0.0.1:5000`; no bridge network, no `host-gateway` tricks – behaves identically under Docker **and** Podman |
| No named volumes | **bind mounts** into `./data/` only | all data sits visibly in the project directory, easy to back up, copy and inspect |
| One compose file for both | `docker compose` and `podman-compose` | identical operation on Debian and RHEL |
| `:Z` on the bind mounts | SELinux relabelling | required on RHEL; a no-op on Debian/Docker (not an error) |

---

## 2. Directory layout

```
.
├── README.md                        # German documentation
├── README_en.md                     # this file
├── docker-compose.yml               # for Docker AND Podman
├── .env.example                     # template -> .env
├── install.sh                       # installs Docker (Debian) / Podman (RHEL)
│
├── scripts/
│   ├── cmk-manage.sh                # stack, backup, restore, import + ncurses GUI
│   ├── migrate_creator.sh           # -> copy this to the SOURCE SERVER
│   └── lib/
│       └── common.sh                # shared functions + TUI helpers
│
├── containers/
│   └── browser/
│       └── Containerfile            # tools image: text browsers, curl, jq, openssl
│
├── systemd/
│   ├── checkmk-stack.service.example # compose-based unit (autostart)
│   └── quadlet/
│       ├── checkmk.container         # Podman Quadlet (alternative)
│       └── nginx-proxy-manager.container
│
├── data/                            # ALL payload data (bind mounts)
│   ├── checkmk/sites/               # -> /omd/sites   (complete OMD sites)
│   └── npm/
│       ├── data/                    # -> /data            (NPM config/DB)
│       ├── letsencrypt/             # -> /etc/letsencrypt (certificates)
│       └── custom-certs/            # self-signed certificates ready to upload
│
├── backups/                         # backups and migration bundles (*.tar.gz)
└── images/                          # downloaded Checkmk image tarballs
```

---

## 3. Requirements & ports

* 64-bit Linux, at least 2 CPU cores and 4 GB RAM (recommended: 4 cores, 8 GB)
* Disk space: rule of thumb 1 GB per 100 monitored hosts and year, plus backups
* root privileges
* Outbound internet access to download the images

### Ports in use (host network!)

| Port | Service | Reachable from |
|---|---|---|
| 80/tcp | NGINX Proxy Manager – HTTP, Let's Encrypt challenge | internet/LAN |
| 443/tcp | NGINX Proxy Manager – HTTPS | internet/LAN |
| 81/tcp | NGINX Proxy Manager – **admin interface** (default; changeable via [`npm-port`](#12-moving-the-admin-interface-to-a-different-port)) | admin network only! |
| 5000/tcp | Checkmk Apache (backend) | **localhost only** – do not expose |
| 8000/tcp | Checkmk agent receiver (TLS-registered agents) | monitored hosts |
| 6557/tcp | Livestatus (only with `CMK_LIVESTATUS_TCP=on`) | distributed sites |

> **Important:** because both containers run on the host network, ports 80, 443
> and 81 must not already be taken on the host (e.g. by an installed
> Apache/NGINX). Check with `ss -tlnp | grep -E ':(80|81|443|5000)\b'`.

---

## 4. Installation on Debian/Ubuntu (Docker)

```bash
# put the project where it should live
sudo mkdir -p /opt/checkmk-stack
sudo cp -r ./* ./.env.example /opt/checkmk-stack/
cd /opt/checkmk-stack

# install Docker CE + compose plugin, prepare the project
sudo ./install.sh --with-systemd
```

`install.sh` does the following:

1. adds the Docker repository from `download.docker.com` (GPG key + sources list)
2. installs `docker-ce`, `docker-ce-cli`, `containerd.io`, `docker-buildx-plugin`,
   `docker-compose-plugin`
3. installs `dialog`, `openssl`, `tar`, `curl` (for the ncurses interface)
4. runs `systemctl enable --now docker`
5. creates the directories under `data/` and `backups/` and generates `.env`
   from `.env.example`
6. optionally sets up the systemd unit `checkmk-stack.service`

Then:

```bash
sudo $EDITOR /opt/checkmk-stack/.env     # at least CMK_PASSWORD and CMK_SITE_ID

# choose the image (Raw needs no login, Enterprise see chapter 8)
sudo ./scripts/cmk-manage.sh image pull --edition raw --version 2.3.0-latest

sudo ./scripts/cmk-manage.sh up
sudo ./scripts/cmk-manage.sh status
```

Under Docker, `restart: unless-stopped` already brings the containers back
after a reboot (the Docker daemon starts them). The systemd unit is still
worthwhile because it additionally starts **all** sites
(see [chapter 18](#18-several-sites-in-one-container)).

---

## 5. Installation on RHEL/Rocky/Alma/Fedora (Podman)

Docker is not available on Red Hat systems – **Podman** is used instead. The
compose file is the same one.

```bash
sudo mkdir -p /opt/checkmk-stack
sudo cp -r ./* ./.env.example /opt/checkmk-stack/
cd /opt/checkmk-stack

sudo ./install.sh --with-systemd
```

`install.sh` installs:

1. `podman`, `tar`, `dialog`, `openssl`
2. `podman-compose` – attempted in this order:
   distribution repository → EPEL → `pip3 install podman-compose`
3. sets up the systemd unit (**always** with Podman, see the next chapter)

Done manually, that is:

```bash
sudo dnf install -y podman podman-compose tar dialog openssl
# if podman-compose is not in the repository:
sudo dnf install -y https://dl.fedoraproject.org/pub/epel/epel-release-latest-9.noarch.rpm
sudo dnf install -y podman-compose
```

Starting up:

```bash
sudo $EDITOR /opt/checkmk-stack/.env

# obtain the image - for Enterprise with credentials, see chapter 8
sudo ./scripts/cmk-manage.sh image pull --edition enterprise --version 2.3.0p23 \
     --user <user> --password <password>

sudo ./scripts/cmk-manage.sh up          # automatically uses podman-compose
sudo ./scripts/cmk-manage.sh status
```

> On RHEL, do read [chapter 8.2](#8-obtaining-the-checkmk-image-raw-enterprise--co):
> short image names and `podman import` instead of `podman load` are the two
> causes of "image not known".

### 5.1 Unregistered RHEL: where the packages come from

A freshly installed Red Hat Enterprise Linux without a subscription hands out
nothing at all:

```
$ sudo dnf install -y podman
This system is not registered with an entitlement server.
You can use "rhc" or "subscription-manager" to register.
Error: There are no enabled repositories in "/etc/yum.repos.d".
```

This affects **every** package – `podman`, `git`, `dialog`, `tar`. There are
three ways out, and `install.sh` now detects the situation and names them.

> Only RHEL itself is affected. **Rocky Linux, AlmaLinux, CentOS Stream and
> Fedora** need no registration – `dnf` works there straight away.

#### Route 1 – free Developer Subscription (recommended)

Red Hat hands out the *Developer Subscription for Individuals* free of charge.
It covers **up to 16 systems**, physical or virtual, and explicitly allows
**small production use** alongside development and testing. It is granted to
the individual, not to a company.

1. Create a free account at <https://developers.redhat.com>
2. Register the server:

```bash
sudo subscription-manager register --username <user>
# you will be prompted for the password; alternatively --password <password>
```

RHEL 9 uses *Simple Content Access* – the repositories are enabled right away,
a `subscription-manager attach` is **no longer** necessary.

3. Verify:

```bash
sudo subscription-manager status      # -> Overall Status: Current
sudo dnf repolist                     # -> rhel-9-for-x86_64-baseos-rpms etc.
sudo dnf install -y git podman
```

If individual repositories are missing:

```bash
sudo subscription-manager repos --list-enabled
sudo subscription-manager repos --enable=rhel-9-for-x86_64-appstream-rpms
```

With a corporate account and an activation key, use this instead:

```bash
sudo subscription-manager register --org <org-id> --activationkey <key>
```

#### Route 2 – installation DVD as a local repository (no account, offline)

Works without any registration and without internet access. You need the
**DVD ISO** (`rhel-9.x-x86_64-dvd.iso`, several GB) – the small `boot.iso`
contains no packages.

```bash
# mount the ISO permanently
sudo mkdir -p /mnt/rhel
sudo mount -o loop,ro /path/rhel-9.8-x86_64-dvd.iso /mnt/rhel

# mount it automatically at boot
echo '/path/rhel-9.8-x86_64-dvd.iso /mnt/rhel iso9660 loop,ro,nofail 0 0' \
  | sudo tee -a /etc/fstab

# create the repository file
sudo tee /etc/yum.repos.d/rhel-local.repo >/dev/null <<'REPO'
[rhel-local-baseos]
name=RHEL local - BaseOS
baseurl=file:///mnt/rhel/BaseOS
enabled=1
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-release

[rhel-local-appstream]
name=RHEL local - AppStream
baseurl=file:///mnt/rhel/AppStream
enabled=1
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-release
REPO

sudo dnf clean all
sudo dnf repolist
sudo dnf install -y podman tar openssl dialog git
```

> **Limitation:** the local repository holds exactly the state of the ISO –
> there are **no security updates**. For a permanently running monitoring
> server that is not acceptable; as a stopgap until registration is in place it
> is fine.

#### Route 3 – switch to Rocky Linux or AlmaLinux

Both are binary-compatible rebuilds of RHEL, permanently free and without
registration. This project runs there unchanged – `install.sh` recognises them
as part of the RHEL family.

```bash
# Rocky Linux
sudo dnf install -y git
sudo git clone https://github.com/rocky-linux/rocky-tools.git
sudo bash rocky-tools/migrate2rocky/migrate2rocky.sh -r

# AlmaLinux
curl -O https://raw.githubusercontent.com/AlmaLinux/almalinux-deploy/master/almalinux-deploy.sh
sudo bash almalinux-deploy.sh
```

> Take a backup or a snapshot first. The conversion replaces the base packages
> on a running system.

#### What this project actually needs

| Package | Purpose | Repository |
|---|---|---|
| `podman` | container runtime | AppStream |
| `tar` | backups and migration bundles | BaseOS |
| `openssl` | self-signed certificates | BaseOS |
| `dialog` | ncurses interface | AppStream |
| `newt` (`whiptail`) | substitute if `dialog` is missing | BaseOS |
| `curl` | image download | BaseOS |
| `podman-compose` | compose support | EPEL or `pip3` |
| `git` | only needed to clone this project | AppStream |

`dialog` is **optional**: if it is missing, the scripts automatically use
`whiptail` from the `newt` package, which lives in BaseOS. `install.sh` tries
exactly this order and therefore no longer aborts.

`git` is not strictly required – the project can just as well be transferred as
a ZIP file or via `scp`.

> **EPEL does not help here:** the EPEL repository depends on Red Hat's base
> repositories. Without a subscription or a local repository you cannot even
> install `epel-release` – and therefore no `podman-compose`. In that case the
> Quadlet route from
> [chapter 6](#6-starting-automatically-after-a-host-reboot-podman) is the
> answer: it needs no compose at all.

### 5.2 SELinux

The bind mounts carry `:Z` in `docker-compose.yml`, so Podman automatically
applies the `container_file_t` context. Should access still fail:

```bash
sudo semanage fcontext -a -t container_file_t "/opt/checkmk-stack/data(/.*)?"
sudo restorecon -Rv /opt/checkmk-stack/data
sudo ausearch -m avc -ts recent          # inspect AVC messages
```

---

## 6. Starting automatically after a host reboot (Podman)

> **This is the most important difference from Docker:** Podman has *no
> daemon*. `restart: unless-stopped` in the compose file therefore does **not**
> survive a reboot. Autostart has to go through systemd. There are two proven
> ways.

### Option A – compose-based systemd unit (recommended, identical to Docker)

Created and enabled automatically by `./install.sh --with-systemd`. Manually:

```bash
sudo tee /etc/systemd/system/checkmk-stack.service >/dev/null <<'UNIT'
[Unit]
Description=Checkmk + NGINX Proxy Manager (container stack)
Wants=network-online.target
After=network-online.target

[Service]
Type=oneshot
RemainAfterExit=yes
WorkingDirectory=/opt/checkmk-stack
EnvironmentFile=/opt/checkmk-stack/.env
TimeoutStartSec=0
ExecStart=/bin/sh -c '/usr/bin/podman-compose up -d'
ExecStartPost=/opt/checkmk-stack/scripts/cmk-manage.sh start-all-sites
ExecStop=/bin/sh -c '/usr/bin/podman-compose down'

[Install]
WantedBy=multi-user.target
UNIT

sudo systemctl daemon-reload
sudo systemctl enable --now checkmk-stack.service
sudo systemctl status checkmk-stack.service
```

Verify that it survives a reboot:

```bash
sudo systemctl is-enabled checkmk-stack     # -> enabled
sudo reboot
# after the restart:
sudo podman ps
sudo /opt/checkmk-stack/scripts/cmk-manage.sh status
```

`ExecStartPost` calls `start-all-sites`. This is necessary because the Checkmk
entrypoint only starts the site named in `CMK_SITE_ID` – with several sites the
remaining ones would stay stopped.

### Option B – Podman Quadlet (Podman 4.4 and newer, no compose)

Quadlet turns simple description files into real systemd units. Advantages: no
`podman-compose` required, clean dependencies, native Podman.

```bash
sudo ./install.sh --quadlet
```

Using the values from `.env`, this writes to `/etc/containers/systemd/`:

* `checkmk.container`
* `nginx-proxy-manager.container`

Then:

```bash
sudo systemctl daemon-reload
sudo systemctl start checkmk nginx-proxy-manager
sudo systemctl status checkmk
```

`WantedBy=multi-user.target default.target` makes the units start
automatically at boot – a separate `systemctl enable` is neither needed nor
possible with Quadlet, because the units are generated.

Templates live in [systemd/quadlet/](systemd/quadlet/).

### Option C – rootless Podman (optional, more effort)

```bash
sudo ./install.sh --rootless
sudo loginctl enable-linger checkmk           # service account
# as a user unit:
mkdir -p ~/.config/systemd/user
cp systemd/checkmk-stack.service.example ~/.config/systemd/user/checkmk-stack.service
systemctl --user daemon-reload
systemctl --user enable --now checkmk-stack
```

`--rootless` sets `net.ipv4.ip_unprivileged_port_start=80` so that ports 80/443
may be bound without root. `loginctl enable-linger` makes user units start
after a reboot even when nobody is logged in.

> Note: in rootless mode the UIDs of the site users are mapped through user
> namespaces. Files under `data/checkmk/sites` then belong to subordinate UIDs
> (e.g. `165536`). That is normal, but it makes working with the files from the
> host awkward. For migrations, **rootful** operation is recommended.

---

## 7. Configuration (`.env`)

```bash
cp .env.example .env
chmod 600 .env
$EDITOR .env
```

The most important settings:

| Variable | Meaning |
|---|---|
| `CMK_RUNTIME` | empty = auto-detect, otherwise `docker` or `podman` |
| `CMK_IMAGE` | **the image actually used**, always fully qualified – see [chapter 8](#8-obtaining-the-checkmk-image-raw-enterprise--co) |
| `CMK_EDITION`, `CMK_VERSION` | inputs for the image downloader only |
| `CMK_REGISTRY_USER/-PASSWORD` | login for `registry.checkmk.com` (Enterprise) |
| `CMK_DOWNLOAD_USER/-PASSWORD` | login for the Checkmk download portal |
| `CMK_SITE_ID` | the site the container creates and starts |
| `CMK_PASSWORD` | initial password for user `cmkadmin` (only on creation) |
| `CMK_LIVESTATUS_TCP` | `on` opens port 6557 for distributed monitoring |
| `MAIL_RELAY_HOST` | SMTP relay for notifications |
| `TZ` | time zone, e.g. `Europe/Berlin` |
| `NPM_ADMIN_PORT` | port of the Proxy Manager interface (default 81) – change only via `npm-port` or the menu |
| `NPM_ADMIN_URL` | start page of the terminal browser; empty = derived from `NPM_ADMIN_PORT` |
| `BACKUP_DIR` | location for backups/bundles (default `./backups`) |
| `CERT_DOMAIN`, `CERT_DAYS` | defaults for self-signed certificates |

### 7.1 Who reads `.env` – and why does it exist alongside the script?

`.env` is the **only** configuration file. There is deliberately no second
configuration inside the management script:

```
                    .env  (single source of truth)
                      |
      +---------------+----------------+------------------+
      |               |                |                  |
docker-compose    systemd unit    cmk-manage.sh      migrate_creator.sh
(variable         (EnvironmentFile)  (source)         (independent, runs on
 substitution)                                         the source server)
```

* **Read** by compose, by the systemd unit and by the script.
* **Written** by the script – either specifically (`image …` sets `CMK_IMAGE`,
  `import --set-default-site` sets `CMK_SITE_ID`, `npm-port` sets
  `NPM_ADMIN_PORT`) or through the settings menu / `config set`
  (see [chapter 14](#14-management-cmk-managesh)). An editor is no longer
  necessary, but remains available as a menu entry.
* The script itself only holds **fallback values** for the case that a key is
  missing from `.env` (in `scripts/lib/common.sh`, function `load_env`). It
  never overwrites a value that is set.

In short: `.env` says *what* runs, the script *does* something with it. Nothing
has to be maintained twice.

---

## 8. Obtaining the Checkmk image (Raw, Enterprise & co.)

### 8.1 Which edition lives where?

This is the most common pitfall: **the Enterprise edition is not on Docker
Hub.** The `checkmk` namespace on Docker Hub contains only:

| Edition (`CMK_EDITION`) | Source | Login |
|---|---|---|
| `raw` (up to 2.4) | `docker.io/checkmk/check-mk-raw` | no |
| `community` (2.5 and later, successor of raw) | `docker.io/checkmk/check-mk-community` | no |
| `cloud` | `docker.io/checkmk/check-mk-cloud` | no |
| `managed` (up to 2.4) | `docker.io/checkmk/check-mk-managed` | no |
| `pro` (2.5 and later, successor of enterprise) | `docker.io/checkmk/check-mk-pro` | no |
| `ultimate`, `ultimatemt` (2.5 and later) | `docker.io/checkmk/check-mk-ultimate[mt]` | no |
| **`enterprise` (2.0 – 2.4)** | **`registry.checkmk.com/enterprise/check-mk-enterprise`** or an image tarball from the customer portal | **yes** |
| `free` (historic, up to 2.1) | `docker.io/checkmk/check-mk-free` | no |

With Checkmk 2.5 the editions were renamed (Raw → Community,
Enterprise → Pro, Managed → Ultimate MT); those images are publicly available
on Docker Hub again. So if you run **2.5 or newer**, simply use `pro`. For
**2.2/2.3/2.4 Enterprise** the only routes are `registry.checkmk.com` or the
tarball.

> Freely downloadable does not mean licence-free – the licence key is entered
> inside Checkmk afterwards.

### 8.2 Two Podman pitfalls (the ones that actually bite)

**1. Short image names.** Podman has no default registry. Short names such as
`checkmk/check-mk-enterprise:2.3.0p23` are resolved through
`unqualified-search-registries` in `/etc/containers/registries.conf` – and on
RHEL `registry.access.redhat.com` is listed **first**. Podman therefore looks
for the image at Red Hat, does not find it and reports `image not known`, or
asks interactively. That is why `.env` and `docker-compose.yml` consistently
use the **fully qualified** name:

```
CMK_IMAGE=registry.checkmk.com/enterprise/check-mk-enterprise:2.3.0p23
```

Inspect the current resolution order:

```bash
grep -A3 unqualified-search-registries /etc/containers/registries.conf
```

**2. `import` instead of `load`.** A downloaded image tarball is installed with
**`load`**:

| Command | Expects | Result |
|---|---|---|
| `podman import file.tar.gz` | a **filesystem** archive (rootfs) | an image **without** entrypoint, environment or tag → unusable, "image unknown" |
| `podman load -i file.tar.gz` | an **image** archive (`docker save`/OCI) | a complete image including metadata and tag ✔ |

Getting rid of a broken image created by `import`:

```bash
podman images | grep '<none>'
podman rmi <image-id>
```

`cmk-manage.sh image load` checks the archive for `manifest.json` or
`oci-layout` beforehand and rejects a wrong archive with exactly this hint.

### 8.3 Storing credentials in `.env`

```bash
CMK_EDITION=enterprise
CMK_VERSION=2.3.0p23

# route 1 - private registry
CMK_REGISTRY=registry.checkmk.com
CMK_REGISTRY_USER=<customer-portal-user>
CMK_REGISTRY_PASSWORD=<password>

# route 2 - tarball from the download portal
CMK_DOWNLOAD_BASEURL=https://download.checkmk.com/checkmk
CMK_DOWNLOAD_USER=<user>
CMK_DOWNLOAD_PASSWORD=<password>
```

The credentials come from the Checkmk customer portal (portal.checkmk.com).
`.env` should therefore be `chmod 600` – `install.sh` takes care of that.

### 8.4 Fetching the image – three ways

**a) From a registry (recommended)**

```bash
# freely available, no login
sudo ./scripts/cmk-manage.sh image pull --edition raw --version 2.3.0-latest

# Enterprise with a login to registry.checkmk.com
sudo ./scripts/cmk-manage.sh image pull --edition enterprise --version 2.3.0p23 \
     --user <user> --password <password>
```

**b) Image tarball from the download portal (with credentials)**

```bash
sudo ./scripts/cmk-manage.sh image download --edition enterprise --version 2.3.0p23 \
     --user <user> --password <password>
```

The downloader tries the usual file names under
`<CMK_DOWNLOAD_BASEURL>/<version>/`
(`check-mk-enterprise-<version>.docker.tar.gz` and
`check-mk-enterprise-docker-<version>.tar.gz`). Should Checkmk have changed the
scheme, the full URL can be given directly:

```bash
sudo ./scripts/cmk-manage.sh image download \
     --url 'https://download.checkmk.com/checkmk/2.3.0p23/check-mk-enterprise-2.3.0p23.docker.tar.gz' \
     --user <user> --password <password>
```

The credentials are handed to `curl` through a temporary `.netrc` file and
therefore do **not** appear in the process list.

**c) Loading a file you already downloaded**

If you fetched the file from the customer portal with a browser:

```bash
sudo ./scripts/cmk-manage.sh image load --file /root/check-mk-enterprise-2.3.0p23.docker.tar.gz
```

In all three cases the script determines the **actual** image name (for `load`
typically `localhost/checkmk/check-mk-enterprise:2.3.0p23`) and writes it to
`CMK_IMAGE` in `.env`. Afterwards:

```bash
sudo ./scripts/cmk-manage.sh up
```

### 8.5 Checking what is in effect

```bash
./scripts/cmk-manage.sh image status     # configured image + local images
./scripts/cmk-manage.sh image list       # local Checkmk images only
./scripts/cmk-manage.sh image use localhost/checkmk/check-mk-enterprise:2.3.0p23
```

`image status` explicitly warns when `CMK_IMAGE` is a short name without a
registry.

The ncurses interface offers the same workflow under the menu entry
*"Checkmk-Image beschaffen/auswählen"* (pick edition → version → credentials →
fetch).

---

## 9. First start

```bash
sudo ./scripts/cmk-manage.sh up
sudo ./scripts/cmk-manage.sh status
```

Output (example, German as printed by the tool):

```
Container
  checkmk                  läuft
  nginx-proxy-manager      läuft

Sites im Container
  SITE                 VERSION            STATUS
  cmk                  2.3.0p23.cre       läuft

Zugang
  Checkmk (direkt)        : http://10.0.0.5:5000/cmk/
  Checkmk (über Proxy)    : https://<FQDN>/cmk/
  Agent-Receiver          : 10.0.0.5:8000
  NGINX Proxy Manager     : http://10.0.0.5:81/  (Erstlogin: admin@example.com / changeme)
```

(`läuft` = running, `gestoppt` = stopped, `Zugang` = access, `Erstlogin` =
initial login.) The port of the admin interface follows `NPM_ADMIN_PORT` from
`.env`; the output always shows the value actually configured.

Logging in to Checkmk: user `cmkadmin`, password from `CMK_PASSWORD`.

---

## 10. Setting up HTTPS – Let's Encrypt or self-signed

### 10.1 Preparing the NGINX Proxy Manager

1. open `http://<server>:81` – or whichever port is set in `NPM_ADMIN_PORT`
   (see [chapter 12](#12-moving-the-admin-interface-to-a-different-port)); if
   that port is blocked on the network, use the terminal browser from
   [chapter 11](#11-configuring-the-proxy-manager-locally-terminal-browser)
2. initial login: **admin@example.com** / **changeme**
3. change the e-mail address and password immediately (enforced on first login)

### 10.2 Creating a proxy host for Checkmk

**Hosts → Proxy Hosts → Add Proxy Host**

| Field | Value |
|---|---|
| Domain Names | `checkmk.example.com` (or your internal name) |
| Scheme | `http` |
| Forward Hostname / IP | `127.0.0.1` |
| Forward Port | `5000` |
| Block Common Exploits | on |
| Websockets Support | **on** (for the live data in the interface) |

Tab **Advanced** (recommended, so Checkmk sees the real client IP and large
agent uploads work):

```nginx
client_max_body_size 0;
proxy_read_timeout 300s;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
```

### 10.3 Option A – Let's Encrypt (publicly reachable server)

Prerequisites: the DNS name resolves publicly to this server and **port 80 is
reachable from the internet** (HTTP-01 challenge).

In the proxy host, tab **SSL**:

* SSL Certificate → **Request a new SSL Certificate**
* *Force SSL* on
* *HTTP/2 Support* on
* *HSTS Enabled* as required
* enter an e-mail address, accept the terms → **Save**

Renewal is handled by the Proxy Manager automatically; the certificates live in
the bind mount `data/npm/letsencrypt/` and are therefore kept inside the
project directory.

For an internal DNS zone without public access, use the **DNS challenge**
(SSL → *Use a DNS Challenge* → select provider, store the API token).

### 10.4 Option B – self-signed certificate (internal environment)

Creating one:

```bash
sudo ./scripts/cmk-manage.sh cert \
     --domain checkmk.intern.example \
     --alt 10.0.0.5 --alt checkmk \
     --days 3650
```

Result in `data/npm/custom-certs/checkmk.intern.example/`:

* `privkey.pem` – private key
* `fullchain.pem` – certificate (with SANs for every name/IP given)

Installing it:

1. **SSL Certificates → Add SSL Certificate → Custom**
2. give it a name, upload `privkey.pem` as *Certificate Key* and
   `fullchain.pem` as *Certificate*
3. select this certificate in the proxy host under **SSL**, enable *Force SSL*
   and *HTTP/2*

> For Checkmk agents and browsers to trust the certificate, `fullchain.pem`
> must be installed as a trusted certificate on the clients (Windows: computer
> certificates → Trusted Root Certification Authorities; Linux:
> `/usr/local/share/ca-certificates/` + `update-ca-certificates`).
> For plain agent communication over port 8000 this is **not** needed – there
> Checkmk uses its own site-internal CA.

The ncurses interface offers the same workflow under *"Selbstsigniertes
Zertifikat erzeugen"*.

---

## 11. Configuring the Proxy Manager locally (terminal browser)

When port 81 cannot be reached from the network, the NGINX Proxy Manager's
admin interface has to be operated **on the machine itself**. For that,
`cmk-manage.sh browser` starts another container on the host network – which
reaches `127.0.0.1:81` just like the Proxy Manager itself.

```bash
sudo ./scripts/cmk-manage.sh browser
```

This is deliberately **not** a service in `docker-compose.yml`: a browser is an
interactive program with a terminal, not a background service. It is started
with `run --rm -it` and disappears without a trace when you quit.

### 11.1 Why not `lynx`

The Proxy Manager's admin interface is a JavaScript application. Classic text
browsers **cannot** render it – you get a blank page. There are therefore two
engines that render a full browser inside the terminal:

| Engine | Based on | JavaScript | Proxy Manager usable |
|---|---|---|---|
| `browsh` | Firefox | yes | **yes** – default, maintained |
| `carbonyl` | Chromium | yes | **yes** – nicer rendering, last released 2023 |
| `links` (links2) | text browser | no | no |
| `w3m` | text browser | no | no |
| `lynx` | text browser | no | no |
| `shell` | Bash + curl/jq/openssl | – | for diagnostics and the API |

The text browsers are included anyway because they are handy for quick checks.
If you pick one, the script warns explicitly.

### 11.2 Usage

```bash
# default: browsh on http://127.0.0.1:81
sudo ./scripts/cmk-manage.sh browser

# a different engine or a different page
sudo ./scripts/cmk-manage.sh browser --engine carbonyl
sudo ./scripts/cmk-manage.sh browser --engine links --url http://127.0.0.1:5000/cmk/

# shell with network tools on the host network
sudo ./scripts/cmk-manage.sh browser --engine shell
```

Key bindings in **browsh**:

| Key | Action |
|---|---|
| `Ctrl+L` | address bar |
| `Ctrl+T` | new tab |
| `Ctrl+W` | close tab |
| `Ctrl+Q` | quit |

The first time a text-browser or `shell` engine is used, the script builds a
small tools image (Debian slim with links2, w3m, lynx, curl, jq, openssl,
roughly 170 MB). Build it in advance – this works without a terminal, e.g. from
an installation script:

```bash
sudo ./scripts/cmk-manage.sh browser --build-only
```

`browsh` and `carbonyl`, by contrast, are pulled as ready-made images; their
names are in `.env` (`BROWSH_IMAGE`, `CARBONYL_IMAGE`) and are fully qualified
so that Podman does not look for them at `registry.access.redhat.com`.

In the ncurses interface everything lives under *"Terminal-Browser (Proxy
Manager lokal bedienen)"*.

### 11.3 Limitation: file upload

A terminal browser cannot open the operating system's file chooser. For
**Let's Encrypt** that does not matter (only form fields are filled in), but
uploading a **self-signed certificate** (*SSL Certificates → Custom*) does not
work reliably this way.

For exactly this case, use the Proxy Manager's REST interface. Create the
certificate as described in
[chapter 10.4](#10-setting-up-https--lets-encrypt-or-self-signed), then:

```bash
NPM=http://127.0.0.1:81
CERTDIR=./data/npm/custom-certs/checkmk.intern.example

# 1) log in and obtain a token
TOKEN=$(curl -s -X POST "$NPM/api/tokens" \
  -H 'Content-Type: application/json' \
  -d '{"identity":"admin@example.com","secret":"<password>"}' | jq -r .token)

# 2) create the certificate entry (provider "other" = your own certificate)
CERT_ID=$(curl -s -X POST "$NPM/api/nginx/certificates" \
  -H "Authorization: Bearer $TOKEN" -H 'Content-Type: application/json' \
  -d '{"provider":"other","nice_name":"checkmk-selfsigned","domain_names":["checkmk.intern.example"]}' \
  | jq -r .id)

# 3) upload key and certificate
curl -s -X POST "$NPM/api/nginx/certificates/$CERT_ID/upload" \
  -H "Authorization: Bearer $TOKEN" \
  -F "certificate=@$CERTDIR/fullchain.pem" \
  -F "certificate_key=@$CERTDIR/privkey.pem"

# 4) create the proxy host for Checkmk
curl -s -X POST "$NPM/api/nginx/proxy-hosts" \
  -H "Authorization: Bearer $TOKEN" -H 'Content-Type: application/json' \
  -d "{\"domain_names\":[\"checkmk.intern.example\"],
       \"forward_scheme\":\"http\",
       \"forward_host\":\"127.0.0.1\",
       \"forward_port\":5000,
       \"certificate_id\":$CERT_ID,
       \"ssl_forced\":true,
       \"http2_support\":true,
       \"block_exploits\":true,
       \"allow_websocket_upgrade\":true,
       \"advanced_config\":\"client_max_body_size 0;\",
       \"enabled\":true,
       \"meta\":{}}"
```

`curl` and `jq` are part of the tools image in case the host lacks them:

```bash
sudo ./scripts/cmk-manage.sh browser --engine shell
```

> On the very first login the Proxy Manager forces you to change the e-mail
> address and password. That is easiest in the browser
> (`cmk-manage.sh browser`); afterwards the API works with the new credentials.

### 11.4 Alternative: SSH tunnel

If SSH access to the machine is possible, a tunnel is the most convenient
route – you then use the interface in your usual browser:

```bash
ssh -N -L 8081:127.0.0.1:81 user@checkmk-server
# then locally:  http://127.0.0.1:8081
```

This changes nothing about the firewall: port 81 stays closed from outside,
access runs through the existing SSH connection.

---

## 12. Moving the admin interface to a different port

Inside the image, the NGINX Proxy Manager's web interface listens on a
**hard-coded port 81** (`listen 81 default;` in
`/etc/nginx/conf.d/production.conf`). No environment variable for this exists
in the published 2.x versions – `NPM_ADMIN_PORT` is already prepared in the
development branch, but is not part of version 2.15.1. And because both
containers run on the host network, the port cannot be moved by port mapping
either.

So if port 81 is blocked on your network:

```bash
sudo ./scripts/cmk-manage.sh npm-port 10443
```

### 12.1 What happens

Within its `http{}` block, nginx includes an official extension point:

```
include /data/nginx/custom/http.conf;
```

`/data` is our bind mount `./data/npm/data`. The command places an additional
server there which listens on the desired port and forwards to the interface on
`127.0.0.1:81`:

```
./data/npm/data/nginx/custom/http.conf

# >>> cmk-manage.sh: Admin-Port - nicht von Hand aendern >>>
server {
    listen 10443;
    listen [::]:10443;
    server_name _;
    client_max_body_size 0;
    location / {
        proxy_pass http://127.0.0.1:81;
        ...
    }
}
# <<< cmk-manage.sh: Admin-Port <<<
```

Only the region between the two markers is managed – your own additions to the
same file are left untouched. The script then restarts the Proxy Manager
container so nginx reads the file, and writes `NPM_ADMIN_PORT` and
`NPM_ADMIN_URL` into `.env`. The terminal browser from
[chapter 11](#11-configuring-the-proxy-manager-locally-terminal-browser) then
starts on the new port automatically.

### 12.2 Important: port 81 stays alive locally

The built-in server on port 81 **cannot** be switched off this way – it keeps
listening on all addresses. That is harmless as long as the firewall blocks it,
and that is exactly the recommendation:

```bash
# firewalld
sudo firewall-cmd --permanent --add-port=10443/tcp
sudo firewall-cmd --permanent --remove-port=81/tcp 2>/dev/null
sudo firewall-cmd --reload

# ufw
sudo ufw allow 10443/tcp
sudo ufw delete allow 81/tcp 2>/dev/null
```

### 12.3 Inspecting and reverting

```bash
./scripts/cmk-manage.sh npm-port          # show the current state
sudo ./scripts/cmk-manage.sh npm-port 81  # back to the default
```

With `81` the managed block is removed again; if the file is empty afterwards
it is deleted entirely. Only the configuration from the image then applies.

The command rejects ports already taken by the stack (80, 443, 5000, 8000,
6557), validates the numeric range, and warns if another service is already
listening on the port you want.

> As soon as an NPM release ships `NPM_ADMIN_PORT`, this detour becomes
> unnecessary – the environment variable will be enough. Until then, the
> extension point is the route that needs no change to the image.

---

## 13. Firewall

**firewalld (RHEL/Rocky/Alma):**

```bash
sudo firewall-cmd --permanent --add-service=http
sudo firewall-cmd --permanent --add-service=https
sudo firewall-cmd --permanent --add-port=8000/tcp          # agent receiver
sudo firewall-cmd --permanent --add-rich-rule='rule family=ipv4 source address=10.0.0.0/24 port port=81 protocol=tcp accept'
sudo firewall-cmd --reload

# With a different admin port (e.g. 10443) open that number instead -
# "cmk-manage.sh npm-port <port>" prints the matching command for you.
```

**ufw (Debian/Ubuntu):**

```bash
sudo ufw allow 80/tcp
sudo ufw allow 443/tcp
sudo ufw allow 8000/tcp
sudo ufw allow from 10.0.0.0/24 to any port 81 proto tcp
sudo ufw enable
```

Port **5000 stays closed** – access goes exclusively through the proxy. Open
port **81** to the administration network only.

---

## 14. Management: `cmk-manage.sh`

The script is both a command-line tool **and** an ncurses interface.

### 14.1 ncurses interface (Midnight Commander style)

```bash
sudo ./scripts/cmk-manage.sh gui
```

The menu labels are German; the command table in
[14.2](#14-management-cmk-managesh) lists the same actions in English.

```
 ┌──────────────── Checkmk Stack Manager ─────────────────┐
 │ Projekt : /opt/checkmk-stack                           │
 │ Runtime : podman     Standard-Site: cmk                │
 │ Backups : /opt/checkmk-stack/backups                   │
 │                                                        │
 │   status    Status von Containern und Sites            │
 │   stack     Stack steuern (Start/Stop/Logs/Update)     │
 │   backup    Backup erstellen                           │
 │   restore   Backup wiederherstellen                    │
 │   import    Site von Direktinstallation importieren    │
 │   manage    Backups verwalten (Liste/Details/Löschen)  │
 │   sites     Sites im Container verwalten               │
 │   image     Checkmk-Image beschaffen/auswählen         │
 │   browser   Terminal-Browser (Proxy Manager lokal)     │
 │   cert      Selbstsigniertes Zertifikat erzeugen       │
 │   config    Einstellungen (.env) - menügeführt         │
 │   shell     Root-Shell im Checkmk-Container            │
 │   quit      Beenden                                    │
 │                  <  OK  >   <Cancel>                   │
 └────────────────────────────────────────────────────────┘
```

Operation: arrow keys, **space** toggles entries in selection lists (e.g.
several sites for a backup), **Enter** confirms, **ESC/Cancel** goes back.
`dialog` is preferred; if only `whiptail` is present, that is used instead.
A file chooser with directory navigation is built in.

### 14.2 Command line

```bash
./scripts/cmk-manage.sh --help
```

| Command | Effect |
|---|---|
| `up` / `down` / `restart` / `pull` | control the stack |
| `status` | containers, sites, access addresses |
| `logs [-f]` | container logs |
| `sites` | list the sites in the container |
| `site-start SITE` / `site-stop SITE` | control a single site |
| `start-all-sites` | start all sites (invoked by systemd) |
| `backup …` | backup as a `.tar.gz` bundle |
| `list` / `verify` / `prune --keep N` | manage backups |
| `restore …` | restore a backup |
| `import …` | import a migration bundle |
| `image status\|list\|pull\|download\|load\|use` | obtain and select the Checkmk image |
| `browser [--engine …]` | terminal browser in a container – operate the Proxy Manager locally |
| `config list\|get\|set\|edit` | show and change settings in `.env` |
| `npm-port [PORT]` | set/show the port of the Proxy Manager interface |
| `cert …` | create a self-signed certificate |
| `shell` | root shell in the Checkmk container |
| `omd …` | any `omd` command inside the container |

Global options: `--env-file FILE`, `-y/--yes` (no prompts, e.g. for cron jobs),
`-v/--verbose`.

### 14.3 Changing settings – without an editor

Every value in `.env` can be maintained through the menu. In the interface:
**Einstellungen (.env) – menügeführt**. The values are grouped by area:

```
 ┌──────────────────── Einstellungen ─────────────────────┐
 │ Alle Werte der Datei .env - direkt hier änderbar.      │
 │ Datei: /opt/checkmk-stack/.env                         │
 │                                                        │
 │   runtime   Runtime und Zeitzone                       │
 │   image     Checkmk-Image und Zugangsdaten             │
 │   checkmk   Checkmk-Site und Container                 │
 │   npm       NGINX Proxy Manager                        │
 │   browser   Terminal-Browser                           │
 │   paths     Verzeichnisse                              │
 │   cert      Selbstsignierte Zertifikate                │
 │   show      Alle Werte auflisten                       │
 │   editor    Datei .env im Editor öffnen                │
 │   back      zurück                                     │
 └────────────────────────────────────────────────────────┘
```

(runtime and time zone / Checkmk image and credentials / Checkmk site and
containers / NGINX Proxy Manager / terminal browser / directories / self-signed
certificates / list all values / open `.env` in an editor / back.)

Each entry shows its current value and, when confirmed, opens the matching
input: free text, number, selection list or password field (passwords appear as
`********` in the overview and stay unchanged if you leave the field empty).
The editor remains available as a menu entry.

The same values from the command line:

```bash
./scripts/cmk-manage.sh config list                  # everything with current values
./scripts/cmk-manage.sh config list --category npm   # one area only
./scripts/cmk-manage.sh config get CMK_SITE_ID
./scripts/cmk-manage.sh config set CMK_SITE_ID prod
./scripts/cmk-manage.sh config set NPM_ADMIN_PORT 10443
```

**Validation happens when setting the value**, not at the next start:

| Key | Check |
|---|---|
| `NPM_ADMIN_PORT` | number 1–65535, not 80/443/5000/8000/6557, warning if in use |
| `CMK_SITE_ID` | letters, digits and `_` only; at most 16 characters |
| `CMK_IMAGE`, `NPM_IMAGE`, … | must be fully qualified (the Podman pitfall) |
| `CMK_RUNTIME` | must be installed on this system |
| `CERT_DAYS` and others | numeric fields |
| URL fields | must start with `http://` or `https://` |

Values containing spaces are quoted automatically so that `.env` remains
readable by compose, systemd **and** the shell. When you change
`NPM_ADMIN_PORT`, the script updates the additional nginx file along with it
(see [chapter 12](#12-moving-the-admin-interface-to-a-different-port)). For
values that only take effect once the containers are recreated, the menu says
so explicitly.

---

## 15. Backup

Backups are created with `omd backup` and packed into a **bundle**:

```
cmkbackup_<site>_<version>_<timestamp>.tar.gz
└── cmkbundle/
    ├── MANIFEST              site, version, source, checksum …
    ├── INFO.txt              human-readable summary
    ├── site.tar.gz           the actual omd backup
    └── site.tar.gz.sha256
```

This way the restore/import tool always knows **which site** and **which
Checkmk version** the archive contains, and can verify its integrity.

```bash
# a single site
sudo ./scripts/cmk-manage.sh backup --site cmk

# all sites, without log files, with a label
sudo ./scripts/cmk-manage.sh backup --all --no-logs --label vor-update

# most consistent backup: the site is stopped briefly
sudo ./scripts/cmk-manage.sh backup --site cmk --stop

# without performance data (considerably smaller)
sudo ./scripts/cmk-manage.sh backup --site cmk --no-rrds --dir /mnt/nas/checkmk
```

Listing and verifying:

```bash
./scripts/cmk-manage.sh list
./scripts/cmk-manage.sh verify --file backups/cmkbackup_cmk_2.3.0p23.cre_20260820-030000.tar.gz
./scripts/cmk-manage.sh prune --keep 7
```

### Nightly backup via cron

```bash
sudo crontab -e
```

```cron
15 2 * * * /opt/checkmk-stack/scripts/cmk-manage.sh -y backup --all --no-logs >>/var/log/cmk-backup.log 2>&1
30 3 * * 0 /opt/checkmk-stack/scripts/cmk-manage.sh -y prune --keep 14   >>/var/log/cmk-backup.log 2>&1
```

---

## 16. Restore

```bash
# simplest case: the site no longer exists
sudo ./scripts/cmk-manage.sh restore --file backups/cmkbackup_cmk_2.3.0p23.cre_20260820-030000.tar.gz

# overwrite an existing site (terminating running processes)
sudo ./scripts/cmk-manage.sh restore --file backups/… --reuse --kill

# restore under a different name (e.g. for a test)
sudo ./scripts/cmk-manage.sh restore --file backups/… --as-site test
```

What the script does:

1. checks the archive structure and the SHA256 checksum
2. checks whether the **Checkmk version of the backup exists in the container**
3. prompts / requires `--reuse` for an existing site, stops the site
4. copies the archive into the container and runs `omd restore`
5. turns off `TMPFS` for the site (not usable inside the container)
6. starts the site and prints the next steps

> **`--as-site` changes the site name** – registered agents would then point
> nowhere and would have to be re-registered. So do **not** use it for a
> migration.

A raw `omd backup` archive (without the bundle wrapper) is accepted as well;
the site name is then read from the archive.

---

## 17. Migrating a bare-metal installation (`migrate_creator.sh`)

Goal: take over an existing Checkmk installation on hardware or a VM into the
container – **without touching the agents**.

### 17.1 Why the agents keep working

An `omd backup` contains the complete site including `etc/ssl` (the site's own
CA and the certificates of the registered agents), `etc/check_mk` (rules,
hosts, passwords) and the monitoring history. If the site is restored **under
the same name** and the server is reachable at **the same IP address**, then

* push/pull agents (`cmk-agent-ctl`) find their registration again,
* the TLS connection to the agent receiver on **port 8000** stays valid,
* all URLs (`https://<ip>/<site>/check_mk/…`) remain correct,
* performance data and history are preserved.

The prerequisite is therefore that the target system **takes over the source
server's IP** (or that the DNS name is repointed) – which is the plan anyway.

### 17.2 Step 1 – copy the script to the source server

```bash
scp scripts/migrate_creator.sh root@old-server:/root/
ssh root@old-server 'chmod +x /root/migrate_creator.sh'
```

The script is **self-contained** – on the source server it only needs `bash`,
`tar` and `omd` (plus `dialog` or `whiptail` for the interface).

### 17.3 Step 2 – create the bundle on the source server

Show the existing sites:

```bash
sudo ./migrate_creator.sh --list
```

```
SITE                 VERSION            STATUS       GROESSE
prod                 2.3.0p23.cre       laeuft       4.7 GiB
test                 2.3.0p23.cre       gestoppt     318.2 MiB
```

(`GROESSE` = size, `laeuft` = running, `gestoppt` = stopped.)

**With the interface** (select sites with the space bar):

```bash
sudo ./migrate_creator.sh --gui
```

The dialog asks in turn for: sites → options (stop the site, skip RRDs/logs) →
target directory → label → optional transfer via `scp`.

**With parameters:**

```bash
# one site, stopping it during the backup (most consistent variant)
sudo ./migrate_creator.sh --site prod --stop --out /var/tmp

# all sites, without logs, transferred straight to the target system
sudo ./migrate_creator.sh --all --no-logs \
     --scp root@10.0.0.5:/opt/checkmk-stack/backups/
```

Result:

```
/var/tmp/cmkmigrate_prod_2.3.0p23.cre_20260820-101500.tar.gz
```

Besides the omd backup, the bundle contains a `MANIFEST` with the site name,
Checkmk version, source hostname, **source IP addresses**, source operating
system and SHA256 checksum, plus a `siteinfo/` directory with `omd config
show`, `omd status` and the IP configuration of the old system for later
reference.

### 17.4 Step 3 – bring the target system to the right version

`omd restore` can only restore a site **if exactly its Checkmk version is
installed on the target system.** In the container, the version comes from the
image – so the image tag has to match:

```bash
tar -xzOf backups/cmkmigrate_prod_*.tar.gz cmkbundle/MANIFEST | grep CMK_VERSION
# CMK_VERSION=2.3.0p23.cre   ->  version 2.3.0p23, suffix .cre = Raw edition

# fetch the matching image and have it written to CMK_IMAGE
sudo ./scripts/cmk-manage.sh image pull --edition raw --version 2.3.0p23
sudo ./scripts/cmk-manage.sh up
```

For an Enterprise source (`.cee`) accordingly:

```bash
sudo ./scripts/cmk-manage.sh image pull --edition enterprise --version 2.3.0p23 \
     --user <user> --password <password>
# or, if the registry is unreachable:
sudo ./scripts/cmk-manage.sh image download --edition enterprise --version 2.3.0p23 \
     --user <user> --password <password>
```

The import script checks this itself and aborts with a corresponding message if
the version is missing.

> **Mind the edition:** the version suffix reveals it – `.cre` = Raw,
> `.cee` = Enterprise, `.cce` = Cloud, `.cme` = Managed. The import script
> reads it from the manifest and, on failure, names the matching `image pull`
> command directly. Enterprise images are **not** on Docker Hub – see
> [chapter 8](#8-obtaining-the-checkmk-image-raw-enterprise--co).

### 17.5 Step 4 – import the bundle

With the interface:

```bash
sudo ./scripts/cmk-manage.sh gui      # -> "import"
```

With parameters:

```bash
sudo ./scripts/cmk-manage.sh import \
     --file backups/cmkmigrate_prod_2.3.0p23.cre_20260820-101500.tar.gz \
     --set-default-site
```

The script

1. recognises the migration bundle and shows the source host and source IP,
2. **warns if this host's IP differs from the source IP** (the agents would
   then have to be adjusted),
3. verifies checksum, version and edition,
4. prompts if the site already exists (`--reuse` overwrites),
5. runs `omd restore` under the **unchanged site name**,
6. optionally sets `CMK_SITE_ID` in `.env` (`--set-default-site`) so the
   container starts the site automatically after a reboot as well,
7. starts the site and prints the next steps.

### 17.6 Step 5 – switch over and verify

```bash
# 1) shut down the old server or stop its site (no double monitoring!)
ssh root@old-server 'omd stop prod'

# 2) move the target system to the IP/DNS name of the old system
#    (take over the static IP or repoint the DNS record)

# 3) recreate the containers if CMK_SITE_ID was changed
sudo ./scripts/cmk-manage.sh up
sudo ./scripts/cmk-manage.sh status

# 4) check inside Checkmk
sudo ./scripts/cmk-manage.sh omd su prod
   OMD[prod]:~$ cmk -O                # activate the configuration
   OMD[prod]:~$ omd status
   OMD[prod]:~$ exit
```

Then check in the interface:

* **Monitor → All hosts** – are the agents reporting again?
* With TLS agents: the service *"Check_MK Agent"* must not report registration
  errors.
* An agent can be tested from the monitored host with `cmk-agent-ctl status` –
  the site `prod` must appear there with the target server and
  "Connection: TLS".

### 17.7 Migration checklist

- [ ] `migrate_creator.sh` copied to the source server
- [ ] bundle created (preferably with `--stop`)
- [ ] bundle copied to the target system (`backups/`)
- [ ] matching image fetched (`image pull`/`image download`), `CMK_IMAGE` set, stack recreated
- [ ] `import --set-default-site` executed, site running
- [ ] old site stopped / old server shut down
- [ ] IP or DNS switched to the target system
- [ ] ports 80/443/8000 open in the firewall
- [ ] proxy host set up in the NGINX Proxy Manager with a certificate
- [ ] agents reporting, `cmk -O` executed
- [ ] backup of the new system set up (cron)

---

## 18. Several sites in one container

At startup the Checkmk entrypoint only creates – and only starts – the site
from `CMK_SITE_ID`. Further sites (e.g. several imported ones) exist in the
bind mount but are not brought up automatically after a restart.

Hence:

```bash
sudo ./scripts/cmk-manage.sh start-all-sites
```

This command is executed automatically by the systemd unit (`ExecStartPost`)
or the Quadlet unit after the container starts, and starts every site present.

> **Recommendation:** for cleanly separated environments, run one project
> directory with its own `.env` per site. Note that because both containers run
> on the host network, only **one** Checkmk container per host can occupy ports
> 5000/8000 – for genuinely multiple instances use separate hosts or a
> bridge-network variant.

---

## 19. Upgrading Checkmk

Because `/omd/sites` is a bind mount, the data survives replacing the image.
The Checkmk software itself lives in the image under `/omd/versions`.

```bash
# 1) ALWAYS back up first
sudo ./scripts/cmk-manage.sh backup --all --label vor-update

# 2) fetch the new image (writes CMK_IMAGE into .env automatically)
sudo ./scripts/cmk-manage.sh image pull --edition raw --version 2.3.0p28

# 3) recreate the containers
sudo ./scripts/cmk-manage.sh up

# 4) update the site (interactive, confirm conflicts)
sudo ./scripts/cmk-manage.sh omd stop cmk
sudo ./scripts/cmk-manage.sh omd update cmk
sudo ./scripts/cmk-manage.sh omd start cmk
```

For a major version jump (e.g. 2.2 → 2.3) be sure to read the Checkmk werks /
release notes beforehand.

---

## 20. Troubleshooting

| Symptom | Cause / solution |
|---|---|
| `Port 80/443 already in use` | a web server is already running on the host: `ss -tlnp \| grep ':80'` → `systemctl disable --now apache2 nginx httpd` |
| Containers do not run after a reboot (Podman) | systemd unit missing: `./install.sh --with-systemd` or `--quadlet`, then `systemctl is-enabled checkmk-stack` |
| `omd restore`: *version is not installed* | fetch the matching image: `cmk-manage.sh image pull --edition <ed> --version <v>`, then `up` |
| Import aborts with an edition warning | an Enterprise backup needs an Enterprise image, see [chapter 8](#8-obtaining-the-checkmk-image-raw-enterprise--co) |
| `image not known` / Podman searches `registry.access.redhat.com` | short image name. Set `CMK_IMAGE` fully qualified: `cmk-manage.sh image use docker.io/…` or `registry.checkmk.com/enterprise/…` |
| `checkmk/check-mk-enterprise` not found on Docker Hub | correct – Enterprise is only on `registry.checkmk.com` or available as a tarball; from 2.5 the edition is called `pro` and is public on Docker Hub |
| Tarball installed with `podman import`, image "unknown" | `import` produces an image without metadata. Remove the broken image with `podman rmi <id>` and use `cmk-manage.sh image load --file …` (which uses `load`) |
| Download of the image tarball fails (401/404) | check the customer portal credentials, give the exact version (no `-latest`), if necessary `image download --url '<full URL>'` |
| Proxy returns `502 Bad Gateway` | the Checkmk container is not running, or the forwarding does not point to `127.0.0.1:5000`: `./scripts/cmk-manage.sh status`, `curl -I http://127.0.0.1:5000/` |
| Let's Encrypt fails | port 80 not reachable from the internet, or wrong DNS; alternative: DNS challenge or a self-signed certificate |
| Agents do not report after the migration | site name changed (`--as-site`), IP not taken over, or port 8000 blocked; check on the agent: `cmk-agent-ctl status` |
| Permission denied on `data/` under RHEL | SELinux: `restorecon -Rv /opt/checkmk-stack/data`, check `:Z` on the mounts |
| `podman-compose` not found | enable EPEL or switch to Quadlet (`./install.sh --quadlet`) |
| Admin interface unreachable from the LAN | port 81 blocked. Move it to a free port: `cmk-manage.sh npm-port 10443` and open that in the firewall |
| New port does not answer after `npm-port` | the Proxy Manager was not restarted: `cmk-manage.sh restart`; check the file `data/npm/data/nginx/custom/http.conf` |
| `cmk-manage.sh` aborts with "CMK_RUNTIME … ist nicht installiert" | it now only warns and detects the runtime itself; set it cleanly with `config set CMK_RUNTIME auto` |
| Proxy Manager interface blank / only text fragments | a text browser without JavaScript was used. Use `cmk-manage.sh browser --engine browsh` (or `carbonyl`) |
| Self-signed certificate cannot be uploaded in the terminal browser | there are no file dialogs there – upload via the API, see [chapter 11.3](#11-configuring-the-proxy-manager-locally-terminal-browser) |
| `browser` reports "braucht ein echtes Terminal" (needs a real terminal) | called from a pipe/script. Call it on the console or over SSH; only the build runs headless, with `--build-only` |
| The TUI does not start | install `dialog`: `apt-get install -y dialog` or `dnf install -y dialog` |
| Site does not start, `tmp` errors | `./scripts/cmk-manage.sh omd config <site> set TMPFS off`, then start the site |

Useful commands:

```bash
./scripts/cmk-manage.sh logs -f                  # container logs
./scripts/cmk-manage.sh omd status cmk           # services of the site
./scripts/cmk-manage.sh shell                    # root shell in the container
docker exec -it checkmk omd su cmk               # or podman exec …
```

---

## 21. Security notes

* Change `CMK_PASSWORD` in the interface right after installation and remove it
  from `.env` (it is only evaluated when the site is created); `chmod 600 .env`
  is already set.
* The Proxy Manager's default login (`admin@example.com` / `changeme`) must be
  changed on first login.
* Make port **81** (admin interface) reachable from the administration network
  only, and never expose port **5000**.
* Backups contain **all** passwords and certificates of the site – store them
  encrypted or restrict access tightly (`chmod 700 backups`).
* Schedule `./scripts/cmk-manage.sh pull` + `up` regularly for image security
  updates.
* Host networking means the container shares the host's network stack. There is
  no additional network isolation – protection comes from the host firewall.

---

## 22. Uninstalling

```bash
sudo ./scripts/cmk-manage.sh down

# systemd
sudo systemctl disable --now checkmk-stack.service
sudo rm -f /etc/systemd/system/checkmk-stack.service
# or Quadlet
sudo rm -f /etc/containers/systemd/checkmk.container \
           /etc/containers/systemd/nginx-proxy-manager.container
sudo systemctl daemon-reload

# remove the images (names from "cmk-manage.sh image list")
docker image rm docker.io/checkmk/check-mk-raw:2.3.0-latest docker.io/jc21/nginx-proxy-manager:2
# podman image rm …

# remove a stored registry login if present
podman logout registry.checkmk.com   # or docker logout

# data (CAUTION: this deletes the entire monitoring setup!)
sudo rm -rf /opt/checkmk-stack/data
```

---

## Licence / notes

This project contains only configuration and helper scripts.
Checkmk (Raw Edition, GPLv2) and NGINX Proxy Manager (MIT) are subject to the
licence terms of their respective vendors.