Add ova2vzdump: OVA -> Proxmox vzdump (.vma.zst) converter
Dockerized converter that turns VirtualBox/VMware OVA appliances into native Proxmox vzdump backups restorable via qmrestore, without needing a PVE host. - OVF parser (CPU/RAM/disks/NICs, SCSI/SATA/IDE controllers) - Streaming OVA extractor - qemu-server.conf generator with required #qmdump#map lines - Orchestrator: qemu-img convert -> vma create -> zstd - Click CLI: convert, create-test-ova, gui subcommands - Flask web UI with Convert + Create-test-OVA tabs, SSE progress - Dockerfile: Debian bookworm + pve-no-subscription for vma/qemu-img, pre-caches Alpine tiny raw image (sha512-verified) for bootable test-OVA generation - End-to-end verified: generated OVA imports into VirtualBox, converted vzdump restores on Proxmox and the VM boots Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Stefan Hacker
parent
5fe65d6b44
commit
dcbc957334
|
|
@ -0,0 +1,10 @@
|
|||
.git
|
||||
.gitignore
|
||||
__pycache__
|
||||
*.pyc
|
||||
*.egg-info
|
||||
.venv
|
||||
venv
|
||||
.pytest_cache
|
||||
.mypy_cache
|
||||
data
|
||||
|
|
@ -0,0 +1,66 @@
|
|||
# ova2vzdump — Debian bookworm + Proxmox no-subscription repo for the
|
||||
# `vma` binary, plus qemu-utils for VMDK->raw conversion.
|
||||
FROM debian:bookworm-slim
|
||||
|
||||
ENV DEBIAN_FRONTEND=noninteractive
|
||||
|
||||
RUN apt-get update && apt-get install -y --no-install-recommends \
|
||||
ca-certificates curl gnupg lsb-release \
|
||||
&& install -d /etc/apt/keyrings \
|
||||
&& curl -fsSL https://enterprise.proxmox.com/debian/proxmox-release-bookworm.gpg \
|
||||
-o /etc/apt/keyrings/proxmox-release-bookworm.gpg \
|
||||
&& echo "deb [signed-by=/etc/apt/keyrings/proxmox-release-bookworm.gpg] \
|
||||
http://download.proxmox.com/debian/pve bookworm pve-no-subscription" \
|
||||
> /etc/apt/sources.list.d/pve.list \
|
||||
&& apt-get update \
|
||||
&& apt-get install -y --no-install-recommends \
|
||||
pve-qemu-kvm \
|
||||
zstd \
|
||||
python3 \
|
||||
python3-pip \
|
||||
python3-venv \
|
||||
&& rm -rf /var/lib/apt/lists/*
|
||||
|
||||
# pve-qemu-kvm provides its own qemu-img and the vma binary; confirm both
|
||||
# are on PATH so the later conversion steps can find them.
|
||||
RUN command -v qemu-img && command -v vma
|
||||
|
||||
# Pre-cache a small bootable Alpine raw image so `create-test-ova --bootable`
|
||||
# can produce a real OVA in seconds without re-downloading. Pinned version +
|
||||
# checksum keeps builds reproducible.
|
||||
ARG ALPINE_VERSION=3.22.4
|
||||
ARG ALPINE_FILE=gcp_alpine-3.22.4-x86_64-bios-tiny-r0.raw.tar.gz
|
||||
ARG ALPINE_URL=https://dl-cdn.alpinelinux.org/alpine/v3.22/releases/cloud
|
||||
RUN set -eux \
|
||||
&& mkdir -p /app/fixtures /tmp/alpine-extract \
|
||||
&& cd /tmp \
|
||||
&& curl -fsSL -o "${ALPINE_FILE}" "${ALPINE_URL}/${ALPINE_FILE}" \
|
||||
&& curl -fsSL -o "${ALPINE_FILE}.hash" "${ALPINE_URL}/${ALPINE_FILE}.sha512" \
|
||||
# Alpine publishes bare hex hashes (no filename). Build a proper
|
||||
# sha512sum-compatible line so we can verify with `-c`.
|
||||
&& printf '%s %s\n' "$(tr -d '[:space:]' < "${ALPINE_FILE}.hash")" "${ALPINE_FILE}" \
|
||||
> "${ALPINE_FILE}.sha512" \
|
||||
&& sha512sum -c "${ALPINE_FILE}.sha512" \
|
||||
&& tar -xzf "${ALPINE_FILE}" -C /tmp/alpine-extract \
|
||||
&& raw="$(find /tmp/alpine-extract -maxdepth 2 -name '*.raw' -print -quit)" \
|
||||
&& test -n "$raw" || (echo "no .raw file inside tarball" && exit 1) \
|
||||
&& mv "$raw" /app/fixtures/alpine-base.raw \
|
||||
&& rm -rf /tmp/alpine-extract "${ALPINE_FILE}" "${ALPINE_FILE}.hash" "${ALPINE_FILE}.sha512" \
|
||||
&& ls -la /app/fixtures/
|
||||
|
||||
WORKDIR /app
|
||||
COPY pyproject.toml requirements.txt README.md ./
|
||||
COPY src ./src
|
||||
COPY scripts ./scripts
|
||||
|
||||
RUN pip install --no-cache-dir --break-system-packages . \
|
||||
&& mkdir -p /data/uploads /data/output
|
||||
|
||||
VOLUME ["/data"]
|
||||
|
||||
EXPOSE 8080
|
||||
|
||||
ENV OVA2VZDUMP_ALPINE_BASE=/app/fixtures/alpine-base.raw
|
||||
|
||||
ENTRYPOINT ["ova2vzdump"]
|
||||
CMD ["gui", "--upload-dir", "/data/uploads", "--output-dir", "/data/output"]
|
||||
|
|
@ -0,0 +1,313 @@
|
|||
# ova2vzdump
|
||||
|
||||
*[English](#english) · [Deutsch](#deutsch)*
|
||||
|
||||
<a id="english"></a>
|
||||
|
||||
Convert OVA appliances (VirtualBox / VMware export) into **Proxmox
|
||||
vzdump** `.vma.zst` backup files that can be restored with `qmrestore`.
|
||||
|
||||
The tricky part is `vma`, the proprietary container format Proxmox uses
|
||||
for QEMU backups. The `vma` binary only ships with Proxmox, so this tool
|
||||
runs inside a Docker image that installs `pve-qemu-kvm` from the Proxmox
|
||||
no-subscription apt repo. That way you can convert appliances on any
|
||||
machine with Docker — no PVE host required.
|
||||
|
||||
## Pipeline
|
||||
|
||||
1. Extract OVA tar → OVF descriptor + VMDK disk(s)
|
||||
2. Parse OVF XML → CPU, memory, disks, NICs
|
||||
3. `qemu-img convert` each VMDK → raw
|
||||
4. Synthesize a `qemu-server.conf`
|
||||
5. `vma create` packs raws + conf → `.vma`
|
||||
6. `zstd` compresses → `vzdump-qemu-<vmid>-<ts>.vma.zst`
|
||||
|
||||
## Usage
|
||||
|
||||
### Build the image
|
||||
|
||||
```bash
|
||||
docker compose build
|
||||
```
|
||||
|
||||
### Web UI
|
||||
|
||||
```bash
|
||||
docker compose up
|
||||
# browse http://localhost:8880
|
||||
```
|
||||
|
||||
Uploads land in `./data/uploads`, results in `./data/output`.
|
||||
|
||||
### Getting a test OVA
|
||||
|
||||
ova2vzdump ships a built-in test OVA generator — both the CLI and the
|
||||
web UI have a dedicated entry point. Two modes:
|
||||
|
||||
- **stub** (default, ~1 KB): valid OVA with an empty VMDK. Fast, exercises
|
||||
the full `extract → parse → qemu-img → vma → zstd` pipeline. Not bootable.
|
||||
- **bootable** (~80 MB): packages a pre-cached **Alpine Linux** raw image
|
||||
(downloaded during `docker build`, sha512-verified). After `qmrestore`
|
||||
on Proxmox the VM actually boots to an Alpine login prompt — this is
|
||||
what you want for a real end-to-end test.
|
||||
|
||||
Existing test OVAs are reused by default; pass `--force` to rebuild.
|
||||
|
||||
#### CLI
|
||||
|
||||
```bash
|
||||
# stub:
|
||||
docker compose run --rm ova2vzdump create-test-ova \
|
||||
/data/uploads/test.ova --name demo
|
||||
|
||||
# bootable Alpine:
|
||||
docker compose run --rm ova2vzdump create-test-ova \
|
||||
/data/uploads/alpine.ova --name alpine --bootable
|
||||
|
||||
# rebuild:
|
||||
docker compose run --rm ova2vzdump create-test-ova \
|
||||
/data/uploads/test.ova --force
|
||||
```
|
||||
|
||||
#### Web UI
|
||||
|
||||
Click the **Create test OVA** tab, pick a name, tick **Bootable** if you
|
||||
want real Alpine, hit Create. The list at the bottom shows what's already
|
||||
in the upload directory and offers direct download. The generated OVA
|
||||
also appears as an available input on the **Convert** tab.
|
||||
|
||||
If you'd rather grab a real-world appliance from elsewhere:
|
||||
|
||||
- **TurnKey Linux Core** (~250 MB):
|
||||
[turnkeylinux.org/core](https://www.turnkeylinux.org/core) → VM build.
|
||||
- **DIY**: export any small VM from VirtualBox via
|
||||
`File → Export Appliance…` → `.ova`.
|
||||
|
||||
### CLI
|
||||
|
||||
```bash
|
||||
docker run --rm \
|
||||
-v "$PWD/data/uploads:/in" \
|
||||
-v "$PWD/data/output:/out" \
|
||||
ova2vzdump:latest convert /in/appliance.ova /out \
|
||||
--vmid 123 --storage local-lvm
|
||||
```
|
||||
|
||||
Output filename looks like `vzdump-qemu-123-2026_04_21-12_30_00.vma.zst`.
|
||||
|
||||
### Restore on Proxmox
|
||||
|
||||
Copy the `.vma.zst` onto a PVE node into a backup-capable storage (for
|
||||
example `/var/lib/vz/dump/`), then:
|
||||
|
||||
```bash
|
||||
qmrestore vzdump-qemu-123-2026_04_21-12_30_00.vma.zst 123 --storage local-lvm
|
||||
```
|
||||
|
||||
## Options
|
||||
|
||||
| Flag | Meaning | Default |
|
||||
| ---- | ------- | ------- |
|
||||
| `--vmid` | **Placeholder** VMID — overridden at restore time | `100` |
|
||||
| `--storage` | **Placeholder** storage name in qemu-server.conf — overridden by `qmrestore --storage` | `local-lvm` |
|
||||
| `--keep-workdir` | Keep intermediate raw disks + vma for debugging | off |
|
||||
|
||||
> `qmrestore <file> <target-vmid> [--storage <name>]` always decides the
|
||||
> real VMID and storage. The values baked into the archive are only
|
||||
> placeholders; you can leave the defaults for most use cases and only
|
||||
> tweak them if you want a meaningful filename like
|
||||
> `vzdump-qemu-<originally-intended-vmid>-*.vma.zst`.
|
||||
|
||||
## Config mapping
|
||||
|
||||
| OVF | qemu-server.conf |
|
||||
| --- | ---------------- |
|
||||
| `VirtualHardwareSection` CPU quantity | `cores` |
|
||||
| `VirtualHardwareSection` memory | `memory` (MiB) |
|
||||
| OS description heuristic (`linux`/`windows`/…) | `ostype` (`l26` / `win10` / `other`) |
|
||||
| Disks w/ SCSI controller | `scsiN` + `scsihw: virtio-scsi-pci` |
|
||||
| Disks w/ SATA controller | `sataN` |
|
||||
| Disks w/ IDE controller | `ideN` |
|
||||
| Ethernet subtype | `netN: <model>=auto,bridge=vmbr0` |
|
||||
|
||||
Heuristics are conservative; review the generated config after restore
|
||||
before booting production workloads.
|
||||
|
||||
## Limitations
|
||||
|
||||
- OVF 1.x / 2.x only (DMTF envelope namespace).
|
||||
- `manifest`, `cert`, and signed OVAs are not verified.
|
||||
- Firmware (BIOS vs. UEFI) is not read from OVF — defaults to SeaBIOS.
|
||||
Edit the config after restore if your appliance needs OVMF.
|
||||
- MAC addresses are not preserved (Proxmox assigns new ones on import).
|
||||
|
||||
## Local development (without Docker)
|
||||
|
||||
You need `qemu-img`, `vma`, and `zstd` on your PATH — i.e. you must be on
|
||||
a Proxmox host. Then:
|
||||
|
||||
```bash
|
||||
pip install -e .
|
||||
ova2vzdump convert appliance.ova ./out
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
<a id="deutsch"></a>
|
||||
|
||||
# ova2vzdump (Deutsch)
|
||||
|
||||
Konvertiert OVA-Appliances (Export aus VirtualBox / VMware) in **Proxmox
|
||||
vzdump** `.vma.zst`-Backup-Dateien, die sich mit `qmrestore` direkt
|
||||
zurückspielen lassen.
|
||||
|
||||
Der knifflige Teil ist `vma` — das proprietäre Container-Format, in dem
|
||||
Proxmox seine QEMU-Backups speichert. Die `vma`-Binary gibt es nur mit
|
||||
Proxmox, deshalb läuft dieses Tool in einem Docker-Image, das
|
||||
`pve-qemu-kvm` aus dem Proxmox-No-Subscription-Apt-Repo installiert.
|
||||
Dadurch kannst du Appliances auf jeder Maschine mit Docker konvertieren —
|
||||
ganz ohne PVE-Host.
|
||||
|
||||
## Pipeline
|
||||
|
||||
1. OVA-Tar entpacken → OVF-Descriptor + VMDK-Disk(s)
|
||||
2. OVF-XML parsen → CPU, RAM, Disks, NICs
|
||||
3. Jede VMDK mit `qemu-img convert` nach raw konvertieren
|
||||
4. Eine `qemu-server.conf` generieren
|
||||
5. `vma create` packt raws + config → `.vma`
|
||||
6. `zstd` komprimiert → `vzdump-qemu-<vmid>-<ts>.vma.zst`
|
||||
|
||||
## Benutzung
|
||||
|
||||
### Image bauen
|
||||
|
||||
```bash
|
||||
docker compose build
|
||||
```
|
||||
|
||||
### Web-UI
|
||||
|
||||
```bash
|
||||
docker compose up
|
||||
# dann http://localhost:8880 öffnen
|
||||
```
|
||||
|
||||
Uploads landen in `./data/uploads`, Ergebnisse in `./data/output`.
|
||||
|
||||
### Test-OVA bekommen
|
||||
|
||||
ova2vzdump bringt einen eingebauten Test-OVA-Generator mit — sowohl die
|
||||
CLI als auch die Web-UI haben einen eigenen Einstiegspunkt. Zwei Modi:
|
||||
|
||||
- **stub** (Default, ~1 KB): valide OVA mit leerer VMDK. Schnell, testet
|
||||
die komplette Pipeline `extract → parse → qemu-img → vma → zstd`.
|
||||
Nicht bootbar.
|
||||
- **bootable** (~80 MB): verpackt ein vorgecachtes **Alpine-Linux**-Raw-
|
||||
Image (wird beim `docker build` heruntergeladen, sha512-verifiziert).
|
||||
Nach `qmrestore` auf Proxmox bootet die VM tatsächlich in einen
|
||||
Alpine-Login — das ist der echte End-to-End-Test.
|
||||
|
||||
Vorhandene Test-OVAs werden standardmäßig wiederverwendet; `--force` baut
|
||||
neu.
|
||||
|
||||
#### CLI
|
||||
|
||||
```bash
|
||||
# Stub:
|
||||
docker compose run --rm ova2vzdump create-test-ova \
|
||||
/data/uploads/test.ova --name demo
|
||||
|
||||
# Bootbares Alpine:
|
||||
docker compose run --rm ova2vzdump create-test-ova \
|
||||
/data/uploads/alpine.ova --name alpine --bootable
|
||||
|
||||
# Neu bauen:
|
||||
docker compose run --rm ova2vzdump create-test-ova \
|
||||
/data/uploads/test.ova --force
|
||||
```
|
||||
|
||||
#### Web-UI
|
||||
|
||||
Auf den Tab **Create test OVA** klicken, Name wählen, Häkchen bei
|
||||
**Bootable** setzen wenn du echtes Alpine willst, auf Create klicken. Die
|
||||
Liste unten zeigt, was im Upload-Verzeichnis liegt, mit direkten
|
||||
Download-Buttons. Die erzeugte OVA erscheint auch im **Convert**-Tab als
|
||||
auswählbare Eingabe.
|
||||
|
||||
Lieber eine echte Appliance von woanders?
|
||||
|
||||
- **TurnKey Linux Core** (~250 MB):
|
||||
[turnkeylinux.org/core](https://www.turnkeylinux.org/core) → VM-Build.
|
||||
- **DIY**: irgendeine kleine VM aus VirtualBox exportieren via
|
||||
`Datei → Appliance exportieren…` → `.ova`.
|
||||
|
||||
### CLI (Konvertierung)
|
||||
|
||||
```bash
|
||||
docker run --rm \
|
||||
-v "$PWD/data/uploads:/in" \
|
||||
-v "$PWD/data/output:/out" \
|
||||
ova2vzdump:latest convert /in/appliance.ova /out \
|
||||
--vmid 123 --storage local-lvm
|
||||
```
|
||||
|
||||
Der Ausgabedateiname sieht etwa so aus:
|
||||
`vzdump-qemu-123-2026_04_21-12_30_00.vma.zst`.
|
||||
|
||||
### Restore auf Proxmox
|
||||
|
||||
Die `.vma.zst` auf einen PVE-Node in ein backup-fähiges Storage kopieren
|
||||
(z.B. `/var/lib/vz/dump/`), dann:
|
||||
|
||||
```bash
|
||||
qmrestore vzdump-qemu-123-2026_04_21-12_30_00.vma.zst 123 --storage local-lvm
|
||||
```
|
||||
|
||||
## Optionen
|
||||
|
||||
| Flag | Bedeutung | Default |
|
||||
| ---- | --------- | ------- |
|
||||
| `--vmid` | **Platzhalter**-VMID — wird beim Restore überschrieben | `100` |
|
||||
| `--storage` | **Platzhalter**-Storage-Name in `qemu-server.conf` — wird durch `qmrestore --storage` überschrieben | `local-lvm` |
|
||||
| `--keep-workdir` | Zwischenergebnisse (raw-Disks, vma) behalten, zum Debuggen | aus |
|
||||
|
||||
> `qmrestore <file> <ziel-vmid> [--storage <name>]` entscheidet immer die
|
||||
> echte VMID und das Storage. Die Werte im Archiv sind nur Platzhalter —
|
||||
> Defaults reichen meist. Du passt sie höchstens an, wenn du einen
|
||||
> sprechenden Dateinamen haben willst, z.B.
|
||||
> `vzdump-qemu-<ursprüngliche-vmid>-*.vma.zst`.
|
||||
|
||||
## Config-Mapping
|
||||
|
||||
| OVF | qemu-server.conf |
|
||||
| --- | ---------------- |
|
||||
| `VirtualHardwareSection` CPU-Anzahl | `cores` |
|
||||
| `VirtualHardwareSection` RAM | `memory` (MiB) |
|
||||
| OS-Description-Heuristik (`linux`/`windows`/…) | `ostype` (`l26` / `win10` / `other`) |
|
||||
| Disks am SCSI-Controller | `scsiN` + `scsihw: virtio-scsi-pci` |
|
||||
| Disks am SATA-Controller | `sataN` |
|
||||
| Disks am IDE-Controller | `ideN` |
|
||||
| Ethernet-Subtyp | `netN: <model>=auto,bridge=vmbr0` |
|
||||
|
||||
Die Heuristiken sind konservativ; bitte die generierte Config nach dem
|
||||
Restore einmal prüfen, bevor du produktive Workloads startest.
|
||||
|
||||
## Limitierungen
|
||||
|
||||
- Nur OVF 1.x / 2.x (DMTF-Envelope-Namespace).
|
||||
- `manifest`, `cert` und signierte OVAs werden nicht verifiziert.
|
||||
- Firmware (BIOS vs. UEFI) wird nicht aus der OVF gelesen — Default ist
|
||||
SeaBIOS. Config nach dem Restore manuell anpassen, falls deine
|
||||
Appliance OVMF braucht.
|
||||
- MAC-Adressen werden nicht übernommen (Proxmox vergibt beim Import
|
||||
neue).
|
||||
|
||||
## Lokale Entwicklung (ohne Docker)
|
||||
|
||||
Du brauchst `qemu-img`, `vma` und `zstd` im PATH — also de facto einen
|
||||
Proxmox-Host. Dann:
|
||||
|
||||
```bash
|
||||
pip install -e .
|
||||
ova2vzdump convert appliance.ova ./out
|
||||
```
|
||||
|
|
@ -0,0 +1,11 @@
|
|||
services:
|
||||
ova2vzdump:
|
||||
build: .
|
||||
image: ova2vzdump:latest
|
||||
container_name: ova2vzdump
|
||||
ports:
|
||||
- "8889:8080"
|
||||
volumes:
|
||||
- ./data/uploads:/data/uploads
|
||||
- ./data/output:/data/output
|
||||
restart: unless-stopped
|
||||
|
|
@ -0,0 +1,26 @@
|
|||
[build-system]
|
||||
requires = ["setuptools>=68", "wheel"]
|
||||
build-backend = "setuptools.build_meta"
|
||||
|
||||
[project]
|
||||
name = "ova2vzdump"
|
||||
version = "0.1.0"
|
||||
description = "Convert OVA (VirtualBox/VMware) templates into Proxmox vzdump (.vma.zst) backups"
|
||||
readme = "README.md"
|
||||
requires-python = ">=3.10"
|
||||
authors = [{ name = "Stefan Hacker", email = "stefanhacker@gmx.de" }]
|
||||
license = { text = "MIT" }
|
||||
dependencies = [
|
||||
"click>=8.1",
|
||||
"flask>=3.0",
|
||||
"lxml>=5.1",
|
||||
]
|
||||
|
||||
[project.scripts]
|
||||
ova2vzdump = "ova2vzdump.cli:main"
|
||||
|
||||
[tool.setuptools.packages.find]
|
||||
where = ["src"]
|
||||
|
||||
[tool.setuptools.package-data]
|
||||
ova2vzdump = ["templates/*.html", "static/*"]
|
||||
|
|
@ -0,0 +1,3 @@
|
|||
click>=8.1
|
||||
flask>=3.0
|
||||
lxml>=5.1
|
||||
|
|
@ -0,0 +1,40 @@
|
|||
#!/usr/bin/env python3
|
||||
"""Thin CLI wrapper so folks can run the test-OVA builder without having
|
||||
the package installed in their active env. Prefer `ova2vzdump create-test-ova`
|
||||
when possible."""
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).parent.parent / "src"))
|
||||
|
||||
from ova2vzdump.test_ova import create_test_ova # noqa: E402
|
||||
|
||||
|
||||
def main() -> None:
|
||||
ap = argparse.ArgumentParser(description=__doc__)
|
||||
ap.add_argument("output", type=Path)
|
||||
ap.add_argument("--name", default="test-vm")
|
||||
ap.add_argument("--size-mib", type=int, default=64)
|
||||
ap.add_argument("--bootable", action="store_true",
|
||||
help="Package pre-cached bootable Alpine image")
|
||||
ap.add_argument("--force", action="store_true",
|
||||
help="Overwrite existing output file")
|
||||
args = ap.parse_args()
|
||||
|
||||
r = create_test_ova(
|
||||
output=args.output, vm_name=args.name, size_mib=args.size_mib,
|
||||
bootable=args.bootable, force=args.force,
|
||||
)
|
||||
if r.reused:
|
||||
print(f"reused existing {r.path} ({r.bytes} bytes) "
|
||||
"(pass --force to rebuild)")
|
||||
else:
|
||||
kind = "bootable Alpine" if r.bootable else "empty stub"
|
||||
print(f"wrote {r.path} ({r.bytes} bytes, {kind})")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
|
|
@ -0,0 +1,3 @@
|
|||
"""ova2vzdump — convert OVA templates to Proxmox vzdump (.vma.zst) backups."""
|
||||
|
||||
__version__ = "0.1.0"
|
||||
|
|
@ -0,0 +1,4 @@
|
|||
from ova2vzdump.cli import main
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
|
|
@ -0,0 +1,112 @@
|
|||
"""Command-line entry point for ova2vzdump."""
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
import click
|
||||
|
||||
from .converter import convert
|
||||
|
||||
|
||||
@click.group()
|
||||
@click.option("-v", "--verbose", count=True, help="-v info, -vv debug")
|
||||
def main(verbose: int) -> None:
|
||||
level = logging.WARNING
|
||||
if verbose == 1:
|
||||
level = logging.INFO
|
||||
elif verbose >= 2:
|
||||
level = logging.DEBUG
|
||||
logging.basicConfig(
|
||||
level=level, format="%(asctime)s %(levelname)s %(name)s: %(message)s"
|
||||
)
|
||||
|
||||
|
||||
@main.command("convert")
|
||||
@click.argument("ova", type=click.Path(exists=True, dir_okay=False, path_type=Path))
|
||||
@click.argument("output_dir", type=click.Path(file_okay=False, path_type=Path))
|
||||
@click.option("--vmid", type=int, default=100, show_default=True,
|
||||
help="Placeholder VMID in filename/config. Overridden at "
|
||||
"restore time by `qmrestore <file> <target-vmid>`.")
|
||||
@click.option("--storage", default="local-lvm", show_default=True,
|
||||
help="Placeholder storage name in qemu-server.conf. "
|
||||
"Override at restore time with `qmrestore --storage`.")
|
||||
@click.option("--keep-workdir", is_flag=True,
|
||||
help="Keep intermediate raw disks and vma file for debugging")
|
||||
def convert_cmd(ova: Path, output_dir: Path, vmid: int, storage: str,
|
||||
keep_workdir: bool) -> None:
|
||||
"""Convert an OVA appliance into a vzdump .vma.zst backup."""
|
||||
def progress(stage: str, pct: float) -> None:
|
||||
click.echo(f" [{stage}] {pct:5.1f}%", err=True)
|
||||
|
||||
try:
|
||||
result = convert(
|
||||
ova_path=ova, output_dir=output_dir, vmid=vmid, storage=storage,
|
||||
progress=progress, keep_workdir=keep_workdir,
|
||||
)
|
||||
except Exception as exc:
|
||||
click.echo(f"error: {exc}", err=True)
|
||||
sys.exit(1)
|
||||
|
||||
click.echo(f"OK: wrote {result.output_path}")
|
||||
click.echo(f" VM '{result.vm_name}', vmid={result.vmid}, "
|
||||
f"{result.disk_count} disk(s), "
|
||||
f"{result.total_bytes / 1024**3:.2f} GiB raw total")
|
||||
click.echo("Restore on Proxmox:")
|
||||
click.echo(f" qmrestore {result.output_path.name} {result.vmid} "
|
||||
f"--storage {storage}")
|
||||
|
||||
|
||||
@main.command("gui")
|
||||
@click.option("--host", default="0.0.0.0", show_default=True)
|
||||
@click.option("--port", type=int, default=8080, show_default=True)
|
||||
@click.option("--upload-dir", type=click.Path(path_type=Path),
|
||||
default=Path("/data/uploads"), show_default=True)
|
||||
@click.option("--output-dir", type=click.Path(path_type=Path),
|
||||
default=Path("/data/output"), show_default=True)
|
||||
def gui_cmd(host: str, port: int, upload_dir: Path, output_dir: Path) -> None:
|
||||
"""Launch the Flask web UI."""
|
||||
from .web import create_app
|
||||
upload_dir.mkdir(parents=True, exist_ok=True)
|
||||
output_dir.mkdir(parents=True, exist_ok=True)
|
||||
app = create_app(upload_dir=upload_dir, output_dir=output_dir)
|
||||
app.run(host=host, port=port, threaded=True)
|
||||
|
||||
|
||||
@main.command("create-test-ova")
|
||||
@click.argument("output", type=click.Path(path_type=Path))
|
||||
@click.option("--name", default="test-vm", show_default=True,
|
||||
help="VM name embedded in the OVF")
|
||||
@click.option("--size-mib", type=int, default=64, show_default=True,
|
||||
help="Disk capacity (ignored when --bootable)")
|
||||
@click.option("--bootable", is_flag=True,
|
||||
help="Package a real bootable Alpine image "
|
||||
"(cached during docker build)")
|
||||
@click.option("--force", is_flag=True,
|
||||
help="Overwrite an existing file instead of reusing it")
|
||||
def create_test_ova_cmd(output: Path, name: str, size_mib: int,
|
||||
bootable: bool, force: bool) -> None:
|
||||
"""Build a small test OVA for smoke-testing the converter."""
|
||||
from .test_ova import create_test_ova
|
||||
try:
|
||||
r = create_test_ova(
|
||||
output=output, vm_name=name, size_mib=size_mib,
|
||||
bootable=bootable, force=force,
|
||||
)
|
||||
except Exception as exc:
|
||||
click.echo(f"error: {exc}", err=True)
|
||||
sys.exit(1)
|
||||
|
||||
if r.reused:
|
||||
click.echo(
|
||||
f"reused existing {r.path} ({r.bytes / 1024:.1f} KiB) "
|
||||
"— pass --force to rebuild"
|
||||
)
|
||||
else:
|
||||
kind = "bootable Alpine" if r.bootable else "empty stub"
|
||||
click.echo(f"wrote {r.path} ({r.bytes / 1024:.1f} KiB, {kind})")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
|
|
@ -0,0 +1,97 @@
|
|||
"""Map an OvfVm into a Proxmox qemu-server.conf string.
|
||||
|
||||
Proxmox's config format is a simple KEY: VALUE text file with disk entries
|
||||
like `scsi0: <storage>:vm-<vmid>-disk-0,size=8G`. For a VMA archive we
|
||||
also need special `#qmdump#map:VIRTDEV:DEVNAME:STOREID:FORMAT:` comment
|
||||
lines — without them, `qmrestore` aborts with "found no device mapping
|
||||
information for device 'drive-XXX'".
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
from .ovf_parser import OvfDisk, OvfVm
|
||||
|
||||
OS_HINT_MAP = [
|
||||
("windows", "win10"),
|
||||
("win", "win10"),
|
||||
("debian", "l26"),
|
||||
("ubuntu", "l26"),
|
||||
("centos", "l26"),
|
||||
("redhat", "l26"),
|
||||
("suse", "l26"),
|
||||
("fedora", "l26"),
|
||||
("linux", "l26"),
|
||||
("bsd", "other"),
|
||||
]
|
||||
|
||||
|
||||
def _guess_ostype(os_desc: str | None) -> str:
|
||||
if not os_desc:
|
||||
return "l26"
|
||||
s = os_desc.lower()
|
||||
for needle, ostype in OS_HINT_MAP:
|
||||
if needle in s:
|
||||
return ostype
|
||||
return "other"
|
||||
|
||||
|
||||
def _bytes_to_gib(b: int) -> int:
|
||||
gib = b // (1024**3)
|
||||
return max(1, gib)
|
||||
|
||||
|
||||
def assign_device_names(disks: list[OvfDisk]) -> list[tuple[str, str]]:
|
||||
"""Return (virtdev, devname) per disk, e.g. ('scsi0', 'drive-scsi0')."""
|
||||
scsi_idx = sata_idx = ide_idx = 0
|
||||
result: list[tuple[str, str]] = []
|
||||
for disk in disks:
|
||||
if disk.controller == "sata":
|
||||
virtdev = f"sata{sata_idx}"; sata_idx += 1
|
||||
elif disk.controller == "ide":
|
||||
virtdev = f"ide{ide_idx}"; ide_idx += 1
|
||||
else:
|
||||
virtdev = f"scsi{scsi_idx}"; scsi_idx += 1
|
||||
result.append((virtdev, f"drive-{virtdev}"))
|
||||
return result
|
||||
|
||||
|
||||
def build_qemu_conf(vm: OvfVm, vmid: int, storage: str = "local-lvm") -> str:
|
||||
devices = assign_device_names(vm.disks)
|
||||
|
||||
lines: list[str] = []
|
||||
lines.append(f"#ova2vzdump import of {vm.name}")
|
||||
if vm.os_type:
|
||||
lines.append(f"#source os: {vm.os_type}")
|
||||
|
||||
# qmdump map lines — REQUIRED for `qmrestore`, otherwise vma extract
|
||||
# errors out with "found no device mapping information for device 'drive-XXX'".
|
||||
for (virtdev, devname), _disk in zip(devices, vm.disks):
|
||||
lines.append(f"#qmdump#map:{virtdev}:{devname}:{storage}:raw:")
|
||||
|
||||
lines.append(f"name: {_sanitize_name(vm.name)}")
|
||||
lines.append(f"cores: {vm.cpus}")
|
||||
lines.append(f"memory: {vm.memory_mb}")
|
||||
lines.append(f"ostype: {_guess_ostype(vm.os_type)}")
|
||||
boot_dev = devices[0][0] if devices else "scsi0"
|
||||
lines.append(f"boot: order={boot_dev};net0")
|
||||
lines.append("scsihw: virtio-scsi-pci")
|
||||
lines.append("agent: 1")
|
||||
|
||||
for i, ((virtdev, _), disk) in enumerate(zip(devices, vm.disks)):
|
||||
size_gib = _bytes_to_gib(disk.capacity_bytes)
|
||||
volume = f"{storage}:vm-{vmid}-disk-{i}"
|
||||
lines.append(f"{virtdev}: {volume},size={size_gib}G")
|
||||
|
||||
for i, nic in enumerate(vm.nics):
|
||||
# Format is `<model>[=<mac>]` — omit the `=<mac>` part to let
|
||||
# Proxmox auto-assign. `=auto` is NOT a valid placeholder; it
|
||||
# would be parsed as a MAC literal and rejected.
|
||||
lines.append(f"net{i}: {nic.model},bridge=vmbr0,firewall=1")
|
||||
|
||||
lines.append("")
|
||||
return "\n".join(lines)
|
||||
|
||||
|
||||
def _sanitize_name(name: str) -> str:
|
||||
safe = "".join(c if c.isalnum() or c in "-_" else "-" for c in name)
|
||||
safe = safe.strip("-") or "imported-vm"
|
||||
return safe[:63]
|
||||
|
|
@ -0,0 +1,155 @@
|
|||
"""Orchestrate the full OVA -> vzdump pipeline.
|
||||
|
||||
Pipeline:
|
||||
1. Extract OVA tar into workdir
|
||||
2. Parse OVF
|
||||
3. Convert each VMDK to raw (vma requires raw inputs)
|
||||
4. Write qemu-server.conf
|
||||
5. Call `vma create` with raw disks + config -> vma file
|
||||
6. Compress with zstd -> vzdump-qemu-<vmid>-<ts>.vma.zst
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
import shutil
|
||||
import subprocess
|
||||
import time
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
from typing import Callable, Optional
|
||||
|
||||
from .config_mapper import assign_device_names, build_qemu_conf
|
||||
from .ova_extractor import extract_ova
|
||||
from .ovf_parser import parse_ovf
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
|
||||
ProgressFn = Callable[[str, float], None]
|
||||
|
||||
|
||||
@dataclass
|
||||
class ConversionResult:
|
||||
output_path: Path
|
||||
vm_name: str
|
||||
vmid: int
|
||||
disk_count: int
|
||||
total_bytes: int
|
||||
|
||||
|
||||
def _noop_progress(stage: str, pct: float) -> None:
|
||||
log.info("[%s] %.1f%%", stage, pct)
|
||||
|
||||
|
||||
def _run(cmd: list[str], **kwargs) -> None:
|
||||
"""Run a subprocess and re-raise with captured stderr on failure."""
|
||||
log.info("run: %s", " ".join(str(c) for c in cmd))
|
||||
try:
|
||||
subprocess.run(cmd, check=True, **kwargs)
|
||||
except subprocess.CalledProcessError as exc:
|
||||
tail = ""
|
||||
if exc.stderr:
|
||||
tail_bytes = exc.stderr if isinstance(exc.stderr, bytes) else exc.stderr.encode()
|
||||
tail = "\n" + tail_bytes.decode(errors="replace").strip()
|
||||
raise RuntimeError(
|
||||
f"{cmd[0]} exited {exc.returncode}: "
|
||||
f"{' '.join(str(c) for c in cmd)}{tail}"
|
||||
) from exc
|
||||
|
||||
|
||||
def _require_tool(name: str) -> str:
|
||||
path = shutil.which(name)
|
||||
if not path:
|
||||
raise RuntimeError(
|
||||
f"Required tool '{name}' not found in PATH. "
|
||||
"Run inside the ova2vzdump Docker image, which ships it."
|
||||
)
|
||||
return path
|
||||
|
||||
|
||||
def convert(
|
||||
ova_path: Path,
|
||||
output_dir: Path,
|
||||
vmid: int = 100,
|
||||
storage: str = "local-lvm",
|
||||
workdir: Optional[Path] = None,
|
||||
progress: ProgressFn = _noop_progress,
|
||||
keep_workdir: bool = False,
|
||||
) -> ConversionResult:
|
||||
_require_tool("qemu-img")
|
||||
_require_tool("vma")
|
||||
_require_tool("zstd")
|
||||
|
||||
ova_path = Path(ova_path).resolve()
|
||||
output_dir = Path(output_dir).resolve()
|
||||
output_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
wd = Path(workdir) if workdir else output_dir / f".work-{int(time.time())}"
|
||||
wd.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
try:
|
||||
progress("extract", 0.0)
|
||||
extracted = extract_ova(ova_path, wd / "ova")
|
||||
progress("extract", 100.0)
|
||||
|
||||
progress("parse", 0.0)
|
||||
vm = parse_ovf(extracted.ovf_path)
|
||||
progress("parse", 100.0)
|
||||
log.info(
|
||||
"Parsed VM '%s': %d CPU, %d MiB RAM, %d disks, %d NICs",
|
||||
vm.name, vm.cpus, vm.memory_mb, len(vm.disks), len(vm.nics),
|
||||
)
|
||||
|
||||
raw_dir = wd / "raw"
|
||||
raw_dir.mkdir(exist_ok=True)
|
||||
raw_disks: list[Path] = []
|
||||
total_bytes = 0
|
||||
for idx, disk in enumerate(vm.disks):
|
||||
src_file = extracted.files.get(disk.file_href)
|
||||
if not src_file or not src_file.exists():
|
||||
raise ValueError(
|
||||
f"Disk file '{disk.file_href}' missing from OVA"
|
||||
)
|
||||
raw_path = raw_dir / f"disk-{idx}.raw"
|
||||
progress(f"convert-disk-{idx}", 0.0)
|
||||
_run([
|
||||
"qemu-img", "convert", "-p", "-O", "raw",
|
||||
str(src_file), str(raw_path),
|
||||
])
|
||||
total_bytes += raw_path.stat().st_size
|
||||
raw_disks.append(raw_path)
|
||||
progress(f"convert-disk-{idx}", 100.0)
|
||||
|
||||
conf_path = wd / "qemu-server.conf"
|
||||
conf_path.write_text(build_qemu_conf(vm, vmid=vmid, storage=storage))
|
||||
|
||||
progress("vma-create", 0.0)
|
||||
vma_path = wd / f"vzdump-qemu-{vmid}.vma"
|
||||
# `-c <path>` — vma derives the archive entry name from basename,
|
||||
# so the conf file must already be named `qemu-server.conf`.
|
||||
# Drives are positional `<bus-device>=<raw-path>`, e.g.
|
||||
# `drive-scsi0=/tmp/disk-0.raw` (no `.raw` in the device name!).
|
||||
devices = assign_device_names(vm.disks)
|
||||
vma_cmd = ["vma", "create", "-v", str(vma_path),
|
||||
"-c", str(conf_path)]
|
||||
for (_, devname), raw_path in zip(devices, raw_disks):
|
||||
vma_cmd.append(f"{devname}={raw_path}")
|
||||
_run(vma_cmd, capture_output=True)
|
||||
progress("vma-create", 100.0)
|
||||
|
||||
progress("compress", 0.0)
|
||||
ts = time.strftime("%Y_%m_%d-%H_%M_%S")
|
||||
out_name = f"vzdump-qemu-{vmid}-{ts}.vma.zst"
|
||||
out_path = output_dir / out_name
|
||||
_run(["zstd", "-T0", "--rm", "-o", str(out_path), str(vma_path)])
|
||||
progress("compress", 100.0)
|
||||
|
||||
return ConversionResult(
|
||||
output_path=out_path,
|
||||
vm_name=vm.name,
|
||||
vmid=vmid,
|
||||
disk_count=len(raw_disks),
|
||||
total_bytes=total_bytes,
|
||||
)
|
||||
finally:
|
||||
if not keep_workdir and wd.exists():
|
||||
shutil.rmtree(wd, ignore_errors=True)
|
||||
|
|
@ -0,0 +1,52 @@
|
|||
"""Extract an OVA tarball into a working directory.
|
||||
|
||||
An OVA is a plain (uncompressed) tar. We stream-extract it so memory stays
|
||||
flat even for 50-GB appliances. Returns paths to the OVF descriptor and a
|
||||
map of file href -> extracted path.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import tarfile
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
@dataclass
|
||||
class ExtractedOva:
|
||||
ovf_path: Path
|
||||
files: dict[str, Path]
|
||||
workdir: Path
|
||||
|
||||
|
||||
def extract_ova(ova_path: Path, workdir: Path) -> ExtractedOva:
|
||||
workdir.mkdir(parents=True, exist_ok=True)
|
||||
ovf_path: Path | None = None
|
||||
files: dict[str, Path] = {}
|
||||
|
||||
with tarfile.open(ova_path, mode="r|*") as tar:
|
||||
for member in tar:
|
||||
if not member.isfile():
|
||||
continue
|
||||
name = Path(member.name).name
|
||||
target = workdir / name
|
||||
if _is_unsafe(target, workdir):
|
||||
raise ValueError(f"Refusing unsafe tar entry: {member.name}")
|
||||
with tar.extractfile(member) as src, open(target, "wb") as dst:
|
||||
while chunk := src.read(1024 * 1024):
|
||||
dst.write(chunk)
|
||||
files[name] = target
|
||||
if target.suffix.lower() == ".ovf":
|
||||
ovf_path = target
|
||||
|
||||
if ovf_path is None:
|
||||
raise ValueError(f"No .ovf descriptor found inside {ova_path}")
|
||||
|
||||
return ExtractedOva(ovf_path=ovf_path, files=files, workdir=workdir)
|
||||
|
||||
|
||||
def _is_unsafe(target: Path, base: Path) -> bool:
|
||||
try:
|
||||
target.resolve().relative_to(base.resolve())
|
||||
return False
|
||||
except ValueError:
|
||||
return True
|
||||
|
|
@ -0,0 +1,169 @@
|
|||
"""Parse OVF 1.x / 2.x XML descriptors into a simple VM dataclass.
|
||||
|
||||
Covers the subset of the DMTF OVF spec that VirtualBox and VMware actually
|
||||
emit: System name, CPU/memory, disks (Disk + File refs), and network
|
||||
adapters. Unknown resources are ignored rather than rejected — real-world
|
||||
OVFs contain a lot of optional cruft.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass, field
|
||||
from pathlib import Path
|
||||
from typing import Optional
|
||||
|
||||
from lxml import etree
|
||||
|
||||
NS = {
|
||||
"ovf": "http://schemas.dmtf.org/ovf/envelope/1",
|
||||
"rasd": "http://schemas.dmtf.org/wbem/wscim/1/cim-schema/2/CIM_ResourceAllocationSettingData",
|
||||
"vssd": "http://schemas.dmtf.org/wbem/wscim/1/cim-schema/2/CIM_VirtualSystemSettingData",
|
||||
"xsi": "http://www.w3.org/2001/XMLSchema-instance",
|
||||
}
|
||||
|
||||
RESOURCE_TYPE_CPU = "3"
|
||||
RESOURCE_TYPE_MEMORY = "4"
|
||||
RESOURCE_TYPE_ETHERNET = "10"
|
||||
RESOURCE_TYPE_DISK = "17"
|
||||
|
||||
|
||||
@dataclass
|
||||
class OvfDisk:
|
||||
disk_id: str
|
||||
file_ref: str
|
||||
file_href: str
|
||||
capacity_bytes: int
|
||||
controller: str = "scsi"
|
||||
|
||||
|
||||
@dataclass
|
||||
class OvfNic:
|
||||
name: str
|
||||
network: str
|
||||
model: str = "virtio"
|
||||
|
||||
|
||||
@dataclass
|
||||
class OvfVm:
|
||||
name: str
|
||||
os_type: Optional[str] = None
|
||||
cpus: int = 1
|
||||
memory_mb: int = 512
|
||||
disks: list[OvfDisk] = field(default_factory=list)
|
||||
nics: list[OvfNic] = field(default_factory=list)
|
||||
|
||||
|
||||
def _text(elem, xpath: str) -> Optional[str]:
|
||||
result = elem.xpath(xpath, namespaces=NS)
|
||||
if not result:
|
||||
return None
|
||||
if isinstance(result[0], str):
|
||||
return result[0].strip() or None
|
||||
return (result[0].text or "").strip() or None
|
||||
|
||||
|
||||
def _parse_capacity(disk_elem) -> int:
|
||||
capacity = disk_elem.get(f"{{{NS['ovf']}}}capacity") or "0"
|
||||
units = disk_elem.get(f"{{{NS['ovf']}}}capacityAllocationUnits") or "byte"
|
||||
multiplier = 1
|
||||
u = units.lower().replace(" ", "")
|
||||
if "byte*2^10" in u or u == "kb" or u == "kib":
|
||||
multiplier = 1024
|
||||
elif "byte*2^20" in u or u == "mb" or u == "mib":
|
||||
multiplier = 1024**2
|
||||
elif "byte*2^30" in u or u == "gb" or u == "gib":
|
||||
multiplier = 1024**3
|
||||
elif "byte*2^40" in u or u == "tb" or u == "tib":
|
||||
multiplier = 1024**4
|
||||
return int(capacity) * multiplier
|
||||
|
||||
|
||||
def parse_ovf(ovf_path: Path) -> OvfVm:
|
||||
tree = etree.parse(str(ovf_path))
|
||||
root = tree.getroot()
|
||||
|
||||
files = {
|
||||
f.get(f"{{{NS['ovf']}}}id"): f.get(f"{{{NS['ovf']}}}href")
|
||||
for f in root.xpath(".//ovf:References/ovf:File", namespaces=NS)
|
||||
}
|
||||
|
||||
disks_by_id: dict[str, OvfDisk] = {}
|
||||
for d in root.xpath(".//ovf:DiskSection/ovf:Disk", namespaces=NS):
|
||||
disk_id = d.get(f"{{{NS['ovf']}}}diskId")
|
||||
file_ref = d.get(f"{{{NS['ovf']}}}fileRef")
|
||||
capacity = _parse_capacity(d)
|
||||
disks_by_id[disk_id] = OvfDisk(
|
||||
disk_id=disk_id,
|
||||
file_ref=file_ref,
|
||||
file_href=files.get(file_ref, ""),
|
||||
capacity_bytes=capacity,
|
||||
)
|
||||
|
||||
vsys = root.xpath(".//ovf:VirtualSystem", namespaces=NS)
|
||||
if not vsys:
|
||||
raise ValueError(f"No VirtualSystem in {ovf_path}")
|
||||
vsys = vsys[0]
|
||||
|
||||
name = vsys.get(f"{{{NS['ovf']}}}id") or "imported-vm"
|
||||
name = _text(vsys, "./ovf:Name/text()") or name
|
||||
|
||||
os_type = None
|
||||
os_section = vsys.xpath("./ovf:OperatingSystemSection", namespaces=NS)
|
||||
if os_section:
|
||||
os_type = _text(os_section[0], "./ovf:Description/text()")
|
||||
|
||||
vm = OvfVm(name=name, os_type=os_type)
|
||||
|
||||
controllers: dict[str, str] = {}
|
||||
items = vsys.xpath(
|
||||
"./ovf:VirtualHardwareSection/ovf:Item", namespaces=NS
|
||||
)
|
||||
for item in items:
|
||||
rtype = _text(item, "./rasd:ResourceType/text()")
|
||||
instance_id = _text(item, "./rasd:InstanceID/text()")
|
||||
subtype = (_text(item, "./rasd:ResourceSubType/text()") or "").lower()
|
||||
if rtype in ("5", "6", "20"):
|
||||
if "sata" in subtype:
|
||||
controllers[instance_id] = "sata"
|
||||
elif "ide" in subtype:
|
||||
controllers[instance_id] = "ide"
|
||||
else:
|
||||
controllers[instance_id] = "scsi"
|
||||
|
||||
for item in items:
|
||||
rtype = _text(item, "./rasd:ResourceType/text()")
|
||||
if rtype == RESOURCE_TYPE_CPU:
|
||||
vm.cpus = int(_text(item, "./rasd:VirtualQuantity/text()") or "1")
|
||||
elif rtype == RESOURCE_TYPE_MEMORY:
|
||||
vm.memory_mb = int(
|
||||
_text(item, "./rasd:VirtualQuantity/text()") or "512"
|
||||
)
|
||||
elif rtype == RESOURCE_TYPE_ETHERNET:
|
||||
nic_name = _text(item, "./rasd:ElementName/text()") or "net0"
|
||||
network = _text(item, "./rasd:Connection/text()") or "default"
|
||||
subtype = _text(item, "./rasd:ResourceSubType/text()") or ""
|
||||
model = _map_nic_model(subtype)
|
||||
vm.nics.append(OvfNic(name=nic_name, network=network, model=model))
|
||||
elif rtype == RESOURCE_TYPE_DISK:
|
||||
host_resource = _text(item, "./rasd:HostResource/text()") or ""
|
||||
disk_key = host_resource.rsplit("/", 1)[-1]
|
||||
if disk_key in disks_by_id:
|
||||
disk = disks_by_id[disk_key]
|
||||
parent = _text(item, "./rasd:Parent/text()")
|
||||
if parent and parent in controllers:
|
||||
disk.controller = controllers[parent]
|
||||
vm.disks.append(disk)
|
||||
|
||||
return vm
|
||||
|
||||
|
||||
def _map_nic_model(subtype: str) -> str:
|
||||
s = subtype.lower()
|
||||
if "virtio" in s:
|
||||
return "virtio"
|
||||
if "e1000" in s:
|
||||
return "e1000"
|
||||
if "vmxnet" in s:
|
||||
return "vmxnet3"
|
||||
if "pcnet" in s:
|
||||
return "rtl8139"
|
||||
return "virtio"
|
||||
|
|
@ -0,0 +1,274 @@
|
|||
<!doctype html>
|
||||
<html lang="en">
|
||||
<head>
|
||||
<meta charset="utf-8">
|
||||
<title>ova2vzdump</title>
|
||||
<style>
|
||||
:root { color-scheme: dark light; }
|
||||
body { font-family: system-ui, sans-serif; max-width: 760px;
|
||||
margin: 2rem auto; padding: 0 1rem; }
|
||||
h1 { margin-bottom: 0.25rem; }
|
||||
h2 { margin-top: 2rem; margin-bottom: 0.5rem; font-size: 1.1rem; }
|
||||
.sub { color: #888; margin-top: 0; margin-bottom: 1.5rem; }
|
||||
nav { display: flex; gap: 0.5rem; margin-bottom: 1rem;
|
||||
border-bottom: 1px solid #8884; }
|
||||
nav button { background: none; color: inherit; border: none;
|
||||
padding: 0.5rem 1rem; cursor: pointer; font: inherit;
|
||||
border-bottom: 2px solid transparent; }
|
||||
nav button.active { border-bottom-color: #2a6; font-weight: 600; }
|
||||
form { display: grid; gap: 0.75rem; padding: 1rem;
|
||||
border: 1px solid #8884; border-radius: 8px; }
|
||||
label { display: grid; gap: 0.25rem; font-size: 0.9rem; }
|
||||
label.row-inline { grid-auto-flow: column; justify-content: start;
|
||||
align-items: center; gap: 0.5rem; }
|
||||
input, button, select { font: inherit; padding: 0.5rem;
|
||||
border-radius: 6px; border: 1px solid #8884;
|
||||
background: transparent; color: inherit; }
|
||||
button[type="submit"] { cursor: pointer; background: #2a6; color: #fff;
|
||||
border: none; font-weight: 600; }
|
||||
button[type="submit"]:disabled { opacity: 0.5; cursor: not-allowed; }
|
||||
.row { display: grid; grid-template-columns: 1fr 1fr; gap: 0.75rem; }
|
||||
.panel { margin-top: 1.5rem; padding: 1rem; border: 1px solid #8884;
|
||||
border-radius: 8px; display: none; }
|
||||
.panel.visible { display: block; }
|
||||
progress { width: 100%; height: 1.5rem; }
|
||||
.stage { font-family: ui-monospace, monospace; color: #888; }
|
||||
.err { color: #e44; white-space: pre-wrap; }
|
||||
.ok { color: #2a6; }
|
||||
.reused { color: #e90; }
|
||||
a.dl { display: inline-block; margin-top: 0.75rem; padding: 0.5rem 1rem;
|
||||
background: #2a6; color: #fff; border-radius: 6px;
|
||||
text-decoration: none; font-weight: 600; }
|
||||
ul.ovalist { list-style: none; padding: 0; margin: 0.5rem 0 0; }
|
||||
ul.ovalist li { padding: 0.4rem 0; border-top: 1px solid #8882;
|
||||
display: flex; justify-content: space-between;
|
||||
gap: 0.5rem; }
|
||||
ul.ovalist code { color: #888; font-size: 0.85rem; }
|
||||
.hidden { display: none; }
|
||||
.hint { font-size: 0.85rem; color: #888; margin: -0.25rem 0 0; }
|
||||
.hint code { background: #8882; padding: 0.1rem 0.3rem;
|
||||
border-radius: 3px; }
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
<h1>ova2vzdump</h1>
|
||||
<p class="sub">OVA → Proxmox vzdump (.vma.zst) converter</p>
|
||||
|
||||
<nav>
|
||||
<button type="button" data-tab="convert" class="active">Convert</button>
|
||||
<button type="button" data-tab="create">Create test OVA</button>
|
||||
</nav>
|
||||
|
||||
<!-- ========== CONVERT TAB ========== -->
|
||||
<section id="tab-convert">
|
||||
<form id="convertForm">
|
||||
<label>OVA file
|
||||
<input type="file" name="ova" accept=".ova,.tar" required>
|
||||
</label>
|
||||
<div class="row">
|
||||
<label>VMID (placeholder)
|
||||
<input type="number" name="vmid" value="100" min="100"
|
||||
max="999999999" required>
|
||||
</label>
|
||||
<label>Storage (placeholder)
|
||||
<input type="text" name="storage" value="local-lvm" required>
|
||||
</label>
|
||||
</div>
|
||||
<p class="hint">Both fields are just placeholders in the generated
|
||||
config — the real values are picked at restore time via
|
||||
<code>qmrestore <file> <target-vmid> --storage <name></code>.
|
||||
Defaults are fine in most cases.</p>
|
||||
<button type="submit" id="goConvert">Convert</button>
|
||||
</form>
|
||||
|
||||
<section class="panel" id="convertStatus">
|
||||
<div><strong>Stage:</strong> <span class="stage" id="stage">—</span></div>
|
||||
<progress id="bar" value="0" max="100"></progress>
|
||||
<div id="msg"></div>
|
||||
<div id="result"></div>
|
||||
</section>
|
||||
</section>
|
||||
|
||||
<!-- ========== CREATE TEST OVA TAB ========== -->
|
||||
<section id="tab-create" class="hidden">
|
||||
<form id="createForm">
|
||||
<label>VM name
|
||||
<input type="text" name="name" value="test-vm" required>
|
||||
</label>
|
||||
<label class="row-inline">
|
||||
<input type="checkbox" name="bootable" id="bootable">
|
||||
<span>Bootable (real Alpine Linux, ~80 MB) — otherwise: empty stub disk</span>
|
||||
</label>
|
||||
<label>Disk size (MiB) — stub only
|
||||
<input type="number" name="size_mib" value="64" min="1" max="102400">
|
||||
</label>
|
||||
<label class="row-inline">
|
||||
<input type="checkbox" name="force" id="force">
|
||||
<span>Rebuild if a matching test OVA already exists</span>
|
||||
</label>
|
||||
<button type="submit" id="goCreate">Create</button>
|
||||
</form>
|
||||
|
||||
<section class="panel" id="createStatus">
|
||||
<div id="createMsg"></div>
|
||||
<div id="createResult"></div>
|
||||
</section>
|
||||
|
||||
<h2>Existing test OVAs</h2>
|
||||
<ul class="ovalist" id="ovalist"><li><em>loading…</em></li></ul>
|
||||
</section>
|
||||
|
||||
<script>
|
||||
// ---------- tab switching ----------
|
||||
const tabs = document.querySelectorAll('nav button');
|
||||
tabs.forEach(btn => btn.addEventListener('click', () => {
|
||||
tabs.forEach(b => b.classList.toggle('active', b === btn));
|
||||
document.getElementById('tab-convert').classList.toggle(
|
||||
'hidden', btn.dataset.tab !== 'convert');
|
||||
document.getElementById('tab-create').classList.toggle(
|
||||
'hidden', btn.dataset.tab !== 'create');
|
||||
if (btn.dataset.tab === 'create') refreshList();
|
||||
}));
|
||||
|
||||
// ---------- convert flow ----------
|
||||
const convertForm = document.getElementById('convertForm');
|
||||
const goConvert = document.getElementById('goConvert');
|
||||
const convertStatus = document.getElementById('convertStatus');
|
||||
const stageEl = document.getElementById('stage');
|
||||
const bar = document.getElementById('bar');
|
||||
const msg = document.getElementById('msg');
|
||||
const result = document.getElementById('result');
|
||||
|
||||
convertForm.addEventListener('submit', async (e) => {
|
||||
e.preventDefault();
|
||||
goConvert.disabled = true;
|
||||
convertStatus.classList.add('visible');
|
||||
stageEl.textContent = 'uploading';
|
||||
bar.value = 0;
|
||||
msg.textContent = '';
|
||||
result.innerHTML = '';
|
||||
|
||||
const fd = new FormData(convertForm);
|
||||
let resp;
|
||||
try { resp = await fetch('/api/jobs', { method: 'POST', body: fd }); }
|
||||
catch (err) {
|
||||
msg.innerHTML = '<span class="err">upload failed: ' + err + '</span>';
|
||||
goConvert.disabled = false;
|
||||
return;
|
||||
}
|
||||
if (!resp.ok) {
|
||||
const body = await resp.text();
|
||||
msg.innerHTML = '<span class="err">upload failed: ' + body + '</span>';
|
||||
goConvert.disabled = false;
|
||||
return;
|
||||
}
|
||||
const { id } = await resp.json();
|
||||
|
||||
const es = new EventSource('/api/jobs/' + id + '/events');
|
||||
es.onmessage = (ev) => {
|
||||
const d = JSON.parse(ev.data);
|
||||
if (d.stage) stageEl.textContent = d.stage;
|
||||
if (typeof d.progress === 'number') bar.value = d.progress;
|
||||
if (d.state === 'error') {
|
||||
msg.innerHTML = '<span class="err">error: ' +
|
||||
(d.message || 'unknown') + '</span>';
|
||||
es.close(); goConvert.disabled = false;
|
||||
}
|
||||
if (d.state === 'done') {
|
||||
msg.innerHTML = '<span class="ok">done — ' + (d.message || '') + '</span>';
|
||||
result.innerHTML = '<a class="dl" href="/download/' +
|
||||
encodeURIComponent(d.output) + '">Download ' + d.output + '</a>';
|
||||
es.close(); goConvert.disabled = false;
|
||||
}
|
||||
};
|
||||
es.onerror = () => { es.close(); goConvert.disabled = false; };
|
||||
});
|
||||
|
||||
// ---------- create test OVA flow ----------
|
||||
const createForm = document.getElementById('createForm');
|
||||
const goCreate = document.getElementById('goCreate');
|
||||
const createStatus = document.getElementById('createStatus');
|
||||
const createMsg = document.getElementById('createMsg');
|
||||
const createResult = document.getElementById('createResult');
|
||||
const ovalist = document.getElementById('ovalist');
|
||||
|
||||
function formatBytes(b) {
|
||||
if (b < 1024) return b + ' B';
|
||||
if (b < 1024**2) return (b / 1024).toFixed(1) + ' KiB';
|
||||
if (b < 1024**3) return (b / 1024**2).toFixed(1) + ' MiB';
|
||||
return (b / 1024**3).toFixed(2) + ' GiB';
|
||||
}
|
||||
|
||||
async function refreshList() {
|
||||
ovalist.innerHTML = '<li><em>loading…</em></li>';
|
||||
try {
|
||||
const r = await fetch('/api/test-ovas');
|
||||
const j = await r.json();
|
||||
if (!j.items || j.items.length === 0) {
|
||||
ovalist.innerHTML = '<li><em>no test OVAs yet</em></li>';
|
||||
return;
|
||||
}
|
||||
ovalist.innerHTML = j.items.map(it =>
|
||||
'<li><span><strong>' + it.name + '</strong> ' +
|
||||
'<code>' + formatBytes(it.bytes) + '</code></span>' +
|
||||
'<a class="dl" href="/api/test-ovas/' +
|
||||
encodeURIComponent(it.name) + '">Download</a></li>'
|
||||
).join('');
|
||||
} catch (e) {
|
||||
ovalist.innerHTML = '<li class="err">failed to list: ' + e + '</li>';
|
||||
}
|
||||
}
|
||||
|
||||
createForm.addEventListener('submit', async (e) => {
|
||||
e.preventDefault();
|
||||
goCreate.disabled = true;
|
||||
createStatus.classList.add('visible');
|
||||
createMsg.innerHTML = '<em>building…</em>';
|
||||
createResult.innerHTML = '';
|
||||
|
||||
const fd = new FormData(createForm);
|
||||
const payload = {
|
||||
name: fd.get('name'),
|
||||
bootable: fd.get('bootable') ? true : false,
|
||||
size_mib: fd.get('size_mib'),
|
||||
force: fd.get('force') ? true : false,
|
||||
};
|
||||
|
||||
let resp, body;
|
||||
try {
|
||||
resp = await fetch('/api/test-ovas', {
|
||||
method: 'POST',
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
body: JSON.stringify(payload),
|
||||
});
|
||||
body = await resp.json();
|
||||
} catch (err) {
|
||||
createMsg.innerHTML = '<span class="err">request failed: ' + err + '</span>';
|
||||
goCreate.disabled = false;
|
||||
return;
|
||||
}
|
||||
if (!resp.ok) {
|
||||
createMsg.innerHTML = '<span class="err">error: ' +
|
||||
(body.error || resp.statusText) + '</span>';
|
||||
goCreate.disabled = false;
|
||||
return;
|
||||
}
|
||||
const kind = body.bootable ? 'bootable Alpine' : 'empty stub';
|
||||
const status = body.reused
|
||||
? '<span class="reused">reused existing ' + kind + '</span>'
|
||||
: '<span class="ok">built ' + kind + '</span>';
|
||||
createMsg.innerHTML = status + ' — ' + body.name +
|
||||
' (' + formatBytes(body.bytes) + ')';
|
||||
createResult.innerHTML = '<a class="dl" href="/api/test-ovas/' +
|
||||
encodeURIComponent(body.name) + '">Download ' + body.name + '</a>';
|
||||
goCreate.disabled = false;
|
||||
refreshList();
|
||||
});
|
||||
|
||||
// initial load when Create tab is active
|
||||
if (!document.getElementById('tab-create').classList.contains('hidden')) {
|
||||
refreshList();
|
||||
}
|
||||
</script>
|
||||
</body>
|
||||
</html>
|
||||
|
|
@ -0,0 +1,231 @@
|
|||
"""Build a small test OVA, either a hollow stub or a real bootable Alpine.
|
||||
|
||||
Two modes:
|
||||
- stub (default): ~1 KB metadata + empty VMDK. Exercises parser +
|
||||
converter pipeline. Not bootable.
|
||||
- bootable: packages a pre-cached Alpine tiny raw image (downloaded at
|
||||
docker build time) as a stream-optimized VMDK inside the OVA. After
|
||||
restore on Proxmox, the VM boots to an Alpine login prompt.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
import os
|
||||
import shutil
|
||||
import subprocess
|
||||
import tarfile
|
||||
import tempfile
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
|
||||
ALPINE_BASE_ENV = "OVA2VZDUMP_ALPINE_BASE"
|
||||
|
||||
|
||||
OVF_TEMPLATE = """<?xml version="1.0" encoding="UTF-8"?>
|
||||
<Envelope xmlns="http://schemas.dmtf.org/ovf/envelope/1"
|
||||
xmlns:ovf="http://schemas.dmtf.org/ovf/envelope/1"
|
||||
xmlns:rasd="http://schemas.dmtf.org/wbem/wscim/1/cim-schema/2/CIM_ResourceAllocationSettingData"
|
||||
xmlns:vssd="http://schemas.dmtf.org/wbem/wscim/1/cim-schema/2/CIM_VirtualSystemSettingData"
|
||||
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
|
||||
xml:lang="en-US">
|
||||
<References>
|
||||
<File ovf:id="file1" ovf:href="{disk_name}" ovf:size="{disk_size}"/>
|
||||
</References>
|
||||
<DiskSection>
|
||||
<Info>Virtual disks</Info>
|
||||
<Disk ovf:diskId="disk1" ovf:fileRef="file1"
|
||||
ovf:capacity="{capacity_bytes}"
|
||||
ovf:capacityAllocationUnits="byte"
|
||||
ovf:format="http://www.vmware.com/interfaces/specifications/vmdk.html#streamOptimized"/>
|
||||
</DiskSection>
|
||||
<NetworkSection>
|
||||
<Info>Logical networks</Info>
|
||||
<Network ovf:name="VM Network">
|
||||
<Description>Default bridged network</Description>
|
||||
</Network>
|
||||
</NetworkSection>
|
||||
<VirtualSystem ovf:id="{vm_name}">
|
||||
<Info>A virtual machine</Info>
|
||||
<Name>{vm_name}</Name>
|
||||
<OperatingSystemSection ovf:id="96" ovf:version="1">
|
||||
<Info>Guest Operating System</Info>
|
||||
<Description>{os_description}</Description>
|
||||
</OperatingSystemSection>
|
||||
<VirtualHardwareSection>
|
||||
<Info>Virtual hardware requirements</Info>
|
||||
<System>
|
||||
<vssd:ElementName>Virtual Hardware Family</vssd:ElementName>
|
||||
<vssd:InstanceID>0</vssd:InstanceID>
|
||||
<vssd:VirtualSystemIdentifier>{vm_name}</vssd:VirtualSystemIdentifier>
|
||||
<vssd:VirtualSystemType>vmx-10</vssd:VirtualSystemType>
|
||||
</System>
|
||||
<Item>
|
||||
<rasd:AllocationUnits>hertz * 10^6</rasd:AllocationUnits>
|
||||
<rasd:ElementName>{cpus} virtual CPU(s)</rasd:ElementName>
|
||||
<rasd:InstanceID>1</rasd:InstanceID>
|
||||
<rasd:ResourceType>3</rasd:ResourceType>
|
||||
<rasd:VirtualQuantity>{cpus}</rasd:VirtualQuantity>
|
||||
</Item>
|
||||
<Item>
|
||||
<rasd:AllocationUnits>byte * 2^20</rasd:AllocationUnits>
|
||||
<rasd:ElementName>{memory_mb} MB of memory</rasd:ElementName>
|
||||
<rasd:InstanceID>2</rasd:InstanceID>
|
||||
<rasd:ResourceType>4</rasd:ResourceType>
|
||||
<rasd:VirtualQuantity>{memory_mb}</rasd:VirtualQuantity>
|
||||
</Item>
|
||||
<Item>
|
||||
<rasd:Address>0</rasd:Address>
|
||||
<rasd:ElementName>SCSIController</rasd:ElementName>
|
||||
<rasd:InstanceID>3</rasd:InstanceID>
|
||||
<rasd:ResourceSubType>VirtualSCSI</rasd:ResourceSubType>
|
||||
<rasd:ResourceType>6</rasd:ResourceType>
|
||||
</Item>
|
||||
<Item>
|
||||
<rasd:AddressOnParent>0</rasd:AddressOnParent>
|
||||
<rasd:ElementName>Hard Disk 1</rasd:ElementName>
|
||||
<rasd:HostResource>ovf:/disk/disk1</rasd:HostResource>
|
||||
<rasd:InstanceID>4</rasd:InstanceID>
|
||||
<rasd:Parent>3</rasd:Parent>
|
||||
<rasd:ResourceType>17</rasd:ResourceType>
|
||||
</Item>
|
||||
<Item>
|
||||
<rasd:AddressOnParent>7</rasd:AddressOnParent>
|
||||
<rasd:AutomaticAllocation>true</rasd:AutomaticAllocation>
|
||||
<rasd:Connection>VM Network</rasd:Connection>
|
||||
<rasd:ElementName>Network adapter 1</rasd:ElementName>
|
||||
<rasd:InstanceID>5</rasd:InstanceID>
|
||||
<rasd:ResourceSubType>E1000</rasd:ResourceSubType>
|
||||
<rasd:ResourceType>10</rasd:ResourceType>
|
||||
</Item>
|
||||
</VirtualHardwareSection>
|
||||
</VirtualSystem>
|
||||
</Envelope>
|
||||
"""
|
||||
|
||||
|
||||
@dataclass
|
||||
class TestOvaResult:
|
||||
path: Path
|
||||
bytes: int
|
||||
bootable: bool
|
||||
reused: bool
|
||||
|
||||
|
||||
def create_test_ova(
|
||||
output: Path,
|
||||
vm_name: str = "test-vm",
|
||||
size_mib: int = 64,
|
||||
bootable: bool = False,
|
||||
force: bool = False,
|
||||
alpine_base: Path | None = None,
|
||||
) -> TestOvaResult:
|
||||
"""Create (or reuse) a test OVA.
|
||||
|
||||
Returns reused=True when `output` already existed and force=False, so
|
||||
callers can surface that to the user instead of silently regenerating.
|
||||
"""
|
||||
output = Path(output)
|
||||
if output.exists() and not force:
|
||||
return TestOvaResult(
|
||||
path=output, bytes=output.stat().st_size,
|
||||
bootable=bootable, reused=True,
|
||||
)
|
||||
|
||||
if not _have("qemu-img"):
|
||||
raise RuntimeError(
|
||||
"qemu-img not found; run inside the ova2vzdump Docker image."
|
||||
)
|
||||
|
||||
output.parent.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
if bootable:
|
||||
base = alpine_base or Path(os.environ.get(
|
||||
ALPINE_BASE_ENV, "/app/fixtures/alpine-base.raw"
|
||||
))
|
||||
if not base.exists():
|
||||
raise RuntimeError(
|
||||
f"Bootable Alpine base image not found at {base}. "
|
||||
"Rebuild the Docker image or set "
|
||||
f"${ALPINE_BASE_ENV}."
|
||||
)
|
||||
_build_bootable(output, vm_name, base)
|
||||
else:
|
||||
_build_stub(output, vm_name, size_mib)
|
||||
|
||||
return TestOvaResult(
|
||||
path=output, bytes=output.stat().st_size,
|
||||
bootable=bootable, reused=False,
|
||||
)
|
||||
|
||||
|
||||
def _build_stub(output: Path, vm_name: str, size_mib: int) -> None:
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
tmpdir = Path(tmp)
|
||||
disk_name = f"{vm_name}-disk-0.vmdk"
|
||||
disk_path = tmpdir / disk_name
|
||||
capacity_bytes = size_mib * 1024 * 1024
|
||||
|
||||
subprocess.run(
|
||||
["qemu-img", "create", "-f", "vmdk",
|
||||
"-o", "subformat=streamOptimized",
|
||||
str(disk_path), f"{size_mib}M"],
|
||||
check=True, capture_output=True,
|
||||
)
|
||||
|
||||
_write_ovf_and_tar(
|
||||
tmpdir, output, vm_name, disk_name, disk_path,
|
||||
capacity_bytes=capacity_bytes,
|
||||
os_description="Debian GNU/Linux 12 (64-bit)",
|
||||
)
|
||||
|
||||
|
||||
def _build_bootable(output: Path, vm_name: str, base_raw: Path) -> None:
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
tmpdir = Path(tmp)
|
||||
disk_name = f"{vm_name}-disk-0.vmdk"
|
||||
disk_path = tmpdir / disk_name
|
||||
|
||||
subprocess.run(
|
||||
["qemu-img", "convert", "-O", "vmdk",
|
||||
"-o", "subformat=streamOptimized",
|
||||
str(base_raw), str(disk_path)],
|
||||
check=True, capture_output=True,
|
||||
)
|
||||
|
||||
# Capacity = virtual size of the raw image (not compressed VMDK size).
|
||||
info = subprocess.run(
|
||||
["qemu-img", "info", "--output=json", str(base_raw)],
|
||||
check=True, capture_output=True, text=True,
|
||||
)
|
||||
import json
|
||||
capacity_bytes = int(json.loads(info.stdout)["virtual-size"])
|
||||
|
||||
_write_ovf_and_tar(
|
||||
tmpdir, output, vm_name, disk_name, disk_path,
|
||||
capacity_bytes=capacity_bytes,
|
||||
os_description="Alpine Linux 3.22 (64-bit)",
|
||||
)
|
||||
|
||||
|
||||
def _write_ovf_and_tar(tmpdir: Path, output: Path, vm_name: str,
|
||||
disk_name: str, disk_path: Path,
|
||||
capacity_bytes: int, os_description: str,
|
||||
cpus: int = 2, memory_mb: int = 1024) -> None:
|
||||
disk_size = disk_path.stat().st_size
|
||||
ovf_path = tmpdir / f"{vm_name}.ovf"
|
||||
ovf_path.write_text(OVF_TEMPLATE.format(
|
||||
vm_name=vm_name, disk_name=disk_name, disk_size=disk_size,
|
||||
capacity_bytes=capacity_bytes, os_description=os_description,
|
||||
cpus=cpus, memory_mb=memory_mb,
|
||||
))
|
||||
# USTAR, not PAX. VirtualBox's OVA importer (VERR_TAR_UNSUPPORTED_PAX_TYPE)
|
||||
# and many other tools reject Python's default PAX extended headers.
|
||||
with tarfile.open(output, "w", format=tarfile.USTAR_FORMAT) as tar:
|
||||
tar.add(ovf_path, arcname=ovf_path.name)
|
||||
tar.add(disk_path, arcname=disk_name)
|
||||
|
||||
|
||||
def _have(tool: str) -> bool:
|
||||
return shutil.which(tool) is not None
|
||||
|
|
@ -0,0 +1,195 @@
|
|||
"""Flask web UI for ova2vzdump.
|
||||
|
||||
Single-page app: upload an OVA, watch the conversion progress via Server-
|
||||
Sent Events, download the resulting .vma.zst. Jobs run in background
|
||||
threads; state is kept in memory (process-local).
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import logging
|
||||
import threading
|
||||
import time
|
||||
import uuid
|
||||
from dataclasses import dataclass, field
|
||||
from pathlib import Path
|
||||
from queue import Queue, Empty
|
||||
from typing import Optional
|
||||
|
||||
from flask import (
|
||||
Flask, Response, jsonify, render_template, request, send_from_directory,
|
||||
stream_with_context,
|
||||
)
|
||||
from werkzeug.utils import secure_filename
|
||||
|
||||
from .converter import convert
|
||||
from .test_ova import create_test_ova
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
|
||||
|
||||
@dataclass
|
||||
class Job:
|
||||
id: str
|
||||
ova_name: str
|
||||
vmid: int
|
||||
storage: str
|
||||
state: str = "pending"
|
||||
stage: str = "init"
|
||||
progress: float = 0.0
|
||||
message: str = ""
|
||||
output_name: Optional[str] = None
|
||||
events: Queue = field(default_factory=Queue)
|
||||
|
||||
def push(self, payload: dict) -> None:
|
||||
self.events.put(payload)
|
||||
|
||||
|
||||
_JOBS: dict[str, Job] = {}
|
||||
_JOBS_LOCK = threading.Lock()
|
||||
|
||||
|
||||
def _run_job(job: Job, ova_path: Path, output_dir: Path) -> None:
|
||||
def progress(stage: str, pct: float) -> None:
|
||||
job.stage = stage
|
||||
job.progress = pct
|
||||
job.push({"stage": stage, "progress": pct, "state": "running"})
|
||||
|
||||
job.state = "running"
|
||||
job.push({"state": "running", "stage": "starting", "progress": 0.0})
|
||||
try:
|
||||
result = convert(
|
||||
ova_path=ova_path, output_dir=output_dir,
|
||||
vmid=job.vmid, storage=job.storage,
|
||||
progress=progress,
|
||||
)
|
||||
job.state = "done"
|
||||
job.output_name = result.output_path.name
|
||||
job.message = (
|
||||
f"{result.vm_name}: {result.disk_count} disk(s), "
|
||||
f"{result.total_bytes / 1024**3:.2f} GiB"
|
||||
)
|
||||
job.push({
|
||||
"state": "done", "stage": "finished", "progress": 100.0,
|
||||
"output": result.output_path.name, "message": job.message,
|
||||
})
|
||||
except Exception as exc:
|
||||
log.exception("job %s failed", job.id)
|
||||
job.state = "error"
|
||||
job.message = str(exc)
|
||||
job.push({"state": "error", "message": str(exc)})
|
||||
finally:
|
||||
try:
|
||||
ova_path.unlink(missing_ok=True)
|
||||
except OSError:
|
||||
pass
|
||||
|
||||
|
||||
def create_app(upload_dir: Path, output_dir: Path) -> Flask:
|
||||
app = Flask(__name__)
|
||||
app.config["MAX_CONTENT_LENGTH"] = 500 * 1024**3
|
||||
|
||||
@app.get("/")
|
||||
def index() -> str:
|
||||
return render_template("index.html")
|
||||
|
||||
@app.post("/api/jobs")
|
||||
def create_job():
|
||||
if "ova" not in request.files:
|
||||
return jsonify(error="no file 'ova' in multipart form"), 400
|
||||
f = request.files["ova"]
|
||||
if not f.filename:
|
||||
return jsonify(error="empty filename"), 400
|
||||
vmid = int(request.form.get("vmid", "100"))
|
||||
storage = request.form.get("storage", "local-lvm")
|
||||
filename = secure_filename(f.filename) or "input.ova"
|
||||
job_id = uuid.uuid4().hex
|
||||
ova_path = upload_dir / f"{job_id}-{filename}"
|
||||
f.save(ova_path)
|
||||
|
||||
job = Job(id=job_id, ova_name=filename, vmid=vmid, storage=storage)
|
||||
with _JOBS_LOCK:
|
||||
_JOBS[job_id] = job
|
||||
t = threading.Thread(
|
||||
target=_run_job, args=(job, ova_path, output_dir), daemon=True,
|
||||
)
|
||||
t.start()
|
||||
return jsonify(id=job_id)
|
||||
|
||||
@app.get("/api/jobs/<job_id>")
|
||||
def job_status(job_id: str):
|
||||
job = _JOBS.get(job_id)
|
||||
if not job:
|
||||
return jsonify(error="not found"), 404
|
||||
return jsonify(
|
||||
id=job.id, state=job.state, stage=job.stage,
|
||||
progress=job.progress, message=job.message,
|
||||
output=job.output_name,
|
||||
)
|
||||
|
||||
@app.get("/api/jobs/<job_id>/events")
|
||||
def job_events(job_id: str):
|
||||
job = _JOBS.get(job_id)
|
||||
if not job:
|
||||
return jsonify(error="not found"), 404
|
||||
|
||||
@stream_with_context
|
||||
def stream():
|
||||
yield f"data: {json.dumps({'state': job.state, 'stage': job.stage, 'progress': job.progress})}\n\n"
|
||||
while True:
|
||||
try:
|
||||
evt = job.events.get(timeout=15)
|
||||
except Empty:
|
||||
yield ": keepalive\n\n"
|
||||
continue
|
||||
yield f"data: {json.dumps(evt)}\n\n"
|
||||
if evt.get("state") in ("done", "error"):
|
||||
return
|
||||
|
||||
return Response(stream(), mimetype="text/event-stream")
|
||||
|
||||
@app.get("/download/<path:name>")
|
||||
def download(name: str):
|
||||
return send_from_directory(output_dir, name, as_attachment=True)
|
||||
|
||||
@app.get("/api/test-ovas")
|
||||
def list_test_ovas():
|
||||
items = []
|
||||
for p in sorted(upload_dir.glob("test-*.ova")):
|
||||
items.append({"name": p.name, "bytes": p.stat().st_size})
|
||||
return jsonify(items=items)
|
||||
|
||||
@app.post("/api/test-ovas")
|
||||
def make_test_ova():
|
||||
data = request.get_json(silent=True) or request.form
|
||||
name = secure_filename(data.get("name", "test-vm")) or "test-vm"
|
||||
bootable = str(data.get("bootable", "")).lower() in ("1", "true", "on")
|
||||
try:
|
||||
size_mib = int(data.get("size_mib", 64))
|
||||
except (TypeError, ValueError):
|
||||
return jsonify(error="size_mib must be an integer"), 400
|
||||
force = str(data.get("force", "")).lower() in ("1", "true", "on")
|
||||
|
||||
suffix = "bootable-alpine" if bootable else f"stub-{size_mib}mib"
|
||||
filename = f"test-{name}-{suffix}.ova"
|
||||
out_path = upload_dir / filename
|
||||
|
||||
try:
|
||||
r = create_test_ova(
|
||||
output=out_path, vm_name=name, size_mib=size_mib,
|
||||
bootable=bootable, force=force,
|
||||
)
|
||||
except Exception as exc:
|
||||
log.exception("test OVA creation failed")
|
||||
return jsonify(error=str(exc)), 500
|
||||
|
||||
return jsonify(
|
||||
name=filename, bytes=r.bytes, bootable=r.bootable,
|
||||
reused=r.reused,
|
||||
)
|
||||
|
||||
@app.get("/api/test-ovas/<path:name>")
|
||||
def download_test_ova(name: str):
|
||||
return send_from_directory(upload_dir, name, as_attachment=True)
|
||||
|
||||
return app
|
||||
Loading…
Reference in New Issue