For senders whose mail body is the receipt (e.g. GoCardless payouts)
there is no attachment to file. The new action "Als PDF anhängen +
nach <Ordner>" renders the mail like a print from a mail client
(header block From/To/Date/Subject, then the content) and replaces it
in the target folder with: original content + PDF + the original mail
byte-for-byte as Original.eml, so the unaltered original is retained
(GoBD).
- Rendering with WeasyPrint. Images are fetched by us and inlined as
data: URIs; WeasyPrint itself may only read data:. Fetching is
restricted to http(s), public addresses (private, loopback,
link-local and PDF_BLOCKED_NETWORKS, default 172.0.0.0/12, are
refused; redirects are checked hop by hop), with time and size
limits. cid: images come from the MIME parts.
- The replacement is APPENDed with the original flags and INTERNALDATE,
verified in the target via its new Message-ID, and only then is the
original deleted. Already converted mails are only moved.
- "PDF-Vorschau" in the rule dialog renders the newest matching mail
read-only and opens it in the browser.
- List/log show 📄 targets; warnings for PDF + move/delete and for
PDF + "Hat Anhang = Ja"; log folder search covers PDF targets.
- Docker image gets Pango and fonts (~316 MB total).
Tested with the real GoCardless mail (3 remote images, rendered both
locally and inside the built image) and a simulated IMAP server for
success, rejected APPEND, missing copy and already converted mails.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
"Fehlgeschlagene neu einlesen" reported the same 29 mails on every
click: it selected all error log entries ever written and never noted
that one had already been retried.
Error log entries now get retried_at when they are picked up; the
endpoint only considers entries without it. If the mail fails again,
the scheduler writes a new, open error entry, so a second click right
after the first reports 0. The log shows retried errors greyed out
with "↻ erneut versucht <Zeit>", and the result dialog says how many
open error entries and mails were requeued.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The existing daily cleanup never effectively ran: it was an interval
job counted from container start (reset on every deploy), it called
asyncio.get_event_loop() from a worker thread, and even when deleting,
SQLite never returns freed pages to the filesystem. Live DB: 4.1 GB,
of which 2.6 GB log and 0.6 GB unused pages.
New maintenance service, scheduled daily at MAINTENANCE_HOUR_UTC:30
and once 5 minutes after every start:
- delete log entries older than LOG_RETENTION_DAYS (30) and keep at
most LOG_MAX_ENTRIES (200k) newest ones
- delete processed_mails markers of deleted rules/accounts (FKs are
not enforced by SQLite)
- VACUUM when >25% of the file is unused, only if free disk space
covers the copy; never runs twice in parallel
GET/POST /api/logs/maintenance expose status and a manual run; the log
menu shows DB size, entry count and last run, with "Datenbank
aufräumen & verkleinern". Removed the dead cleanup_old_logs.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The log had 6.45M rows, of which 434 were actual actions: every mail x
every non-matching rule wrote "Keine Übereinstimmung" (5.3M rows, ~1 GB),
plus "keine neuen Mails" per rule every 30s and a "Poll gestartet" row.
Free-text searches took 50-70s.
- Routine entries are only written with LOG_VERBOSE=true; the poll
summary is logged only when there were new mails, actions or errors.
"Why didn't my rule match" is now answered by the rule tester.
- The scheduler commits every 25 evaluated mails and after each rule,
since the dropped log calls used to be the implicit commit points.
- Migration rebuilds filter_logs without the routine rows (live:
6.45M -> ~38k) via copy instead of DELETE.
- /api/logs: keyset pagination (before_id) instead of OFFSET, count
capped at 1000, since/until range, multi-level filter, per-field
filters; ordner: also matches the move target stored in details.
- New log page: one search field with shortcuts (von:, betreff:,
regel:, ordner:, text:), live search with debounce and abort, active
filters as removable chips, click on rule/sender/folder in a row to
filter, time range (default 7 days), level quick filter, expandable
row details, "Ältere laden", filters kept in the URL.
- Disable Pico's colour transition on segmented buttons (they briefly
flashed in the primary colour on load).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- fix: "Log leeren" without account filter used a plain DELETE over
millions of rows. That held the sqlite write lock long enough that
the scheduler's own log writes started failing with
"database is locked", and the request itself eventually 500'd back
as HTML "Internal Server Error" — which the frontend then tried to
JSON.parse, producing the popup the user reported. Now uses
DROP+CREATE for full clears (near-instant) and wal_checkpoint to
reclaim space; per-account clear still uses a normal DELETE.
- Raise sqlite busy_timeout from 10s to 60s so a legitimately slow
writer no longer starves smaller ones.
- retry-failed: single bulk DELETE with mail_uid IN (...) per account
instead of one query per uid.
- Both endpoints now return proper HTTPException(500, ...) with detail
instead of letting the exception bubble as plain text.
- Frontend: parseResponse() reads text first, then tries JSON. Both
clearLogs() and retryFailed() surface the actual server message
instead of crashing on non-JSON responses.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
duffyduckandClaude Opus 5.5