Compare commits

..

4 Commits

Author SHA1 Message Date
duffyduck 921656090b remove db 2026-03-06 08:26:03 +01:00
duffyduck a43bfa80d6 remove data 2026-03-06 08:25:30 +01:00
duffyduck e221df2013 dockerignore 2026-03-06 08:22:49 +01:00
Stefan Hacker 05f814c5b7 first commit 2026-03-06 08:20:07 +01:00
21 changed files with 300 additions and 5105 deletions
-4
View File
@@ -1,4 +0,0 @@
data/
__pycache__/
*.pyc
.env
+2 -26
View File
@@ -1,39 +1,15 @@
FROM python:3.12-slim
RUN apt-get update && apt-get install -y --no-install-recommends \
libzbar0 \
# Chromium dependencies for Playwright
libglib2.0-0t64 \
libnss3 \
libnspr4 \
libatk1.0-0t64 \
libatk-bridge2.0-0t64 \
libcups2t64 \
libdrm2 \
libexpat1 \
libxcomposite1 \
libxdamage1 \
libxext6 \
libxfixes3 \
libxrandr2 \
libgbm1 \
libxkbcommon0 \
libpango-1.0-0 \
libcairo2 \
libasound2t64 \
libatspi2.0-0t64 \
fonts-liberation \
&& rm -rf /var/lib/apt/lists/*
RUN apt-get update && apt-get install -y --no-install-recommends libzbar0 && rm -rf /var/lib/apt/lists/*
WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
RUN playwright install chromium
COPY app/ ./app/
RUN mkdir -p /data/uploads /data/amazon_session
RUN mkdir -p /data/uploads
EXPOSE 8000
-196
View File
@@ -1,196 +0,0 @@
# Belegimport
Automatischer Import von Belegen (Rechnungen, Gutschriften) aus verschiedenen Quellen und Weiterleitung per SMTP an Buchhaltungssoftware (z.B. Buchhaltungsbutler).
## Features
- **Scan-Upload**: PDF hochladen, automatische Trennung per QR-Code-Trennseiten
- **IMAP**: Automatischer Abruf von Belegen aus Email-Postfachern
- **SMB/Netzlaufwerk**: Automatischer Abruf von Belegen aus Netzwerkordnern
- **FTP / SFTP**: Automatischer Abruf von Belegen via FTP (passiv, unverschluesselt) oder SFTP (SSH)
- **Amazon Business**: Automatischer Abruf von Amazon-Rechnungen per API
- **Eingangs-/Ausgangsbelege**: Getrennte Import-Adressen fur Einkauf und Verkauf
- **Scheduler**: Automatischer Abruf in konfigurierbaren Intervallen
- **Verarbeitungslog**: Ubersicht aller importierten Belege mit Status
## Technologie
- Python 3.12, FastAPI, Jinja2, SQLite (aiosqlite)
- Docker / docker-compose
- Playwright (optional, fur Amazon Browser-Automation als Fallback)
## Installation
### Docker (empfohlen)
```bash
git clone <repository-url>
cd lex-office-belegimport-mail
sudo docker-compose up --build -d
```
Die Webanwendung ist erreichbar unter: `http://localhost:8081`
### Konfiguration
Alle Einstellungen werden uber die Weboberflache vorgenommen:
1. **Einstellungen** (`/settings`): SMTP-Server, IMAP, SMB, Import-Emailadressen
2. **Plattformen** (`/platforms`): Amazon Business API-Zugangsdaten
3. **Scan-Upload** (`/`): Manueller PDF-Upload mit Belegart-Auswahl
---
## Amazon Business API Einrichtung
Die Amazon-Integration nutzt die offizielle Amazon Business API (Reconciliation + Document API) um Rechnungen automatisch abzurufen. Kein Browser-Login, keine CAPTCHAs, vollautomatisch.
### Voraussetzungen
- Amazon Business Konto (mit Business Prime)
- Zugang zum [Amazon Solution Provider Portal](https://solutionproviderportal.amazon.com/)
### Schritt 1: Als Entwickler registrieren
1. Offne das [Solution Provider Portal](https://solutionproviderportal.amazon.com/)
2. Wahle **"Private seller applications"** (fur eigene Integrationen)
3. Wahle **"Erstellen Sie Anwendungen, die SP-APIs verwenden"**
4. Fulle die Unternehmensdaten aus (Name, Handelsregisternummer, Adresse)
5. Verifiziere per SMS
### Schritt 2: Rollen auswahlen
Wahle folgende Rollen:
- **Abgleichen von Business-Einkaufen** (Reconciliation API)
- **Amazon Business-Bestellung** (Business Orders API)
### Schritt 3: Sicherheitskontrollen
Beantworte alle Sicherheitsfragen mit **"Ja"**.
Bei den Textfeldern:
- **Externe Parteien**: `Keine. Die Daten werden ausschliesslich intern fur die eigene Buchhaltung verwendet.`
- **Externe Quellen**: `Keine.`
### Schritt 4: App registrieren
Nach der Freischaltung (kann einige Tage dauern):
1. Im Developer Central: **"+ Neuen App-Client hinzufugen"**
2. Einstellungen:
- **App-Name**: `Beleg import` (oder eigener Name)
- **API-Typ**: `SP-API`
- **App-Typ**: `Produktion`
- **Amazon Business**: angehakt
- **Verkaufer**: nicht angehakt
- **Rollen**: Abgleichen von Business-Einkaufen + Amazon Business-Bestellung
- **RDT**: Nein
- **OAuth-Anmeldungs-URI**: `https://ihre-domain.de/api/amazon-oauth-callback`
- **OAuth-Umleitungs-URI**: `https://ihre-domain.de/api/amazon-oauth-callback`
3. Nach dem Speichern: **"Anmeldedaten fur Login mit Amazon" -> "Anzeigen"**
- Notiere **Client-ID** (`amzn1.application-oa2-client.xxxxx`)
- Notiere **Client-Sicherheitsschluessel** (`amzn1.oa2-cs.v1.xxxxx`)
4. Die **App-ID** (`amzn1.sp.solution.xxxxx`) steht in der App-Ubersicht unter dem App-Namen
> **Hinweis**: Die OAuth-Umleitungs-URI muss eine echte Domain mit Top-Level-Domain sein.
> `localhost` und `.local` Domains werden von Amazon nicht akzeptiert.
> Die URI muss nicht offentlich erreichbar sein - Amazon leitet nur den Browser des Benutzers dorthin weiter.
> **Fehler SPSA0404**: Falls beim Autorisieren der Fehler "Keine unterstuetzte Geschaeftseinheit" erscheint,
> muss die Autorisierung uber den OAuth-Flow (Website) statt uber Self-Authorization erfolgen.
> Der Belegimport unterstutzt dies automatisch.
### Schritt 5: Im Belegimport konfigurieren
1. Offne die Plattformen-Seite im Belegimport
2. Setze **Abruf-Modus** auf **"API (empfohlen)"**
3. Trage ein:
- **App-ID**: `amzn1.sp.solution.xxxxx`
- **Client-ID**: `amzn1.application-oa2-client.xxxxx`
- **Client-Sicherheitsschluessel**: Der Secret-Wert
4. **Einstellungen speichern**
5. Klicke **"Bei Amazon autorisieren"**
6. Melde dich bei Amazon an und erlaube den Zugriff
7. Kopiere den `spapi_oauth_code` (oder die ganze URL) aus der Browser-Adressleiste
8. Trage den Code im Belegimport ein und klicke **"Token tauschen"**
9. Status sollte auf **"API autorisiert"** wechseln
### Schritt 6: Rechnungen abrufen
- **Manuell**: Klicke "Jetzt Rechnungen abrufen"
- **Automatisch**: Aktiviere den Scheduler unter Einstellungen (z.B. alle 60 Minuten)
Die Rechnungen werden als PDF per SMTP an die konfigurierte Eingangsbeleg-Adresse gesendet.
Bereits abgerufene Rechnungen werden automatisch ubersprungen.
### OAuth Redirect URI (lokale Installation)
Da Amazon keine `localhost`-URIs akzeptiert, gibt es zwei Optionen:
**Option A: Eigene Domain verwenden (empfohlen)**
Trage eine echte Domain ein (z.B. `https://ihre-domain.de/api/amazon-oauth-callback`).
Nach der Amazon-Autorisierung leitet der Browser dorthin weiter - die Seite ladt nicht,
aber der Auth-Code steht in der URL-Leiste. Diesen Code im Belegimport eintragen.
**Option B: /etc/hosts Eintrag**
Falls der Server lokal erreichbar sein soll:
```bash
# In /etc/hosts eintragen:
127.0.0.1 app.belegimport.de
```
Dann in der Amazon App als Redirect URI eintragen:
`https://app.belegimport.de/api/amazon-oauth-callback`
> **Achtung**: Amazon pruft ob die Domain eine Top-Level-Domain hat.
> `.local` funktioniert nicht, aber `.de` schon.
### Umgebungsvariablen
In `docker-compose.yml`:
```yaml
environment:
- OAUTH_REDIRECT_BASE=https://ihre-domain.de # Muss zur Amazon App passen
```
---
## Eingangs- und Ausgangsbelege
Der Belegimport unterscheidet zwischen:
- **Eingangsbelege (Einkauf)**: Rechnungen die Sie von Lieferanten erhalten
- **Ausgangsbelege (Verkauf/Gutschrift)**: Rechnungen die Sie an Kunden senden
Fur beide Typen konnen separate Import-Emailadressen konfiguriert werden (z.B. fur Buchhaltungsbutler).
Amazon-Rechnungen werden automatisch als Eingangsbelege klassifiziert.
Bei IMAP und SMB konnen jeweils getrennte Quell- und Verarbeitungsordner fur Eingangs- und Ausgangsbelege konfiguriert werden.
Beim Scan-Upload kann die Belegart per Radio-Button ausgewahlt werden.
---
## Verarbeitungslog
Unter `/log` werden alle verarbeiteten Belege angezeigt mit:
- Zeitpunkt, Betreff, Absender
- Belegart (Eingang/Ausgang)
- Anzahl Anhange
- Status (OK/Fehler)
- Fehlermeldung (falls vorhanden)
- SMTP-Protokoll (anzeigbar)
---
## Lizenz
Privates Projekt.
-515
View File
@@ -1,515 +0,0 @@
"""Amazon Business API client using SP-API (Reconciliation + Document API).
This module provides API-based invoice retrieval as an alternative to browser automation.
Uses OAuth2 with LWA (Login with Amazon) for authentication.
Document API workflow (EU):
1. POST /reports/.../reports → reportId
2. GET /reports/.../reports/{reportId} → poll until DONE → reportDocumentId
3. GET /reports/.../documents/{reportDocumentId} → presigned URL
4. Download + decompress (gzip then zip) → PDF
"""
import asyncio
import gzip
import io
import logging
import urllib.parse
import zipfile
from datetime import datetime, timedelta
from pathlib import Path
import httpx
from app.database import get_settings, save_settings, add_log_entry, is_invoice_downloaded, mark_invoice_downloaded
from app.mail_processor import _connect_smtp, _build_forward_email, _send_with_log
logger = logging.getLogger(__name__)
# Amazon LWA (Login with Amazon) endpoints
LWA_TOKEN_URL = "https://api.amazon.com/auth/o2/token"
# Amazon Business OAuth consent URLs per domain (NOT sellercentral!)
AB_OAUTH_URLS = {
"amazon.de": "https://www.amazon.de/b2b/abws/oauth",
"amazon.at": "https://www.amazon.de/b2b/abws/oauth", # AT uses DE
"amazon.fr": "https://www.amazon.fr/b2b/abws/oauth",
"amazon.it": "https://www.amazon.it/b2b/abws/oauth",
"amazon.es": "https://www.amazon.es/b2b/abws/oauth",
"amazon.co.uk": "https://www.amazon.co.uk/b2b/abws/oauth",
"amazon.com": "https://www.amazon.com/b2b/abws/oauth",
}
# Amazon Business API endpoints per region
AB_API_ENDPOINTS = {
"eu": "https://eu.business-api.amazon.com",
"na": "https://na.business-api.amazon.com",
}
# API versions
RECONCILIATION_VERSION = "2021-01-08"
REPORTS_VERSION = "2021-09-30"
# Domain to region mapping
DOMAIN_REGION = {
"amazon.de": "eu",
"amazon.at": "eu",
"amazon.fr": "eu",
"amazon.it": "eu",
"amazon.es": "eu",
"amazon.co.uk": "eu",
"amazon.com": "na",
}
# Domain to marketplace ID
DOMAIN_MARKETPLACE = {
"amazon.de": "A1PA6795UKMFR9",
"amazon.at": "A2NODRKZP88ZB9",
"amazon.fr": "A13V1IB3VIYZZH",
"amazon.it": "APJ6JRA9NG5V4",
"amazon.es": "A1RKKUPIHCS9HS",
"amazon.co.uk": "A1F83G8C2ARO7P",
"amazon.com": "ATVPDKIKX0DER",
}
def get_oauth_authorize_url(application_id: str, redirect_uri: str, domain: str = "amazon.de", state: str = "") -> str:
"""Generate the OAuth authorization URL for Amazon Business API consent."""
base_url = AB_OAUTH_URLS.get(domain, AB_OAUTH_URLS["amazon.de"])
params = {
"applicationId": application_id,
"state": state or "auth",
"redirect_uri": redirect_uri,
}
return f"{base_url}?{urllib.parse.urlencode(params)}"
async def exchange_auth_code(code: str, client_id: str, client_secret: str, redirect_uri: str) -> dict:
"""Exchange authorization code for refresh token via LWA."""
async with httpx.AsyncClient() as client:
resp = await client.post(LWA_TOKEN_URL, data={
"grant_type": "authorization_code",
"code": code,
"client_id": client_id,
"client_secret": client_secret,
"redirect_uri": redirect_uri,
})
if resp.status_code != 200:
logger.error(f"LWA Token-Exchange fehlgeschlagen: {resp.status_code} {resp.text}")
return {"error": f"Token-Exchange fehlgeschlagen: {resp.status_code} - {resp.text}"}
data = resp.json()
logger.info("LWA Token-Exchange erfolgreich")
return data
async def get_access_token(client_id: str, client_secret: str, refresh_token: str) -> str | None:
"""Get a fresh access token using the refresh token."""
async with httpx.AsyncClient() as client:
resp = await client.post(LWA_TOKEN_URL, data={
"grant_type": "refresh_token",
"refresh_token": refresh_token,
"client_id": client_id,
"client_secret": client_secret,
})
if resp.status_code != 200:
logger.error(f"Access-Token-Refresh fehlgeschlagen: {resp.status_code} {resp.text}")
return None
data = resp.json()
return data.get("access_token")
async def check_api_configured() -> dict:
"""Check if API credentials are configured and valid."""
settings = await get_settings()
client_id = settings.get("amazon_client_id", "")
client_secret = settings.get("amazon_client_secret", "")
refresh_token = settings.get("amazon_refresh_token", "")
if not client_id or not client_secret:
return {"configured": False, "authorized": False, "error": "Client-ID oder Client-Secret fehlt"}
if not refresh_token:
return {"configured": True, "authorized": False, "error": "Noch nicht autorisiert (Refresh-Token fehlt)"}
# Try to get an access token to verify credentials
access_token = await get_access_token(client_id, client_secret, refresh_token)
if not access_token:
return {"configured": True, "authorized": False, "error": "Autorisierung abgelaufen - bitte erneut autorisieren"}
return {"configured": True, "authorized": True}
async def _get_api_client(settings: dict) -> tuple[httpx.AsyncClient, str] | None:
"""Create an authenticated API client. Returns (client, region) or None."""
client_id = settings.get("amazon_client_id", "")
client_secret = settings.get("amazon_client_secret", "")
refresh_token = settings.get("amazon_refresh_token", "")
if not all([client_id, client_secret, refresh_token]):
return None
access_token = await get_access_token(client_id, client_secret, refresh_token)
if not access_token:
return None
domain = settings.get("amazon_domain", "amazon.de")
region = DOMAIN_REGION.get(domain, "eu")
client = httpx.AsyncClient(
base_url=AB_API_ENDPOINTS.get(region, AB_API_ENDPOINTS["eu"]),
headers={
"x-amz-access-token": access_token,
"Content-Type": "application/json",
"user-agent": "Belegimport/1.0 (Language=Python/3.12)",
},
timeout=30.0,
)
return client, region
async def get_transactions(settings: dict, since_date: datetime) -> list[dict]:
"""Get transactions via Reconciliation API."""
result = await _get_api_client(settings)
if not result:
return []
client, region = result
transactions = []
try:
# feedEndDate must not exceed current UTC time
now_utc = datetime.utcnow()
params = {
"feedStartDate": since_date.strftime("%Y-%m-%dT00:00:00Z"),
"feedEndDate": now_utc.strftime("%Y-%m-%dT%H:%M:%SZ"),
}
next_token = None
page = 0
while True:
page += 1
if next_token:
params["nextPageToken"] = next_token
logger.info(f"Amazon API: Reconciliation-Abfrage Seite {page}...")
resp = await client.get(
f"/reconciliation/{RECONCILIATION_VERSION}/transactions",
params=params,
)
if resp.status_code != 200:
logger.error(f"Amazon API: Reconciliation fehlgeschlagen: {resp.status_code} {resp.text}")
break
data = resp.json()
page_transactions = data.get("transactions", [])
transactions.extend(page_transactions)
logger.info(f"Amazon API: Seite {page}: {len(page_transactions)} Transaktionen")
next_token = data.get("nextPageToken")
if not next_token:
break
except Exception as e:
logger.error(f"Amazon API: Reconciliation-Fehler: {e}")
finally:
await client.aclose()
logger.info(f"Amazon API: {len(transactions)} Transaktionen gesamt")
return transactions
async def _create_invoice_report(client: httpx.AsyncClient, order_id: str, marketplace_id: str) -> str | None:
"""Step 1: Create a report request for invoice PDF."""
body = {
"reportType": "GET_AB_INVOICE_PDF",
"marketplaceIds": [marketplace_id],
"reportOptions": {
"orderId": order_id,
"documentType": "Invoice",
},
}
try:
resp = await client.post(f"/reports/{REPORTS_VERSION}/reports", json=body)
if resp.status_code in (200, 202):
data = resp.json()
report_id = data.get("reportId")
logger.info(f"Amazon API: Report erstellt für {order_id}: {report_id}")
return report_id
else:
logger.warning(f"Amazon API: Report-Erstellung fehlgeschlagen für {order_id}: {resp.status_code} {resp.text}")
return None
except Exception as e:
logger.error(f"Amazon API: Report-Erstellung Fehler: {e}")
return None
async def _poll_report_status(client: httpx.AsyncClient, report_id: str, max_wait: int = 120) -> str | None:
"""Step 2: Poll report status until DONE. Returns reportDocumentId."""
for i in range(max_wait // 15 + 1):
try:
resp = await client.get(f"/reports/{REPORTS_VERSION}/reports/{report_id}")
if resp.status_code != 200:
logger.warning(f"Amazon API: Report-Status fehlgeschlagen: {resp.status_code}")
return None
data = resp.json()
status = data.get("processingStatus", "")
if status == "DONE":
doc_id = data.get("reportDocumentId")
logger.info(f"Amazon API: Report {report_id} fertig: documentId={doc_id}")
return doc_id
elif status in ("CANCELLED", "FATAL"):
logger.warning(f"Amazon API: Report {report_id} fehlgeschlagen: {status}")
return None
else:
logger.debug(f"Amazon API: Report {report_id} Status: {status}, warte...")
await asyncio.sleep(15)
except Exception as e:
logger.error(f"Amazon API: Report-Status Fehler: {e}")
return None
logger.warning(f"Amazon API: Report {report_id} Timeout nach {max_wait}s")
return None
async def _download_report_document(client: httpx.AsyncClient, document_id: str) -> bytes | None:
"""Step 3: Get presigned URL and download + decompress the PDF."""
try:
resp = await client.get(f"/reports/{REPORTS_VERSION}/documents/{document_id}")
if resp.status_code != 200:
logger.warning(f"Amazon API: Document-URL fehlgeschlagen: {resp.status_code}")
return None
data = resp.json()
url = data.get("url", "")
compression = data.get("compressionAlgorithm", "")
if not url:
logger.warning(f"Amazon API: Keine Download-URL für Document {document_id}")
return None
# Download the document (presigned S3 URL, expires in 5 min)
async with httpx.AsyncClient(timeout=60.0) as dl_client:
dl_resp = await dl_client.get(url)
if dl_resp.status_code != 200:
logger.warning(f"Amazon API: Document-Download fehlgeschlagen: {dl_resp.status_code}")
return None
content = dl_resp.content
# Decompress: EU documents are gzip-compressed, then the content is a zip file
if compression == "GZIP" or content[:2] == b'\x1f\x8b':
try:
content = gzip.decompress(content)
except Exception:
pass # might not be gzipped
# Check if it's a zip file containing the PDF
if content[:2] == b'PK':
try:
with zipfile.ZipFile(io.BytesIO(content)) as zf:
for name in zf.namelist():
if name.lower().endswith('.pdf'):
content = zf.read(name)
break
except Exception:
pass # might not be a zip
# Verify it's a PDF
if content[:4] == b'%PDF':
logger.info(f"Amazon API: PDF heruntergeladen: {len(content)} Bytes")
return content
else:
logger.warning(f"Amazon API: Heruntergeladenes Dokument ist kein PDF (starts: {content[:20]})")
return None
except Exception as e:
logger.error(f"Amazon API: Document-Download Fehler: {e}")
return None
async def download_invoice(settings: dict, order_id: str) -> bytes | None:
"""Download invoice PDF via Document API (3-step async process)."""
result = await _get_api_client(settings)
if not result:
return None
client, region = result
domain = settings.get("amazon_domain", "amazon.de")
marketplace_id = DOMAIN_MARKETPLACE.get(domain, DOMAIN_MARKETPLACE["amazon.de"])
try:
# Step 1: Create report
report_id = await _create_invoice_report(client, order_id, marketplace_id)
if not report_id:
return None
# Step 2: Poll until done
document_id = await _poll_report_status(client, report_id)
if not document_id:
return None
# Step 3: Download document
return await _download_report_document(client, document_id)
except Exception as e:
logger.error(f"Amazon API: Invoice-Download-Fehler für {order_id}: {e}")
return None
finally:
await client.aclose()
async def process_amazon_api() -> dict:
"""Process Amazon invoices via API (Reconciliation + Document API)."""
settings = await get_settings()
if settings.get("amazon_enabled") != "true":
return {"processed": 0, "skipped": 0, "errors": 0}
# Check API credentials
status = await check_api_configured()
if not status.get("authorized"):
error_msg = status.get("error", "API nicht konfiguriert")
logger.warning(f"Amazon API: {error_msg}")
return {"processed": 0, "skipped": 0, "errors": 0, "error": error_msg}
domain = settings.get("amazon_domain", "amazon.de")
# Determine date range
since_str = settings.get("amazon_since_date", "")
if since_str:
try:
since_date = datetime.strptime(since_str, "%Y-%m-%d")
except ValueError:
since_date = datetime.now() - timedelta(days=30)
else:
since_date = datetime.now() - timedelta(days=30)
logger.info(f"Amazon API: Import gestartet: domain={domain}, seit={since_date.strftime('%Y-%m-%d')}")
# Connect SMTP
import_email = settings.get("import_email_eingang") or settings.get("import_email", "")
if not import_email:
error_msg = "Keine Import-Email für Eingangsbelege konfiguriert"
logger.error(f"Amazon API: {error_msg}")
await add_log_entry("Amazon-Import", f"Amazon ({domain})", 0, "error", error_msg, beleg_type="eingang")
return {"processed": 0, "skipped": 0, "errors": 1, "error": error_msg}
smtp = _connect_smtp(settings)
if not smtp:
error_msg = "SMTP-Verbindung fehlgeschlagen"
logger.error(f"Amazon API: {error_msg}")
await add_log_entry("Amazon-Import", f"Amazon ({domain})", 0, "error", error_msg, beleg_type="eingang")
return {"processed": 0, "skipped": 0, "errors": 1, "error": error_msg}
processed = 0
skipped = 0
errors = 0
try:
# Get transactions via Reconciliation API
transactions = await get_transactions(settings, since_date)
if not transactions:
logger.info("Amazon API: Keine Transaktionen gefunden")
await save_settings({"amazon_last_sync": datetime.now().strftime("%Y-%m-%d %H:%M")})
await add_log_entry(
"Amazon-Import (API)", f"Amazon ({domain})", 0,
"success", "Keine neuen Rechnungen gefunden", beleg_type="eingang",
)
smtp.quit()
return {"processed": 0, "skipped": 0, "errors": 0}
# Extract unique orders with their line items
orders = {}
for txn in transactions:
line_items = txn.get("transactionLineItems", [])
for item in line_items:
oid = item.get("orderId", "")
if oid and oid not in orders:
orders[oid] = {
"orderId": oid,
"invoiceNumber": txn.get("invoiceNumber", ""),
"transactionDate": txn.get("transactionDate", ""),
}
# Fallback: if no line items, use transaction-level orderId
if not line_items:
oid = txn.get("orderId", "")
if oid and oid not in orders:
orders[oid] = {
"orderId": oid,
"invoiceNumber": txn.get("invoiceNumber", ""),
"transactionDate": txn.get("transactionDate", ""),
}
logger.info(f"Amazon API: {len(orders)} eindeutige Bestellungen gefunden")
for oid, order_info in orders.items():
# Check if already downloaded
if await is_invoice_downloaded(oid, oid):
skipped += 1
continue
# Download invoice PDF
pdf_data = await download_invoice(settings, oid)
if pdf_data:
# Save debug copy if enabled
if settings.get("debug_save_amazon_pdfs") == "true":
debug_dir = Path("/data/uploads") / "amazon_invoices"
debug_dir.mkdir(parents=True, exist_ok=True)
debug_path = debug_dir / f"Amazon_Rechnung_{oid}.pdf"
debug_path.write_bytes(pdf_data)
logger.info(f"Amazon API: Debug-PDF gespeichert: {debug_path}")
# Send via SMTP
filename = f"Amazon_Rechnung_{oid}.pdf"
subject = f"Amazon Rechnung - {oid}"
from_addr = settings.get("smtp_username", "belegimport@local")
msg = _build_forward_email(
from_addr=from_addr,
to_addr=import_email,
original_subject=subject,
original_from=f"Amazon ({domain})",
attachments=[(filename, pdf_data)],
)
smtp_log = _send_with_log(smtp, msg)
await add_log_entry(
subject, f"Amazon ({domain})", 1,
"success", "", import_email, smtp_log, beleg_type="eingang",
)
await mark_invoice_downloaded(oid, oid)
processed += 1
logger.info(f"Amazon API: Rechnung für {oid} gesendet")
else:
# No invoice available for this order
await mark_invoice_downloaded(oid, oid)
skipped += 1
logger.debug(f"Amazon API: Keine Rechnung für {oid}")
except Exception as e:
logger.error(f"Amazon API: Import-Fehler: {e}", exc_info=True)
errors += 1
await add_log_entry(
"Amazon-Import (API)", f"Amazon ({domain})", 0,
"error", str(e), beleg_type="eingang",
)
finally:
try:
smtp.quit()
except Exception:
pass
await save_settings({"amazon_last_sync": datetime.now().strftime("%Y-%m-%d %H:%M")})
if processed > 0 or errors > 0:
summary = f"{processed} verarbeitet, {skipped} übersprungen, {errors} Fehler"
await add_log_entry(
"Amazon-Import (API, Zusammenfassung)", f"Amazon ({domain})", processed,
"success" if errors == 0 else "warning", summary, beleg_type="eingang",
)
logger.info(f"Amazon API: Import fertig: {processed} verarbeitet, {skipped} übersprungen, {errors} Fehler")
return {"processed": processed, "skipped": skipped, "errors": errors}
File diff suppressed because it is too large Load Diff
+10 -270
View File
@@ -1,16 +1,12 @@
import logging
import os
import aiosqlite
from cryptography.fernet import Fernet
DB_PATH = os.environ.get("DB_PATH", "/data/belegimport.db")
SCHEMA_VERSION = 9
logger = logging.getLogger(__name__)
_fernet = None
ENCRYPTED_KEYS = {"imap_password", "smtp_password", "smb_password", "amazon_password", "amazon_client_secret", "amazon_refresh_token", "ftp_password"}
ENCRYPTED_KEYS = {"imap_password", "smtp_password", "smb_password"}
DEFAULT_SETTINGS = {
"imap_server": "",
@@ -23,13 +19,9 @@ DEFAULT_SETTINGS = {
"smtp_ssl": "starttls",
"smtp_username": "",
"smtp_password": "",
"import_email": "",
"import_email_eingang": "",
"import_email_ausgang": "",
"lexoffice_email": "",
"source_folder": "Rechnungen",
"processed_folder": "Rechnungen/Verarbeitet",
"source_folder_ausgang": "",
"processed_folder_ausgang": "",
"interval_minutes": "5",
"scheduler_enabled": "false",
"fetch_since_date": "",
@@ -43,36 +35,7 @@ DEFAULT_SETTINGS = {
"smb_share": "",
"smb_source_path": "",
"smb_processed_path": "Verarbeitet",
"smb_source_path_ausgang": "",
"smb_processed_path_ausgang": "",
"smb_mode": "forward",
# FTP / SFTP
"ftp_enabled": "false",
"ftp_protocol": "sftp", # "sftp" or "ftp"
"ftp_server": "",
"ftp_port": "22",
"ftp_username": "",
"ftp_password": "",
"ftp_source_path": "",
"ftp_processed_path": "Verarbeitet",
"ftp_source_path_ausgang": "",
"ftp_processed_path_ausgang": "",
"ftp_mode": "forward",
# Amazon
"amazon_enabled": "false",
"amazon_email": "",
"amazon_password": "",
"amazon_domain": "amazon.de",
"amazon_last_sync": "",
"amazon_since_date": "",
# Amazon API (SP-API / Business API)
"amazon_app_id": "", # amzn1.sp.solution.xxxxx (from Developer Portal)
"amazon_client_id": "", # amzn1.application-oa2-client.xxxxx (LWA Client ID)
"amazon_client_secret": "", # LWA Client Secret
"amazon_refresh_token": "",
"amazon_mode": "browser", # "browser" or "api"
# Debug
"debug_save_amazon_pdfs": "false",
}
@@ -115,157 +78,9 @@ def _decrypt(fernet: Fernet, value: str) -> str:
return ""
async def _get_schema_version(db: aiosqlite.Connection) -> int:
"""Read current schema version from DB. Returns 0 if not set."""
try:
cursor = await db.execute(
"SELECT value FROM settings WHERE key = 'schema_version'"
)
row = await cursor.fetchone()
return int(row[0]) if row else 0
except Exception:
return 0
async def _set_schema_version(db: aiosqlite.Connection, version: int):
await db.execute(
"INSERT OR REPLACE INTO settings (key, value) VALUES ('schema_version', ?)",
(str(version),),
)
async def _run_migrations(db: aiosqlite.Connection, current_version: int):
"""Run all pending migrations sequentially."""
if current_version < 1:
logger.info("Migration v1: Initiale Tabellenstruktur")
# v1: Base tables (idempotent via IF NOT EXISTS)
await db.execute("""
CREATE TABLE IF NOT EXISTS settings (
key TEXT PRIMARY KEY,
value TEXT NOT NULL DEFAULT ''
)
""")
await db.execute("""
CREATE TABLE IF NOT EXISTS processing_log (
id INTEGER PRIMARY KEY AUTOINCREMENT,
timestamp TEXT NOT NULL DEFAULT (datetime('now', 'localtime')),
email_subject TEXT,
email_from TEXT,
attachments_count INTEGER DEFAULT 0,
status TEXT NOT NULL,
error_message TEXT,
sent_to TEXT DEFAULT '',
smtp_log TEXT DEFAULT '',
beleg_type TEXT DEFAULT 'eingang'
)
""")
await db.commit()
await _set_schema_version(db, 1)
if current_version < 2:
logger.info("Migration v2: lexoffice_email -> import_email")
# v2: Rename lexoffice_email -> import_email
cursor = await db.execute(
"SELECT value FROM settings WHERE key = 'lexoffice_email'"
)
row = await cursor.fetchone()
if row and row[0]:
# Copy value to import_email if it's empty
cursor2 = await db.execute(
"SELECT value FROM settings WHERE key = 'import_email'"
)
row2 = await cursor2.fetchone()
if not row2 or not row2[0]:
await db.execute(
"INSERT OR REPLACE INTO settings (key, value) VALUES ('import_email', ?)",
(row[0],),
)
logger.info(" lexoffice_email Wert nach import_email übertragen")
await db.execute("DELETE FROM settings WHERE key = 'lexoffice_email'")
await db.commit()
await _set_schema_version(db, 2)
if current_version < 3:
logger.info("Migration v3: Amazon-Plattform hinzugefügt")
await db.execute("""
CREATE TABLE IF NOT EXISTS amazon_downloaded (
order_id TEXT PRIMARY KEY,
downloaded_at TEXT NOT NULL DEFAULT (datetime('now', 'localtime'))
)
""")
await db.commit()
await _set_schema_version(db, 3)
if current_version < 4:
logger.info("Migration v4: sent_to Spalte im Verarbeitungslog")
await db.execute("""
ALTER TABLE processing_log ADD COLUMN sent_to TEXT DEFAULT ''
""")
await db.commit()
await _set_schema_version(db, 4)
if current_version < 5:
logger.info("Migration v5: SMTP-Protokoll im Verarbeitungslog")
await db.execute("""
ALTER TABLE processing_log ADD COLUMN smtp_log TEXT DEFAULT ''
""")
await db.commit()
await _set_schema_version(db, 5)
if current_version < 6:
logger.info("Migration v6: Per-Invoice Tracking statt per-Order")
try:
await db.execute("""
ALTER TABLE amazon_downloaded ADD COLUMN invoice_url TEXT DEFAULT ''
""")
except Exception:
pass # column already exists
await db.commit()
await _set_schema_version(db, 6)
if current_version < 8:
logger.info("Migration v7/8: Eingangs-/Ausgangsbelege Unterscheidung")
# Add beleg_type column to processing_log (check if it exists first)
cursor = await db.execute("PRAGMA table_info(processing_log)")
columns = [row[1] for row in await cursor.fetchall()]
if "beleg_type" not in columns:
await db.execute("""
ALTER TABLE processing_log ADD COLUMN beleg_type TEXT DEFAULT 'eingang'
""")
logger.info(" beleg_type Spalte hinzugefügt")
# Migrate import_email -> import_email_eingang
cursor = await db.execute(
"SELECT value FROM settings WHERE key = 'import_email'"
)
row = await cursor.fetchone()
if row and row[0]:
cursor2 = await db.execute(
"SELECT value FROM settings WHERE key = 'import_email_eingang'"
)
row2 = await cursor2.fetchone()
if not row2 or not row2[0]:
await db.execute(
"INSERT OR REPLACE INTO settings (key, value) VALUES ('import_email_eingang', ?)",
(row[0],),
)
logger.info(" import_email nach import_email_eingang übertragen")
await db.commit()
await _set_schema_version(db, 8)
if current_version < 9:
logger.info("Migration v9: FTP/SFTP-Settings hinzugefuegt (defaults werden eingefuegt)")
# No table changes needed - new settings are added via DEFAULT_SETTINGS loop in init_db
await db.commit()
await _set_schema_version(db, 9)
await db.commit()
async def init_db():
os.makedirs(os.path.dirname(DB_PATH) or ".", exist_ok=True)
os.makedirs(os.path.dirname(DB_PATH), exist_ok=True)
async with aiosqlite.connect(DB_PATH) as db:
# Ensure base tables exist (needed before we can read schema_version)
await db.execute("""
CREATE TABLE IF NOT EXISTS settings (
key TEXT PRIMARY KEY,
@@ -280,34 +95,12 @@ async def init_db():
email_from TEXT,
attachments_count INTEGER DEFAULT 0,
status TEXT NOT NULL,
error_message TEXT,
sent_to TEXT DEFAULT '',
smtp_log TEXT DEFAULT '',
beleg_type TEXT DEFAULT 'eingang'
)
""")
await db.execute("""
CREATE TABLE IF NOT EXISTS amazon_downloaded (
order_id TEXT NOT NULL,
downloaded_at TEXT NOT NULL DEFAULT (datetime('now', 'localtime')),
invoice_url TEXT DEFAULT '',
PRIMARY KEY (order_id, invoice_url)
error_message TEXT
)
""")
await db.commit()
# Check version and run migrations
current_version = await _get_schema_version(db)
if current_version < SCHEMA_VERSION:
logger.info(
f"DB-Schema v{current_version} -> v{SCHEMA_VERSION}, starte Migrationen..."
)
await _run_migrations(db, current_version)
logger.info(f"DB-Schema auf v{SCHEMA_VERSION} aktualisiert")
else:
logger.info(f"DB-Schema v{current_version} ist aktuell")
# Insert default settings for any new keys (never overwrites existing values)
# Insert default settings if not present
for key, value in DEFAULT_SETTINGS.items():
await db.execute(
"INSERT OR IGNORE INTO settings (key, value) VALUES (?, ?)",
@@ -323,7 +116,7 @@ async def get_settings() -> dict:
fernet = await _get_fernet()
async with aiosqlite.connect(DB_PATH) as db:
cursor = await db.execute(
"SELECT key, value FROM settings WHERE key NOT IN ('encryption_key', 'schema_version')"
"SELECT key, value FROM settings WHERE key != 'encryption_key'"
)
rows = await cursor.fetchall()
@@ -340,7 +133,7 @@ async def save_settings(data: dict):
fernet = await _get_fernet()
async with aiosqlite.connect(DB_PATH) as db:
for key, value in data.items():
if key in ("encryption_key", "schema_version"):
if key == "encryption_key":
continue
store_value = _encrypt(fernet, value) if key in ENCRYPTED_KEYS else value
await db.execute(
@@ -350,29 +143,19 @@ async def save_settings(data: dict):
await db.commit()
def get_import_email(settings: dict, beleg_type: str = "eingang") -> str:
"""Resolve the correct import email address based on document type."""
if beleg_type == "ausgang":
return settings.get("import_email_ausgang", "")
return settings.get("import_email_eingang", "") or settings.get("import_email", "")
async def add_log_entry(
email_subject: str,
email_from: str,
attachments_count: int,
status: str,
error_message: str = "",
sent_to: str = "",
smtp_log: str = "",
beleg_type: str = "eingang",
):
async with aiosqlite.connect(DB_PATH) as db:
await db.execute(
"""INSERT INTO processing_log
(email_subject, email_from, attachments_count, status, error_message, sent_to, smtp_log, beleg_type)
VALUES (?, ?, ?, ?, ?, ?, ?, ?)""",
(email_subject, email_from, attachments_count, status, error_message, sent_to, smtp_log, beleg_type),
(email_subject, email_from, attachments_count, status, error_message)
VALUES (?, ?, ?, ?, ?)""",
(email_subject, email_from, attachments_count, status, error_message),
)
await db.commit()
@@ -385,46 +168,3 @@ async def get_log_entries(limit: int = 100) -> list[dict]:
)
rows = await cursor.fetchall()
return [dict(row) for row in rows]
async def clear_log_entries() -> int:
async with aiosqlite.connect(DB_PATH) as db:
cursor = await db.execute("SELECT COUNT(*) FROM processing_log")
count = (await cursor.fetchone())[0]
await db.execute("DELETE FROM processing_log")
await db.commit()
return count
async def is_invoice_downloaded(order_id: str, invoice_url: str = "") -> bool:
"""Check if a specific invoice has been downloaded.
If invoice_url is given, check per-URL. Otherwise check per order_id."""
async with aiosqlite.connect(DB_PATH) as db:
if invoice_url:
cursor = await db.execute(
"SELECT 1 FROM amazon_downloaded WHERE order_id = ? AND invoice_url = ?",
(order_id, invoice_url),
)
else:
cursor = await db.execute(
"SELECT 1 FROM amazon_downloaded WHERE order_id = ?", (order_id,)
)
return await cursor.fetchone() is not None
async def mark_invoice_downloaded(order_id: str, invoice_url: str = ""):
async with aiosqlite.connect(DB_PATH) as db:
await db.execute(
"INSERT OR IGNORE INTO amazon_downloaded (order_id, invoice_url) VALUES (?, ?)",
(order_id, invoice_url),
)
await db.commit()
async def reset_downloaded_invoices() -> int:
async with aiosqlite.connect(DB_PATH) as db:
cursor = await db.execute("SELECT COUNT(*) FROM amazon_downloaded")
count = (await cursor.fetchone())[0]
await db.execute("DELETE FROM amazon_downloaded")
await db.commit()
return count
-614
View File
@@ -1,614 +0,0 @@
"""FTP / SFTP file import processor.
Same design as smb_processor but for FTP (passive, unencrypted) and SFTP (SSH).
Reads PDF files from a remote source folder, forwards them via SMTP, then moves
them to a processed folder.
"""
import asyncio
import ftplib
import io
import logging
import os
import posixpath
import tempfile
import paramiko
from app.database import get_settings, add_log_entry, get_import_email
from app.mail_processor import _connect_smtp, _build_forward_email, _send_with_log
from app.scanner import detect_separator_pages, split_pdf
logger = logging.getLogger(__name__)
# ---------------------------------------------------------------------------
# Generic adapter interface
# ---------------------------------------------------------------------------
class _FtpAdapter:
"""Common interface for FTP and SFTP backends."""
def list_pdfs(self, path: str) -> list[str]:
raise NotImplementedError
def list_dirs(self, path: str, max_depth: int = 5) -> list[str]:
raise NotImplementedError
def read_file(self, path: str) -> bytes:
raise NotImplementedError
def ensure_dir(self, path: str):
raise NotImplementedError
def stat_exists(self, path: str) -> bool:
raise NotImplementedError
def rename(self, src: str, dst: str):
raise NotImplementedError
def close(self):
pass
# ---------------------------------------------------------------------------
# FTP (passive, unencrypted)
# ---------------------------------------------------------------------------
class _PlainFtpAdapter(_FtpAdapter):
def __init__(self, server: str, port: int, username: str, password: str):
self.ftp = ftplib.FTP()
self.ftp.connect(server, port, timeout=15)
self.ftp.login(username or "anonymous", password or "")
self.ftp.set_pasv(True)
# Remember initial CWD - all subsequent operations should resolve relative to this
try:
self._initial_cwd = self.ftp.pwd()
except Exception:
self._initial_cwd = None
logger.debug(f"FTP initial CWD: {self._initial_cwd}")
def _reset_cwd(self):
"""Reset CWD to initial directory after stateful operations."""
if self._initial_cwd:
try:
self.ftp.cwd(self._initial_cwd)
except Exception:
pass
def list_pdfs(self, path: str) -> list[str]:
"""List PDF files via LIST (more reliable than NLST across FTP servers)."""
self._reset_cwd()
target = path if path else "."
lines = []
try:
self.ftp.retrlines(f"LIST {target}", lines.append)
except ftplib.error_perm:
return []
files = []
for line in lines:
if not line or line[0:1] == "d":
continue # skip directories
parts = line.split(maxsplit=8)
if len(parts) < 9:
continue
name = parts[-1]
if name.lower().endswith(".pdf") and not name.startswith("."):
files.append(name)
return sorted(files)
def list_dirs(self, path: str, max_depth: int = 5) -> list[str]:
self._reset_cwd()
base = path or ""
logger.debug(f"FTP list_dirs: base={base!r}, max_depth={max_depth}")
return self._list_dirs_rec(base, max_depth, 0, "")
def _list_dirs_rec(self, base: str, max_depth: int, depth: int, prefix: str) -> list[str]:
result = []
try:
entries = []
target = base if base else "."
self.ftp.retrlines(f"LIST {target}", entries.append)
logger.debug(f"FTP LIST {target!r} -> {len(entries)} entries")
except ftplib.error_perm as e:
logger.warning(f"FTP LIST {base!r} failed: {e}")
return []
for line in entries:
# Try to detect dirs (line starts with 'd') - works for unix-style listings
if not line or not line[0:1] == "d":
continue
parts = line.split(maxsplit=8)
if len(parts) < 9:
continue
name = parts[-1]
if name in (".", "..") or name.startswith("."):
continue
rel = f"{prefix}/{name}" if prefix else name
result.append(rel)
if depth < max_depth - 1:
sub = posixpath.join(base, name) if base else name
result.extend(self._list_dirs_rec(sub, max_depth, depth + 1, rel))
return result
def read_file(self, path: str) -> bytes:
self._reset_cwd()
buf = io.BytesIO()
self.ftp.retrbinary(f"RETR {path}", buf.write)
return buf.getvalue()
def ensure_dir(self, path: str):
"""Create directory tree, walking step by step from initial CWD.
IMPORTANT: FTP's cwd() is stateful - it changes the current working
directory for ALL subsequent operations. We must walk the path one
segment at a time relative to the current position, not concatenate
and re-cwd from initial each iteration.
"""
if not path:
return
try:
self._reset_cwd()
parts = [p for p in path.split("/") if p]
for p in parts:
try:
self.ftp.cwd(p)
except ftplib.error_perm:
# Doesn't exist - create and enter
try:
self.ftp.mkd(p)
self.ftp.cwd(p)
except ftplib.error_perm as e:
logger.warning(f"FTP mkd({p}) failed: {e}")
return
finally:
self._reset_cwd()
def stat_exists(self, path: str) -> bool:
"""Check if a file or directory exists at path."""
try:
self.ftp.size(path)
return True
except (ftplib.error_perm, ftplib.error_temp):
pass
# Try as directory - cwd then immediately reset
try:
self.ftp.cwd(path)
self._reset_cwd()
return True
except ftplib.error_perm:
self._reset_cwd()
return False
def rename(self, src: str, dst: str):
self._reset_cwd()
self.ftp.rename(src, dst)
def close(self):
try:
self.ftp.quit()
except Exception:
try:
self.ftp.close()
except Exception:
pass
# ---------------------------------------------------------------------------
# SFTP (paramiko)
# ---------------------------------------------------------------------------
class _SftpAdapter(_FtpAdapter):
def __init__(self, server: str, port: int, username: str, password: str):
self.transport = paramiko.Transport((server, port))
self.transport.connect(username=username, password=password)
self.sftp = paramiko.SFTPClient.from_transport(self.transport)
def _resolve(self, path: str) -> str:
"""Resolve path - empty/None means user's home/root directory."""
if not path:
try:
return self.sftp.normalize(".")
except IOError:
return "."
return path
def list_pdfs(self, path: str) -> list[str]:
try:
entries = self.sftp.listdir(self._resolve(path))
except IOError:
return []
return sorted(
e for e in entries if e.lower().endswith(".pdf") and not e.startswith(".")
)
def list_dirs(self, path: str, max_depth: int = 5) -> list[str]:
base = self._resolve(path)
logger.debug(f"SFTP list_dirs: base={base!r}, max_depth={max_depth}")
return self._list_dirs_rec(base, max_depth, 0, "")
def _list_dirs_rec(self, base: str, max_depth: int, depth: int, prefix: str) -> list[str]:
from stat import S_ISDIR
result = []
try:
entries = self.sftp.listdir_attr(base)
logger.debug(f"SFTP listdir_attr({base!r}) -> {[e.filename for e in entries]}")
except IOError as e:
logger.warning(f"SFTP listdir_attr({base!r}) failed: {e}")
return result
for entry in entries:
if entry.filename.startswith(".") or entry.filename in ("..", "."):
continue
if entry.st_mode and S_ISDIR(entry.st_mode):
rel = f"{prefix}/{entry.filename}" if prefix else entry.filename
result.append(rel)
if depth < max_depth - 1:
sub = posixpath.join(base, entry.filename)
result.extend(self._list_dirs_rec(sub, max_depth, depth + 1, rel))
return result
def read_file(self, path: str) -> bytes:
with self.sftp.open(path, "rb") as f:
return f.read()
def ensure_dir(self, path: str):
if not path:
return
parts = [p for p in path.split("/") if p]
cur = ""
for p in parts:
cur = f"{cur}/{p}" if cur else p
try:
self.sftp.stat(cur)
except IOError:
try:
self.sftp.mkdir(cur)
except IOError:
pass
def stat_exists(self, path: str) -> bool:
try:
self.sftp.stat(path)
return True
except IOError:
return False
def rename(self, src: str, dst: str):
self.sftp.rename(src, dst)
def close(self):
try:
self.sftp.close()
except Exception:
pass
try:
self.transport.close()
except Exception:
pass
# ---------------------------------------------------------------------------
# Helpers
# ---------------------------------------------------------------------------
def _make_adapter(settings: dict) -> _FtpAdapter:
protocol = settings.get("ftp_protocol", "sftp").lower()
server = settings["ftp_server"]
username = settings.get("ftp_username", "")
password = settings.get("ftp_password", "")
if protocol == "sftp":
port = int(settings.get("ftp_port") or 22)
return _SftpAdapter(server, port, username, password)
else:
port = int(settings.get("ftp_port") or 21)
return _PlainFtpAdapter(server, port, username, password)
def _join_path(*parts: str) -> str:
"""Join FTP/SFTP path segments using forward slash."""
result = ""
for p in parts:
if not p:
continue
p = p.replace("\\", "/").strip("/")
if not p:
continue
result = f"{result}/{p}" if result else p
return result
def _move_with_dedup(adapter: _FtpAdapter, src: str, dest_dir: str, filename: str):
"""Move file to dest_dir, renaming if a duplicate exists."""
dest = _join_path(dest_dir, filename)
if adapter.stat_exists(dest):
name, ext = os.path.splitext(filename)
counter = 1
while True:
new_name = f"{name}_{counter}{ext}"
new_dest = _join_path(dest_dir, new_name)
if not adapter.stat_exists(new_dest):
dest = new_dest
break
counter += 1
adapter.rename(src, dest)
# ---------------------------------------------------------------------------
# Processing pipeline
# ---------------------------------------------------------------------------
async def _process_ftp_folder(
smtp_conn, settings: dict, adapter: _FtpAdapter,
source_path: str, processed_path: str,
import_email: str, beleg_type: str, mode: str,
) -> dict:
"""Process one FTP folder pair. Returns counts dict."""
smtp_from = settings.get("smtp_username", "")
protocol = settings.get("ftp_protocol", "sftp").upper()
processed = 0
skipped = 0
errors = 0
await asyncio.to_thread(adapter.ensure_dir, processed_path)
pdf_files = await asyncio.to_thread(adapter.list_pdfs, source_path)
if not pdf_files:
logger.info(f"Keine PDF-Dateien im {protocol}-Ordner '{source_path}' ({beleg_type})")
return {"processed": 0, "skipped": 0, "errors": 0}
logger.info(f"{len(pdf_files)} PDF-Datei(en) im {protocol}-Ordner '{source_path}' ({beleg_type})")
for filename in pdf_files:
file_path = _join_path(source_path, filename)
try:
pdf_data = await asyncio.to_thread(adapter.read_file, file_path)
if mode == "separator":
with tempfile.NamedTemporaryFile(suffix=".pdf", delete=False) as tmp:
tmp.write(pdf_data)
tmp_path = tmp.name
try:
separator_pages = await asyncio.to_thread(detect_separator_pages, tmp_path, None)
documents = await asyncio.to_thread(split_pdf, tmp_path, separator_pages)
finally:
os.unlink(tmp_path)
if not documents:
skipped += 1
continue
smtp_log_parts = []
for i, doc_bytes in enumerate(documents):
doc_filename = f"{os.path.splitext(filename)[0]}_Teil_{i + 1}.pdf"
subject = f"{protocol}-Import: {filename} (Dokument {i + 1}/{len(documents)})"
msg = _build_forward_email(
from_addr=smtp_from,
to_addr=import_email,
original_subject=subject,
original_from=f"{protocol}-Import",
attachments=[(doc_filename, doc_bytes)],
)
smtp_log_parts.append(_send_with_log(smtp_conn, msg))
await add_log_entry(
email_subject=f"{protocol}: {filename}",
email_from=f"{protocol}-Import",
attachments_count=len(documents),
status="success",
sent_to=import_email,
smtp_log="\n---\n".join(smtp_log_parts),
beleg_type=beleg_type,
)
logger.info(f"{protocol} verarbeitet ({beleg_type}): {filename} -> {len(documents)} Dokument(e)")
else:
msg = _build_forward_email(
from_addr=smtp_from,
to_addr=import_email,
original_subject=f"{protocol}-Import: {filename}",
original_from=f"{protocol}-Import",
attachments=[(filename, pdf_data)],
)
smtp_log = _send_with_log(smtp_conn, msg)
await add_log_entry(
email_subject=f"{protocol}: {filename}",
email_from=f"{protocol}-Import",
attachments_count=1,
status="success",
sent_to=import_email,
smtp_log=smtp_log,
beleg_type=beleg_type,
)
logger.info(f"{protocol} verarbeitet ({beleg_type}): {filename}")
await asyncio.to_thread(_move_with_dedup, adapter, file_path, processed_path, filename)
processed += 1
except Exception as e:
errors += 1
logger.error(f"Fehler bei {protocol}-Datei {filename}: {e}")
try:
await add_log_entry(
email_subject=f"{protocol}: {filename}",
email_from=f"{protocol}-Import",
attachments_count=0,
status="error",
error_message=str(e),
beleg_type=beleg_type,
)
except Exception:
pass
return {"processed": processed, "skipped": skipped, "errors": errors}
async def process_ftp() -> dict:
"""Process PDF files from FTP/SFTP server - main pipeline."""
settings = await get_settings()
if settings.get("ftp_enabled") != "true":
return {"processed": 0, "skipped": 0, "errors": 0}
if not settings.get("ftp_server"):
return {"processed": 0, "skipped": 0, "errors": 0, "error": "FTP nicht konfiguriert"}
import_email_eingang = get_import_email(settings, "eingang")
if not import_email_eingang:
return {"processed": 0, "skipped": 0, "errors": 0, "error": "Import-Email nicht konfiguriert"}
mode = settings.get("ftp_mode", "forward")
protocol = settings.get("ftp_protocol", "sftp").upper()
total = {"processed": 0, "skipped": 0, "errors": 0}
smtp_conn = None
adapter = None
try:
adapter = await asyncio.to_thread(_make_adapter, settings)
smtp_conn = _connect_smtp(settings)
# Eingangsbelege
source = settings.get("ftp_source_path", "")
processed_path = settings.get("ftp_processed_path", "Verarbeitet")
result = await _process_ftp_folder(
smtp_conn, settings, adapter,
source, processed_path,
import_email_eingang, "eingang", mode,
)
for k in total:
total[k] += result[k]
# Ausgangsbelege (optional)
import_email_ausgang = get_import_email(settings, "ausgang")
source_ausgang = settings.get("ftp_source_path_ausgang", "")
processed_ausgang = settings.get("ftp_processed_path_ausgang", "")
if import_email_ausgang and source_ausgang:
if not processed_ausgang:
processed_ausgang = source_ausgang + "/Verarbeitet"
result = await _process_ftp_folder(
smtp_conn, settings, adapter,
source_ausgang, processed_ausgang,
import_email_ausgang, "ausgang", mode,
)
for k in total:
total[k] += result[k]
except Exception as e:
logger.error(f"{protocol}-Verbindungsfehler: {e}")
try:
await add_log_entry(
email_subject="",
email_from=f"{protocol}-Import",
attachments_count=0,
status="error",
error_message=f"{protocol}-Verbindungsfehler: {e}",
)
except Exception:
pass
return {**total, "errors": total["errors"] + 1, "error": str(e)}
finally:
if adapter:
try:
await asyncio.to_thread(adapter.close)
except Exception:
pass
if smtp_conn:
try:
smtp_conn.quit()
except Exception:
pass
logger.info(f"{protocol} fertig: {total['processed']} verarbeitet, {total['skipped']} übersprungen, {total['errors']} Fehler")
return total
async def test_ftp_connection() -> dict:
"""Test FTP/SFTP connection and return TOP-LEVEL folders only (lazy loading)."""
settings = await get_settings()
if not settings.get("ftp_server"):
return {"success": False, "error": "FTP-Server nicht konfiguriert", "folders": []}
adapter = None
try:
adapter = await asyncio.to_thread(_make_adapter, settings)
folders = await asyncio.to_thread(adapter.list_dirs, "", 1)
return {"success": True, "folders": sorted(folders)}
except Exception as e:
logger.error(f"FTP-Test fehlgeschlagen: {e}")
return {"success": False, "error": str(e), "folders": []}
finally:
if adapter:
try:
await asyncio.to_thread(adapter.close)
except Exception:
pass
async def create_ftp_folder(folder_path: str) -> dict:
"""Create a folder on the FTP/SFTP server."""
settings = await get_settings()
if not settings.get("ftp_server"):
return {"success": False, "error": "FTP nicht konfiguriert"}
if not folder_path or not folder_path.strip():
return {"success": False, "error": "Ordnername darf nicht leer sein"}
folder_path = folder_path.strip().replace("\\", "/")
adapter = None
try:
adapter = await asyncio.to_thread(_make_adapter, settings)
await asyncio.to_thread(adapter.ensure_dir, folder_path)
return {"success": True}
except Exception as e:
logger.error(f"FTP-Ordner erstellen fehlgeschlagen: {e}")
return {"success": False, "error": str(e)}
finally:
if adapter:
try:
await asyncio.to_thread(adapter.close)
except Exception:
pass
async def list_ftp_folders() -> dict:
"""Return TOP-LEVEL folder list from FTP/SFTP server (lazy loading)."""
settings = await get_settings()
if not settings.get("ftp_server"):
return {"folders": []}
adapter = None
try:
adapter = await asyncio.to_thread(_make_adapter, settings)
folders = await asyncio.to_thread(adapter.list_dirs, "", 1)
return {"folders": sorted(folders)}
except Exception:
return {"folders": []}
finally:
if adapter:
try:
await asyncio.to_thread(adapter.close)
except Exception:
pass
async def list_ftp_subfolders(parent_path: str) -> dict:
"""List direct subfolders of a path (one level deep, for lazy tree expansion)."""
settings = await get_settings()
if not settings.get("ftp_server"):
return {"success": False, "error": "FTP nicht konfiguriert", "folders": []}
adapter = None
try:
adapter = await asyncio.to_thread(_make_adapter, settings)
rel_folders = await asyncio.to_thread(adapter.list_dirs, parent_path, 1)
# Prefix with parent_path so the frontend has full paths
if parent_path:
folders = [f"{parent_path}/{f}" for f in rel_folders]
else:
folders = rel_folders
return {"success": True, "folders": sorted(folders)}
except Exception as e:
logger.error(f"FTP-Subfolder-Liste fehlgeschlagen: {e}")
return {"success": False, "error": str(e), "folders": []}
finally:
if adapter:
try:
await asyncio.to_thread(adapter.close)
except Exception:
pass
+113 -183
View File
@@ -9,30 +9,11 @@ from email.mime.text import MIMEText
from email import encoders
import logging
from app.database import get_settings, add_log_entry, get_import_email
from app.database import get_settings, add_log_entry
logger = logging.getLogger(__name__)
def _send_with_log(smtp_conn: smtplib.SMTP, msg) -> str:
"""Send email and capture SMTP protocol exchange."""
log_lines = []
original_print_debug = smtp_conn._print_debug
def _capture(*args):
log_lines.append(" ".join(str(a) for a in args))
smtp_conn._print_debug = _capture
old_level = smtp_conn.debuglevel
smtp_conn.set_debuglevel(1)
try:
smtp_conn.send_message(msg)
finally:
smtp_conn.set_debuglevel(old_level)
smtp_conn._print_debug = original_print_debug
return "\n".join(log_lines)
def _connect_imap(settings: dict) -> imaplib.IMAP4_SSL | imaplib.IMAP4:
server = settings["imap_server"]
port = int(settings.get("imap_port", 993))
@@ -104,7 +85,7 @@ def _build_forward_email(
msg["Subject"] = f"Belegimport: {original_subject}"
body = (
f"Automatisch weitergeleitet von Belegimport.\n"
f"Automatisch weitergeleitet von LexOffice Belegimport.\n"
f"Original-Absender: {original_from}\n"
f"Original-Betreff: {original_subject}\n"
f"Anzahl Anhänge: {len(attachments)}"
@@ -137,117 +118,21 @@ def _move_email(conn: imaplib.IMAP4, msg_uid: bytes, dest_folder: str):
conn.expunge()
async def _process_folder(
imap_conn, smtp_conn, settings: dict,
source_folder: str, processed_folder: str,
import_email: str, beleg_type: str, fetch_since: str,
) -> dict:
"""Process one IMAP folder pair. Returns counts dict."""
smtp_from = settings.get("smtp_username", "")
processed = 0
skipped = 0
errors = 0
_ensure_folder_exists(imap_conn, processed_folder)
status, _ = imap_conn.select(f'"{source_folder}"')
if status != "OK":
logger.warning(f"Ordner '{source_folder}' konnte nicht geöffnet werden")
return {"processed": 0, "skipped": 0, "errors": 0}
search_criteria = "ALL"
if fetch_since:
try:
from datetime import datetime
dt = datetime.strptime(fetch_since, "%Y-%m-%d")
imap_date = dt.strftime("%d-%b-%Y")
search_criteria = f'(SINCE {imap_date})'
except ValueError:
logger.warning(f"Ungültiges Datum: {fetch_since}, verwende ALL")
status, data = imap_conn.uid("SEARCH", None, search_criteria)
if status != "OK" or not data[0]:
logger.info(f"Keine Emails im Ordner '{source_folder}' ({beleg_type})")
return {"processed": 0, "skipped": 0, "errors": 0}
msg_uids = data[0].split()
logger.info(f"{len(msg_uids)} Email(s) im Ordner '{source_folder}' ({beleg_type})")
for msg_uid in msg_uids:
subject = "?"
from_addr = "?"
try:
status, msg_data = imap_conn.uid("FETCH", msg_uid, "(RFC822)")
if status != "OK":
continue
raw_email = msg_data[0][1]
msg = email.message_from_bytes(raw_email, policy=policy.default)
subject = str(msg.get("Subject", "(Kein Betreff)"))
from_addr = str(msg.get("From", "(Unbekannt)"))
attachments = _extract_attachments(msg)
if not attachments:
skipped += 1
logger.debug(f"Übersprungen (keine Anhänge): {subject}")
continue
forward_msg = _build_forward_email(
from_addr=smtp_from,
to_addr=import_email,
original_subject=subject,
original_from=from_addr,
attachments=attachments,
)
smtp_log = _send_with_log(smtp_conn, forward_msg)
imap_conn.select(f'"{source_folder}"')
_move_email(imap_conn, msg_uid, processed_folder)
imap_conn.select(f'"{source_folder}"')
processed += 1
logger.info(f"Verarbeitet ({beleg_type}): {subject} ({len(attachments)} Anhänge)")
await add_log_entry(
email_subject=subject,
email_from=from_addr,
attachments_count=len(attachments),
status="success",
sent_to=import_email,
smtp_log=smtp_log,
beleg_type=beleg_type,
)
except Exception as e:
errors += 1
logger.error(f"Fehler bei Email UID {msg_uid}: {e}")
try:
await add_log_entry(
email_subject=subject,
email_from=from_addr,
attachments_count=0,
status="error",
error_message=str(e),
beleg_type=beleg_type,
)
except Exception:
pass
return {"processed": processed, "skipped": skipped, "errors": errors}
async def process_mailbox() -> dict:
settings = await get_settings()
import_email_eingang = get_import_email(settings, "eingang")
if not settings.get("imap_server") or not import_email_eingang:
logger.warning("IMAP oder Import-Email nicht konfiguriert")
if not settings.get("imap_server") or not settings.get("lexoffice_email"):
logger.warning("IMAP oder LexOffice-Email nicht konfiguriert")
return {"processed": 0, "skipped": 0, "errors": 0, "error": "Nicht konfiguriert"}
fetch_since = settings.get("fetch_since_date", "")
total = {"processed": 0, "skipped": 0, "errors": 0}
source_folder = settings.get("source_folder", "INBOX")
processed_folder = settings.get("processed_folder", "INBOX/Verarbeitet")
lexoffice_email = settings["lexoffice_email"]
smtp_from = settings.get("smtp_username", "")
processed = 0
skipped = 0
errors = 0
imap_conn = None
smtp_conn = None
@@ -255,31 +140,92 @@ async def process_mailbox() -> dict:
imap_conn = _connect_imap(settings)
smtp_conn = _connect_smtp(settings)
# Eingangsbelege
source = settings.get("source_folder", "INBOX")
processed_folder = settings.get("processed_folder", "INBOX/Verarbeitet")
result = await _process_folder(
imap_conn, smtp_conn, settings,
source, processed_folder,
import_email_eingang, "eingang", fetch_since,
)
for k in total:
total[k] += result[k]
_ensure_folder_exists(imap_conn, processed_folder)
# Ausgangsbelege (optional)
import_email_ausgang = get_import_email(settings, "ausgang")
source_ausgang = settings.get("source_folder_ausgang", "")
processed_ausgang = settings.get("processed_folder_ausgang", "")
if import_email_ausgang and source_ausgang:
if not processed_ausgang:
processed_ausgang = source_ausgang + "/Verarbeitet"
result = await _process_folder(
imap_conn, smtp_conn, settings,
source_ausgang, processed_ausgang,
import_email_ausgang, "ausgang", fetch_since,
)
for k in total:
total[k] += result[k]
status, _ = imap_conn.select(f'"{source_folder}"')
if status != "OK":
raise Exception(f"Ordner '{source_folder}' konnte nicht geöffnet werden")
# Build IMAP search criteria
search_criteria = "ALL"
fetch_since = settings.get("fetch_since_date", "")
if fetch_since:
try:
from datetime import datetime
dt = datetime.strptime(fetch_since, "%Y-%m-%d")
imap_date = dt.strftime("%d-%b-%Y")
search_criteria = f'(SINCE {imap_date})'
except ValueError:
logger.warning(f"Ungültiges Datum: {fetch_since}, verwende ALL")
status, data = imap_conn.uid("SEARCH", None, search_criteria)
if status != "OK" or not data[0]:
logger.info("Keine Emails im Ordner gefunden")
return {"processed": 0, "skipped": 0, "errors": 0}
msg_uids = data[0].split()
logger.info(f"{len(msg_uids)} Email(s) im Ordner '{source_folder}' gefunden")
for msg_uid in msg_uids:
try:
status, msg_data = imap_conn.uid("FETCH", msg_uid, "(RFC822)")
if status != "OK":
continue
raw_email = msg_data[0][1]
msg = email.message_from_bytes(raw_email, policy=policy.default)
subject = str(msg.get("Subject", "(Kein Betreff)"))
from_addr = str(msg.get("From", "(Unbekannt)"))
attachments = _extract_attachments(msg)
if not attachments:
skipped += 1
logger.debug(f"Übersprungen (keine Anhänge): {subject}")
continue
forward_msg = _build_forward_email(
from_addr=smtp_from,
to_addr=lexoffice_email,
original_subject=subject,
original_from=from_addr,
attachments=attachments,
)
smtp_conn.send_message(forward_msg)
# Re-select source folder before move (in case _ensure_folder changed it)
imap_conn.select(f'"{source_folder}"')
_move_email(imap_conn, msg_uid, processed_folder)
# Re-select after expunge to keep UIDs valid
imap_conn.select(f'"{source_folder}"')
processed += 1
logger.info(
f"Verarbeitet: {subject} ({len(attachments)} Anhänge)"
)
await add_log_entry(
email_subject=subject,
email_from=from_addr,
attachments_count=len(attachments),
status="success",
)
except Exception as e:
errors += 1
logger.error(f"Fehler bei Email UID {msg_uid}: {e}")
try:
await add_log_entry(
email_subject=subject if "subject" in dir() else "?",
email_from=from_addr if "from_addr" in dir() else "?",
attachments_count=0,
status="error",
error_message=str(e),
)
except Exception:
pass
except Exception as e:
logger.error(f"Verbindungsfehler: {e}")
@@ -290,7 +236,7 @@ async def process_mailbox() -> dict:
status="error",
error_message=f"Verbindungsfehler: {e}",
)
return {**total, "errors": total["errors"] + 1, "error": str(e)}
return {"processed": processed, "skipped": skipped, "errors": errors + 1, "error": str(e)}
finally:
if imap_conn:
@@ -304,52 +250,36 @@ async def process_mailbox() -> dict:
except Exception:
pass
logger.info(f"Fertig: {total['processed']} verarbeitet, {total['skipped']} übersprungen, {total['errors']} Fehler")
return total
logger.info(
f"Fertig: {processed} verarbeitet, {skipped} übersprungen, {errors} Fehler"
)
return {"processed": processed, "skipped": skipped, "errors": errors}
async def send_test_email() -> dict:
settings = await get_settings()
import_email_eingang = get_import_email(settings, "eingang")
import_email_ausgang = get_import_email(settings, "ausgang")
if not settings.get("smtp_server") or not import_email_eingang:
return {"success": False, "error": "SMTP oder Import-Email (Eingang) nicht konfiguriert"}
if not settings.get("smtp_server") or not settings.get("lexoffice_email"):
return {"success": False, "error": "SMTP oder LexOffice-Email nicht konfiguriert"}
try:
smtp_conn = _connect_smtp(settings)
smtp_logs = []
# Test Eingangsbelege
msg = MIMEMultipart()
msg["From"] = settings["smtp_username"]
msg["To"] = import_email_eingang
msg["Subject"] = "Belegimport - Test-Email (Eingangsbelege)"
msg["To"] = settings["lexoffice_email"]
msg["Subject"] = "LexOffice Belegimport - Test-Email"
msg.attach(MIMEText(
"Dies ist eine Test-Email vom Belegimport Service.\n"
"Ziel: Eingangsbelege",
"plain", "utf-8",
"Dies ist eine Test-Email vom LexOffice Belegimport Service.\n"
"Wenn Sie diese Email erhalten, funktioniert die SMTP-Verbindung.",
"plain",
"utf-8",
))
smtp_logs.append("=== Eingangsbelege ===")
smtp_logs.append(_send_with_log(smtp_conn, msg))
# Test Ausgangsbelege (if configured)
if import_email_ausgang:
msg2 = MIMEMultipart()
msg2["From"] = settings["smtp_username"]
msg2["To"] = import_email_ausgang
msg2["Subject"] = "Belegimport - Test-Email (Ausgangsbelege)"
msg2.attach(MIMEText(
"Dies ist eine Test-Email vom Belegimport Service.\n"
"Ziel: Ausgangsbelege",
"plain", "utf-8",
))
smtp_logs.append("=== Ausgangsbelege ===")
smtp_logs.append(_send_with_log(smtp_conn, msg2))
smtp_conn.send_message(msg)
smtp_conn.quit()
return {"success": True, "smtp_log": "\n".join(smtp_logs)}
return {"success": True}
except Exception as e:
logger.error(f"Test-Email fehlgeschlagen: {e}")
+10 -323
View File
@@ -6,41 +6,19 @@ from contextlib import asynccontextmanager
from pathlib import Path
from fastapi import FastAPI, Request, Form, UploadFile
from fastapi.responses import HTMLResponse, JSONResponse, Response
from fastapi.responses import HTMLResponse, JSONResponse, Response, StreamingResponse
from fastapi.staticfiles import StaticFiles
from fastapi.templating import Jinja2Templates
from sse_starlette.sse import EventSourceResponse
from app.database import init_db, get_settings, save_settings, get_log_entries, clear_log_entries, reset_downloaded_invoices
from app.database import init_db, get_settings, save_settings, get_log_entries
from app.mail_processor import process_mailbox, send_test_email, test_imap_connection, create_imap_folder
from app.scheduler import start_scheduler, configure_job, get_scheduler_status
from app.scanner import process_scanned_pdf, generate_separator_pdf, UPLOAD_DIR
from app.smb_processor import process_smb_share, test_smb_connection, create_smb_folder, list_smb_folders, list_smb_subfolders
from app.ftp_processor import process_ftp, test_ftp_connection, create_ftp_folder, list_ftp_folders, list_ftp_subfolders
from app.amazon_processor import (
start_login as amazon_start_login,
submit_otp as amazon_submit_otp,
get_login_state as amazon_get_login_state,
check_session_valid as amazon_check_session,
clear_session as amazon_clear_session,
process_amazon,
start_interactive_login as amazon_start_interactive,
get_browser_screenshot as amazon_get_screenshot,
send_browser_click as amazon_browser_click,
send_browser_type as amazon_browser_type,
send_browser_key as amazon_browser_key,
close_interactive_login as amazon_close_interactive,
is_interactive_login_active as amazon_login_active,
)
from app.amazon_api import (
get_oauth_authorize_url,
exchange_auth_code,
check_api_configured,
process_amazon_api,
)
from app.smb_processor import process_smb_share, test_smb_connection, create_smb_folder, list_smb_folders
logging.basicConfig(
level=getattr(logging, os.environ.get("LOG_LEVEL", "INFO").upper(), logging.INFO),
level=logging.INFO,
format="%(asctime)s [%(levelname)s] %(name)s: %(message)s",
)
logger = logging.getLogger(__name__)
@@ -54,23 +32,18 @@ async def lifespan(app: FastAPI):
interval = int(settings.get("interval_minutes", 5))
enabled = settings.get("scheduler_enabled", "false") == "true"
configure_job(interval, enabled)
logger.info("Belegimport gestartet")
logger.info("LexOffice Belegimport gestartet")
yield
logger.info("Belegimport beendet")
logger.info("LexOffice Belegimport beendet")
app = FastAPI(title="Belegimport", lifespan=lifespan)
app = FastAPI(title="LexOffice Belegimport", lifespan=lifespan)
app.mount("/static", StaticFiles(directory="app/static"), name="static")
templates = Jinja2Templates(directory="app/templates")
@app.get("/", response_class=HTMLResponse)
async def index(request: Request):
return templates.TemplateResponse("scan.html", {"request": request})
@app.get("/settings", response_class=HTMLResponse)
async def settings_page(request: Request):
settings = await get_settings()
logs = await get_log_entries(limit=20)
status = get_scheduler_status()
@@ -100,13 +73,9 @@ async def _save_form_settings(request: Request) -> dict:
"smtp_ssl": form.get("smtp_ssl", "starttls"),
"smtp_username": form.get("smtp_username", ""),
"smtp_password": form.get("smtp_password") or current.get("smtp_password", ""),
"import_email": form.get("import_email", ""),
"import_email_eingang": form.get("import_email_eingang", ""),
"import_email_ausgang": form.get("import_email_ausgang", ""),
"lexoffice_email": form.get("lexoffice_email", ""),
"source_folder": form.get("source_folder", "Rechnungen"),
"processed_folder": form.get("processed_folder", "Rechnungen/Verarbeitet"),
"source_folder_ausgang": form.get("source_folder_ausgang", ""),
"processed_folder_ausgang": form.get("processed_folder_ausgang", ""),
"interval_minutes": form.get("interval_minutes", "5"),
"scheduler_enabled": form.get("scheduler_enabled", "false"),
"fetch_since_date": form.get("fetch_since_date", ""),
@@ -120,23 +89,7 @@ async def _save_form_settings(request: Request) -> dict:
"smb_share": form.get("smb_share", ""),
"smb_source_path": form.get("smb_source_path", ""),
"smb_processed_path": form.get("smb_processed_path", "Verarbeitet"),
"smb_source_path_ausgang": form.get("smb_source_path_ausgang", ""),
"smb_processed_path_ausgang": form.get("smb_processed_path_ausgang", ""),
"smb_mode": form.get("smb_mode", "forward"),
# FTP / SFTP
"ftp_enabled": form.get("ftp_enabled", "false"),
"ftp_protocol": form.get("ftp_protocol", "sftp"),
"ftp_server": form.get("ftp_server", ""),
"ftp_port": form.get("ftp_port", "22"),
"ftp_username": form.get("ftp_username", ""),
"ftp_password": form.get("ftp_password") or current.get("ftp_password", ""),
"ftp_source_path": form.get("ftp_source_path", ""),
"ftp_processed_path": form.get("ftp_processed_path", "Verarbeitet"),
"ftp_source_path_ausgang": form.get("ftp_source_path_ausgang", ""),
"ftp_processed_path_ausgang": form.get("ftp_processed_path_ausgang", ""),
"ftp_mode": form.get("ftp_mode", "forward"),
# Debug
"debug_save_amazon_pdfs": form.get("debug_save_amazon_pdfs", "false"),
}
await save_settings(data)
@@ -232,45 +185,6 @@ async def api_create_smb_folder(request: Request):
return JSONResponse(result)
@app.get("/api/list-smb-subfolders")
async def api_list_smb_subfolders(request: Request):
parent = request.query_params.get("path", "")
result = await list_smb_subfolders(parent)
return JSONResponse(result)
@app.post("/api/test-ftp")
async def api_test_ftp(request: Request):
await _save_form_settings(request)
result = await test_ftp_connection()
return JSONResponse(result)
@app.post("/api/process-ftp")
async def api_process_ftp(request: Request):
await _save_form_settings(request)
result = await process_ftp()
return JSONResponse(result)
@app.post("/api/create-ftp-folder")
async def api_create_ftp_folder(request: Request):
body = await request.json()
folder_name = body.get("folder_name", "")
result = await create_ftp_folder(folder_name)
if result["success"]:
folders_result = await list_ftp_folders()
result["folders"] = folders_result.get("folders", [])
return JSONResponse(result)
@app.get("/api/list-ftp-subfolders")
async def api_list_ftp_subfolders(request: Request):
parent = request.query_params.get("path", "")
result = await list_ftp_subfolders(parent)
return JSONResponse(result)
@app.get("/log", response_class=HTMLResponse)
async def log_page(request: Request):
logs = await get_log_entries(limit=500)
@@ -280,12 +194,6 @@ async def log_page(request: Request):
})
@app.post("/api/clear-log")
async def api_clear_log():
count = await clear_log_entries()
return JSONResponse({"success": True, "count": count})
@app.get("/api/status")
async def api_status():
return get_scheduler_status()
@@ -339,7 +247,6 @@ async def scan_upload_chunk(
async def scan_process(request: Request):
body = await request.json()
upload_id = body.get("upload_id", "")
beleg_type = body.get("beleg_type", "eingang")
try:
uuid.UUID(upload_id)
@@ -362,7 +269,7 @@ async def scan_process(request: Request):
# Process in background task
async def _process():
try:
result = await process_scanned_pdf(str(pdf_path), progress_callback, beleg_type=beleg_type)
result = await process_scanned_pdf(str(pdf_path), progress_callback)
_scan_progress.setdefault(upload_id, []).append({
"stage": "done", "result": result
})
@@ -416,225 +323,5 @@ async def separator_pdf():
return Response(
content=pdf_bytes,
media_type="application/pdf",
headers={"Content-Disposition": "attachment; filename=Trennseite.pdf"},
headers={"Content-Disposition": "attachment; filename=Trennseite_LexOffice.pdf"},
)
# --- Plattformen (Amazon) ---
@app.get("/platforms", response_class=HTMLResponse)
async def platforms_page(request: Request):
settings = await get_settings()
status = get_scheduler_status()
return templates.TemplateResponse("platforms.html", {
"request": request,
"settings": settings,
"status": status,
"message": None,
"message_type": None,
})
@app.post("/api/amazon-settings")
async def api_amazon_settings(request: Request):
body = await request.json()
current = await get_settings()
data = {
"amazon_enabled": body.get("amazon_enabled", "false"),
"amazon_domain": body.get("amazon_domain", "amazon.de"),
"amazon_email": body.get("amazon_email", ""),
"amazon_password": body.get("amazon_password") or current.get("amazon_password", ""),
"amazon_since_date": body.get("amazon_since_date", ""),
"amazon_mode": body.get("amazon_mode", "browser"),
"amazon_app_id": body.get("amazon_app_id", ""),
"amazon_client_id": body.get("amazon_client_id", ""),
"amazon_client_secret": body.get("amazon_client_secret") or current.get("amazon_client_secret", ""),
}
await save_settings(data)
return JSONResponse({"success": True})
@app.get("/api/amazon-status")
async def api_amazon_status():
settings = await get_settings()
mode = settings.get("amazon_mode", "browser")
if mode == "api":
api_status = await check_api_configured()
return JSONResponse({
"mode": "api",
"session_valid": api_status.get("authorized", False),
"login_active": False,
"api_configured": api_status.get("configured", False),
"api_authorized": api_status.get("authorized", False),
})
else:
valid = await amazon_check_session()
login_active = amazon_login_active()
return JSONResponse({
"mode": "browser",
"session_valid": valid,
"login_active": login_active,
})
def _get_oauth_redirect_uri(request: Request) -> str:
"""Get OAuth redirect URI from env var or request."""
base = os.environ.get("OAUTH_REDIRECT_BASE", "").rstrip("/")
if not base:
base = str(request.base_url).rstrip("/")
return f"{base}/api/amazon-oauth-callback"
@app.get("/api/amazon-oauth-url")
async def api_amazon_oauth_url(request: Request):
"""Generate OAuth authorization URL for Amazon Business API."""
settings = await get_settings()
app_id = settings.get("amazon_app_id", "")
if not app_id:
return JSONResponse({"error": "App-ID nicht konfiguriert"}, status_code=400)
redirect_uri = _get_oauth_redirect_uri(request)
domain = settings.get("amazon_domain", "amazon.de")
state = str(uuid.uuid4())
url = get_oauth_authorize_url(app_id, redirect_uri, domain, state)
return JSONResponse({"url": url, "state": state})
@app.get("/api/amazon-oauth-callback")
async def api_amazon_oauth_callback(request: Request):
"""Handle OAuth callback from Amazon."""
code = request.query_params.get("spapi_oauth_code") or request.query_params.get("code", "")
error = request.query_params.get("error", "")
if error:
return HTMLResponse(f"<h2>Autorisierung fehlgeschlagen</h2><p>{error}</p><p>Fenster kann geschlossen werden.</p>")
if not code:
return HTMLResponse("<h2>Fehler: Kein Autorisierungscode erhalten</h2><p>Fenster kann geschlossen werden.</p>")
settings = await get_settings()
client_id = settings.get("amazon_client_id", "")
client_secret = settings.get("amazon_client_secret", "")
redirect_uri = _get_oauth_redirect_uri(request)
result = await exchange_auth_code(code, client_id, client_secret, redirect_uri)
if "error" in result:
return HTMLResponse(f"<h2>Token-Exchange fehlgeschlagen</h2><p>{result['error']}</p>")
refresh_token = result.get("refresh_token", "")
if refresh_token:
await save_settings({"amazon_refresh_token": refresh_token})
return HTMLResponse(
"<h2>Autorisierung erfolgreich!</h2>"
"<p>Refresh-Token wurde gespeichert. Dieses Fenster kann geschlossen werden.</p>"
"<script>window.close();</script>"
)
return HTMLResponse("<h2>Fehler: Kein Refresh-Token erhalten</h2>")
@app.post("/api/amazon-oauth-exchange")
async def api_amazon_oauth_exchange(request: Request):
"""Manual OAuth code exchange - user pastes the code from the redirect URL."""
body = await request.json()
code = body.get("code", "").strip()
if not code:
return JSONResponse({"error": "Kein Code angegeben"}, status_code=400)
settings = await get_settings()
client_id = settings.get("amazon_client_id", "")
client_secret = settings.get("amazon_client_secret", "")
redirect_uri = _get_oauth_redirect_uri(request)
result = await exchange_auth_code(code, client_id, client_secret, redirect_uri)
if "error" in result:
return JSONResponse({"error": result["error"]}, status_code=400)
refresh_token = result.get("refresh_token", "")
if refresh_token:
await save_settings({"amazon_refresh_token": refresh_token})
return JSONResponse({"success": True})
return JSONResponse({"error": "Kein Refresh-Token erhalten"}, status_code=400)
@app.post("/api/amazon-login")
async def api_amazon_login():
"""Start interactive browser login."""
await amazon_start_interactive()
return JSONResponse({"success": True})
@app.get("/api/amazon-login-state")
async def api_amazon_login_state():
return JSONResponse(amazon_get_login_state())
@app.get("/api/amazon-browser-screenshot")
async def api_amazon_browser_screenshot():
img = await amazon_get_screenshot()
if img is None:
return JSONResponse({"error": "Kein Browser offen"}, status_code=404)
return Response(content=img, media_type="image/png")
@app.post("/api/amazon-browser-click")
async def api_amazon_browser_click(request: Request):
body = await request.json()
await amazon_browser_click(int(body["x"]), int(body["y"]))
return JSONResponse({"success": True})
@app.post("/api/amazon-browser-type")
async def api_amazon_browser_type(request: Request):
body = await request.json()
await amazon_browser_type(body["text"])
return JSONResponse({"success": True})
@app.post("/api/amazon-browser-key")
async def api_amazon_browser_key(request: Request):
body = await request.json()
await amazon_browser_key(body["key"])
return JSONResponse({"success": True})
@app.post("/api/amazon-login-close")
async def api_amazon_login_close():
await amazon_close_interactive()
return JSONResponse({"success": True})
@app.post("/api/amazon-otp")
async def api_amazon_otp(request: Request):
body = await request.json()
ok = await amazon_submit_otp(body.get("code", ""))
return JSONResponse({"success": ok})
@app.post("/api/amazon-logout")
async def api_amazon_logout():
await amazon_clear_session()
return JSONResponse({"success": True})
@app.post("/api/amazon-process")
async def api_amazon_process():
settings = await get_settings()
mode = settings.get("amazon_mode", "browser")
if mode == "api":
result = await process_amazon_api()
else:
result = await process_amazon()
return JSONResponse(result)
@app.post("/api/amazon-reset")
async def api_amazon_reset():
count = await reset_downloaded_invoices()
return JSONResponse({"success": True, "count": count})
+11 -17
View File
@@ -11,12 +11,12 @@ from pypdf import PdfReader, PdfWriter
import qrcode
from qrcode.constants import ERROR_CORRECT_H
from app.database import get_settings, add_log_entry, get_import_email
from app.mail_processor import _connect_smtp, _build_forward_email, _send_with_log
from app.database import get_settings, add_log_entry
from app.mail_processor import _connect_smtp, _build_forward_email
logger = logging.getLogger(__name__)
SEPARATOR_QR_CONTENT = "BELEGIMPORT-TRENNUNG"
SEPARATOR_QR_CONTENT = "LEXOFFICE-TRENNUNG"
UPLOAD_DIR = Path(os.environ.get("UPLOAD_DIR", "/data/uploads"))
@@ -84,13 +84,12 @@ def split_pdf(pdf_path: str, separator_pages: list[int]) -> list[bytes]:
return documents
async def process_scanned_pdf(pdf_path: str, progress_callback=None, beleg_type: str = "eingang") -> dict:
"""Full pipeline: detect separators, split, send each document via email."""
async def process_scanned_pdf(pdf_path: str, progress_callback=None) -> dict:
"""Full pipeline: detect separators, split, send each document to LexOffice."""
settings = await get_settings()
import_email = get_import_email(settings, beleg_type)
if not settings.get("smtp_server") or not import_email:
return {"error": "SMTP oder Import-Email nicht konfiguriert", "total_pages": 0, "documents": 0, "sent": 0, "errors": 1}
if not settings.get("smtp_server") or not settings.get("lexoffice_email"):
return {"error": "SMTP oder LexOffice-Email nicht konfiguriert", "total_pages": 0, "documents": 0, "sent": 0, "errors": 1}
# Step 1: Detect separator pages (CPU-bound, run in thread)
if progress_callback:
@@ -117,7 +116,7 @@ async def process_scanned_pdf(pdf_path: str, progress_callback=None, beleg_type:
if not documents:
return {"error": "Keine Dokumente nach dem Splitting gefunden", "total_pages": total_pages, "documents": 0, "sent": 0, "errors": 1}
# Step 3: Send each document via email
# Step 3: Send each document to LexOffice
if progress_callback:
progress_callback("status", 0, 0, f"{len(documents)} Dokument(e) erkannt, starte Versand...")
@@ -136,12 +135,12 @@ async def process_scanned_pdf(pdf_path: str, progress_callback=None, beleg_type:
filename = f"Scan_Dokument_{i + 1}.pdf"
msg = _build_forward_email(
from_addr=settings["smtp_username"],
to_addr=import_email,
to_addr=settings["lexoffice_email"],
original_subject=f"Scan-Upload Dokument {i + 1}/{len(documents)}",
original_from="Scan-Upload",
attachments=[(filename, doc_bytes)],
)
smtp_log = _send_with_log(smtp_conn, msg)
smtp_conn.send_message(msg)
sent += 1
await add_log_entry(
@@ -149,9 +148,6 @@ async def process_scanned_pdf(pdf_path: str, progress_callback=None, beleg_type:
email_from="Scan-Upload",
attachments_count=1,
status="success",
sent_to=import_email,
smtp_log=smtp_log,
beleg_type=beleg_type,
)
except Exception as e:
@@ -163,8 +159,6 @@ async def process_scanned_pdf(pdf_path: str, progress_callback=None, beleg_type:
attachments_count=1,
status="error",
error_message=str(e),
sent_to=import_email,
beleg_type=beleg_type,
)
except Exception as e:
@@ -223,7 +217,7 @@ def generate_separator_pdf() -> bytes:
# Title text
_centered_textbox(page, 120, "TRENNSEITE", 36, (0, 0, 0))
_centered_textbox(page, 170, "Belegimport", 16, (0.4, 0.4, 0.4))
_centered_textbox(page, 170, "LexOffice Belegimport", 16, (0.4, 0.4, 0.4))
# Insert QR code image centered
qr_bytes = io.BytesIO()
-24
View File
@@ -5,10 +5,6 @@ from apscheduler.triggers.interval import IntervalTrigger
from app.mail_processor import process_mailbox
from app.smb_processor import process_smb_share
from app.ftp_processor import process_ftp
from app.amazon_processor import process_amazon
from app.amazon_api import process_amazon_api
from app.database import get_settings
logger = logging.getLogger(__name__)
@@ -25,7 +21,6 @@ async def _run_processor():
return
_is_processing = True
try:
# Email and SMB first - these are fast and must not be blocked by Amazon
logger.info("Starte automatische Email-Verarbeitung...")
result = await process_mailbox()
logger.info(f"Email-Verarbeitung abgeschlossen: {result}")
@@ -33,25 +28,6 @@ async def _run_processor():
logger.info("Starte automatische SMB-Verarbeitung...")
smb_result = await process_smb_share()
logger.info(f"SMB-Verarbeitung abgeschlossen: {smb_result}")
logger.info("Starte automatische FTP-Verarbeitung...")
ftp_result = await process_ftp()
logger.info(f"FTP-Verarbeitung abgeschlossen: {ftp_result}")
# Amazon separately with timeout - must not block next scheduler runs
logger.info("Starte automatische Amazon-Verarbeitung...")
try:
settings = await get_settings()
amazon_mode = settings.get("amazon_mode", "browser")
if amazon_mode == "api":
amazon_result = await asyncio.wait_for(process_amazon_api(), timeout=300)
else:
amazon_result = await asyncio.wait_for(process_amazon(), timeout=300)
logger.info(f"Amazon-Verarbeitung abgeschlossen: {amazon_result}")
except asyncio.TimeoutError:
logger.error("Amazon-Verarbeitung nach 5 Minuten abgebrochen (Timeout)")
except Exception as e:
logger.error(f"Fehler bei Amazon-Verarbeitung: {e}")
except Exception as e:
logger.error(f"Fehler bei automatischer Verarbeitung: {e}")
finally:
+110 -180
View File
@@ -5,8 +5,8 @@ import tempfile
import smbclient
from app.database import get_settings, add_log_entry, get_import_email
from app.mail_processor import _connect_smtp, _build_forward_email, _send_with_log
from app.database import get_settings, add_log_entry
from app.mail_processor import _connect_smtp, _build_forward_email
from app.scanner import detect_separator_pages, split_pdf
logger = logging.getLogger(__name__)
@@ -66,14 +66,8 @@ def _list_pdf_files(source_path: str) -> list[str]:
def _read_smb_file(filepath: str) -> bytes:
"""Read a file from SMB share into memory.
share_access="rwd" allows concurrent read/write/delete operations on the same
file, which is required because smbclient keeps the underlying SMB connection
in a pool. Without this, subsequent rename/delete on the file fails with
STATUS_ACCESS_DENIED until the session is closed.
"""
with smbclient.open_file(filepath, mode="rb", share_access="rwd") as f:
"""Read a file from SMB share into memory."""
with smbclient.open_file(filepath, mode="rb") as f:
return f.read()
@@ -99,7 +93,7 @@ def _move_smb_file(source: str, dest_dir: str, filename: str):
def _list_smb_folders_recursive(
base_path: str, max_depth: int = 5, _current_depth: int = 0, _prefix: str = ""
base_path: str, max_depth: int = 3, _current_depth: int = 0, _prefix: str = ""
) -> list[str]:
"""Recursively list folders on the SMB share, returning relative paths."""
folders = []
@@ -120,119 +114,6 @@ def _list_smb_folders_recursive(
return folders
async def _process_smb_folder(
smtp_conn, settings: dict, base_path: str,
source_rel: str, processed_rel: str,
import_email: str, beleg_type: str, mode: str,
) -> dict:
"""Process one SMB folder pair. Returns counts dict."""
smtp_from = settings.get("smtp_username", "")
processed = 0
skipped = 0
errors = 0
source_path = _smb_unc_path(base_path, source_rel)
processed_path = _smb_unc_path(base_path, processed_rel)
await asyncio.to_thread(_ensure_smb_folder, processed_path)
pdf_files = await asyncio.to_thread(_list_pdf_files, source_path)
if not pdf_files:
logger.info(f"Keine PDF-Dateien im SMB-Ordner '{source_rel}' ({beleg_type})")
return {"processed": 0, "skipped": 0, "errors": 0}
logger.info(f"{len(pdf_files)} PDF-Datei(en) im SMB-Ordner '{source_rel}' ({beleg_type})")
for filename in pdf_files:
file_path = _smb_unc_path(source_path, filename)
try:
pdf_data = await asyncio.to_thread(_read_smb_file, file_path)
if mode == "separator":
with tempfile.NamedTemporaryFile(suffix=".pdf", delete=False) as tmp:
tmp.write(pdf_data)
tmp_path = tmp.name
try:
separator_pages = await asyncio.to_thread(
detect_separator_pages, tmp_path, None
)
documents = await asyncio.to_thread(
split_pdf, tmp_path, separator_pages
)
finally:
os.unlink(tmp_path)
if not documents:
skipped += 1
continue
smtp_log_parts = []
for i, doc_bytes in enumerate(documents):
doc_filename = f"{os.path.splitext(filename)[0]}_Teil_{i + 1}.pdf"
subject = f"SMB-Import: {filename} (Dokument {i + 1}/{len(documents)})"
msg = _build_forward_email(
from_addr=smtp_from,
to_addr=import_email,
original_subject=subject,
original_from="SMB-Import",
attachments=[(doc_filename, doc_bytes)],
)
smtp_log_parts.append(_send_with_log(smtp_conn, msg))
await add_log_entry(
email_subject=f"SMB: {filename}",
email_from="SMB-Import",
attachments_count=len(documents),
status="success",
sent_to=import_email,
smtp_log="\n---\n".join(smtp_log_parts),
beleg_type=beleg_type,
)
logger.info(
f"SMB verarbeitet ({beleg_type}): {filename} -> {len(documents)} Dokument(e)"
)
else:
msg = _build_forward_email(
from_addr=smtp_from,
to_addr=import_email,
original_subject=f"SMB-Import: {filename}",
original_from="SMB-Import",
attachments=[(filename, pdf_data)],
)
smtp_log = _send_with_log(smtp_conn, msg)
await add_log_entry(
email_subject=f"SMB: {filename}",
email_from="SMB-Import",
attachments_count=1,
status="success",
sent_to=import_email,
smtp_log=smtp_log,
beleg_type=beleg_type,
)
logger.info(f"SMB verarbeitet ({beleg_type}): {filename}")
await asyncio.to_thread(_move_smb_file, file_path, processed_path, filename)
processed += 1
except Exception as e:
errors += 1
logger.error(f"Fehler bei SMB-Datei {filename}: {e}")
try:
await add_log_entry(
email_subject=f"SMB: {filename}",
email_from="SMB-Import",
attachments_count=0,
status="error",
error_message=str(e),
beleg_type=beleg_type,
)
except Exception:
pass
return {"processed": processed, "skipped": skipped, "errors": errors}
async def process_smb_share() -> dict:
"""Process PDF files from SMB share - main pipeline."""
settings = await get_settings()
@@ -243,43 +124,113 @@ async def process_smb_share() -> dict:
if not settings.get("smb_server") or not settings.get("smb_share"):
return {"processed": 0, "skipped": 0, "errors": 0, "error": "SMB nicht konfiguriert"}
import_email_eingang = get_import_email(settings, "eingang")
if not import_email_eingang:
return {"processed": 0, "skipped": 0, "errors": 0, "error": "Import-Email nicht konfiguriert"}
if not settings.get("lexoffice_email"):
return {"processed": 0, "skipped": 0, "errors": 0, "error": "LexOffice-Email nicht konfiguriert"}
mode = settings.get("smb_mode", "forward")
total = {"processed": 0, "skipped": 0, "errors": 0}
smtp_from = settings.get("smtp_username", "")
lexoffice_email = settings["lexoffice_email"]
processed = 0
skipped = 0
errors = 0
smtp_conn = None
try:
base_path = await asyncio.to_thread(_smb_register_session, settings)
source_path = _smb_unc_path(base_path, settings.get("smb_source_path", ""))
processed_path = _smb_unc_path(base_path, settings.get("smb_processed_path", "Verarbeitet"))
await asyncio.to_thread(_ensure_smb_folder, processed_path)
pdf_files = await asyncio.to_thread(_list_pdf_files, source_path)
if not pdf_files:
logger.info("Keine PDF-Dateien im SMB-Ordner gefunden")
return {"processed": 0, "skipped": 0, "errors": 0}
logger.info(f"{len(pdf_files)} PDF-Datei(en) im SMB-Ordner gefunden")
smtp_conn = _connect_smtp(settings)
# Eingangsbelege
source = settings.get("smb_source_path", "")
processed_rel = settings.get("smb_processed_path", "Verarbeitet")
result = await _process_smb_folder(
smtp_conn, settings, base_path,
source, processed_rel,
import_email_eingang, "eingang", mode,
)
for k in total:
total[k] += result[k]
for filename in pdf_files:
file_path = _smb_unc_path(source_path, filename)
try:
pdf_data = await asyncio.to_thread(_read_smb_file, file_path)
# Ausgangsbelege (optional)
import_email_ausgang = get_import_email(settings, "ausgang")
source_ausgang = settings.get("smb_source_path_ausgang", "")
processed_ausgang = settings.get("smb_processed_path_ausgang", "")
if import_email_ausgang and source_ausgang:
if not processed_ausgang:
processed_ausgang = source_ausgang + "/Verarbeitet"
result = await _process_smb_folder(
smtp_conn, settings, base_path,
source_ausgang, processed_ausgang,
import_email_ausgang, "ausgang", mode,
)
for k in total:
total[k] += result[k]
if mode == "separator":
with tempfile.NamedTemporaryFile(suffix=".pdf", delete=False) as tmp:
tmp.write(pdf_data)
tmp_path = tmp.name
try:
separator_pages = await asyncio.to_thread(
detect_separator_pages, tmp_path, None
)
documents = await asyncio.to_thread(
split_pdf, tmp_path, separator_pages
)
finally:
os.unlink(tmp_path)
if not documents:
skipped += 1
continue
for i, doc_bytes in enumerate(documents):
doc_filename = f"{os.path.splitext(filename)[0]}_Teil_{i + 1}.pdf"
subject = f"SMB-Import: {filename} (Dokument {i + 1}/{len(documents)})"
msg = _build_forward_email(
from_addr=smtp_from,
to_addr=lexoffice_email,
original_subject=subject,
original_from="SMB-Import",
attachments=[(doc_filename, doc_bytes)],
)
smtp_conn.send_message(msg)
await add_log_entry(
email_subject=f"SMB: {filename}",
email_from="SMB-Import",
attachments_count=len(documents),
status="success",
)
logger.info(
f"SMB verarbeitet: {filename} -> {len(documents)} Dokument(e) "
f"({len(separator_pages)} Trennseite(n))"
)
else:
msg = _build_forward_email(
from_addr=smtp_from,
to_addr=lexoffice_email,
original_subject=f"SMB-Import: {filename}",
original_from="SMB-Import",
attachments=[(filename, pdf_data)],
)
smtp_conn.send_message(msg)
await add_log_entry(
email_subject=f"SMB: {filename}",
email_from="SMB-Import",
attachments_count=1,
status="success",
)
logger.info(f"SMB verarbeitet: {filename}")
await asyncio.to_thread(_move_smb_file, file_path, processed_path, filename)
processed += 1
except Exception as e:
errors += 1
logger.error(f"Fehler bei SMB-Datei {filename}: {e}")
try:
await add_log_entry(
email_subject=f"SMB: {filename}",
email_from="SMB-Import",
attachments_count=0,
status="error",
error_message=str(e),
)
except Exception:
pass
except Exception as e:
logger.error(f"SMB-Verbindungsfehler: {e}")
@@ -293,7 +244,7 @@ async def process_smb_share() -> dict:
)
except Exception:
pass
return {**total, "errors": total["errors"] + 1, "error": str(e)}
return {"processed": processed, "skipped": skipped, "errors": errors + 1, "error": str(e)}
finally:
if smtp_conn:
@@ -302,12 +253,12 @@ async def process_smb_share() -> dict:
except Exception:
pass
logger.info(f"SMB fertig: {total['processed']} verarbeitet, {total['skipped']} übersprungen, {total['errors']} Fehler")
return total
logger.info(f"SMB fertig: {processed} verarbeitet, {skipped} übersprungen, {errors} Fehler")
return {"processed": processed, "skipped": skipped, "errors": errors}
async def test_smb_connection() -> dict:
"""Test SMB connection and return TOP-LEVEL folders only (lazy loading)."""
"""Test SMB connection and return folder list."""
settings = await get_settings()
if not settings.get("smb_server") or not settings.get("smb_share"):
@@ -315,7 +266,7 @@ async def test_smb_connection() -> dict:
try:
base_path = await asyncio.to_thread(_smb_register_session, settings)
folders = await asyncio.to_thread(_list_smb_folders_recursive, base_path, 1)
folders = await asyncio.to_thread(_list_smb_folders_recursive, base_path, 3)
return {"success": True, "folders": sorted(folders)}
except Exception as e:
@@ -347,34 +298,13 @@ async def create_smb_folder(folder_path: str) -> dict:
async def list_smb_folders() -> dict:
"""Return TOP-LEVEL folder list from SMB share (lazy loading)."""
"""Return current folder list from SMB share."""
settings = await get_settings()
if not settings.get("smb_server") or not settings.get("smb_share"):
return {"folders": []}
try:
base_path = await asyncio.to_thread(_smb_register_session, settings)
folders = await asyncio.to_thread(_list_smb_folders_recursive, base_path, 1)
folders = await asyncio.to_thread(_list_smb_folders_recursive, base_path, 3)
return {"folders": sorted(folders)}
except Exception:
return {"folders": []}
async def list_smb_subfolders(parent_path: str) -> dict:
"""List direct subfolders of a path (one level deep, for lazy tree expansion)."""
settings = await get_settings()
if not settings.get("smb_server") or not settings.get("smb_share"):
return {"success": False, "error": "SMB nicht konfiguriert", "folders": []}
try:
base_path = await asyncio.to_thread(_smb_register_session, settings)
full_path = _smb_unc_path(base_path, parent_path) if parent_path else base_path
# max_depth=1 returns only direct children
rel_folders = await asyncio.to_thread(_list_smb_folders_recursive, full_path, 1)
# Prefix with parent_path so the frontend has full paths
if parent_path:
folders = [f"{parent_path}/{f}" for f in rel_folders]
else:
folders = rel_folders
return {"success": True, "folders": sorted(folders)}
except Exception as e:
logger.error(f"SMB-Subfolder-Liste fehlgeschlagen: {e}")
return {"success": False, "error": str(e), "folders": []}
-44
View File
@@ -200,11 +200,6 @@ main {
color: #856404;
}
.badge-info {
background: #d1ecf1;
color: #0c5460;
}
.badge-inactive {
background: rgba(255, 255, 255, 0.2);
color: rgba(255, 255, 255, 0.7);
@@ -241,16 +236,6 @@ main {
border: 1px solid #bee5eb;
}
/* Allow cards with tables to scroll horizontally */
.card-table {
overflow-x: auto;
}
/* Wider main container for pages with large tables */
.main-wide {
max-width: 95%;
}
table {
width: 100%;
border-collapse: collapse;
@@ -424,35 +409,6 @@ small.text-muted {
align-items: center;
}
/* Tree-View Toggle */
.folder-tree-toggle {
flex-shrink: 0;
background: none;
border: none;
cursor: pointer;
width: 1.5rem;
text-align: center;
color: var(--text-muted);
font-size: 0.75rem;
padding: 0.5rem 0;
user-select: none;
}
.folder-tree-toggle:hover {
color: var(--primary);
}
.folder-tree-toggle.empty {
cursor: default;
visibility: hidden;
}
.folder-tree-children {
margin-left: 1.5rem;
border-left: 1px dashed var(--border);
padding-left: 0.25rem;
}
.folder-item {
display: flex;
align-items: center;
+5 -6
View File
@@ -3,16 +3,15 @@
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Belegimport</title>
<title>LexOffice Belegimport</title>
<link rel="stylesheet" href="/static/style.css">
</head>
<body>
<nav>
<div class="nav-brand">Belegimport</div>
<div class="nav-brand">LexOffice Belegimport</div>
<div class="nav-links">
<a href="/" class="{% if active_page == 'scan' %}active{% endif %}">Scan-Upload</a>
<a href="/settings" class="{% if active_page == 'settings' %}active{% endif %}">Einstellungen</a>
<a href="/platforms" class="{% if active_page == 'platforms' %}active{% endif %}">Plattformen</a>
<a href="/" class="{% if active_page == 'settings' %}active{% endif %}">Einstellungen</a>
<a href="/scan" class="{% if active_page == 'scan' %}active{% endif %}">Scan-Upload</a>
<a href="/log" class="{% if active_page == 'log' %}active{% endif %}">Verarbeitungslog</a>
</div>
<div class="nav-status">
@@ -30,7 +29,7 @@
</div>
</nav>
<main class="{% if main_class is defined %}{{ main_class }}{% endif %}">
<main>
{% if message %}
<div class="alert alert-{{ message_type or 'info' }}">
{{ message }}
+2 -75
View File
@@ -1,30 +1,21 @@
{% extends "base.html" %}
{% set active_page = "log" %}
{% set main_class = "main-wide" %}
{% set message = None %}
{% block content %}
<div class="card card-table">
<div style="display:flex;align-items:center;justify-content:space-between;margin-bottom:1rem;padding-bottom:0.5rem;border-bottom:1px solid var(--border);">
<h2 style="margin:0;border:none;padding:0;">Verarbeitungslog</h2>
{% if logs %}
<button type="button" class="btn btn-secondary btn-small" onclick="clearLog()">Log leeren</button>
{% endif %}
</div>
<div class="card">
<h2>Verarbeitungslog</h2>
{% if logs %}
<table>
<thead>
<tr>
<th>ID</th>
<th>Zeitpunkt</th>
<th>Art</th>
<th>Betreff</th>
<th>Absender</th>
<th>Anhänge</th>
<th>Gesendet an</th>
<th>Status</th>
<th>Fehlermeldung</th>
<th>SMTP</th>
</tr>
</thead>
<tbody>
@@ -32,17 +23,9 @@
<tr class="{% if log.status == 'error' %}row-error{% endif %}">
<td>{{ log.id }}</td>
<td>{{ log.timestamp }}</td>
<td>
{% if log.get('beleg_type', 'eingang') == 'ausgang' %}
<span class="badge badge-warning">Ausgang</span>
{% else %}
<span class="badge badge-info">Eingang</span>
{% endif %}
</td>
<td>{{ log.email_subject or '-' }}</td>
<td>{{ log.email_from or '-' }}</td>
<td>{{ log.attachments_count }}</td>
<td>{{ log.sent_to or '-' }}</td>
<td>
{% if log.status == 'success' %}
<span class="badge badge-success">OK</span>
@@ -51,12 +34,6 @@
{% endif %}
</td>
<td>{{ log.error_message or '-' }}</td>
<td>
{% if log.smtp_log %}
<button type="button" class="btn btn-small btn-secondary" onclick="showSmtpLog({{ log.id }})">Anzeigen</button>
<script>window._smtpLogs = window._smtpLogs || {}; window._smtpLogs[{{ log.id }}] = {{ log.smtp_log | tojson }};</script>
{% else %}-{% endif %}
</td>
</tr>
{% endfor %}
</tbody>
@@ -65,54 +42,4 @@
<p class="text-muted">Noch keine Einträge vorhanden.</p>
{% endif %}
</div>
<!-- SMTP Log Modal -->
<div id="smtpModal" class="modal-overlay" style="display:none;" onclick="if(event.target===this)closeSmtpModal()">
<div class="modal" style="max-width:700px;">
<div class="modal-header">
<h3>SMTP-Protokoll</h3>
<button type="button" class="modal-close" onclick="closeSmtpModal()">&times;</button>
</div>
<div class="modal-body" style="padding:1.25rem;background:var(--bg,#f5f5f5);">
<pre id="smtpModalBody" style="margin:0;font-size:0.8rem;white-space:pre-wrap;word-break:break-all;"></pre>
</div>
</div>
</div>
<style>
.btn-small {
padding: 0.25rem 0.6rem;
font-size: 0.8rem;
}
</style>
<script>
function showSmtpLog(logId) {
const log = window._smtpLogs && window._smtpLogs[logId];
if (!log) return;
document.getElementById('smtpModalBody').textContent = log;
document.getElementById('smtpModal').style.display = 'flex';
}
function closeSmtpModal() {
document.getElementById('smtpModal').style.display = 'none';
}
document.addEventListener('keydown', function(e) {
if (e.key === 'Escape') closeSmtpModal();
});
async function clearLog() {
if (!confirm('Verarbeitungslog wirklich leeren?')) return;
try {
const resp = await fetch('/api/clear-log', {method: 'POST'});
const data = await resp.json();
if (data.success) {
location.reload();
}
} catch (e) {
alert('Fehler: ' + e.message);
}
}
</script>
{% endblock %}
-530
View File
@@ -1,530 +0,0 @@
{% extends "base.html" %}
{% set active_page = "platforms" %}
{% block content %}
{% if message %}
<div class="alert alert-{{ message_type or 'info' }}">{{ message }}</div>
{% endif %}
<div class="card">
<h2>Amazon Business - Einstellungen</h2>
<div class="form-grid">
<div class="form-group">
<label for="amazon_enabled">Status</label>
<select id="amazon_enabled" name="amazon_enabled">
<option value="false" {% if settings.get('amazon_enabled') != 'true' %}selected{% endif %}>Deaktiviert</option>
<option value="true" {% if settings.get('amazon_enabled') == 'true' %}selected{% endif %}>Aktiviert</option>
</select>
</div>
<div class="form-group">
<label for="amazon_mode">Abruf-Modus</label>
<select id="amazon_mode" name="amazon_mode" onchange="toggleAmazonMode()">
<option value="api" {% if settings.get('amazon_mode') == 'api' %}selected{% endif %}>API (empfohlen)</option>
<option value="browser" {% if settings.get('amazon_mode', 'browser') == 'browser' %}selected{% endif %}>Browser-Automation</option>
</select>
</div>
<div class="form-group">
<label for="amazon_domain">Amazon-Domain</label>
<select id="amazon_domain" name="amazon_domain">
<option value="amazon.de" {% if settings.get('amazon_domain') == 'amazon.de' %}selected{% endif %}>amazon.de</option>
<option value="amazon.at" {% if settings.get('amazon_domain') == 'amazon.at' %}selected{% endif %}>amazon.at</option>
<option value="amazon.com" {% if settings.get('amazon_domain') == 'amazon.com' %}selected{% endif %}>amazon.com</option>
</select>
</div>
<div class="form-group">
<label for="amazon_since_date">Rechnungen ab Datum</label>
<input type="date" id="amazon_since_date" name="amazon_since_date"
value="{{ settings.get('amazon_since_date', '') }}">
<small class="text-muted">Leer = letzte 30 Tage</small>
</div>
</div>
<!-- API Mode Fields -->
<div id="apiFields" style="{% if settings.get('amazon_mode', 'browser') != 'api' %}display:none;{% endif %}margin-top:1rem;">
<h3 style="font-size:1rem;margin-bottom:0.75rem;">API-Zugangsdaten (Amazon Business API)</h3>
<div class="form-grid">
<div class="form-group">
<label for="amazon_app_id">App-ID</label>
<input type="text" id="amazon_app_id" name="amazon_app_id"
value="{{ settings.get('amazon_app_id', '') }}"
placeholder="amzn1.sp.solution.xxxxx">
<small class="text-muted">Aus dem Solution Provider Portal</small>
</div>
<div class="form-group">
<label for="amazon_client_id">Client-ID (LWA)</label>
<input type="text" id="amazon_client_id" name="amazon_client_id"
value="{{ settings.get('amazon_client_id', '') }}"
placeholder="amzn1.application-oa2-client.xxxxx">
</div>
<div class="form-group">
<label for="amazon_client_secret">Client-Sicherheitsschluessel (LWA)</label>
<input type="password" id="amazon_client_secret" name="amazon_client_secret"
placeholder="{% if settings.get('amazon_client_secret') %}(gespeichert){% else %}Client Secret eingeben{% endif %}">
</div>
<div class="form-group" style="align-self:end;">
<small class="text-muted">
{% if settings.get('amazon_refresh_token') %}
Refresh-Token: gespeichert
{% else %}
Refresh-Token: fehlt - bitte autorisieren
{% endif %}
</small>
</div>
</div>
</div>
<!-- Browser Mode Fields -->
<div id="browserFields" style="{% if settings.get('amazon_mode', 'browser') != 'browser' %}display:none;{% endif %}margin-top:1rem;">
<h3 style="font-size:1rem;margin-bottom:0.75rem;">Browser-Zugangsdaten</h3>
<div class="form-grid">
<div class="form-group">
<label for="amazon_email">Amazon E-Mail</label>
<input type="email" id="amazon_email" name="amazon_email"
value="{{ settings.get('amazon_email', '') }}"
placeholder="email@example.com">
</div>
<div class="form-group">
<label for="amazon_password">Amazon Passwort</label>
<input type="password" id="amazon_password" name="amazon_password"
placeholder="{% if settings.get('amazon_password') %}(gespeichert){% else %}Passwort eingeben{% endif %}">
</div>
</div>
</div>
<div style="margin-top:1rem;">
{% if settings.get('amazon_last_sync') %}
<small class="text-muted">Letzter Abruf: {{ settings.get('amazon_last_sync') }}</small>
{% endif %}
</div>
<div class="form-actions" style="margin-top:1rem;">
<button type="button" class="btn btn-primary" onclick="saveAmazonSettings()">Einstellungen speichern</button>
</div>
<div id="settingsMsg" style="margin-top:0.75rem;"></div>
</div>
<div class="card">
<h2>Anmeldung &amp; Abruf</h2>
<div style="display:flex;align-items:center;gap:0.75rem;margin-bottom:1rem;">
<span>Status:</span>
<span id="sessionBadge" class="badge badge-inactive">Wird geprüft...</span>
</div>
<!-- API Mode Buttons -->
<div id="apiButtons" style="display:none;">
<div style="display:flex;gap:0.75rem;flex-wrap:wrap;align-items:center;">
<button type="button" id="btnOAuth" class="btn btn-primary" onclick="doOAuth()">Bei Amazon autorisieren</button>
<button type="button" id="btnProcessApi" class="btn btn-success" onclick="doProcess()" style="display:none;">Jetzt Rechnungen abrufen</button>
<button type="button" class="btn btn-secondary" onclick="doReset()">Importierte zurücksetzen</button>
</div>
<!-- Manual code exchange for local setups -->
<div id="oauthCodeBox" style="display:none;margin-top:1rem;">
<p class="text-muted" style="font-size:0.85rem;">
Nach der Autorisierung bei Amazon werden Sie zu einer Seite weitergeleitet die nicht laden wird.
Kopieren Sie den Wert von <code>spapi_oauth_code</code> aus der URL-Leiste und tragen ihn hier ein:
</p>
<div style="display:flex;gap:0.5rem;align-items:center;">
<input type="text" id="oauthCode" placeholder="spapi_oauth_code hier einfuegen..."
style="flex:1;padding:0.5rem;border:1px solid var(--border-color);border-radius:4px;background:var(--input-bg);color:var(--text-color);">
<button type="button" class="btn btn-primary" onclick="exchangeCode()">Token tauschen</button>
</div>
</div>
</div>
<!-- Browser Mode Buttons -->
<div id="browserButtons">
<div style="display:flex;gap:0.75rem;flex-wrap:wrap;">
<button type="button" id="btnLogin" class="btn btn-primary" onclick="doLogin()">Bei Amazon anmelden</button>
<button type="button" id="btnLogout" class="btn btn-secondary" onclick="doLogout()" style="display:none;">Session löschen</button>
<button type="button" id="btnProcess" class="btn btn-success" onclick="doProcess()" style="display:none;">Jetzt Rechnungen abrufen</button>
<button type="button" class="btn btn-secondary" onclick="doReset()">Importierte zurücksetzen</button>
</div>
</div>
<div id="processMsg" style="margin-top:0.75rem;"></div>
</div>
<!-- Interactive Browser Modal -->
<div id="browserModal" style="display:none;position:fixed;inset:0;background:rgba(0,0,0,0.7);z-index:1000;overflow:auto;">
<div style="max-width:1340px;margin:2rem auto;background:var(--card-bg);border-radius:8px;padding:1.5rem;position:relative;">
<div style="display:flex;justify-content:space-between;align-items:center;margin-bottom:1rem;">
<h3 style="margin:0;">Amazon Login - Interaktiver Browser</h3>
<button type="button" class="btn btn-secondary" onclick="closeBrowser()" style="padding:0.25rem 0.75rem;">Schließen</button>
</div>
<div id="browserInfo" class="alert alert-info" style="margin-bottom:1rem;">
Klicken und tippen Sie im Browser-Bild, um sich bei Amazon anzumelden. CAPTCHAs und 2FA können Sie direkt lösen.
</div>
<div style="position:relative;display:inline-block;border:1px solid var(--border-color);cursor:crosshair;">
<img id="browserImg" src="" alt="Browser" style="display:block;max-width:100%;height:auto;"
onclick="onBrowserClick(event)">
</div>
<div style="margin-top:1rem;display:flex;gap:0.5rem;align-items:center;flex-wrap:wrap;">
<input type="text" id="browserInput" placeholder="Text eingeben und Enter drücken..."
style="flex:1;min-width:200px;padding:0.5rem;border:1px solid var(--border-color);border-radius:4px;background:var(--input-bg);color:var(--text-color);"
onkeydown="onBrowserKeydown(event)">
<button type="button" class="btn btn-primary" onclick="sendBrowserText()">Senden</button>
<button type="button" class="btn btn-secondary" onclick="sendKey('Tab')">Tab</button>
<button type="button" class="btn btn-secondary" onclick="sendKey('Escape')">Esc</button>
<button type="button" class="btn btn-secondary" onclick="sendKey('Backspace')">&#9003;</button>
</div>
</div>
</div>
<script>
// --- Mode Toggle ---
function toggleAmazonMode() {
const mode = document.getElementById('amazon_mode').value;
document.getElementById('apiFields').style.display = mode === 'api' ? '' : 'none';
document.getElementById('browserFields').style.display = mode === 'browser' ? '' : 'none';
document.getElementById('apiButtons').style.display = mode === 'api' ? '' : 'none';
document.getElementById('browserButtons').style.display = mode === 'browser' ? '' : 'none';
checkSession();
}
// --- Settings ---
async function saveAmazonSettings() {
const btn = event.target;
btn.disabled = true;
btn.textContent = 'Speichert...';
const msgEl = document.getElementById('settingsMsg');
const data = {
amazon_enabled: document.getElementById('amazon_enabled').value,
amazon_mode: document.getElementById('amazon_mode').value,
amazon_domain: document.getElementById('amazon_domain').value,
amazon_email: document.getElementById('amazon_email').value,
amazon_password: document.getElementById('amazon_password').value,
amazon_since_date: document.getElementById('amazon_since_date').value,
amazon_app_id: document.getElementById('amazon_app_id').value,
amazon_client_id: document.getElementById('amazon_client_id').value,
amazon_client_secret: document.getElementById('amazon_client_secret').value,
};
try {
const resp = await fetch('/api/amazon-settings', {
method: 'POST',
headers: {'Content-Type': 'application/json'},
body: JSON.stringify(data),
});
const result = await resp.json();
if (result.success) {
msgEl.innerHTML = '<div class="alert alert-success">Einstellungen gespeichert</div>';
if (data.amazon_password) {
document.getElementById('amazon_password').value = '';
document.getElementById('amazon_password').placeholder = '(gespeichert)';
}
} else {
msgEl.innerHTML = '<div class="alert alert-error">' + escapeHtml(result.error || 'Fehler') + '</div>';
}
} catch (e) {
msgEl.innerHTML = '<div class="alert alert-error">Verbindungsfehler</div>';
} finally {
btn.disabled = false;
btn.textContent = 'Einstellungen speichern';
}
}
// --- Session Status ---
async function checkSession() {
const badge = document.getElementById('sessionBadge');
try {
const resp = await fetch('/api/amazon-status');
const data = await resp.json();
if (data.mode === 'api') {
// API mode
if (data.api_authorized) {
badge.className = 'badge badge-success';
badge.textContent = 'API autorisiert';
document.getElementById('btnOAuth').style.display = 'none';
document.getElementById('btnProcessApi').style.display = '';
} else if (data.api_configured) {
badge.className = 'badge badge-warning';
badge.textContent = 'Nicht autorisiert';
document.getElementById('btnOAuth').style.display = '';
document.getElementById('btnProcessApi').style.display = 'none';
} else {
badge.className = 'badge badge-inactive';
badge.textContent = 'API nicht konfiguriert';
document.getElementById('btnOAuth').style.display = 'none';
document.getElementById('btnProcessApi').style.display = 'none';
}
} else {
// Browser mode
if (data.login_active) {
badge.className = 'badge badge-warning';
badge.textContent = 'Login läuft...';
document.getElementById('btnLogout').style.display = 'none';
document.getElementById('btnProcess').style.display = 'none';
} else if (data.session_valid) {
badge.className = 'badge badge-success';
badge.textContent = 'Angemeldet';
document.getElementById('btnLogout').style.display = '';
document.getElementById('btnProcess').style.display = '';
} else {
badge.className = 'badge badge-inactive';
badge.textContent = 'Nicht angemeldet';
document.getElementById('btnLogout').style.display = 'none';
document.getElementById('btnProcess').style.display = 'none';
}
}
} catch (e) {
badge.className = 'badge badge-inactive';
badge.textContent = 'Unbekannt';
}
}
// --- OAuth Authorization (API mode) ---
async function doOAuth() {
const msgEl = document.getElementById('processMsg');
try {
const resp = await fetch('/api/amazon-oauth-url');
const data = await resp.json();
if (data.url) {
window.open(data.url, '_blank');
// Show code input box for manual exchange
document.getElementById('oauthCodeBox').style.display = '';
msgEl.innerHTML = '<div class="alert alert-info">Amazon-Autorisierungsseite wurde geöffnet. Nach der Autorisierung den Code aus der URL kopieren und unten eintragen.</div>';
} else {
msgEl.innerHTML = '<div class="alert alert-error">' + escapeHtml(data.error || 'Fehler') + '</div>';
}
} catch (e) {
msgEl.innerHTML = '<div class="alert alert-error">Verbindungsfehler</div>';
}
}
async function exchangeCode() {
const codeInput = document.getElementById('oauthCode');
let code = codeInput.value.trim();
const msgEl = document.getElementById('processMsg');
if (!code) {
msgEl.innerHTML = '<div class="alert alert-error">Bitte den Code eintragen</div>';
return;
}
// If user pasted the full URL, extract the code
if (code.includes('spapi_oauth_code=')) {
const url = new URL(code.startsWith('http') ? code : 'https://x.com?' + code);
code = url.searchParams.get('spapi_oauth_code') || code;
}
msgEl.innerHTML = '<div class="alert alert-info">Token wird getauscht...</div>';
try {
const resp = await fetch('/api/amazon-oauth-exchange', {
method: 'POST',
headers: {'Content-Type': 'application/json'},
body: JSON.stringify({code}),
});
const data = await resp.json();
if (data.success) {
msgEl.innerHTML = '<div class="alert alert-success">Erfolgreich autorisiert! Refresh-Token gespeichert.</div>';
document.getElementById('oauthCodeBox').style.display = 'none';
checkSession();
} else {
msgEl.innerHTML = '<div class="alert alert-error">' + escapeHtml(data.error || 'Token-Exchange fehlgeschlagen') + '</div>';
}
} catch (e) {
msgEl.innerHTML = '<div class="alert alert-error">Verbindungsfehler</div>';
}
}
// --- Interactive Browser Login ---
let screenshotInterval = null;
let loginPollInterval = null;
async function doLogin() {
const msgEl = document.getElementById('processMsg');
msgEl.innerHTML = '<div class="alert alert-info">Browser wird gestartet...</div>';
try {
await fetch('/api/amazon-login', {method: 'POST'});
// Open browser modal
document.getElementById('browserModal').style.display = '';
msgEl.innerHTML = '';
startScreenshotPolling();
startLoginStatePolling();
} catch (e) {
msgEl.innerHTML = '<div class="alert alert-error">Browser konnte nicht gestartet werden</div>';
}
}
function startScreenshotPolling() {
if (screenshotInterval) clearInterval(screenshotInterval);
refreshScreenshot();
screenshotInterval = setInterval(refreshScreenshot, 1500);
}
async function refreshScreenshot() {
try {
const resp = await fetch('/api/amazon-browser-screenshot');
if (resp.ok) {
const blob = await resp.blob();
document.getElementById('browserImg').src = URL.createObjectURL(blob);
}
} catch (e) {}
}
function startLoginStatePolling() {
if (loginPollInterval) clearInterval(loginPollInterval);
loginPollInterval = setInterval(async () => {
try {
const resp = await fetch('/api/amazon-login-state');
const data = await resp.json();
const info = document.getElementById('browserInfo');
if (data.status === 'logged_in') {
info.className = 'alert alert-success';
info.textContent = 'Erfolgreich angemeldet! Sie können das Fenster schließen.';
} else if (data.status === 'login_failed') {
info.className = 'alert alert-error';
info.textContent = data.message || 'Login fehlgeschlagen';
} else if (data.status === 'interactive') {
info.className = 'alert alert-info';
info.textContent = data.message || 'Bitte im Browser anmelden...';
}
} catch (e) {}
}, 2000);
}
async function onBrowserClick(event) {
const img = event.target;
const rect = img.getBoundingClientRect();
// Scale click coordinates to actual browser viewport (1280x800)
const scaleX = 1280 / img.clientWidth;
const scaleY = 800 / img.clientHeight;
const x = Math.round((event.clientX - rect.left) * scaleX);
const y = Math.round((event.clientY - rect.top) * scaleY);
try {
await fetch('/api/amazon-browser-click', {
method: 'POST',
headers: {'Content-Type': 'application/json'},
body: JSON.stringify({x, y}),
});
setTimeout(refreshScreenshot, 500);
} catch (e) {}
}
async function sendBrowserText() {
const input = document.getElementById('browserInput');
const text = input.value;
if (!text) return;
try {
await fetch('/api/amazon-browser-type', {
method: 'POST',
headers: {'Content-Type': 'application/json'},
body: JSON.stringify({text}),
});
input.value = '';
setTimeout(refreshScreenshot, 500);
} catch (e) {}
}
async function sendKey(key) {
try {
await fetch('/api/amazon-browser-key', {
method: 'POST',
headers: {'Content-Type': 'application/json'},
body: JSON.stringify({key}),
});
setTimeout(refreshScreenshot, 500);
} catch (e) {}
}
function onBrowserKeydown(event) {
if (event.key === 'Enter') {
event.preventDefault();
sendBrowserText();
// Also send Enter to the browser
sendKey('Enter');
}
}
async function closeBrowser() {
if (screenshotInterval) { clearInterval(screenshotInterval); screenshotInterval = null; }
if (loginPollInterval) { clearInterval(loginPollInterval); loginPollInterval = null; }
document.getElementById('browserModal').style.display = 'none';
try {
await fetch('/api/amazon-login-close', {method: 'POST'});
} catch (e) {}
checkSession();
}
// --- Logout ---
async function doLogout() {
if (!confirm('Amazon-Session wirklich löschen? Sie müssen sich danach neu anmelden.')) return;
const btn = document.getElementById('btnLogout');
btn.disabled = true;
try {
await fetch('/api/amazon-logout', {method: 'POST'});
document.getElementById('processMsg').innerHTML = '<div class="alert alert-info">Session gelöscht</div>';
} catch (e) {
document.getElementById('processMsg').innerHTML = '<div class="alert alert-error">Fehler</div>';
} finally {
btn.disabled = false;
checkSession();
}
}
// --- Process ---
async function doProcess() {
const btn = document.getElementById('btnProcess');
const msgEl = document.getElementById('processMsg');
btn.disabled = true;
btn.textContent = 'Rechnungen werden abgerufen...';
msgEl.innerHTML = '';
try {
const resp = await fetch('/api/amazon-process', {method: 'POST'});
const data = await resp.json();
if (data.error) {
msgEl.innerHTML = '<div class="alert alert-error">' + escapeHtml(data.error) + '</div>';
} else if (data.processed > 0 || data.errors > 0) {
let msg = data.processed + ' Rechnung(en) importiert';
if (data.skipped > 0) msg += ', ' + data.skipped + ' übersprungen';
if (data.errors > 0) msg += ', ' + data.errors + ' Fehler';
const cls = data.errors > 0 ? 'warning' : 'success';
msgEl.innerHTML = '<div class="alert alert-' + cls + '">' + escapeHtml(msg) + '</div>';
} else {
msgEl.innerHTML = '<div class="alert alert-info">Keine neuen Rechnungen gefunden</div>';
}
} catch (e) {
msgEl.innerHTML = '<div class="alert alert-error">Verbindungsfehler</div>';
} finally {
btn.disabled = false;
btn.textContent = 'Jetzt Rechnungen abrufen';
checkSession();
}
}
// --- Reset ---
async function doReset() {
if (!confirm('Alle als importiert markierten Rechnungen zurücksetzen? Beim nächsten Abruf werden alle Rechnungen erneut heruntergeladen und gesendet.')) return;
const msgEl = document.getElementById('processMsg');
try {
const resp = await fetch('/api/amazon-reset', {method: 'POST'});
const data = await resp.json();
if (data.success) {
msgEl.innerHTML = '<div class="alert alert-info">' + data.count + ' Bestellung(en) zurückgesetzt</div>';
} else {
msgEl.innerHTML = '<div class="alert alert-error">' + escapeHtml(data.error || 'Fehler') + '</div>';
}
} catch (e) {
msgEl.innerHTML = '<div class="alert alert-error">Verbindungsfehler</div>';
}
}
function escapeHtml(str) {
const d = document.createElement('div');
d.textContent = str;
return d.innerHTML;
}
// Initial setup
toggleAmazonMode();
checkSession();
</script>
{% endblock %}
+3 -10
View File
@@ -5,16 +5,9 @@
<div class="card">
<h2>Scan-Upload</h2>
<p class="text-muted" style="margin-bottom:1rem;">
Mehrseitige PDF hochladen. Trennseiten mit QR-Code werden automatisch erkannt und die einzelnen Dokumente gesendet.
Mehrseitige PDF hochladen. Trennseiten mit QR-Code werden automatisch erkannt und die einzelnen Dokumente an LexOffice gesendet.
</p>
<!-- Belegart -->
<div style="margin-bottom:1rem;display:flex;gap:1rem;align-items:center;">
<label style="margin:0;font-weight:600;">Belegart:</label>
<label style="margin:0;cursor:pointer;"><input type="radio" name="beleg_type" value="eingang" checked> Eingangsbeleg (Einkauf)</label>
<label style="margin:0;cursor:pointer;"><input type="radio" name="beleg_type" value="ausgang"> Ausgangsbeleg (Verkauf/Gutschrift)</label>
</div>
<!-- Upload Zone -->
<div id="uploadZone" class="upload-zone" onclick="document.getElementById('fileInput').click()">
<div class="upload-icon">&#128196;</div>
@@ -182,7 +175,7 @@ async function startProcessing(uploadId) {
const resp = await fetch('/api/scan-process', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ upload_id: uploadId, beleg_type: document.querySelector('input[name="beleg_type"]:checked').value }),
body: JSON.stringify({ upload_id: uploadId }),
});
if (!resp.ok) {
@@ -241,7 +234,7 @@ function listenProgress(uploadId) {
if (result.separator_pages > 0) {
msg += ' (' + result.separator_pages + ' Trennseite(n))';
}
msg += ', ' + result.sent + ' gesendet';
msg += ', ' + result.sent + ' an LexOffice gesendet';
if (result.errors > 0) {
msg += ', ' + result.errors + ' Fehler';
showResult(msg, 'warning');
+33 -625
View File
@@ -77,14 +77,13 @@
</div>
<div class="card">
<h2>Import - Eingangsbelege</h2>
<h2>LexOffice & Ordner</h2>
<div class="form-grid">
<div class="form-group form-group-wide">
<label for="import_email_eingang">Import-Email Eingangsbelege</label>
<input type="email" id="import_email_eingang" name="import_email_eingang"
value="{{ settings.get('import_email_eingang', '') or settings.get('import_email', '') }}" placeholder="eingang@buchhaltung.example.com">
<label for="lexoffice_email">LexOffice Import-Emailadresse</label>
<input type="email" id="lexoffice_email" name="lexoffice_email"
value="{{ settings.get('lexoffice_email', '') }}" placeholder="import-xyz@lexoffice.de">
</div>
<input type="hidden" id="import_email" name="import_email" value="{{ settings.get('import_email_eingang', '') or settings.get('import_email', '') }}">
<div class="form-group">
<label for="source_folder">Eingangsordner (IMAP)</label>
<div class="input-with-btn">
@@ -102,37 +101,9 @@
</div>
</div>
</div>
</div>
<div class="card">
<h2>Import - Ausgangsbelege <small style="font-weight:normal;color:var(--text-muted);">(optional)</small></h2>
<div class="form-grid">
<div class="form-group form-group-wide">
<label for="import_email_ausgang">Import-Email Ausgangsbelege</label>
<input type="email" id="import_email_ausgang" name="import_email_ausgang"
value="{{ settings.get('import_email_ausgang', '') }}" placeholder="ausgang@buchhaltung.example.com">
<small class="text-muted">Leer lassen wenn keine Ausgangsbelege importiert werden sollen</small>
</div>
<div class="form-group">
<label for="source_folder_ausgang">Eingangsordner Ausgangsbelege (IMAP)</label>
<div class="input-with-btn">
<input type="text" id="source_folder_ausgang" name="source_folder_ausgang"
value="{{ settings.get('source_folder_ausgang', '') }}" placeholder="Ausgangsrechnungen">
<button type="button" class="btn btn-icon" onclick="openFolderPicker('source_folder_ausgang')" title="Ordner auswählen">&#128193;</button>
</div>
</div>
<div class="form-group">
<label for="processed_folder_ausgang">Verarbeitet-Ordner Ausgangsbelege (IMAP)</label>
<div class="input-with-btn">
<input type="text" id="processed_folder_ausgang" name="processed_folder_ausgang"
value="{{ settings.get('processed_folder_ausgang', '') }}" placeholder="Ausgangsrechnungen/Verarbeitet">
<button type="button" class="btn btn-icon" onclick="openFolderPicker('processed_folder_ausgang')" title="Ordner auswählen">&#128193;</button>
</div>
</div>
</div>
<div class="form-actions">
<button type="button" class="btn btn-secondary" onclick="testEmail()">
<span class="btn-text">Test-Email senden</span>
<span class="btn-text">Test-Email an LexOffice senden</span>
<span class="btn-spinner" style="display:none;">Sende...</span>
</button>
</div>
@@ -187,7 +158,7 @@
value="{{ settings.get('smb_share', '') }}" placeholder="Scans">
</div>
<div class="form-group">
<label for="smb_source_path">Quellordner Eingangsbelege</label>
<label for="smb_source_path">Quellordner</label>
<div class="input-with-btn">
<input type="text" id="smb_source_path" name="smb_source_path"
value="{{ settings.get('smb_source_path', '') }}" placeholder="(Wurzel der Freigabe)">
@@ -195,29 +166,13 @@
</div>
</div>
<div class="form-group">
<label for="smb_processed_path">Verarbeitet-Ordner Eingangsbelege</label>
<label for="smb_processed_path">Verarbeitet-Ordner</label>
<div class="input-with-btn">
<input type="text" id="smb_processed_path" name="smb_processed_path"
value="{{ settings.get('smb_processed_path', 'Verarbeitet') }}" placeholder="Verarbeitet">
<button type="button" class="btn btn-icon" onclick="openSmbFolderPicker('smb_processed_path')" title="Ordner auswählen">&#128193;</button>
</div>
</div>
<div class="form-group">
<label for="smb_source_path_ausgang">Quellordner Ausgangsbelege</label>
<div class="input-with-btn">
<input type="text" id="smb_source_path_ausgang" name="smb_source_path_ausgang"
value="{{ settings.get('smb_source_path_ausgang', '') }}" placeholder="(optional)">
<button type="button" class="btn btn-icon" onclick="openSmbFolderPicker('smb_source_path_ausgang')" title="Ordner auswählen">&#128193;</button>
</div>
</div>
<div class="form-group">
<label for="smb_processed_path_ausgang">Verarbeitet-Ordner Ausgangsbelege</label>
<div class="input-with-btn">
<input type="text" id="smb_processed_path_ausgang" name="smb_processed_path_ausgang"
value="{{ settings.get('smb_processed_path_ausgang', '') }}" placeholder="(optional)">
<button type="button" class="btn btn-icon" onclick="openSmbFolderPicker('smb_processed_path_ausgang')" title="Ordner auswählen">&#128193;</button>
</div>
</div>
</div>
<div class="form-actions">
<button type="button" class="btn btn-secondary" onclick="testSmb()">
@@ -227,97 +182,6 @@
</div>
</div>
<div class="card">
<h2>FTP / SFTP-Server</h2>
<div class="form-grid">
<div class="form-group">
<label for="ftp_enabled">FTP-Import</label>
<select id="ftp_enabled" name="ftp_enabled">
<option value="true" {% if settings.get('ftp_enabled') == 'true' %}selected{% endif %}>Aktiviert</option>
<option value="false" {% if settings.get('ftp_enabled') != 'true' %}selected{% endif %}>Deaktiviert</option>
</select>
</div>
<div class="form-group">
<label for="ftp_protocol">Protokoll</label>
<select id="ftp_protocol" name="ftp_protocol" onchange="updateFtpDefaultPort()">
<option value="sftp" {% if settings.get('ftp_protocol', 'sftp') == 'sftp' %}selected{% endif %}>SFTP (SSH, verschluesselt)</option>
<option value="ftp" {% if settings.get('ftp_protocol') == 'ftp' %}selected{% endif %}>FTP (passiv, unverschluesselt)</option>
</select>
</div>
<div class="form-group">
<label for="ftp_mode">Verarbeitungsmodus</label>
<select id="ftp_mode" name="ftp_mode">
<option value="forward" {% if settings.get('ftp_mode', 'forward') == 'forward' %}selected{% endif %}>Direkt weiterleiten</option>
<option value="separator" {% if settings.get('ftp_mode') == 'separator' %}selected{% endif %}>Trennseiten-Erkennung</option>
</select>
<small class="text-muted">Direkt: jede PDF als ein Beleg. Trennseiten: QR-Splitting wie bei Scan-Upload.</small>
</div>
<div class="form-group">
<label for="ftp_server">Server</label>
<input type="text" id="ftp_server" name="ftp_server"
value="{{ settings.get('ftp_server', '') }}" placeholder="ftp.example.com oder 192.168.1.100">
</div>
<div class="form-group">
<label for="ftp_port">Port</label>
<input type="number" id="ftp_port" name="ftp_port"
value="{{ settings.get('ftp_port', '22') }}">
<small class="text-muted">SFTP=22, FTP=21</small>
</div>
<div class="form-group">
<label for="ftp_username">Benutzername</label>
<input type="text" id="ftp_username" name="ftp_username"
value="{{ settings.get('ftp_username', '') }}" placeholder="user">
</div>
<div class="form-group">
<label for="ftp_password">Passwort</label>
<input type="password" id="ftp_password" name="ftp_password"
placeholder="{% if settings.get('ftp_password') %}(gespeichert){% else %}Passwort eingeben{% endif %}">
</div>
<div class="form-group">
<label for="ftp_source_path">Quellordner Eingangsbelege</label>
<div class="input-with-btn">
<input type="text" id="ftp_source_path" name="ftp_source_path"
value="{{ settings.get('ftp_source_path', '') }}" placeholder="(Wurzel)">
<button type="button" class="btn btn-icon" onclick="openFtpFolderPicker('ftp_source_path')" title="Ordner auswählen">&#128193;</button>
</div>
</div>
<div class="form-group">
<label for="ftp_processed_path">Verarbeitet-Ordner Eingangsbelege</label>
<div class="input-with-btn">
<input type="text" id="ftp_processed_path" name="ftp_processed_path"
value="{{ settings.get('ftp_processed_path', 'Verarbeitet') }}" placeholder="Verarbeitet">
<button type="button" class="btn btn-icon" onclick="openFtpFolderPicker('ftp_processed_path')" title="Ordner auswählen">&#128193;</button>
</div>
</div>
<div class="form-group">
<label for="ftp_source_path_ausgang">Quellordner Ausgangsbelege</label>
<div class="input-with-btn">
<input type="text" id="ftp_source_path_ausgang" name="ftp_source_path_ausgang"
value="{{ settings.get('ftp_source_path_ausgang', '') }}" placeholder="(optional)">
<button type="button" class="btn btn-icon" onclick="openFtpFolderPicker('ftp_source_path_ausgang')" title="Ordner auswählen">&#128193;</button>
</div>
</div>
<div class="form-group">
<label for="ftp_processed_path_ausgang">Verarbeitet-Ordner Ausgangsbelege</label>
<div class="input-with-btn">
<input type="text" id="ftp_processed_path_ausgang" name="ftp_processed_path_ausgang"
value="{{ settings.get('ftp_processed_path_ausgang', '') }}" placeholder="(optional)">
<button type="button" class="btn btn-icon" onclick="openFtpFolderPicker('ftp_processed_path_ausgang')" title="Ordner auswählen">&#128193;</button>
</div>
</div>
</div>
<div class="form-actions">
<button type="button" class="btn btn-secondary" onclick="testFtp()">
<span class="btn-text">Verbindung testen &amp; Ordner laden</span>
<span class="btn-spinner" style="display:none;">Verbinde...</span>
</button>
<button type="button" class="btn btn-secondary" onclick="processFtp()">
<span class="btn-text">Jetzt abrufen</span>
<span class="btn-spinner" style="display:none;">Verarbeite...</span>
</button>
</div>
</div>
<div class="card">
<h2>Zeitplan</h2>
<div class="form-grid">
@@ -342,20 +206,6 @@
</div>
</div>
<div class="card">
<h2>Debug</h2>
<div class="form-grid">
<div class="form-group">
<label for="debug_save_amazon_pdfs">Amazon-PDFs zwischenspeichern</label>
<select id="debug_save_amazon_pdfs" name="debug_save_amazon_pdfs">
<option value="false" {% if settings.get('debug_save_amazon_pdfs') != 'true' %}selected{% endif %}>Aus</option>
<option value="true" {% if settings.get('debug_save_amazon_pdfs') == 'true' %}selected{% endif %}>An</option>
</select>
<small class="text-muted">Speichert heruntergeladene Amazon-Rechnungen in /data/uploads/amazon_invoices/</small>
</div>
</div>
</div>
<div class="form-actions-main">
<button type="submit" class="btn btn-primary">Einstellungen speichern</button>
<button type="button" class="btn btn-success" onclick="manualProcess()">
@@ -378,7 +228,6 @@
<th>Betreff</th>
<th>Absender</th>
<th>Anhänge</th>
<th>Art</th>
<th>Status</th>
</tr>
</thead>
@@ -389,13 +238,6 @@
<td>{{ log.email_subject or '-' }}</td>
<td>{{ log.email_from or '-' }}</td>
<td>{{ log.attachments_count }}</td>
<td>
{% if log.get('beleg_type', 'eingang') == 'ausgang' %}
<span class="badge badge-warning">Ausgang</span>
{% else %}
<span class="badge badge-info">Eingang</span>
{% endif %}
</td>
<td>
{% if log.status == 'success' %}
<span class="badge badge-success">OK</span>
@@ -432,10 +274,7 @@
<div class="modal">
<div class="modal-header">
<h3 id="smbFolderModalTitle">SMB-Ordner auswählen</h3>
<div style="display:flex;gap:0.5rem;align-items:center;">
<button type="button" class="btn btn-secondary btn-sm" onclick="reloadSmbFolders()" title="Ordner-Liste neu laden">&#8634; Neu laden</button>
<button type="button" class="modal-close" onclick="closeSmbFolderModal()">&times;</button>
</div>
<button type="button" class="modal-close" onclick="closeSmbFolderModal()">&times;</button>
</div>
<div class="modal-body">
<div id="smbFolderList" class="folder-list"></div>
@@ -447,176 +286,12 @@
</div>
</div>
<div id="ftpFolderModal" class="modal-overlay" style="display:none;" onclick="closeFtpFolderModal(event)">
<div class="modal">
<div class="modal-header">
<h3 id="ftpFolderModalTitle">FTP-Ordner auswählen</h3>
<div style="display:flex;gap:0.5rem;align-items:center;">
<button type="button" class="btn btn-secondary btn-sm" onclick="reloadFtpFolders()" title="Ordner-Liste neu laden">&#8634; Neu laden</button>
<button type="button" class="modal-close" onclick="closeFtpFolderModal()">&times;</button>
</div>
</div>
<div class="modal-body">
<div id="ftpFolderList" class="folder-list"></div>
<div id="ftpFolderLoading" class="text-muted" style="display:none;padding:1rem;">
Verbinde und lade Ordner...
</div>
<div id="ftpFolderError" class="alert alert-error" style="display:none;margin:1rem;"></div>
</div>
</div>
</div>
<script>
let cachedFolders = null;
let cachedDelimiter = '.';
let folderTargetField = null;
let createOpenFor = null;
// --- Generic Tree-View Helpers (with Lazy Loading) ---
// Used by both SMB and FTP folder pickers
const expandedTreeNodes = { smb: new Set(), ftp: new Set() };
const loadedTreeNodes = { smb: new Set(['']), ftp: new Set([''])}; // root is always loaded
const loadingTreeNodes = { smb: new Set(), ftp: new Set() };
function buildFolderTree(paths) {
// Build nested tree from flat paths like ["A", "A/B", "A/B/C", "X"]
const root = { name: '', path: '', children: {} };
paths.forEach(p => {
if (!p) return;
const parts = p.split('/');
let node = root;
let curPath = '';
for (const part of parts) {
curPath = curPath ? curPath + '/' + part : part;
if (!node.children[part]) {
node.children[part] = { name: part, path: curPath, children: {} };
}
node = node.children[part];
}
});
return root;
}
function renderFolderTree(node, ns, currentValue, selectFn, addBtnFn, depth) {
let html = '';
const childKeys = Object.keys(node.children).sort((a, b) => a.localeCompare(b));
childKeys.forEach(key => {
const child = node.children[key];
const hasChildren = Object.keys(child.children).length > 0;
const isLoaded = loadedTreeNodes[ns].has(child.path);
const isLoading = loadingTreeNodes[ns].has(child.path);
const isExpanded = expandedTreeNodes[ns].has(child.path);
const isSelected = child.path === currentValue;
const escapedPath = child.path.replace(/'/g, "\\'");
// Always show toggle button - we don't know yet if there are children until loaded
const arrow = isLoading ? '&#8987;' // ⌛
: isExpanded ? '&#9660;' // ▼
: '&#9654;'; // ▶
html += '<div class="folder-row">';
html += '<button type="button" class="folder-tree-toggle" onclick="toggleTreeNode(\'' + ns + '\',\'' + escapedPath + '\')">' + arrow + '</button>';
html += '<button type="button" class="folder-item' + (isSelected ? ' selected' : '') + '" onclick="' + selectFn + '(\'' + escapedPath + '\')">';
html += '<span class="folder-icon">&#128193;</span> ' + esc(child.name);
if (isSelected) html += ' <span class="badge badge-success">ausgewählt</span>';
html += '</button>';
html += '<button type="button" class="folder-add-btn" onclick="event.stopPropagation(); ' + addBtnFn + '(\'' + escapedPath + '\')" title="Unterordner erstellen">&#128193;+</button>';
html += '</div>';
html += '<div id="' + ns + '-create-row-' + CSS.escape(child.path) + '" class="create-inline" style="display:none;"></div>';
if (isExpanded && (hasChildren || isLoaded)) {
html += '<div class="folder-tree-children">';
if (hasChildren) {
html += renderFolderTree(child, ns, currentValue, selectFn, addBtnFn, depth + 1);
} else if (isLoaded) {
html += '<p class="text-muted" style="padding:0.25rem 0.5rem;font-size:0.85rem;">(keine Unterordner)</p>';
}
html += '</div>';
}
});
return html;
}
async function toggleTreeNode(ns, path) {
if (expandedTreeNodes[ns].has(path)) {
// Collapse
expandedTreeNodes[ns].delete(path);
if (ns === 'smb') showSmbFolderModal(smbFolderTargetField);
else if (ns === 'ftp') showFtpFolderModal(ftpFolderTargetField);
return;
}
// Expand - load subfolders if not loaded yet
if (!loadedTreeNodes[ns].has(path)) {
loadingTreeNodes[ns].add(path);
if (ns === 'smb') showSmbFolderModal(smbFolderTargetField);
else if (ns === 'ftp') showFtpFolderModal(ftpFolderTargetField);
try {
const endpoint = '/api/list-' + ns + '-subfolders?path=' + encodeURIComponent(path);
const resp = await fetch(endpoint);
const data = await resp.json();
if (data.success && data.folders) {
// Merge new folders into cache
const cacheKey = ns === 'smb' ? 'cachedSmbFolders' : 'cachedFtpFolders';
const existingSet = new Set(window[cacheKey] || []);
data.folders.forEach(f => existingSet.add(f));
window[cacheKey] = Array.from(existingSet).sort();
loadedTreeNodes[ns].add(path);
}
} catch (e) {
console.error('Lazy load failed:', e);
} finally {
loadingTreeNodes[ns].delete(path);
}
}
expandedTreeNodes[ns].add(path);
if (ns === 'smb') showSmbFolderModal(smbFolderTargetField);
else if (ns === 'ftp') showFtpFolderModal(ftpFolderTargetField);
}
function expandTreePathsForValue(ns, value) {
if (!value) return;
const parts = value.split('/');
let cur = '';
for (let i = 0; i < parts.length - 1; i++) {
cur = cur ? cur + '/' + parts[i] : parts[i];
expandedTreeNodes[ns].add(cur);
}
}
async function preloadTreePathForValue(ns, value) {
// Lazy-load all ancestor paths so the tree displays the selected value
if (!value) return;
const parts = value.split('/');
let cur = '';
const cacheKey = ns === 'smb' ? 'cachedSmbFolders' : 'cachedFtpFolders';
for (let i = 0; i < parts.length - 1; i++) {
cur = cur ? cur + '/' + parts[i] : parts[i];
if (loadedTreeNodes[ns].has(cur)) continue;
try {
const endpoint = '/api/list-' + ns + '-subfolders?path=' + encodeURIComponent(cur);
const resp = await fetch(endpoint);
const data = await resp.json();
if (data.success && data.folders) {
const existingSet = new Set(window[cacheKey] || []);
data.folders.forEach(f => existingSet.add(f));
window[cacheKey] = Array.from(existingSet).sort();
loadedTreeNodes[ns].add(cur);
}
} catch (e) { /* ignore */ }
}
}
function resetTreeState(ns) {
expandedTreeNodes[ns].clear();
loadedTreeNodes[ns].clear();
loadedTreeNodes[ns].add('');
loadingTreeNodes[ns].clear();
}
function showAlert(message, type) {
const el = document.getElementById('jsAlert');
el.textContent = message;
@@ -668,11 +343,8 @@ async function testEmail() {
const resp = await fetch('/api/test-email', { method: 'POST', body: getFormData() });
const data = await resp.json();
if (data.success) {
const eingang = document.getElementById('import_email_eingang').value;
const ausgang = document.getElementById('import_email_ausgang').value;
let targets = eingang;
if (ausgang) targets += ' + ' + ausgang;
showAlert('Test-Email erfolgreich an ' + targets + ' gesendet! Einstellungen gespeichert.', 'success');
const addr = document.getElementById('lexoffice_email').value;
showAlert('Test-Email erfolgreich an ' + addr + ' gesendet! Einstellungen gespeichert.', 'success');
} else {
showAlert('Test-Email fehlgeschlagen: ' + data.error, 'error');
}
@@ -775,10 +447,8 @@ function showFolderModal(targetField) {
const currentValue = folderTargetField ? document.getElementById(folderTargetField).value : '';
let html = '<div class="folder-picker-fields">';
html += '<button type="button" class="folder-field-btn ' + (folderTargetField === 'source_folder' ? 'active' : '') + '" onclick="switchFolderTarget(\'source_folder\')">Eingang Quelle: <strong>' + esc(document.getElementById('source_folder').value) + '</strong></button>';
html += '<button type="button" class="folder-field-btn ' + (folderTargetField === 'processed_folder' ? 'active' : '') + '" onclick="switchFolderTarget(\'processed_folder\')">Eingang Verarbeitet: <strong>' + esc(document.getElementById('processed_folder').value) + '</strong></button>';
html += '<button type="button" class="folder-field-btn ' + (folderTargetField === 'source_folder_ausgang' ? 'active' : '') + '" onclick="switchFolderTarget(\'source_folder_ausgang\')">Ausgang Quelle: <strong>' + esc(document.getElementById('source_folder_ausgang').value || '(nicht gesetzt)') + '</strong></button>';
html += '<button type="button" class="folder-field-btn ' + (folderTargetField === 'processed_folder_ausgang' ? 'active' : '') + '" onclick="switchFolderTarget(\'processed_folder_ausgang\')">Ausgang Verarbeitet: <strong>' + esc(document.getElementById('processed_folder_ausgang').value || '(nicht gesetzt)') + '</strong></button>';
html += '<button type="button" class="folder-field-btn ' + (folderTargetField === 'source_folder' ? 'active' : '') + '" onclick="switchFolderTarget(\'source_folder\')">Eingangsordner: <strong>' + esc(document.getElementById('source_folder').value) + '</strong></button>';
html += '<button type="button" class="folder-field-btn ' + (folderTargetField === 'processed_folder' ? 'active' : '') + '" onclick="switchFolderTarget(\'processed_folder\')">Verarbeitet-Ordner: <strong>' + esc(document.getElementById('processed_folder').value) + '</strong></button>';
html += '</div>';
html += '<div class="folder-items">';
if (cachedFolders && cachedFolders.length > 0) {
@@ -940,19 +610,12 @@ async function testSmb() {
function openSmbFolderPicker(targetField) {
smbFolderTargetField = targetField;
// Always reload when opening to ensure fresh state
cachedSmbFolders = null;
window.cachedSmbFolders = null;
resetTreeState('smb');
showSmbFolderModalLoading(targetField);
fetchSmbFolders(targetField);
}
function reloadSmbFolders() {
cachedSmbFolders = null;
resetTreeState('smb');
showSmbFolderModalLoading(smbFolderTargetField);
fetchSmbFolders(smbFolderTargetField);
if (cachedSmbFolders) {
showSmbFolderModal(targetField);
} else {
showSmbFolderModalLoading(targetField);
fetchSmbFolders(targetField);
}
}
async function fetchSmbFolders(targetField) {
@@ -961,12 +624,6 @@ async function fetchSmbFolders(targetField) {
const data = await resp.json();
if (data.success) {
cachedSmbFolders = data.folders;
window.cachedSmbFolders = data.folders;
// Preload ancestor paths if a value is already selected
if (targetField) {
const currentValue = document.getElementById(targetField).value;
await preloadTreePathForValue('smb', currentValue);
}
showSmbFolderModal(targetField);
} else {
showSmbFolderModalError('SMB-Verbindung fehlgeschlagen: ' + data.error);
@@ -994,10 +651,8 @@ function showSmbFolderModal(targetField) {
const currentValue = smbFolderTargetField ? document.getElementById(smbFolderTargetField).value : '';
let html = '<div class="folder-picker-fields">';
html += '<button type="button" class="folder-field-btn ' + (smbFolderTargetField === 'smb_source_path' ? 'active' : '') + '" onclick="switchSmbFolderTarget(\'smb_source_path\')">Eingang Quelle: <strong>' + esc(document.getElementById('smb_source_path').value || '(Wurzel)') + '</strong></button>';
html += '<button type="button" class="folder-field-btn ' + (smbFolderTargetField === 'smb_processed_path' ? 'active' : '') + '" onclick="switchSmbFolderTarget(\'smb_processed_path\')">Eingang Verarbeitet: <strong>' + esc(document.getElementById('smb_processed_path').value) + '</strong></button>';
html += '<button type="button" class="folder-field-btn ' + (smbFolderTargetField === 'smb_source_path_ausgang' ? 'active' : '') + '" onclick="switchSmbFolderTarget(\'smb_source_path_ausgang\')">Ausgang Quelle: <strong>' + esc(document.getElementById('smb_source_path_ausgang').value || '(nicht gesetzt)') + '</strong></button>';
html += '<button type="button" class="folder-field-btn ' + (smbFolderTargetField === 'smb_processed_path_ausgang' ? 'active' : '') + '" onclick="switchSmbFolderTarget(\'smb_processed_path_ausgang\')">Ausgang Verarbeitet: <strong>' + esc(document.getElementById('smb_processed_path_ausgang').value || '(nicht gesetzt)') + '</strong></button>';
html += '<button type="button" class="folder-field-btn ' + (smbFolderTargetField === 'smb_source_path' ? 'active' : '') + '" onclick="switchSmbFolderTarget(\'smb_source_path\')">Quellordner: <strong>' + esc(document.getElementById('smb_source_path').value || '(Wurzel)') + '</strong></button>';
html += '<button type="button" class="folder-field-btn ' + (smbFolderTargetField === 'smb_processed_path' ? 'active' : '') + '" onclick="switchSmbFolderTarget(\'smb_processed_path\')">Verarbeitet-Ordner: <strong>' + esc(document.getElementById('smb_processed_path').value) + '</strong></button>';
html += '</div>';
html += '<div class="folder-items">';
@@ -1012,12 +667,19 @@ function showSmbFolderModal(targetField) {
html += '</div>';
}
const smbFolders = window.cachedSmbFolders || cachedSmbFolders;
if (smbFolders && smbFolders.length > 0) {
// Auto-expand path to current value, then render tree
expandTreePathsForValue('smb', currentValue);
const tree = buildFolderTree(smbFolders);
html += renderFolderTree(tree, 'smb', currentValue, 'selectSmbFolder', 'toggleSmbCreateInput', 0);
if (cachedSmbFolders && cachedSmbFolders.length > 0) {
cachedSmbFolders.forEach(folder => {
const isSelected = folder === currentValue;
const escapedFolder = folder.replace(/'/g, "\\'");
html += '<div class="folder-row">';
html += '<button type="button" class="folder-item' + (isSelected ? ' selected' : '') + '" onclick="selectSmbFolder(\'' + escapedFolder + '\')">';
html += '<span class="folder-icon">&#128193;</span> ' + esc(folder);
if (isSelected) html += ' <span class="badge badge-success">ausgewählt</span>';
html += '</button>';
html += '<button type="button" class="folder-add-btn" onclick="event.stopPropagation(); toggleSmbCreateInput(\'' + escapedFolder + '\')" title="Unterordner erstellen">&#128193;+</button>';
html += '</div>';
html += '<div id="smb-create-row-' + CSS.escape(folder) + '" class="create-inline" style="display:none;"></div>';
});
} else {
html += '<p class="text-muted" style="padding:0.5rem;">Keine Ordner gefunden.</p>';
}
@@ -1139,259 +801,5 @@ function closeSmbFolderModal(event) {
document.getElementById('smbFolderModal').style.display = 'none';
cachedSmbFolders = null;
}
// --- FTP / SFTP ---
let cachedFtpFolders = null;
let ftpFolderTargetField = null;
function updateFtpDefaultPort() {
const proto = document.getElementById('ftp_protocol').value;
const portInput = document.getElementById('ftp_port');
const current = portInput.value;
if (proto === 'sftp' && (current === '21' || !current)) {
portInput.value = '22';
} else if (proto === 'ftp' && (current === '22' || !current)) {
portInput.value = '21';
}
}
async function testFtp() {
const btn = event.currentTarget;
setButtonLoading(btn, true);
try {
const resp = await fetch('/api/test-ftp', { method: 'POST', body: getFormData() });
const data = await resp.json();
if (data.success) {
cachedFtpFolders = data.folders;
showAlert('FTP-Verbindung erfolgreich! Einstellungen gespeichert.', 'success');
showFtpFolderModal(null);
} else {
showAlert('FTP-Verbindung fehlgeschlagen: ' + data.error, 'error');
}
} catch (e) {
showAlert('Fehler: ' + e.message, 'error');
} finally {
setButtonLoading(btn, false);
}
}
async function processFtp() {
const btn = event.currentTarget;
setButtonLoading(btn, true);
try {
const resp = await fetch('/api/process-ftp', { method: 'POST', body: getFormData() });
const data = await resp.json();
if (data.error) {
showAlert('FTP-Fehler: ' + data.error, 'error');
} else {
showAlert(`FTP-Abruf fertig: ${data.processed} verarbeitet, ${data.skipped || 0} uebersprungen, ${data.errors} Fehler`, data.errors > 0 ? 'warning' : 'success');
}
} catch (e) {
showAlert('Fehler: ' + e.message, 'error');
} finally {
setButtonLoading(btn, false);
}
}
function openFtpFolderPicker(targetField) {
ftpFolderTargetField = targetField;
// Always reload when opening to ensure fresh state
cachedFtpFolders = null;
window.cachedFtpFolders = null;
resetTreeState('ftp');
showFtpFolderModalLoading(targetField);
fetchFtpFolders(targetField);
}
function reloadFtpFolders() {
cachedFtpFolders = null;
resetTreeState('ftp');
showFtpFolderModalLoading(ftpFolderTargetField);
fetchFtpFolders(ftpFolderTargetField);
}
async function fetchFtpFolders(targetField) {
try {
const resp = await fetch('/api/test-ftp', { method: 'POST', body: getFormData() });
const data = await resp.json();
if (data.success) {
cachedFtpFolders = data.folders;
window.cachedFtpFolders = data.folders;
if (targetField) {
const currentValue = document.getElementById(targetField).value;
await preloadTreePathForValue('ftp', currentValue);
}
showFtpFolderModal(targetField);
} else {
showFtpFolderModalError('FTP-Verbindung fehlgeschlagen: ' + data.error);
}
} catch (e) {
showFtpFolderModalError('Fehler: ' + e.message);
}
}
function showFtpFolderModalLoading(targetField) {
const modal = document.getElementById('ftpFolderModal');
document.getElementById('ftpFolderList').innerHTML = '';
document.getElementById('ftpFolderLoading').style.display = '';
document.getElementById('ftpFolderError').style.display = 'none';
modal.style.display = 'flex';
}
function showFtpFolderModal(targetField) {
if (targetField) ftpFolderTargetField = targetField;
const modal = document.getElementById('ftpFolderModal');
document.getElementById('ftpFolderLoading').style.display = 'none';
document.getElementById('ftpFolderError').style.display = 'none';
const list = document.getElementById('ftpFolderList');
const currentValue = ftpFolderTargetField ? document.getElementById(ftpFolderTargetField).value : '';
let html = '<div class="folder-picker-fields">';
html += '<button type="button" class="folder-field-btn ' + (ftpFolderTargetField === 'ftp_source_path' ? 'active' : '') + '" onclick="switchFtpFolderTarget(\'ftp_source_path\')">Eingang Quelle: <strong>' + esc(document.getElementById('ftp_source_path').value || '(Wurzel)') + '</strong></button>';
html += '<button type="button" class="folder-field-btn ' + (ftpFolderTargetField === 'ftp_processed_path' ? 'active' : '') + '" onclick="switchFtpFolderTarget(\'ftp_processed_path\')">Eingang Verarbeitet: <strong>' + esc(document.getElementById('ftp_processed_path').value) + '</strong></button>';
html += '<button type="button" class="folder-field-btn ' + (ftpFolderTargetField === 'ftp_source_path_ausgang' ? 'active' : '') + '" onclick="switchFtpFolderTarget(\'ftp_source_path_ausgang\')">Ausgang Quelle: <strong>' + esc(document.getElementById('ftp_source_path_ausgang').value || '(nicht gesetzt)') + '</strong></button>';
html += '<button type="button" class="folder-field-btn ' + (ftpFolderTargetField === 'ftp_processed_path_ausgang' ? 'active' : '') + '" onclick="switchFtpFolderTarget(\'ftp_processed_path_ausgang\')">Ausgang Verarbeitet: <strong>' + esc(document.getElementById('ftp_processed_path_ausgang').value || '(nicht gesetzt)') + '</strong></button>';
html += '</div>';
html += '<div class="folder-items">';
// Root option for source path
if (ftpFolderTargetField === 'ftp_source_path') {
const isRoot = currentValue === '';
html += '<div class="folder-row">';
html += '<button type="button" class="folder-item' + (isRoot ? ' selected' : '') + '" onclick="selectFtpFolder(\'\')">';
html += '<span class="folder-icon">&#128193;</span> (Wurzel des Servers)';
if (isRoot) html += ' <span class="badge badge-success">ausgewählt</span>';
html += '</button>';
html += '</div>';
}
const ftpFolders = window.cachedFtpFolders || cachedFtpFolders;
if (ftpFolders && ftpFolders.length > 0) {
// Auto-expand path to current value, then render tree
expandTreePathsForValue('ftp', currentValue);
const tree = buildFolderTree(ftpFolders);
html += renderFolderTree(tree, 'ftp', currentValue, 'selectFtpFolder', 'toggleFtpCreateInput', 0);
} else {
html += '<p class="text-muted" style="padding:0.5rem;">Keine Ordner gefunden.</p>';
}
// Create new root folder option
html += '<div class="folder-row" style="border-top:1px solid var(--border);margin-top:0.5rem;padding-top:0.5rem;">';
html += '<button type="button" class="folder-item" onclick="event.stopPropagation(); toggleFtpCreateInput(\'\')" style="color:var(--primary);">';
html += '<span class="folder-icon">&#128193;+</span> Neuen Ordner erstellen';
html += '</button>';
html += '</div>';
html += '<div id="ftp-create-row-root" class="create-inline" style="display:none;"></div>';
html += '</div>';
list.innerHTML = html;
modal.style.display = 'flex';
}
function switchFtpFolderTarget(field) {
ftpFolderTargetField = field;
showFtpFolderModal(field);
}
function selectFtpFolder(folder) {
if (ftpFolderTargetField) {
document.getElementById(ftpFolderTargetField).value = folder;
}
showFtpFolderModal(ftpFolderTargetField);
}
function showFtpFolderModalError(msg) {
document.getElementById('ftpFolderLoading').style.display = 'none';
document.getElementById('ftpFolderError').textContent = msg;
document.getElementById('ftpFolderError').style.display = '';
}
function toggleFtpCreateInput(parentFolder) {
document.querySelectorAll('.create-inline[id^="ftp-create-row"]').forEach(el => {
const rowId = parentFolder === '' ? 'ftp-create-row-root' : 'ftp-create-row-' + CSS.escape(parentFolder);
if (el.id !== rowId) {
el.style.display = 'none';
el.innerHTML = '';
}
});
const rowId = parentFolder === '' ? 'ftp-create-row-root' : 'ftp-create-row-' + CSS.escape(parentFolder);
const row = document.getElementById(rowId);
if (!row) return;
if (row.style.display !== 'none') {
row.style.display = 'none';
row.innerHTML = '';
return;
}
const prefix = parentFolder ? parentFolder + '/' : '';
row.innerHTML =
'<div class="create-folder-inline">' +
'<span class="create-folder-prefix">' + esc(prefix) + '</span>' +
'<input type="text" class="create-folder-input" id="newFtpSubfolderInput" placeholder="Name" autofocus>' +
'<button type="button" class="btn btn-primary btn-sm" onclick="doCreateFtpFolder(\'' + parentFolder.replace(/'/g, "\\'") + '\')">OK</button>' +
'<button type="button" class="btn btn-secondary btn-sm" onclick="toggleFtpCreateInput(\'' + parentFolder.replace(/'/g, "\\'") + '\')">Abbrechen</button>' +
'</div>' +
'<div id="ftpCreateError" class="text-error" style="display:none;"></div>';
row.style.display = '';
const input = document.getElementById('newFtpSubfolderInput');
if (input) {
input.focus();
input.addEventListener('keydown', function(e) {
if (e.key === 'Enter') { e.preventDefault(); doCreateFtpFolder(parentFolder); }
if (e.key === 'Escape') { toggleFtpCreateInput(parentFolder); }
});
}
}
async function doCreateFtpFolder(parentFolder) {
const input = document.getElementById('newFtpSubfolderInput');
const errorEl = document.getElementById('ftpCreateError');
if (!input) return;
const subName = input.value.trim();
if (!subName) {
errorEl.textContent = 'Bitte einen Namen eingeben.';
errorEl.style.display = '';
return;
}
const fullName = parentFolder ? parentFolder + '/' + subName : subName;
errorEl.style.display = 'none';
input.disabled = true;
try {
const resp = await fetch('/api/create-ftp-folder', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ folder_name: fullName })
});
const data = await resp.json();
if (data.success) {
cachedFtpFolders = data.folders;
if (ftpFolderTargetField) {
document.getElementById(ftpFolderTargetField).value = fullName;
}
showFtpFolderModal(ftpFolderTargetField);
} else {
errorEl.textContent = data.error;
errorEl.style.display = '';
input.disabled = false;
}
} catch (e) {
errorEl.textContent = e.message;
errorEl.style.display = '';
input.disabled = false;
}
}
function closeFtpFolderModal(event) {
if (event && event.target !== document.getElementById('ftpFolderModal')) return;
document.getElementById('ftpFolderModal').style.display = 'none';
cachedFtpFolders = null;
}
</script>
{% endblock %}
Binary file not shown.
+1 -3
View File
@@ -1,7 +1,7 @@
services:
belegimport:
build: .
container_name: belegimport
container_name: lexoffice-belegimport
ports:
- "8081:8000"
volumes:
@@ -9,6 +9,4 @@ services:
environment:
- DB_PATH=/data/belegimport.db
- TZ=Europe/Berlin
- LOG_LEVEL=DEBUG
- OAUTH_REDIRECT_BASE=https://hacker-net.de
restart: unless-stopped
-4
View File
@@ -12,7 +12,3 @@ PyMuPDF==1.25.3
qrcode==8.0
sse-starlette==2.2.1
smbprotocol==1.14.0
paramiko==3.5.0
playwright==1.49.1
playwright-stealth==2.0.2
httpx==0.28.1