Skip to content

★ Enterprise Edition — not included in the Community Edition. Compare editions

Monitoring Configuration

Metric endpoints, the shipped alert rules, and Alertmanager routing.


Mailyte's monitoring has two layers: Prometheus scraping every worker's /metrics endpoint (plus the MySQL and Redis exporters), and the monitoring worker service, which actively health-checks services, publishes backup-freshness gauges, auto-heals containers through the docker-socket proxy, and emits health.* webhooks.

Metric Endpoints

Every worker exposes GET /metrics (Prometheus text format, generated by shared/metrics.py). Metric names are prefixed with the service name (api_http_requests_total, monitoring_backup_age_seconds, …) — see Prometheus Metrics for the complete list.

Target Port (container) Notes
api 8080
webhooks 8081
rate_limiter 8082
archiver 8083
encryption 8084
monitoring 8085 also /metrics/prometheus (aggregation)
analytics 8085 scrape target fixed 2026-08-30 (was host port 8087)
tracking 8086
dashboard 8088
delivery_optimizer 8088
templates 8089 scrape job added 2026-08-30
queue_manager 8090
rag 8090 scrape target fixed 2026-08-30 (was host port 8091)
url_protection 8090 scrape job added 2026-08-30
oauth 8091 scrape job added 2026-08-30
storage_usage 8092 metric prefix is storage_
jmap / migration / autoconfig / caldav 8098–8101
rspamd 11334 native Rspamd metrics
mysql-exporter 9104 prom/mysqld-exporter
redis-exporter 9121 oliver006/redis_exporter

[!WARNING] There is no Postfix or Dovecot exporter and no node-exporter in the stack. The postfix-exporter:9154 / dovecot-exporter:9166 scrape jobs that pointed at those nonexistent containers are commented out in prometheus.yml as of 2026-08-30, as are the alert rules on postfix_* and node_filesystem_* series. Postfix delivery visibility comes from the log ingestor writing mail_logs instead. Add the exporters yourself (and re-enable the commented jobs/rules) if you want queue-depth and host metrics in Prometheus.

The Monitoring Worker

worker/monitoring (port 8085) is a producer, not a scraper replacement:

  • Service health checks — HTTP and protocol-level (SMTP/IMAP) probes; state transitions emit health.service.down / health.service.up webhooks.
  • Auto-heal — restarts unhealthy containers through the scoped socket proxy (DOCKER_HOST=tcp://docker-proxy:2375), emitting health.recovery.
  • Backup monitor — publishes monitoring_backup_age_seconds, monitoring_backup_last_status, monitoring_backup_last_size_bytes, monitoring_backup_unencrypted_runs (labels host, backup_type) from the backup_history table. BACKUP_EXPECTED_HOSTS declares which hosts must report.
  • System alerts — CPU/memory/bounce-rate/SLA thresholds emit health.system.alert webhooks.

Shipped Alert Rules

Rules live in monitoring/prometheus/rules/ (mounted read-only into Prometheus).

mail_alerts.yml

Active rules (every live expression matches a real series as of the 2026-08-30 cleanup):

Alert Expression For Severity
ServiceDown up == 0 2m critical
HighErrorRate sum by (job) (rate({__name__=~".+_http_errors_total"}[5m])) / sum by (job) (rate({__name__=~".+_http_requests_total"}[5m])) > 0.05 5m warning
DatabaseConnectionPoolExhausted mysql threads_connected / max_connections > 0.80 5m warning
SlowQueries rate(mysql_global_status_slow_queries[5m]) > 10 10m warning
RedisMemoryHigh used/max > 0.80 10m warning

[!NOTE] HighErrorRate sweeps the service-prefixed counters via a __name__ regex, because every service metric carries its service prefix (e.g. api_http_errors_total) and the bare name matches nothing. The rules whose series have no producer at all — MailQueueBackup/Critical and the bounce-rate pair (postfix_*, no Postfix exporter), DiskSpaceWarning/Critical (node_filesystem_*, no node-exporter), KafkaConsumerLag (kafka-exporter not deployed), and BruteForceDetected/DLPViolation (auth_failures_total / dlp_violations_total never exported) — are commented out in mail_alerts.yml with dated notes; restore each alongside its producer.

backup_alerts.yml

These are written against real series and all function:

Alert Fires when Severity
NoRecentFullBackup monitoring_backup_age_seconds{backup_type="full"} > 93600 (26 h) critical
NoRecentIncrementalBackup > 7200 (2 h) warning
BackupMonitoringGone absent(monitoring_backup_age_seconds) for 15m critical
BackupLastRunFailed last run incomplete on a host that has succeeded before warning
BackupNeverRun age at the 10-year sentinel for 1 h critical
UnencryptedBackupsPresent monitoring_backup_unencrypted_runs > 0 warning
ArchiveSpoolNotDraining / ArchiveSpoolBacklogCritical archiver_archive_spool_depth > 0 (30m) / > 5000 (15m) warning / critical
ArchiveStoreFailing rate(archiver_archive_store_failures_total[15m]) > 0 critical

They alert on absence deliberately — a backup that quietly stopped running produces no errors, only rising age.

Alertmanager

monitoring/alertmanager/alertmanager.yml routes everything to the webhooks service (POST /alertmanager, added 2026-08-30), which forwards each alert through the global webhook dispatcher as signed system.alert.firing / system.alert.resolved events to WEBHOOK_URL:

YAML
route:
  receiver: 'webhook-notifications'
  group_by: ['alertname', 'severity']
  group_wait: 30s
  group_interval: 5m
  repeat_interval: 4h
  routes:
    - receiver: 'critical-alerts'
      match: { severity: critical }
      group_wait: 10s
      group_interval: 1m
      repeat_interval: 1h

receivers:
  - name: 'webhook-notifications'
    webhook_configs:
      - url: 'http://webhooks:8081/alertmanager'
        send_resolved: true
  - name: 'critical-alerts'
    webhook_configs:
      - url: 'http://webhooks:8081/alertmanager'
        send_resolved: true

Inhibit rules: critical suppresses warning for the same alertname; ServiceDown suppresses other alerts for the same job.

[!WARNING] If your downstream of the webhooks service is email through Mailyte itself, you won't get alerts when Mailyte is down. Wire at least the critical route to an external channel (PagerDuty, Slack, etc.) by editing the receiver.

Grafana

Provisioned from monitoring/grafana/: the Prometheus datasource (http://prometheus:9090) and two dashboards in the Mailyte folder — mail_overview and security_dashboard. Login: GRAFANA_ADMIN_USER (default admin) / GRAFANA_ADMIN_PASSWORD; sign-up disabled. In production, Grafana is published on 127.0.0.1:3000 and via Traefik at grafana.$DOMAIN.

Key Health Checks That Don't Need Prometheus

Bash
./start.sh health                             # per-service health summary
docker exec postfix postqueue -p | tail -1    # queue depth
curl -s localhost:8083/health                 # API health (host port)
docker compose ps                             # container states