Skip to content

ADR 0003: Production resource, logging, and retention policy

  • Status: Accepted
  • Date: 2026-09-28
  • Scope: OVH host resource limits, bounded logs, and safe disk retention

Context

AndreaWeb shares a 1-vCPU, 3.7-GiB VPS with the host proxy and system services. The six long-running Compose services need explicit resource ceilings and bounded logs. Persistent volumes, cutover backups, release sources, and rollback images also compete for limited disk. Cleanup must preserve the data and artifacts needed to recover production.

The September 28 final verification recorded 39G total disk, 15G used, 24G free (39%), 13% inode use, 3.7 GiB RAM with 2.6 GiB available, and 1 GiB active swap with about 7 MiB used. Disk improved from about 39G total, 26G used, and 14G free (67%) before cleanup. Docker images changed from 6.228 GB total/3.057 GB reclaimable to 5.126 GB total/1.955 GB reclaimable; 2.016 GB build cache was pruned to zero after verification. All six application/data services were healthy after about one hour, with no OOM kills and zero restarts. These observations are a light-load snapshot, not a sizing guarantee.

Decision

  1. Keep the existing container memory and PID limits: web 384 MiB/128, storefront 512 MiB/128, medusa 1,024 MiB/256, postgres 512 MiB/128, redis 256 MiB/64, garage 256 MiB/128, and garage-init 128 MiB/64. Redis remains noeviction with maxmemory=0; no CPU caps or arbitrary Node heap flags are introduced without workload evidence.
  2. Bound all seven services' Docker json-file logs to 10 MiB per file and three files. Keep host journald at SystemMaxUse=512M, SystemKeepFree=2G, and MaxRetentionSec=21day; retain the normal daily logrotate timer and Nginx rotation behavior.
  3. Keep the active runtime configuration, deployed web/storefront source, backend rollback source, previous runtime configuration, and known-good rollback images. Remove only release directories/images confirmed obsolete and only build cache confirmed idle and reclaimable. Do not automate volume deletion.
  4. Verify off-server backup copies before deleting temporary VPS copies. A missing on-VPS backup is an expected warning when custody is off-server only. These manual copies do not replace scheduled backups or a restore rehearsal.
  5. Use the read-only production audit script to inspect host capacity, container status/configuration, log rotation, backup age, and release-pointer validity. It does not send alerts. Recheck the cloud-init logrotate configuration after package upgrades.
  6. Keep the root filesystem passno=0 cloud-image entry unchanged after review; do not add a swap passno or treat swap as application memory capacity. The 1-GiB swapfile remains mode 0600, active and persistent, with vm.swappiness=10.

Safe retention procedure

Before removing recovery or rollback material:

  1. Confirm the exact target path, ownership, and what it contains without exposing private contents in logs, Git, screenshots, or chat.
  2. Verify backup integrity and the retained off-server copy independently.
  3. Check the active deployment marker, current release pointer, known-good image identities, and whether the target is part of the rollback set.
  4. Check Docker's actual reclaimable status and volume identities. Preserve named volumes; never use down --volumes or volume pruning as a cleanup shortcut.
  5. Remove only artifacts confirmed outside the recovery/rollback set, then record post-cleanup disk, inode, service-health, and volume results.

The September cleanup deleted both exact VPS cutover backup directories only after those checks; both verified off-server copies remain. It pruned 2.016 GB of idle build cache to zero, removed four obsolete release directories and redundant images, and preserved all four named volumes and intentional rollback images.

Consequences and open work

The service, container-log, journald, and emergency-swap bounds are now explicit and verified. Docker retained 1.955 GB of reclaimable known-good rollback images after cleanup. The final host had 24G free, up from about 14G before this cleanup.

Representative peak-load tests, external monitoring/alerts, backup destination and schedule, retention objectives, and a full Garage restore rehearsal remain open. They need an agreed workload, destinations, operators, cadence, and recovery objectives before automation is designed. The current audit script reports status only and is not an alerting service.