Appearance
How to recover from a full outage
Symptoms
ping 147.93.186.68fails, or- Host boots but no HTTPS services answer, or
- Multiple interdependent stacks failed after reboot/disk pressure
Steps
- Confirm reachability from more than one network path:bash
ping -c 3 147.93.186.68 - Out-of-band console via the hosting provider if SSH is dead.
- Boot errors:bash
journalctl -b -p err --no-pager | tail -100 - Disk full (common silent killer):bashClear safe space: old logs,
df -h docker system dfdocker image prune,/tmpjunk — not application bind mounts under~/nextcloud,~/puter, etc. - Failed units:bash
systemctl --failed systemctl restart docker caddy - Containers:bash
docker ps -a # restart stacks intentionally, one at a time - If filesystem corruption is evident, stop writes and restore from backup (restic how-to).
Verify
bash
systemctl is-active caddy docker
docker ps
curl -sI https://cloud.cadenceworks.tech | head -5