Resilience Engineering · High Availability · Infrastructure — 2024
Vœrynth Sentinel
Out-of-band recovery supervisor and watchdog isolating failure domains.
Concept visual
Overview
Vœrynth Sentinel is an autonomous Raspberry Pi recovery guardian that sits strictly outside the primary automation host's failure domain to monitor health, distinguish maintenance states, power-cycle frozen hardware, and orchestrate SSH disaster recovery.
Challenge
When a home automation server or host kernel locks up, internal watchdog scripts are rendered inert because they share CPU registers and memory space with the frozen operating system.
Approach
Enforce strict failure-domain separation. The Sentinel runs on separate dedicated silicon, communicates out-of-band, controls an independent smart relay for hard cold reboots, and verifies system health before and after escalation.
Architecture
- 01
Observer Engine: Periodic non-intrusive health checks via HTTP API, Ping, and Supervisor Docker socket.
- 02
State Machine: Multi-tier escalation algorithm preventing reboot loops during legitimate system updates.
- 03
Out-of-Band Power Cycle: Tasmota/Tuya isolated relay cuts primary power cleanly for 15 seconds to discharge capacitance.
- 04
SSH Rescue Payload: Automated terminal execution post-boot to repair corrupted databases or roll back snapshots.
- 05
Escalation Circuit: Graceful alerting dispatch via SMS/local buzzer if host remains unresponsive after max power cycles.
Specifications
- Silicon Host
- Dedicated Raspberry Pi (Zero / 4B)
- Failure Domain
- 100% Out-of-Band Isolation
- Heartbeat Interval
- Configurable 30s - 300s window
- Recovery Mechanism
- Cold Power Cut + Automated SSH Diagnostics
Stack
Python 3 · Raspberry Pi OS · Docker Socket API · SSH Automation · Relay GPIO · Systemd
Attribution
Open-source resilience tool designed and authored by KK Achari.
Imagery marked “Concept visual” is art direction, not documentary photography.