# Alerts and audit

The server watches itself and emails you through any SMTP server (`[alerts.smtp]` in its [config](https://hakobu.dev/docs/servers/index.md)): one email when a problem starts and one when it's over, never a stream. Without SMTP, problems show only in `status`.

## What it watches

- an app that hasn't answered its health check for 3 minutes;
- a failed deploy (closed by the next one that works) or backup, and a backup that didn't prove it restores;
- the disk 85% full (closed under 80%);
- PostgreSQL not answering;
- an update of hakobu that didn't install (closed once it does), and a newer minor release of hakobu out for 14 days that this server hasn't updated to;
- the **audit**, hourly:
  - a port listening beyond loopback and the tailnet that isn't in `alerts.allow_ports` (80 and 443 are allowed for `public` ingress by themselves, and 22, as SSH is the way to hakobu);
  - a container hakobu didn't start;
  - unattended upgrades that haven't run for 3 days;
  - your owner's key still on the server, where the installer made it (`ssh -t root@SERVER hakobu server owner-key` takes it).

What's fixed closes by itself. Problems survive a restart of hakobu, so you aren't mailed twice.

The audit sees what listens on the machine, not a firewall in front of it (a cloud firewall is outside its view), and only TCP.

## The pulse

Every 15 minutes hakobu fetches `alerts.pulse_url`, or `alerts.pulse_fail_url` while a problem is open. Point them at a dead man's switch ([healthchecks.io](https://healthchecks.io) and the like): it emails you when the pulses stop, which is how you hear that hakobu, or the whole server, is down.

## Your apps' errors

hakobu doesn't collect errors or traces. Give an app a `SENTRY_DSN` [secret](https://hakobu.dev/docs/secrets/index.md) and any Sentry-compatible service has them.
