One of my machines ran out of disk due to a scraper during the week, so I finally set up:

  • email alerting
  • having alerts on disk space per machine
  • having some alerts on services not running.
  • having some probing.

Hopefully all will improve. I found this a useful collection of prom rules.