Solaris SMF and FMA Notifications

I never realized that there is a really easy way to “monitor” your Solaris in a way to use build-in SMF and FMA monitors and send a mail in case of diagnoses.

# svccfg setnotify problem-diagnosed mailto:pressy@solaris.wtf
# svcadm enable http:apache24
# mv /etc/apache2/2.4/httpd.conf /etc/apache2/2.4/httpd.conf_bu
# pkill httpd
# svcs -xv
svc:/network/http:apache24 (Apache 2.4 HTTP server)
State: maintenance since Fri May 24 11:51:19 2019
Reason: Method failed.
See: http://support.oracle.com/msg/SMF-8000-8Q
See: http://httpd.apache.org
See: man -M /usr/apache2/2.4/man -s 8 httpd
See: /var/svc/log/network-http:apache24.log
Impact: This service is not running.

uhhh… got mail:

SUNW-MSG-ID: SMF-8000-YX, TYPE: Defect, VER: 1, SEVERITY: Major
EVENT-TIME: Fri May 24 11:51:19 CEST 2019
PLATFORM: ORCL,SPARC-T4-1, CSN: AKBLABLA42, HOSTNAME: sparc-server
SOURCE: software-diagnosis, REV: 0.2
EVENT-ID: e0114186-cd70-4085-84aa-802b091a399e
DESC: Service svc:/network/http:apache24 failed - a start, stop or refresh method failed.
AUTO-RESPONSE: The service has been placed into the maintenance state.
IMPACT: svc:/network/http:apache24 is unavailable.
REC-ACTION: Run 'svcs -xv svc:/network/http:apache24' to determine the generic reason why the service failed, the location of any logfiles, and a list of other services impacted. Please refer to the associated reference document at http://support.oracle.com/msg/SMF-8000-YX for the latest service procedures and policies regarding this diagnosis.

Nice… you could set several tags like problem-diagnosed, problem-updated, problem-repaired, problem-resolved, to- or from- (maintenance, from-online, to-degraded) or all for every transition.

And it would also work for specific services:

# svccfg -s application/myservice setnotify problem-diagnosed mailto:pressy@solaris.wtf

Easy, isn’t it?