Problem: Sar data is usable, but not visually pleasing. We want to create a solution that works well with reports and allows us to show quickly and effortlessly to upper management how well (or poorly) our systems are responding. How do we create reports that are visual and informative?
Analysis: First, we need to look at our goal. Do we want something that we can run on-the-fly or that's regularly scheduled? For this case, I want to have something that's regularly scheduled and archived; we'll create graphs that we can put behind our intranet to view progression. While doing this, we'll keep in mind that we may want to run a report from within Webmin.
For our graphs, we can do one of two things: we can reformat the data to be imported to an Exchell spreadsheet, where we can generate the graph, and possibly use pivot tables (if we had enough data); or we can download and install gnuplot. We want automation at its best, and because the Domain Administrator controls whether or not we can run macros, it's best to stay native to the OS we love: Solaris. Unfortunately, Solaris doesn't ship with gnuplot, but we can easily get a stable version through our blastwave resources.
# /opt/csw/bin/pkgutil -i gnuplot
Now that we have our graphing program installed, we need to configure sar:
# svcadm enable sar
# svcs -xv sar
svc:/system/sar:default (system activity reporting package)
State: online since Mon Mar 23 10:03:06 2009
See: man -M /usr/share/man -s 1M sar
See: /var/svc/log/system-sar:default.log
Impact: None.
Running "sar" should give us data, but we haven't populated it yet. User "sys" already has templates in its crontab for sar, but we'll schedule our own for every 10 minutes:
# EDITOR=vi;export EDITOR; crontab -e sys
#ident "@(#)sys 1.5 92/07/14 SMI" /* SVr4.0 1.2 */
#
# The sys crontab should be used to do performance collection. See cron
# and performance manual pages for details on startup.
#
1,11,21,31,41,51 * * * 1-6 /usr/lib/sa/sa1
# 0 * * * 0-6 /usr/lib/sa/sa1
# 20,40 8-17 * * 1-5 /usr/lib/sa/sa1
# 5 18 * * 1-5 /usr/lib/sa/sa2 -s 8:00 -e 18:01 -i 1200 -A
Now, we'll let that populate at its regular intervals. We don't collect data on Sunday because there's no batch processes running on that day. This will make our scripting a little difficult, but we have ways of getting around it. When working on our script, we're going to want to run a report on the previous [sar] day's data--not today's. Now that we've got everything in place, let's work on the solution.
Solution: If it we didn't skip a day, we could just use gnu's date command with yesterday's date. First, let's create a timestamp script that takes care of this:
#!/bin/ksh
#########################################################
# sarstamp.ksh #
# A program to create a timestamp based on the last #
# (previous day) sar file generated #
#########################################################
YEAR=`date +%Y`
MODAY=`ls -ltr /var/adm/sa|tail -2 |head -1| sed -e 's/Jan/01/g;s/Feb/02/g;s/Mar/03/g;s/Apr/04/g;s/May/05/g;s/Jun/06/g;s/Jul/07/g;s/Aug/08/g;s/Sep/09
/g;s/Oct/10/g;s/Nov/11/g;s/Dec/12/g' | awk '{print $6$7}'`
echo ${YEAR}${MODAY}
If we didn't want each sar graph to have the timestamp match its sar-file creation date, we wouldn't have needed the timestamp script. We'll create a gnuplot script called "sar.gpl," and save our files as png files. We'll also want a grid, titles, and times for our x-axis. Using the sar data, we'll plot in this example, the %USR, %IDLE, Freemem, and Freeswap data as an 800x600 png file.
#!/opt/csw/bin/gnuplot -persist
set terminal png size 800,600
set grid
# Set time formats
set xdata time
set timefmt "%H:%M:%S"
set xrange ["00:00:00":"24:00:00"]
set xlabel "Time"
set key left box
set yrange [0:20]
set ylabel "% Usr"
set output "/tmp/sar-`hostname`-`/root/bin/sarstamp.ksh`-usr.png"
plot "/tmp/sar-cpu.dat" using 1:2 title 'Usr' with lines
set yrange [50:100]
set ylabel "% Idle"
set output "/tmp/sar-`hostname`-`/root/bin/sarstamp.ksh`-idle.png"
plot "/tmp/sar-cpu.dat" using 1:5 title 'Idle' with lines
unset yrange
set yrange [*:*]
set ylabel "FreeMem"
set output "/tmp/sar-`hostname`-`/root/bin/sarstamp.ksh`-mem.png"
plot "/tmp/sar-mem.dat" using 1:2 title 'Freemem' with lines
set ylabel "Swap"
set output "/tmp/sar-`hostname`-`/root/bin/sarstamp.ksh`-swap.png"
plot "/tmp/sar-mem.dat" using 1:3 title 'Freeswap' with lines
Now that we've got our sar script created, we'll create a nice wrapper around the whole thing to re-format the sar data to usable output (columned output with data and no text).
#!/bin/ksh
########################################################################
# #
# sar_graphs.ksh #
# #
# Author: Alan T. Landucci-Ruiz #
# #
# Program to generate graphical representation of the previous #
# day's sar data #
# #
########################################################################
HOST=`/bin/hostname`
MAILTO="myname@host.com"
SARFILE=`ls -1tr /var/adm/sa | tail -2 | head -1`
sar -f /var/adm/sa/${SARFILE} | egrep -v '[a-z]|^$' > /tmp/sar-cpu.dat
sar -r -f /var/adm/sa/${SARFILE} | egrep -v '[a-z]|^$'> /tmp/sar-mem.dat
/opt/csw/bin/gnuplot sar.gpl
rm /tmp/sar-cpu.dat
rm /tmp/sar-mem.dat
Easy enough. Now we can copy our sar data from /tmp to any location on our web server. Alternatively, we can have our web-server's location NFS-mounted and save directly. Later on, we'll use this same tool to plot drive usage.
Sun Update Tools