Sysstat

From wiki.mikejung.biz
Jump to: navigation, search

Liquidweb 728x90.jpg

sar -r memory usage and cache

This command will give you a good idea on the amount of overcommiting that is going on. It also shows you how much memory is allocated to buffers and as a cache. A commit of over 100% is normal so don't worry if you see a slightly high commit. If you see this get over 150 or 200 and also notice lots of paging in and out then you might need more RAM.

sar -r

09:50:01 AM kbmemfree kbmemused  %memused kbbuffers  kbcached  kbcommit   %commit
10:00:01 AM     84416    911188     91.52     47060    446024   3397748    111.64
10:10:01 AM     82360    913244     91.73     44292    497640   3277184    107.67
10:20:01 AM     69916    925688     92.98     44668    473864   3454444    113.50
10:30:01 AM    137604    858000     86.18     45364    424992   3323636    109.20


       -r     Report memory utilization statistics.  The following values are displayed:

              kbmemfree
                     Amount of free memory available in kilobytes.

              kbmemused
                     Amount of used memory in kilobytes. This does not take into account memory  used  by
                     the kernel itself.

              %memused
                     Percentage of used memory.

              kbbuffers
                     Amount of memory used as buffers by the kernel in kilobytes.

              kbcached
                     Amount of memory used to cache data by the kernel in kilobytes.

              kbcommit
                     Amount  of  memory  in kilobytes needed for current workload. This is an estimate of
                     how much RAM/swap is needed to guarantee that there never is out of memory.

              %commit
                     Percentage of memory needed for current workload in relation to the total amount  of
                     memory  (RAM+swap).  This number may be greater than 100% because the kernel usually
                     overcommits memory.


sar -B paging and swap usage

To view paging activity use -B. This gives you an idea of active swap usage which can reduce performance. Some activity is fine and normal, but if you notice lots of activity you might want to free up some memory.

sar -B

09:50:01 AM  pgpgin/s pgpgout/s   fault/s  majflt/s  pgfree/s pgscank/s pgscand/s pgsteal/s    %vmeff
10:00:01 AM      2.64     22.96    772.80      0.06    560.78      0.00      0.00      0.00      0.00
10:10:01 AM    740.69     30.93    828.12      1.01    759.45    130.66      0.00    128.25     98.15
10:20:01 AM      5.62     28.01    829.80      0.13    576.62     31.18      0.00     29.40     94.28
10:30:01 AM      5.47     21.67    823.66      0.19    582.78      5.39      0.00      5.22     96.75


       -B     Report  paging  statistics. Some of the metrics below are available only with post 2.5 ker-
              nels. The following values are displayed:

              pgpgin/s
                     Total number of kilobytes the system paged in from disk per second.  Note: With  old
                     kernels (2.2.x) this value is a number of blocks per second (and not kilobytes).

              pgpgout/s
                     Total  number  of kilobytes the system paged out to disk per second.  Note: With old
                     kernels (2.2.x) this value is a number of blocks per second (and not kilobytes).

              fault/s
                     Number of page faults (major + minor) made by the system per second.  This is not  a
                     count  of  page  faults  that generate I/O, because some page faults can be resolved
                     without I/O.

              majflt/s
                     Number of major faults the system has made per second,  those  which  have  required
                     loading a memory page from disk.

              pgfree/s
                     Number of pages placed on the free list by the system per second.

              pgscank/s
                     Number of pages scanned by the kswapd daemon per second.

              pgscand/s
                     Number of pages scanned directly per second.

              pgsteal/s
                     Number  of  pages  the system has reclaimed from cache (pagecache and swapcache) per
                     second to satisfy its memory demands.

              %vmeff
                     Calculated as pgsteal / pgscan, this is a metric of the efficiency of page  reclaim.
                     If  it  is near 100% then almost every page coming off the tail of the inactive list
                     is being reaped. If it gets too low (e.g. less than 30%) then the virtual memory  is
                     having  some  difficulty.   This  field  is  displayed as zero if no pages have been
                     scanned during the interval of time.


sar -b block device io and transfer rates

To view Block Device io summary stats use the -b flag.

sar -b

09:50:01 AM       tps      rtps      wtps   bread/s   bwrtn/s
10:00:01 AM      2.71      0.20      2.51      5.29     42.91
10:10:01 AM     15.38     12.78      2.60   1479.24     58.18
10:20:01 AM      2.87      0.32      2.55     11.25     53.07
10:30:01 AM      2.63      0.29      2.34     10.93     40.99
 -b


              tps    Total number of transfers per second that were issued to physical devices.  A trans-
                     fer  is  an  I/O request to a physical device. Multiple logical requests can be com-
                     bined into a single I/O request to the device.  A transfer is of indeterminate size.

              rtps
                     Total number of read requests per second issued to physical devices.

              wtps
                     Total number of write requests per second issued to physical devices.

              bread/s
                     Total amount of data read from the devices in blocks per second.  Blocks are equiva-
                     lent to sectors with 2.4 kernels and newer and therefore have a size of  512  bytes.
                     With older kernels, a block is of indeterminate size.

              bwrtn/s
                     Total amount of data written to devices in blocks per second.

sar -d block device activity

If you want more detailed io information with sar, use the -d flag to get

sar -d

10:00:01 AM       DEV       tps  rd_sec/s  wr_sec/s  avgrq-sz  avgqu-sz     await     svctm     %util
10:10:01 AM  dev252-0     15.38   1479.24     58.18     99.95      0.01      0.45      0.32      0.50
10:20:01 AM  dev252-0      2.87     11.25     53.07     22.44      0.00      0.44      0.21      0.06
10:30:01 AM  dev252-0      2.63     10.93     40.99     19.72      0.00      0.43      0.23      0.06
10:40:01 AM  dev252-0      2.74     35.72     47.50     30.36      0.00      0.50      0.26      0.07
      -d     Report  activity  for  each  block  device (kernels 2.4 and newer only).  When data is dis-
             played, the device specification dev m-n is generally used ( DEV column).  m is  the  major
             number of the device.  With recent kernels (post 2.5), n is the minor number of the device,
             but is only a sequence number with pre 2.5 kernels. Device names may also be pretty-printed
             if  option  -p  is used or persistent device names can be printed if option -j is used (see
             below). Values for fields avgqu-sz, await, svctm and %util may be unavailable and displayed
             as  0.00  with some 2.4 kernels.  Note that disk activity depends on sadc options "-S DISK"
             and "-S XDISK" to be collected. The following values are displayed:
             tps
                    Indicate the number of transfers per second that were issued to the device.   Multi-
                    ple  logical  requests  can  be  combined into a single I/O request to the device. A
                    transfer is of indeterminate size.
             rd_sec/s
                    Number of sectors read from the device. The size of a sector is 512 bytes.
             wr_sec/s
                    Number of sectors written to the device. The size of a sector is 512 bytes.
             avgrq-sz
                    The average size (in sectors) of the requests that were issued to the device.
             avgqu-sz
                    The average queue length of the requests that were issued to the device.
             await
                    The average time (in milliseconds) for I/O requests  issued  to  the  device  to  be
                    served.  This  includes  the  time spent by the requests in queue and the time spent
                    servicing them.
             svctm
                    The average service time (in milliseconds) for I/O requests that were issued to  the
                    device.
             %util
                    Percentage  of  CPU  time during which I/O requests were issued to the device (band-
                    width utilization for the device). Device saturation occurs when this value is close
                    to 100%.

sar per cpu

To view detailed, Per CPU metrics use -P ALL

sar -P ALL

Average:        CPU     %user     %nice   %system   %iowait    %steal     %idle
Average:        all      0.84      0.02      0.30      0.08      0.22     98.54
Average:          0      0.60      0.03      0.29      0.14      0.27     98.67
Average:          1      1.08      0.02      0.30      0.02      0.16     98.41

Previous days metrics are located in this directory (CentOS 6.6).

/var/log/sa/

To view a specific day's metrics you can use this command, change "sa04" to whatever day you want.

sar -f /var/log/sa/sa30


/var/log/sa/sa30 This file seems to be the oldest, this is an example of what it's output looks like.

12:00:01 AM     CPU     %user     %nice   %system   %iowait    %steal     %idle
12:10:01 AM     all      0.51      0.01      0.35      1.88      0.50     96.75
12:20:01 AM     all      0.41      0.05      0.27      1.02      0.31     97.95
12:30:01 AM     all      0.30      0.01      0.17      0.64      0.03     98.85

/var/log/sa/sar30 This file is slightly different than the other version, this is an example of what it's output looks like.

12:00:01 AM     CPU      %usr     %nice      %sys   %iowait    %steal      %irq     %soft    %guest     %idle
12:10:01 AM     all      0.51      0.01      0.35      1.88      0.50      0.00      0.01      0.00     96.75
12:10:01 AM       0      0.69      0.02      0.43      3.28      0.62      0.00      0.01      0.00     94.95
12:10:01 AM       1      0.32      0.00      0.26      0.48      0.39      0.00      0.00      0.00     98.55

sysstat configuration file and cron location

/etc/cron.d/sysstat

This is systat's cronjob. You can edit this if you want to change the default settings. If you want to increase or decrease the frequency of the cron then this is your file.

vim /etc/cron.d/sysstat

# Run system activity accounting tool every 10 minutes
*/10 * * * * root /usr/lib64/sa/sa1 1 1
# 0 * * * * root /usr/lib64/sa/sa1 600 6 &
# Generate a daily summary of process accounting at 23:53
53 23 * * * root /usr/lib64/sa/sa2 -A


/etc/sysconfig/sysstat

This file lets you configure the retention period for the logs, when to compress them, what parameters to use and what compression program to use.

vim /etc/sysconfig/sysstat

# sysstat-9.0.4 configuration file.

# How long to keep log files (in days).
# If value is greater than 28, then log files are kept in
# multiple directories, one for each month.
HISTORY=28

# Compress (using gzip or bzip2) sa and sar files older than (in days):
COMPRESSAFTER=31

# Parameters for the system activity data collector (see sadc manual page)
# which are used for the generation of log files.
SADC_OPTIONS="-S DISK"

# Compression program to use.
ZIP="bzip2"

man SADC

 
SADC(8)                       Linux User’s Manual                      SADC(8)



NAME
       sadc - System activity data collector.

SYNOPSIS
       /usr/lib64/sa/sadc  [  -C  comment  ] [ -S { INT | DISK | SNMP | IPV6 |
       POWER | XDISK | ALL | XALL } ] [ -F ] [ -L ] [ -V ] [ interval [  count
       ] ] [ outfile ]

DESCRIPTION
       The  sadc  command  samples  system  data  a  specified number of times
       (count) at a specified interval  measured  in  seconds  (interval).  It
       writes in binary format to the specified outfile or to standard output.
       If outfile is set to -, then sadc uses  the  standard  system  activity
       daily  data  file,  the  /var/log/sa/sadd  file, where the dd parameter
       indicates the current day.  In this case, sadc will overwrite the  file
       if  it is from a previous month.  By default sadc collects all the data
       available from the kernel.  Exceptions are interrupts  and  disk  data,
       for  which  the relevant options must be explicitly passed to sadc (see
       options below).

       When the count parameter is not specified, sadc writes  its  data  end-
       lessly.   When both interval and count are not specified, and option -C
       is not used, a dummy record, which is used at system  startup  to  mark
       the  time when the counter restarts from 0, will be written.  For exam-
       ple, one of the system startup script may write the restart mark to the
       daily data file by the command entry:

       /usr/lib64/sa/sadc -

       The  sadc  command  is intended to be used as a backend to the sar com-
       mand.

       Note: The sadc command only reports on local activities.


OPTIONS
       -C comment
              When neither the interval nor the count  parameters  are  speci-
              fied,  this option tells sadc to write a dummy record containing
              the specified comment string.  This comment  can  then  be  dis-
              played with option -C of sar.

       -F     The  creation  of  outfile  will  be forced. If the file already
              exists and has not the format expected by sadc then it  will  be
              truncated. This may be useful for daily data files created by an
              older version of sadc and whose format is no  longer  compatible
              with current one.

       -L     sadc  will  try  to  get an exclusive lock on the outfile before
              writing to it or truncating it.  Failure  to  get  the  lock  is
              fatal,  except in the case of trying to write a normal (i.e. not
              a dummy and not a header) record to an existing file,  in  which
              case sadc will try again at the next interval. Usually, the only
              reason a lock would fail would be if another sadc  process  were
              also  writing  to the file. This can happen when cron is used to
              launch sadc.  If the system is under heavy  load,  an  old  sadc
              might still be running when cron starts a new one. Without lock-
              ing, this situation can result in a  corrupted  system  activity
              file.

       -S { INT | DISK | SNMP | IPV6 | POWER | XDISK | ALL | XALL }
              Specify  which  optional activities should be collected by sadc.
              Some activities are optional to prevent data files from  growing
              too  large.   The INT keyword indicates that sadc should collect
              data for system interrupts.  The  DISK  keyword  indicates  that
              sadc  should  collect data for block devices.  The SNMP and IPV6
              keywords indicate respectively that  SNMP  and  IPv6  statistics
              should  be  collected by sadc.  The POWER keyword indicates that
              sadc should collect power management statistics.  The  ALL  key-
              word  is  equivalent  to  specifying  all the keywords above and
              therefore all previous activities are collected.

              The XDISK keyword is an extension to the DISK one and  indicates
              that  partition  statistics should be collected by sadc in addi-
              tion to disk statistics. This option  works  only  with  kernels
              2.6.25  and later.  The XALL keyword is equivalent to specifying
              all the keywords above (including keyword extensions) and there-
              fore all possible activities are collected.

              Important  note:  The activities (including optional ones) saved
              in an existing data file prevail over those selected with option
              -S.   As  a consequence, appending data to an existing data file
              will result in option -S being ignored.

       -V     Print version number then exit.