Product SiteDocumentation Site

Chapter 4. Monitoring System Performance

4.1. The pmstat Command
4.2. The pmdumptext Command
4.3. The pmval Command
4.4. The pminfo Command
4.5. The pmstore Command
This chapter describes the performance monitoring tools available in Performance Co-Pilot (PCP). This product provides a group of commands and tools for measuring system performance. Each tool is described completely by its own man page. The man pages are accessible through the man command. For example, the man page for the tool pmdumptext is viewed by entering the following command:
man pmdumptext
The following major sections are covered in this chapter:
The following sections describe the various graphical and text-based PCP tools used to monitor local or remote system performance.

4.1. The pmstat Command

The pmstat command provides a periodic, one-line summary of system performance. This command is intended to monitor system performance at the highest level, after which other tools may be used for examining subsystems to observe potential performance problems in greater detail. After entering the pmstat command, you see output similar to the following, with successive lines appearing periodically:
@ Thu Aug 15 09:25:56 2013
 loadavg                      memory      swap        io    system         cpu
   1 min   swpd   free   buff  cache   pi   po   bi   bo   in   cs  us  sy  id
    1.29 833960  5614m 144744 265824    0    0    0 1664  13K  23K   6   7  81
    1.51 833956  5607m 144744 265712    0    0    0 1664  13K  24K   5   7  83
    1.55 833956  5595m 145196 271908    0    0  14K 1056  13K  24K   7   7  74
An additional line of output is added every five seconds. The -t interval option may be used to vary the update interval (i.e. the sampling interval).
The output from pmstat is directed to standard output, and the columns in the report are interpreted as follows:
The 1-minute load average (runnable processes).
The swpd column indicates average swap space used during the interval (all columns reported in Kbytes unless otherwise indicated). The free column indicates average free memory during the interval. The buff column indicates average buffer memory in use during the interval. The cache column indicates average cached memory in use during the interval.
Reports the average number of pages that are paged-in (pi) and paged-out (po) per second during the interval. It is normal for the paged-in values to be non-zero, but the system is suffering memory stress if the paged-out values are non-zero over an extended period.
The bi and bo columns indicate the average rate per second of block input and block output operations respectfully, during the interval. These rates are independent of the I/O block size. If the values become large, they are reported as thousands of operations per second (K suffix) or millions of operations per second (M suffix).
Context switch rate (cs) and interrupt rate (in). Rates are expressed as average operations per second during the interval. Note that the interrupt rate is normally at least HZ (the clock interrupt rate, and kernel.all.hz metric) interrupts per second.
Percentage of CPU time spent executing user code (us), system and interrupt code (sy), idle loop (id).
As with most PCP utilities, real-time metric, and archive logs are interchangeable.
For example, the following command uses a local system PCP archive log 20130731 and the timezone of the host (smash) from which performance metrics in the archive were collected:
pmstat -a ${PCP_LOG_DIR}/pmlogger/smash/20130731 -t 2hour -A 1hour -z
Note: timezone set to local timezone of host "smash"
@ Wed Jul 31 10:00:00 2013
 loadavg                      memory      swap        io    system         cpu
   1 min   swpd   free   buff  cache   pi   po   bi   bo   in   cs  us  sy  id
    3.90  24648  6234m 239176  2913m    ?    ?    ?    ?    ?    ?   ?   ?   ?
    1.72  24648  5273m 239320  2921m    0    0    4   86  11K  19K   5   5  84
    3.12  24648  5194m 241428  2969m    0    0    0   84  10K  19K   5   5  85
    1.97  24644  4945m 244004  3146m    0    0    0   84  10K  19K   5   5  84
    3.82  24640  4908m 244116  3147m    0    0    0   83  10K  18K   5   5  85
    3.38  24620  4860m 244116  3148m    0    0    0   83  10K  18K   5   4  85
    2.89  24600  4804m 244120  3149m    0    0    0   83  10K  18K   5   4  85
pmFetch: End of PCP archive log
For complete information on pmstat usage and command line options, see the pmstat(1) man page.