Edition 3
Copyright © 2000, 2013 Silicon Graphics, Inc.
Copyright © 2013, 2015, 2016 Red Hat, Inc.
LICENSE
TRADEMARKS AND ATTRIBUTIONS
man PCPIntro
man command, for example:
man pcp
man man
pminfo(1) refers to the man page in section 1 for the pminfo command.
${PCP_VARIABLE}/etc/pcp.conf file. These special variables indicate parameters that affect all PCP commands, and are likely to be different between platforms.
commanduser inputMonitoring tools such as pmval and pminfo report on metrics, but have minimal interaction with target systems. Collection tools, called PMDAs, extract performance values from target systems, but do not provide user interfaces.
libpcp, below the PMAPI level; see the PMAPI(3) man page. Connections between PMDAs and PMCD are managed by the PMDA functions; see the PMDA(3) and pmcd(1) man pages. There can be multiple monitor clients and multiple PMDAs on the one host, but there may be only one PMCD process.
-n pmnsfile command line option may be used with many PCP monitoring tools to force use of a local PMNS file in preference to the PMNS at the source of the metrics.
pmlogger utility, and are replayed in an architecture as shown in Figure 1.3, “Architecture for Retrospective Analysis”.
man command. For example, to see the pminfo(1) man page for the pminfo command, enter this command:
man pminfochkhelp dbpmda mmv newhelp pmapi pmclient pmda pmgenmap Note
${PCP_PMDAS_DIR} directory.
dlopen(3), at initialization time. This is the highest performance option (there is no context switching and no interprocess communication (IPC) between the PMCD and the PMDA), but is operationally intractable in some situations. For example, difficulties arise where special access permissions are required to read the instrumentation behind the performance metrics (pmcd does not run as root), or where the performance metrics are provided by an existing process with a different protocol interface. The DSO PMDA effectively executes as part of PMCD; so great care is required when crafting a PMDA in this manner. Calls to exit(1) in the PMDA, or a library it uses, would cause PMCD to exit and end monitoring of that host. Other implications are discussed in Section 2.2.3, “Daemon PMDA”.
main routine conversion wrapper so that communication with PMCD uses message passing rather than direct procedure calls. For some very basic examples, see the ${PCP_PMDAS_DIR}/trivial/trivial.c and ${PCP_PMDAS_DIR}/simple/simple.c source files.
Note
${PCP_PMDAS_DIR}/simple PMDA has implementations in C, Perl and Python.
libpcp) is thread safe, the layered PMDA library (libpcp_pmda) is not. This is a deliberate design decision to trade-off commonly required performance and efficiency against the less common requirement for multiple threads of control to call the PCP libraries.
libpcp_pmda library, as documented in the PMDA(3) and associated man pages.
mmap file; a sequential log file (where the PMDA parses the tail of the log file to extract the information); a snapshot file (the PMDA rereads the file as required); or application-specific communication services (IPC).
Note
Procedure 2.1. Creating a PMDA
pthreads or single threaded).
pmcd(1) man page.
PDU_FETCHPDU_PROFILEPDU_INSTANCE_REQPDU_DESC_REQPDU_TEXT_REQPDU_RESULTPDU_PMNS_NAMESPDU_PMNS_CHILDPDU_PMNS_TRAVERSEPDU_PMNS_IDSPDU_AUTHdlopen; see the dlopen(3) man page. This eliminates the need for an IPC layer (typically a pipe) between each PMDA and PMCD, because each request becomes a function call rather than a message exchange. The required library functions are detailed in Section 2.5, “PMDA Interface”.
${PCP_PMCDCONF_PATH}, is updated to reflect the domain and name of the PMDA, the location of the shared object, and the name of the initialization function. The initialization sequence is discussed in Section 2.6, “Initializing a PMDA”.
Example 2.1. Simple PMDA as a DSO
cat ${PCP_PMCDCONF_PATH}
# Performance Metrics Domain Specifications
#
# This file is automatically generated during the build
# Name Id IPC IPC Params File/Cmd
pmcd 2 dso pmcd_init ${PCP_PMDAS_DIR}/pmcd/pmda_pmcd.so
linux 60 dso linux_init ${PCP_PMDAS_DIR}/linux/pmda_linux.so
proc 3 pipe binary ${PCP_PMDAS_DIR}/linux/pmda_proc.so -d 3
simple 254 dso simple_init ${PCP_PMDAS_DIR}/simple/pmda_simple.so${PCP_PMCDCONF_PATH}, the DSO version of the simple PMDA is in a library named pmda_simple.so and has an initialization function called simple_init. The domain of the simple PMDA is 254, as shown in the column headed Id.
Note
pmda_simple.so. On Mac OS X it is pmda_simple.dylib and on Windows it is pmda_simple.dll.
libpcp_pmda library is designed to allow simple implementation of a PMDA that runs as a separate process. The library functions provide a message passing layer acting as a generic wrapper that accepts PDUs, makes library calls using the standard DSO PMDA interface, and sends PDUs. Therefore, you can implement a PMDA as a DSO and then install it as either a daemon or a DSO, depending on the presence or absence of the generic wrapper.
fork and execv (or CreateProcess on Windows). You can easily connect a pipe to the PMDA using standard input and output. The PMCD process may also connect to a daemon PMDA using IPv4 or IPv6 TCP/IP, or UNIX domain sockets if the platform supports that; see the tcp(7), ip(7), ipv6(7) or unix(7) man pages.
Example 2.2. Simple PMDA as a Daemon
pipe to PMCD and the executable image for the PMDA is ${PCP_PMDAS_DIR}/simple/pmdasimple, using domain number 253.
#cd ${PCP_PMDAS_DIR}/simple#./Install... Install simple as a daemon or dso agent? [daemon] daemon PMCD should communicate with the daemon via pipe or socket? [pipe] pipe ... #cat ${PCP_PMCDCONF_PATH}# Performance Metrics Domain Specifications # # This file is automatically generated during the build # Name Id IPC IPC Params File/Cmd pmcd 2 dso pmcd_init ${PCP_PMDAS_DIR}/pmcd/pmda_pmcd.so linux 60 dso linux_init ${PCP_PMDAS_DIR}/linux/pmda_linux.so proc 3 pipe binary ${PCP_PMDAS_DIR}/linux/pmda_proc.so -d 3 simple 253 pipe binary ${PCP_PMDAS_DIR}/simple/pmdasimple -d 253
${PCP_PMDAS_DIR}/trivial and ${PCP_PMDAS_DIR}/simple directories, respectively, where both binaries and source code are available.
hinv.ndisk counts the number of disks and has only one value on a host, whereas the metric disk.dev.total counts disk I/O operations and has one value for each disk on the host.
disk.dev.total is defined over an instance domain that has one member per disk spindle.
${PCP_VAR_DIR}/pmns/stdpmid. This file is a simple table of PMDA names and their corresponding domain number. However, a PMDA does not have to use this domain number--the file is only a guide to help avoid domain number clashes when PMDAs are installed and activated.
Tip
${PCP_PMCDCONF_PATH} control file used by PMCD is all that is required for successful starting of PMCD and the associated PMDAs, the developer of a new PMDA is encouraged to add the default domain number for each new PMDA to the ${PCP_VAR_DIR}/pmns/stdpmid.local file and then to run the Make.stdpmid script in ${PCP_VAR_DIR}/pmns to recreate ${PCP_VAR_DIR}/pmns/stdpmid; this file acts as a repository for documenting the known default domain numbers.
hinv.ninterface, network.interface.speed, network.interface.duplex, and so on)
pmstore(1) man page) into the delay metric, so that the delay used by the PMDA can be altered dynamically.
typedef struct {
pmID pmid; /* unique identifier */
int type; /* base data type */
pmInDom indom; /* instance domain */
int sem; /* semantics of value */
pmUnits units; /* dimension and units */
} pmDesc;pmid type indom sem units Note
pminfo command line options, for example:
$ pminfo -d -m network.interface.out.drops
network.interface.out.drops PMID: 60.3.11
Data Type: 64-bit unsigned int InDom: 60.3 0xf000003
Semantics: counter Units: count
PM_TYPE_*, PM_SEM_*, PM_SPACE_*, PM_TIME_*, and PM_COUNT_* are defined in the <pcp/pmapi.h> header file. You may use them to initialize the elements of a pmDesc structure. The pmID type is an unsigned integer that can be safely cast to a __pmID_int structure, which contains fields defining the metric's (PMDA's) domain, cluster, and item number as shown in Example 2.3, “ __pmID_int Structure”:
Example 2.3. __pmID_int Structure
typedef struct {
int flag:1;
unsigned int domain:9;
unsigned int cluster:12;
unsigned int item:10;
} __pmID_int;<pcp/impl.h> file.
flag field should be ignored. The domain number should be set at run time when the PMDA is initialized. The PMDA_PMID macro defined in <pcp/pmapi.h> can be used to set the cluster and item fields at compile time, as these should always be known and fixed for a particular metric.
Note
pmdaMetric structures should be defined within the PMDA, with one structure per metric as shown in Example 2.4, “ pmdaMetric Structure”.
Example 2.4. pmdaMetric Structure
typedef struct {
void *m_user; /* for users external use */
pmDesc m_desc; /* metric description */
} pmdaMetric;pmDesc structure and a handle that allows PMDA-specific structures to be associated with each metric. For example, m_user could be a pointer to a global variable containing the metric value, or a pointer to a function that may be called to instantiate the metric's value.
Example 2.5. Trivial PMDA
static pmdaMetric metrictab[] = {
/* time */
{ NULL,
{ PMDA_PMID(0, 1), PM_TYPE_U32, PM_INDOM_NULL, PM_SEM_INSTANT,
PMDA_PMUNITS(0, 1, 0, 0, PM_TIME_SEC, 0) }, },
};trivial.time) has the following:
PM_TYPE_U32)
Example 2.6. Effect of Semantics on a Metric
Timestamps: 1 3 5 7 9 11 Value: 10 30 60 80 90 N/A
Timestamps: 1 3 5 7 9 11 Semantics: Counter N/A 10 15 10 5 N/A Instantaneous 10 30 60 80 90 N/A Discrete 10 30 60 80 90 90
Note
pmdaCache routines to assist.
M[1] M[2] M[3] M[4] M[5] M[6]
M1[1] M1[2] M1[3] M2[1] M2[2] M2[3]
PM_TYPE_AGGREGATE). However, this requires the development of special PMAPI client tools, because the standard PCP tools have no knowledge of the structure and interpretation of the binary data. The usual issues of platform-depdendence must also be kept in mind for this case - endianness, word-size, alignment and so on - the (possibly remote) special PMAPI client tools may need this information in order to decode the data successfully.
pmdaInstid structure as shown in Example 2.7, “ pmdaInstid Structure”:
Example 2.7. pmdaInstid Structure
typedef struct {
int i_inst; /* internal instance identifier */
char *i_name; /* external instance identifier */
} pmdaInstid;i_inst instance identifier must be a unique integer within a particular instance domain.
pmdaIndom structure as shown in Example 2.8, “ pmdaIndom Structure”:
Example 2.8. pmdaIndom Structure
typedef struct {
pmInDom it_indom; /* indom, filled in */
int it_numinst; /* number of instances */
pmdaInstid *it_set; /* instance identifiers */
} pmdaIndom;it_indom element contains a pmInDom that must be unique across every PMDA. The other fields of the pmdaIndom structure are the number of instances in the instance domain and a pointer to an array of instance descriptions.
__pmInDom_int Structure” shows that the pmInDom can be safely cast to __pmInDom_int, which specifies the PMDA's domain and the instance number within the PMDA:
Example 2.9. __pmInDom_int Structure
typedef struct {
int flag:1;
unsigned int domain:9; /* the administrative PMD */
unsigned int serial:22; /* unique within PMD */
} __pmInDom_int;domain field must be set up when the PMDA is initialized.
pmdaInit(3) man page.
Example 2.10. Simple PMDA
/*
* list of instances
*/
static pmdaInstid color[] = {
{ 0, “red” }, { 1, “green” }, { 2, “blue” }
};
static pmdaInstid *timenow = NULL;
static unsigned int timesize = 0;
/*
* list of instance domains
*/
static pmdaIndom indomtab[] = {
#define COLOR_INDOM 0
{ COLOR_INDOM, 3, color },
#define NOW_INDOM 1
{ NOW_INDOM, 0, NULL },
};
/*
* all metrics supported in this PMDA - one table entry for each
*/
static pmdaMetric metrictab[] = {
/* numfetch */
{ NULL,
{ PMDA_PMID(0, 0), PM_TYPE_U32, PM_INDOM_NULL, PM_SEM_INSTANT,
PMDA_PMUNITS(0, 0, 0, 0, 0, 0) }, },
/* color */
{ NULL,
{ PMDA_PMID(0, 1), PM_TYPE_32, COLOR_INDOM, PM_SEM_INSTANT,
PMDA_PMUNITS(0, 0, 0, 0, 0, 0) }, },
/* time.user */
{ NULL,
{ PMDA_PMID(1, 2), PM_TYPE_DOUBLE, PM_INDOM_NULL, PM_SEM_COUNTER,
PMDA_PMUNITS(0, 1, 0, 0, PM_TIME_SEC, 0) }, },
/* time.sys */
{ NULL,
{ PMDA_PMID(1,3), PM_TYPE_DOUBLE, PM_INDOM_NULL, PM_SEM_COUNTER,
PMDA_PMUNITS(0, 1, 0, 0, PM_TIME_SEC, 0) }, },
/* now */
{ NULL,
{ PMDA_PMID(2,4), PM_TYPE_U32, NOW_INDOM, PM_SEM_INSTANT,
PMDA_PMUNITS(0, 0, 0, 0, 0, 0) }, },
};simple.color is associated, via COLOR_INDOM, with the first instance domain listed in indomtab. PMDA initialization assigns the correct domain portion of the instance domain identifier in indomtab[0].it_indom and metrictab[1].m_desc.indom. This instance domain has three instances: red, green, and blue.
simple.now is associated, via NOW_INDOM, with the second instance domain listed in indomtab. PMDA initialization assigns the correct domain portion of the instance domain identifier in indomtab[1].it_indom and metrictab[4].m_desc.indom. This instance domain is dynamic and initially has no instances.
simple.now), and this requires some refinement of the default functions and data structures of the libpcp_pmda library. Briefly, this involves providing new functions that act as wrappers for pmdaInstance and pmdaFetch while understanding the dynamics of the instance domain, and then overriding the instance and fetch methods in the pmdaInterface structure during PMDA initialization.
simple_fetch and simple_instance, and defaults are over-ridden by the following assignments in the simple_init function:
dp->version.any.fetch = simple_fetch; dp->version.any.instance = simple_instance;
pthreads(7) man page.
Warning
pmns(5) man page and its hierarchical nature, syntax, and helper tools are further described in the Performance Co-Pilot User's and Administrator's Guide.
${PCP_VAR_DIR}/pmns/root.
pmns File for the Simple PMDA” shows the simple PMDA, which has five metrics:
simple node
simple.time
Example 2.11. pmns File for the Simple PMDA
simple {
numfetch SIMPLE:0:0
color SIMPLE:0:1
time
now SIMPLE:2:4
}
simple.time {
user SIMPLE:1:2
sys SIMPLE:1:3
}pmns File for the Simple PMDA” shows an alternative PMNS for the simple PMDA:
Example 2.12. Alternate pmns File for the Simple PMDA
simple {
numfetch SIMPLE:0:0
color SIMPLE:0:1
usertime SIMPLE:1:2
systime SIMPLE:1:3
}SIMPLE macro is replaced by the domain number listed in ${PCP_VAR_DIR}/pmns/stdpmid for the corresponding PMDA during installation (for the simple PMDA, this would normally be the value 253).
pmns File for the Simple PMDA” shows this dynamic namespace syntax, for all metrics in the simple PMDA:
Example 2.13. Dynamic metrics pmns File for the Simple PMDA
simple SIMPLE:*:*
SIMPLE macro is replaced by the domain number, and all (simple.*) metric namespace operations must be handled by the PMDA. This is in contrast to the static metric name model earlier, where the host-wide PMNS file is updated and used by PMCD, acting on behalf of the agent.
pminfo with the -T option. The help text, such as the help text for the simple PMDA in Example 2.14, “Help Text for the Simple PMDA”, is specified in a specially formatted file, normally called help. This file is converted to the expected run-time format using the newhelp command; see the newhelp(1) man page. Converted help text files are usually placed in the PMDA's directory below ${PCP_PMDAS_DIR} as part of the PMDA installation procedure.
Example 2.14. Help Text for the Simple PMDA
simple.numfetch) or a symbolic reference to the instance domain number (SIMPLE.1), followed by the short description. The verbose description is on the following lines, terminated by the next line starting with “@” or end of file:
@ SIMPLE.0 Instance domain “colour” for simple PMDA
Universally 3 instances, “red” (0), “green” (1) and “blue” (3).
@ SIMPLE.1 Dynamic instance domain “time” for simple PMDA
An instance domain is computed on-the-fly for exporting current time
information. Refer to the help text for simple.now for more details.
@ simple.numfetch Number of pmFetch operations.
The cumulative number of pmFetch operations directed to “simple” PMDA.
This counter may be modified with pmstore(1).
@ simple.color Metrics which increment with each fetch
This metric has 3 instances, designated “red”, “green” and “blue”.
The value of the metric is monotonic increasing in the range 0 to
255, then back to 0. The different instances have different starting
values, namely 0 (red), 100 (green) and 200 (blue).
The metric values my be altered using pmstore(1).
@ simple.time.user Time agent has spent executing user code
The time in seconds that the CPU has spent executing agent user code.
@ simple.time.sys Time agent has spent executing system code
The time in seconds that the CPU has spent executing agent system code.
@ simple.now Time of day with a configurable instance domain
The value reflects the current time of day through a dynamically
reconfigurable instance domain. On each metric value fetch request,
the agent checks to see whether the configuration file in
${PCP_PMDAS_DIR}/simple/simple.conf has been modified - if it has then
the file is re-parsed and the instance domain for this metric is again
constructed according to its contents.
This configuration file contains a single line of comma-separated time
tokens from this set:
“sec” (seconds after the minute),
“min” (minutes after the hour),
“hour” (hour since midnight).
An example configuration file could be: sec,min,hour
and in this case the simple.now metric would export values for the
three instances “sec”, “min” and “hour” corresponding respectively to
the components seconds, minutes and hours of the current time of day.
The instance domain reflects each token present in the file, and the
values reflect the time at which the PMDA processes the fetch.pmLookupDesc requests with a pmDesc structure whose type field has the special value PM_TYPE_NOSUPPORT. Values of fields other than pmid and type are immaterial, but Example 2.15, “ Setting Values” is typically benign:
Example 2.15. Setting Values
pmDesc dummy = {
.pmid = PMDA_PMID(3,0), /* pmid, fill this in */
.type = PM_TYPE_NOSUPPORT, /* this is the important part */
.indom = PM_INDOM_NULL, /* singular,causes no problems */
.sem = 0, /* no semantics */
.units = PMDA_PMUNITS(0,0,0,0,0,0) /* no units */
};pmFetch requests with a pmResult in which no values are returned for the unsupported metric. This is marginally friendlier than the other semantically acceptable option of returning an illegal PMID error or PM_ERR_PMID.
pmExtractValue, pmConvScale, pmAtomStr, pmTypeStr, and pmPrintValue) return the PM_ERR_CONV error or an appropriate diagnostic string, if an attempt is made to operate on a value for which type is PM_TYPE_NOSUPPORT.
type field in the pmDesc structure, they should not manipulate values for unsupported metrics. Even if tools ignore type in the metric's description, following these development guidelines ensures that no misleading value is ever returned; so there is no reason to call the extraction, conversion, and printing functions.
pmdaMain.
pmdaMain can be used by a daemon PMDA as a wrapper to handle the communication protocol using the same callbacks as a DSO PMDA. This allows a PMDA to be built as both a daemon and a DSO, and then to be installed as either.
<pcp/pmda.h>:
pmdaExt structure as its last argument. This structure contains all the information that is required by the default callbacks in most cases. The one exception is pmdaFetch, which needs an additional callback to instantiate the current value for each supported combination of a performance metric and an instance.
libpcp.so and libpcp_pmda.so.
pmdaFetch is defined as trivial_fetchCallBack:
Example 2.16. Request Handling Callbacks in the Trivial PMDA
static int
trivial_fetchCallBack(pmdaMetric *mdesc, unsigned int inst, pmAtomValue *atom)
{
__pmID_int *idp = (__pmID_int *)&(mdesc->m_desc.pmid);
if (idp->cluster != 0 || idp->item != 0)
return PM_ERR_PMID;
if (inst != PM_IN_NULL)
return PM_ERR_INST;
atom->l = time(NULL);
return 0;
}pmdaSetFetchCallBack in trivial_init. As a rule of thumb, the API routines with named ending with CallBack are helpers for the higher PDU handling routines like pmdaFetch. The latter are set directly using the PMDA Interface Structures, as described in Section 2.5.2, “PMDA Structures”.
pmdaFetch is more complicated because it supports more metrics, some metrics are instantiated with each fetch, and one instance domain is dynamic. The default pmdaFetch callback, shown in Example 2.17, “Request Handling Callbacks in the Simple PMDA”, is replaced by simple_fetch in simple_init, which increments the number of fetches and updates the instance domain for INDOM_NOW before calling pmdaFetch:
Example 2.17. Request Handling Callbacks in the Simple PMDA
static int
simple_fetch(int numpmid, pmID pmidlist[], pmResult **resp, pmdaExt *pmda)
{
numfetch++;
simple_timenow_check();
simple_timenow_refresh();
return pmdaFetch(numpmid, pmidlist, resp, pmda);
}pmdaFetch is defined as simple_fetchCallBack. The PMID is extracted from the pmdaMetric structure, and if valid, the appropriate field in the pmAtomValue structure is set. The available types and associated fields are described further in Section 3.4, “Performance Metric Descriptions” and Example 3.16, “ pmAtomValue Structure”.
Note
simple.numfetch metric has no instance domain and is easily handled first as shown in Example 2.18, “ simple.numfetch Metric”:
Example 2.18. simple.numfetch Metric
static int
simple_fetchCallBack(pmdaMetric *mdesc, unsigned int inst, pmAtomValue *atom)
{
int i;
static int oldfetch;
static double usr, sys;
__pmID_int *idp = (__pmID_int *)&(mdesc->m_desc.pmid);
if (inst != PM_IN_NULL &&
!(idp->cluster == 0 && idp->item == 1) &&
!(idp->cluster == 2 && idp->item == 4))
return PM_ERR_INST;
if (idp->cluster == 0) {
if (idp->item == 0) { /* simple.numfetch */
atom->l = numfetch;
}simple.color Metric”, the inst parameter is used to specify which instance is required for the simple.color metric:
Example 2.19. simple.color Metric
else if (idp->item == 1) { /* simple.color */
switch (inst) {
case 0: /* red */
red = (red + 1) % 256;
atom->l = red;
break;
case 1: /* green */
green = (green + 1) % 256;
atom->l = green;
break;
case 2: /* blue */
blue = (blue + 1) % 256;
atom->l = blue;
break;
default:
return PM_ERR_INST;
}
}
else
return PM_ERR_PMID;simple.time Metric”, the simple.time metric is in a second cluster and has a simple optimization to reduce the overhead of calling times twice on the same fetch and return consistent values from a single call to times when both metrics simple.time.user and simple.time.sys are requested in a single pmFetch. The previous fetch count is used to determine if the usr and sys values should be updated:
Example 2.20. simple.time Metric
else if (idp->cluster == 1) { /* simple.time */
if (oldfetch < numfetch) {
__pmProcessRunTimes(&usr, &sys);
oldfetch = numfetch;
}
if (idp->item == 2) /* simple.time.user */
atom->d = usr;
else if (idp->item == 3) /* simple.time.sys */
atom->d = sys;
else
return PM_ERR_PMID;
}simple.now Metric”, the simple.now metric is in a third cluster and uses inst again to select a specific instance from the INDOM_NOW instance domain. The values associated with instances in this instance domain are managed using the pmdaCache(3) helper routines, which provide efficient interfaces for managing more complex instance domains:
Example 2.21. simple.now Metric
else if (idp->cluster == 2) {
if (idp->item == 4) { /* simple.now */
struct timeslice *tsp;
sts = pmdaCacheLookup(*now_indom, inst, NULL, (void *)&tsp);
if (sts != PMDA_CACHE_ACTIVE) {
if (sts < 0)
__pmNotifyErr(LOG_ERR, "pmdaCacheLookup failed: inst=%d: %s",
inst, pmErrStr(sts));
return PM_ERR_INST;
}
atom->l = tsp->tm_field;
}
else
return PM_ERR_PMID;
}simple_store in the Simple PMDApmStore as shown in Example 2.22, “ simple_store in the Simple PMDA”. For additional information, see the pmstore(1) and pmStore(3) man pages.
Example 2.22. simple_store in the Simple PMDA
pmdaStore callback (which returns PM_ERR_PERMISSION to indicate no metrics can be altered) is replaced by simple_store in simple_init. This replacement function must take the same arguments so that it can be assigned to the function pointer in the pmdaInterface structure.
pmResult and checks the cluster and unit of each PMID to ensure that it corresponds to a metric that can be changed. Checks are made on the values to ensure they are within range before being assigned to variables in the PMDA that hold the current values for exported metrics:
static int
simple_store(pmResult *result, pmdaExt *pmda)
{
int i, j, val, sts = 0;
pmAtomValue av;
pmValueSet *vsp = NULL;
__pmID_int *pmidp = NULL;
/* a store request may affect multiple metrics at once */
for (i = 0; i < result->numpmid; i++) {
vsp = result->vset[i];
pmidp = (__pmID_int *)&vsp->pmid;
if (pmidp->cluster == 0) { /* storable metrics are cluster 0 */
switch (pmidp->item) {
case 0: /* simple.numfetch */
val = vsp->vlist[0].value.lval;
if (val < 0) {
sts = PM_ERR_SIGN;
val = 0;
}
numfetch = val;
break;
case 1: /* simple.color */
/* a store request may affect multiple instances at once */
for (j = 0; j < vsp->numval && sts == 0; j++) {
val = vsp->vlist[j].value.lval;
if (val < 0) {
sts = PM_ERR_SIGN;
val = 0;
} if (val > 255) {
sts = PM_ERR_CONV;
val = 255;
}simple.color metric has an instance domain that must be searched because any or all instances may be specified. Any instances that are not supported in this instance domain should cause an error value of PM_ERR_INST to be returned as shown in Example 2.23, “ simple.color and PM_ERR_INST Errors”:
Example 2.23. simple.color and PM_ERR_INST Errors
switch (vsp->vlist[j].inst) {
case 0: /* red */
red = val;
break;
case 1: /* green */
green = val;
break;
case 2: /* blue */
blue = val;
break;
default:
sts = PM_ERR_INST;
}PM_ERR_PMID as shown in Example 2.24, “ PM_ERR_PMID Errors”:
PM_ERR_PERMISSION, and metrics not supported by the PMDA should result in an error value of PM_ERR_PMID as shown in Example 2.25, “ PM_ERR_PERMISSION and PM_ERR_PMID Errors”:
Example 2.25. PM_ERR_PERMISSION and PM_ERR_PMID Errors
else if ((pmidp->cluster == 1 &&
(pmidp->item == 2 || pmidp->item == 3)) ||
(pmidp->cluster == 2 && pmidp->item == 4)) {
sts = PM_ERR_PERMISSION;
break;
}
else {
sts = PM_ERR_PMID;
break;
}
}
return sts;
}pmdaExt pmda argument is not used by the simple_store function above.
Note
pmlogger is actively sampling the metric being modified, for example, which may cause unexpected results to be persisted in an archive. Consider also the use of client credentials, available via the attribute callback of the pmdaInterface structure, to appropriately limit access to any modifications that might be made via your storable metrics.
pmdaFetch CallbacksPMDA_INTERFACE_1 and PMDA_INTERFACE_2, the return codes for the pmdaFetch callback function are defined:
PM_ERR_PMID, PM_ERR_INST or PM_ERR_AGAIN)
PMDA_INTERFACE_3 and all later versions, the return codes for the pmdaFetch callback function are defined:
PM_ERR_PMID, PM_ERR_INST)
pcp_pmda library are defined in <pcp/pmda.h>. Example 2.26, “ pmdaInterface Structure Header” and Example 2.28, “ pmdaExt Stucture” describe the pmdaInterface and pmdaExt structures.
Example 2.26. pmdaInterface Structure Header
typedef struct {
int domain; /* set/return performance metrics domain id here */
struct {
unsigned int pmda_interface : 8; /* PMDA DSO version */
unsigned int pmapi_version : 8; /* PMAPI version */
unsigned int flags : 16; /* optional feature flags */
} comm; /* set/return communication and version info */
int status; /* return initialization status here */
union {
...
pmdaExt structure. Protocol version one is for backwards compatibility only, and should not be used in any new PMDA.
pmdaExt structure, as mentioned above.
pmdaFetch Callbacks”.
pmdaPMID, pmdaName, pmdaChildren interfaces)
pmdaGetContext helper routine). At the same time, support for PM_TYPE_EVENT metrics was implemented, which relies on the per-client context concepts (pmdaEvent* helper routines).
pmdaAttribute interface)
Example 2.27. pmdaInterface Structure, Latest Version
...
union {
...
/*
* PMDA_INTERFACE6
*/
struct {
pmdaExt *ext;
int (*profile)(pmdaInProfile *, pmdaExt *);
int (*fetch)(int, pmID *, pmResult **, pmdaExt *);
int (*desc)(pmID, pmDesc *, pmdaExt *);
int (*instance)(pmInDom, int, char *, pmdaInResult **, pmdaExt *);
int (*text)(int, int, char **, pmdaExt *);
int (*store)(pmResult *, pmdaExt *);
int (*pmid)(const char *, pmID *, pmdaExt *);
int (*name)(pmID, char ***, pmdaExt *);
int (*children)(const char *, int, char ***, int **, pmdaExt *);
int (*attribute)(int, int, const char *, int, pmdaExt *);
} six;
} version;
} pmdaInterface;Note
And thus it shall remain. For brevity, we have shown only the latest interface version (six) above, but all prior versions still exist, build, and function. In other words, PMDAs built against earlier versions of this header structure (and PMDA library) function correctly with the latest version of the PMDA library.
Example 2.28. pmdaExt Stucture
typedef struct {
unsigned int e_flags; /* PMDA_EXT_FLAG_* bit field */
void *e_ext; /* used internally within libpcp_pmda */
char *e_sockname; /* socket name to pmcd */
char *e_name; /* name of this pmda */
char *e_logfile; /* path to log file */
char *e_helptext; /* path to help text */
int e_status; /* =0 is OK */
int e_infd; /* input file descriptor from pmcd */
int e_outfd; /* output file descriptor to pmcd */
int e_port; /* port to pmcd */
int e_singular; /* =0 for singular values */
int e_ordinal; /* >=0 for non-singular values */
int e_direct; /* =1 if pmid map to meta table */
int e_domain; /* metrics domain */
int e_nmetrics; /* number of metrics */
int e_nindoms; /* number of instance domains */
int e_help; /* help text comes via this handle */
__pmProfile *e_prof; /* last received profile */
pmdaIoType e_io; /* connection type to pmcd */
pmdaIndom *e_indoms; /* instance domain table */
pmdaIndom *e_idp; /* instance domain expansion */
pmdaMetric *e_metrics; /* metric description table */
pmdaResultCallBack e_resultCallBack; /* to clean up pmResult after fetch */
pmdaFetchCallBack e_fetchCallBack; /* to assign metric values in fetch */
pmdaCheckCallBack e_checkCallBack; /* callback on receipt of a PDU */
pmdaDoneCallBack e_doneCallBack; /* callback after PDU is processed */
/* added for PMDA_INTERFACE_5 */
int e_context; /* client context id from pmcd */
pmdaEndContextCallBack e_endCallBack; /* callback after client context closed */
} pmdaExt;pmdaExt structure contains filenames, pointers to tables, and some variables shared by several functions in the pcp_pmda library. All fields of the pmdaInterface and pmdaExt structures can be correctly set by PMDA initialization functions; see the pmdaDaemon(3), pmdaDSO(3), pmdaGetOptions(3), pmdaInit(3), and pmdaConnect(3) man pages for a full description of how various fields in these structures may be set or used by pcp_pmda library functions.
pmdaInterface structure is initialized by main or the PMCD process, respectively.
pmdaInterface
pmdaExt
pcp_pmda library, most of these requirements can be handled by the default pmdaInit function; see the pmdaInit(3) man page.
pmdaDSO function; see the pmdaDSO(3) man page.
pmdaFetch function called trivial_fetchCallBack; see the pmdaFetch(3) man page:
Example 2.29. Initialization in the Trivial PMDA
static char *username;
static int isDSO = 1; /* ==0 if I am a daemon */
void trivial_init(pmdaInterface *dp)
{
if (isDSO)
pmdaDSO(dp, PMDA_INTERFACE_2, “trivial DSO”,
“${PCP_PMDAS_DIR}/trivial/help”);
else
__pmSetProcessIdentity(username);
if (dp->status != 0)
return;
pmdaSetFetchCallBack(dp, trivial_fetchCallBack);
pmdaInit(dp, NULL, 0,
metrictab, sizeof(metrictab)/sizeof(metrictab[0]));
}main routine clears isDSO and sets username accordingly.
trivial_init routine provides the opportunity to do any extra DSO or daemon setup before calling the library pmdaInit. In the example, the help text is setup for DSO mode and the daemon is switched to run as an unprivileged user (default is root, but it is generally good form for PMDAs to run with the least privileges possible). If dp->status is non-zero after the pmdaDSO call, the PMDA will be removed by PMCD and cannot safely continue to use the pmdaInterface structure.
PDU_FETCH and PDU_RESULT request PDUs (for pmFetch and pmStore operations respectively), as well as providing pmdaFetch with the callback simple_fetchCallBack.
Example 2.30. Initialization in the Simple PMDA
static int isDSO = 1; /* =0 I am a daemon */
static char *username;
void simple_init(pmdaInterface *dp)
{
if (isDSO)
pmdaDSO(dp, PMDA_INTERFACE_2, “simple DSO”,
“${PCP_PMDAS_DIR}/simple/help”);
else
__pmSetProcessIdentity(username);
if (dp->status != 0)
return;
dp->version.any.fetch = simple_fetch;
dp->version.any.store = simple_store;
dp->version.any.instance = simple_instance;
pmdaSetFetchCallBack(dp, simple_fetchCallBack);
pmdaInit(dp, indomtab, sizeof(indomtab)/sizeof(indomtab[0]),
metrictab, sizeof(metrictab)/sizeof(metrictab[0]));
}main, for PMDAs that can operate in both modes.
simple_fetch is dealing with a single request for (possibly many) values for metrics from the PMDA, and simple_fetchCallBack is its little helper, dealing with just one metric and one instance (optionally, if the metric happens to have an instance domain) within that larger request.
pmdaInterface structure that is passed to the initialization function
pcp_pmda library; see the pmdaDaemon(3), pmdaGetOptions(3), pmdaOpenLog(3), pmdaConnect(3), and pmdaMain(3) man pages.
Note
require the extraction process to run as a specific user in order to access the instrumentation. Many domains require the default root level of access for a daemon PMDA.
pmdaGetOptions, as shown in Example 2.31, “ main in the Simple PMDA”. For additional information, see the pmdaGetOptions(3) man page.
Example 2.31. main in the Simple PMDA
static pmLongOptions longopts[] = {
PMDA_OPTIONS_HEADER(“Options”),
PMOPT_DEBUG,
PMDAOPT_DOMAIN,
PMDAOPT_LOGFILE,
PMDAOPT_USERNAME,
PMOPT_HELP,
PMDA_OPTIONS_TEXT(“\nExactly one of the following options may appear:”),
PMDAOPT_INET,
PMDAOPT_PIPE,
PMDAOPT_UNIX,
PMDAOPT_IPV6,
PMDA_OPTIONS_END
};
static pmdaOptions opts = {
.short_options = “D:d:i:l:pu:U:6:?”,
.long_options = longopts,
};
int
main(int argc, char **argv)
{
pmdaInterface dispatch;
isDSO = 0;
__pmSetProgname(argv[0]);
__pmGetUsername(&username);
pmdaDaemon(&dispatch, PMDA_INTERFACE_2, pmProgname, SIMPLE,
“simple.log”, “${PCP_PMDAS_DIR}/simple/help”);
pmdaGetOptions(argc, argv, &opts, &dispatch);
if (opts.errors) {
pmdaUsageMessage(&opts);
exit(1);
}
if (opts.username)
username = opts.username;
pmdaOpenLog(&dispatch);
simple_init(&dispatch);
simple_timenow_check();
pmdaConnect(&dispatch);
pmdaMain(&dispatch);
exit(0);
}pmdaMain will return are either unexpected error conditions (often from failed initialisation, which would already have been logged), or when PMCD closes the connection to the PMDA. In all cases the correct action to take is simply to exit cleanly, possibly after any final cleanup the PMDA may need to perform.
${PCP_PMCDOPTIONS_PATH}, attaching a debugger (such as gdb) to the PMDA process might cause an already running PMCD to close its connection with the PMDA. If timeouts are disabled, PMCD could wait forever to connect with the PMDA.
${PCP_LOG_DIR}/pmcd/pmcd.log.
dbpmda tool, which is similar to PMCD except that dbpmda provides complete control over the PDUs that are sent to the PMDA, and there are no time limits--it is essentially an interactive debugger for exercising a PMDA. See the dbpmda(3) man page for details.
-D command-line option. Supported options can be listed with the pmdbg command; see the pmdbg(1) man page. Setting the debug options for PMCD in ${PCP_PMCDOPTIONS_PATH} might generate too much information to be useful, especially if there are other clients and PMDAs connected to the PMCD process.
pmcd.control.debug:
# pmstore pmcd.control.debug 5pcp_pmda library functions log additional information if the libpmda option is set within the PMDA; see the PMDA(3) man page. The command-line argument -D is trapped by pmdaGetOptions to set the global debugging control options. Adding tests within the PMDA for the appl0, appl1 and appl2 trace flags permits different levels of information to be logged to the PMDA's log file.
simple_store metric causes a timestamped log message to be sent to the current log file whenever pmstore attempts to change simple.numfetch and the PCP debugging options have the appl0 option set as shown in Example 2.32, “ simple.numfetch in the Simple PMDA”:
Example 2.32. simple.numfetch in the Simple PMDA
case 0: /* simple.numfetch */
x
val = vsp->vlist[0].value.lval;
if (val < 0) {
sts = PM_ERR_SIGN;
val = 0;
}
if (pmDebugOptions.appl0__) {
__pmNotifyErr(LOG_DEBUG,
"simple: %d stored into numfetch", val);
}
numfetch = val;
break;dbpmda Debug Utilitydbpmda utility provides a simple interface to the PDU communication protocol. It allows daemon and DSO PMDAs to be tested with most request types, while the PMDA process may be monitored with a debugger, tracing utilities, and other diagnostic tools. The dbpmda(1) man page contains a sample session with the simple PMDA.
${PCP_PMDAS_DIR} into which several files would be installed. In the description in Section 2.8.1, “Installing a PMDA”, the PMDA of interest is assumed to be known by the name newbie, hence the PMDA directory would be ${PCP_PMDAS_DIR}/newbie.
Note
${PCP_VAR_DIR}/pmns directory.
${PCP_PMDAS_DIR}/newbie.
${PCP_PMCDCONF_PATH} file has been updated.
SIGHUP signal) that the new PMDA exists.
Makefile should include an install target to compile and link the PMDA (as a DSO, or a daemon or both) in the PMDA directory. The clobber target should remove any files created as a by-product of the install target.
${PCP_PMDAS_DIR}/simple/Makefile as a template for constructing a new PMDA Makefile; changing the assignment of IAM from simple to newbie would account for most of the required changes.
Install script should make use of the generic procedures defined in the script ${PCP_SHARE_DIR}/lib/pmdaproc.sh, and may be as straightforward as the one used for the trivial PMDA, shown in Example 2.33, “ Install Script for the Trivial PMDA”:
Example 2.33. Install Script for the Trivial PMDA
. ${PCP_DIR}/etc/pcp.env
. ${PCP_SHARE_DIR}/lib/pmdaproc.sh
iam=trivial
pmda_interface=2
pmdaSetup
pmdainstall
exitpmdaproc.sh Procedures”, may be assigned values to modify the behavior of the pmdaSetup and pmdainstall procedures from ${PCP_SHARE_DIR}/lib/pmdaproc.sh.
Table 2.1. Variables to Control Behavior of Generic pmdaproc.sh Procedures
|
Shell Variable
|
Use
|
Default
|
|---|---|---|
$iam
|
Name of the PMDA; assignment to this variable is mandatory.
Example:
iam=newbie
| |
$dso_opt
|
Can this PMDA be installed as a DSO?
| false
|
$daemon_opt
|
Can this PMDA be installed as a daemon?
| true
|
$perl_opt
|
Is this PMDA a perl script?
| false
|
$python_opt
|
Is this PMDA a python script?
| false
|
$pipe_opt
|
If installed as a daemon PMDA, is the default IPC via pipes?
| true
|
$socket_opt
|
If installed as a daemon PMDA, is the default IPC via an Internet socket?
| false
|
$socket_inet_def
|
If installed as a daemon PMDA, and the IPC method uses an Internet socket, the default port number.
| |
$ipc_prot
|
IPC style for PDU exchanges involving a daemon PMDA;
binary or text.
| binary
|
$check_delay
|
Delay in seconds between installing PMDA and checking if metrics are available.
| 3
|
$args
|
Additional command-line arguments passed to a daemon PMDA.
| |
$pmda_interface
|
Version of the
libpcp_pmda library required, used to determine the version for generating help text files.
| 1
|
$pmns_source
|
The name of the PMNS file (by default relative to the PMDA directory).
| pmns
|
$pmns_name
|
First-level name for this PMDA's metrics in the PMNS.
| $iam
|
$help_source
|
The name of the help file (by default relative to the PMDA directory).
| help
|
$pmda_name
|
The name of the executable for a daemon PMDA.
| pmda$iam
|
$dso_name
|
The name of the shared library for a DSO PMDA.
| pmda$iam.$dso_suffix
|
$dso_entry
|
The name of the initialization function for a DSO PMDA.
| ${iam}_init
|
$domain
|
The numerical PMDA domain number (from
domain.h).
| |
$SYMDOM
|
The symbolic name of the PMDA domain number (from
domain.h).
| |
$status
|
Exit status for the shell script
| 0
|
do_pmda and do_check will be set to reflect the intention to install the PMDA (as opposed to install just the PMNS) and to check the availability of the metrics once the PMDA is installed. By default, each variable is true; however, the command-line options -N and -Q to Install may be used to set the variables to false, as follows: do_pmda (-N) and do_check (-N or -Q).
Example 2.34. Changing Variable Assignments
You will need to choose an appropriate configuration for installation of the ... Performance Metrics Domain Agent (PMDA). collector collect performance statistics on this system monitor allow this system to monitor local and/or remote systems both collector and monitor configuration for this system
pmdaSetup procedure, the Install script should also prompt for any PMDA-specific parameters, which are typically accumulated in the args variable and used by the pmdainstall procedure.
pmdainstall procedure involves the following tasks:
$pmns_source file, replacing symbolic references to SYMDOM by the desired numeric domain number from domain.
$pmns_name.
pmchart views can be found (files with names ending in “.pmchart”), copy these to the standard directory (${PCP_VAR_DIR}/config/pmchart) with the “.pmchart” suffix removed.
Makefile to build the appropriate executables.
${PCP_PMCDCONF_PATH}).
SIGHUP to PMCD forces it to reread the configuration file and start, restart, or remove any PMDAs that have changed since the file was last read. However, if the newly installed PMDA must run using a different privilege level to PMCD then PMCD must be restarted. This is because PMCD runs unprivileged after initially starting the PMDAs.
# ${PCP_RC_DIR}/pcp start-N command-line option, the PMDA Install script may be used to update the PMNS without installing the PMDA. This functionality is rarely, if ever, used in modern versions of PCP, but allows one to populate the local PMNS with the names of the performance metrics from a PMDA installed on a remote host. The -N option can also install pmchart views useful on a monitoring system, although this also is rarely used now with each platforms package management tools handling this task.
${PCP_PMCDCONF_PATH} and signal PMCD (with SIGHUP) to reread its configuration file. To completely remove a PMDA requires the reverse process of the installation, including an update of the Performance Metrics Name Space (PMNS).
Remove script in the PMDA directory that uses the same common procedures as the Install script described Section 2.8.1, “Installing a PMDA”.
hinv.physmem has a single 32-bit unsigned integer value, representing the number of megabytes of physical memory in the system, while the performance metric disk.dev.total has one 32-bit unsigned integer value per disk spindle, representing the cumulative count of I/O operations involving each associated disk spindle. These concepts are described in greater detail in Section 2.3, “Domains, Metrics, and Instances”.
pmID.
metric instances is really a number of related concepts, as follows:
Example 3.1. Metrics Sharing the Same Instance Domain
$ pminfo -f filesys.free
filesys.free
inst [1 or “/dev/disk0”] value 1803
inst [2 or “/dev/disk1”] value 22140
inst [3 or “/dev/disk2”] value 157938filesys.free has three values, currently 1803, 22140, and 157938. These values are respectively associated with the instances identified by the internal identifiers 1, 2 and 3, and the external identifiers /dev/disk0, /dev/disk1, and /dev/disk2. These instances form an instance domain that is shared by the performance metrics filesys.capacity, filesys.used, filesys.free, filesys.mountdir, and so on.
typedef declaration:
typedef unsigned long pmInDom;
PM_INDOM_NULL is reserved to indicate that the metric has a single value (a singular instance domain). For example, the performance metric mem.freemem always has exactly one value. Note that this is semantically different to a performance metric like kernel.percpu.cpu.sys that has a non-singular instance domain, but may have only one value available; for example, on a system with a single processor.
PM_IN_NULL is reserved for the single value in a singular instance domain. Performance metric values are delivered across the PMAPI as a set of instance identifier and value pairs.
Note
proc.memory.rss (one instance per process), swap.free (one instance per swap partition) and kernel.percpu.cpu.intr (one instance per CPU).
pmlogger at some remote host or earlier time (a retrospective or archive source). The metric's source is specified when the PMAPI context is created by calling the pmNewContext function. This function returns an opaque handle which can be used to identify the context.
pmSetMode function.
pmAddProfile and pmDelProfile functions.
pmUseContext. If a live context connection fails, the pmReconnectContext function can be used to attempt to reconnect it.
pmDesc structure) that describes the format and semantics of the performance metric. The pmDesc structure, in Example 3.2, “ pmDesc Structure”, provides all of the information required to interpret and manipulate a performance metric through the PMAPI. It has the following declaration:
Example 3.2. pmDesc Structure
/* Performance Metric Descriptor */
typedef struct {
pmID pmid; /* unique identifier */
int type; /* base data type (see below) */
pmInDom indom; /* instance domain */
int sem; /* semantics of value (see below) */
pmUnits units; /* dimension and units (see below) */
} pmDesc;type field in the pmDesc structure describes various encodings of a metric's value. Its value will be one of the following constants:
/* pmDesc.type - data type of metric values */ #define PM_TYPE_NOSUPPORT -1 /* not in this version */ #define PM_TYPE_32 0 /* 32-bit signed integer */ #define PM_TYPE_U32 1 /* 32-bit unsigned integer */ #define PM_TYPE_64 2 /* 64-bit signed integer */ #define PM_TYPE_U64 3 /* 64-bit unsigned integer */ #define PM_TYPE_FLOAT 4 /* 32-bit floating point */ #define PM_TYPE_DOUBLE 5 /* 64-bit floating point */ #define PM_TYPE_STRING 6 /* array of char */ #define PM_TYPE_AGGREGATE 7 /* arbitrary binary data */ #define PM_TYPE_AGGREGATE_STATIC 8 /* static pointer to aggregate */ #define PM_TYPE_EVENT 9 /* packed pmEventArray */ #define PM_TYPE_UNKNOWN 255 /* used in pmValueBlock not pmDesc */
pmResult structure, using a container metric with a value of type PM_TYPE_EVENT.
PM_TYPE_STRING, PM_TYPE_AGGREGATE, PM_TYPE_AGGREGATE_STATIC, or PM_TYPE_EVENT, the interpretation of that value is unknown to many PCP components. In the case of the aggregate types, the application using the value and the Performance Metrics Domain Agent (PMDA) providing the value must have some common understanding about how the value is structured and interpreted. Strings can be manipulated using the standard C libraries. Event records contain timestamps, event flags and event parameters, and the PMAPI provides support for unpacking an event record - see the pmUnpackEventRecords(3) man page for details. Further discussion on event metrics and event records can be found in Section 3.6, “Performance Event Metrics”.
PM_TYPE_NOSUPPORT indicates that the PCP collection framework knows about the metric, but the corresponding service or application is either not configured or is at a revision level that does not provide support for this performance metric.
sem field of a pmDesc structure and uses the following constants:
/* pmDesc.sem - semantics of metric values */ #define PM_SEM_COUNTER 1 /* cumulative count, monotonic increasing */ #define PM_SEM_INSTANT 3 /* instantaneous value continuous domain */ #define PM_SEM_DISCRETE 4 /* instantaneous value discrete domain */
pmUnits data structure, shown in Example 3.3, “ pmUnits and pmDesc Structures”. It is embedded in the pmDesc structure :
Example 3.3. pmUnits and pmDesc Structures
/*
* Encoding for the units (dimensions and
* scale) for Performance Metric Values
*
* For example, a pmUnits struct of
* { 1, -1, 0, PM_SPACE_MBYTE, PM_TIME_SEC, 0 }
* represents Mbytes/sec, while
* { 0, 1, -1, 0, PM_TIME_HOUR, 6 }
* represents hours/million-events
*/
typedef struct {
int pad:8;
int scaleCount:4; /* one of PM_COUNT_* below */
int scaleTime:4; /* one of PM_TIME_* below */
int scaleSpace:4; /* one of PM_SPACE_* below */
int dimCount:4; /* event dimension */
int dimTime:4; /* time dimension */
int dimSpace:4; /* space dimension
} pmUnits; /* dimensional units and scale of value */
/* pmUnits.scaleSpace */
#define PM_SPACE_BYTE 0 /* bytes */
#define PM_SPACE_KBYTE 1 /* Kilobytes (1024) */
#define PM_SPACE_MBYTE 2 /* Megabytes (1024^2) */
#define PM_SPACE_GBYTE 3 /* Gigabytes (1024^3) */
#define PM_SPACE_TBYTE 4 /* Terabytes (1024^4) */
/* pmUnits.scaleTime */
#define PM_TIME_NSEC 0 /* nanoseconds */
#define PM_TIME_USEC 1 /* microseconds */
#define PM_TIME_MSEC 2 /* milliseconds */
#define PM_TIME_SEC 3 /* seconds */
#define PM_TIME_MIN 4 /* minutes */
#define PM_TIME_HOUR 5 /* hours */
/*
* pmUnits.scaleCount (e.g. count events, syscalls,
* interrupts, etc.) -- these are simply powers of 10,
* and not enumerated here.
* e.g. 6 for 10^6, or -3 for 10^-3
*/
#define PM_COUNT_ONE 0 /* 1 */pmFetch (or pmStore) function call. To accommodate this, values are delivered across the PMAPI in the form of a tree data structure, rooted at a pmResult structure. This encoding is illustrated in Figure 3.1, “A Structured Result for Performance Metrics from pmFetch”, and uses the component data structures in Example 3.4, “ pmValueBlock and pmValue Structures”:
Example 3.4. pmValueBlock and pmValue Structures
typedef struct {
int inst; /* instance identifier */
union {
pmValueBlock *pval; /* pointer to value-block */
int lval; /* integer value insitu */
} value;
} pmValue;inst element. If a value for a particular metric-instance pair is a 32-bit integer (signed or unsigned), then it will be stored in the lval element. If not, the value will be in a pmValueBlock structure, as shown in Example 3.5, “pmValueBlock Structure”, and will be located via pval:
pmValueBlock structure is as follows:
Example 3.5. pmValueBlock Structure
typedef struct {
unsigned int vlen : 24; /* bytes for vtype/vlen + vbuf */
unsigned int vtype : 8; /* value type */
char vbuf[1]; /* the value */
} pmValueBlock;pmValueBlock (including the vtype and vlen fields) is stored in vlen. Despite the prototype declaration of vbuf, this array really accommodates vlen minus sizeof(vlen) bytes. The vtype field encodes the type of the value in the vbuf[] array, and is one of the PM_TYPE_* macros defined in <pcp/pmapi.h>.
pmValueSet structure, as shown in Example 3.6, “ pmValueSet Structure”, contains all of the values to be returned from pmFetch for a single performance metric identified by the pmid field.
Example 3.6. pmValueSet Structure
typedef struct {
pmID pmid; /* metric identifier */
int numval; /* number of values */
int valfmt; /* value style, insitu or ptr */
pmValue vlist[1]; /* set of instances/values */
} pmValueSet;numval field identifies the number of value-instance pairs in the vlist array (despite the prototype declaration of size 1). If numval is zero, there are no values available for the associated performance metric and vlist[0] is undefined. A negative value for numval indicates an error condition (see the pmErrStr(3) man page) and vlist[0] is undefined. The valfmt field has the value PM_VAL_INSITU to indicate that the values for the performance metrics should be located directly via the lval member of the value union embedded in the elements of vlist; otherwise, metric values are located indirectly via the pval member of the elements of vlist.
pmResult structure, as shown in Example 3.7, “ pmResult Structure”, contains a time stamp and an array of numpmid pointers to pmValueSet.
Example 3.7. pmResult Structure
/* Result returned by pmFetch() */
typedef struct {
struct timeval timestamp; /* stamped by collector */
int numpmid; /* number of PMIDs */
pmValueSet *vset[1]; /* set of value sets */
} pmResultpmValueSet pointer per PMID, with a one-to-one correspondence to the set of requested PMIDs passed to pmFetch.
pmResult that serves to identify when the performance metric values were collected. The time is in the format returned by gettimeofday and is typically very close to the time when the metric values were extracted from their respective domains.
Note
write(2) syscall entry point encoding”.
pmlogger for subsequent replay.
pmResult structure seen previously in Example 3.7, “ pmResult Structure”.
pmFetch(3) specifying an event metric PMID are such that all events observed on the collector since the previous fetch (by this specific monitor client) are to transfered to the monitor. Each event will have the metadata described earlier encoded with it (timestamps, flags, and so on) for each event. The encoding of the series of events involves a compound data structure within the pmValueSet associated with the event metric PMID, as illustrated in Figure 3.3, “Result Format for Event Performance Metrics from pmFetch”.
pmEventArray structure, as in Example 3.8, “ pmEventArray and pmEventRecord Structures”:
Example 3.8. pmEventArray and pmEventRecord Structures
typedef struct {
__pmTimeval er_timestamp; /* 2 x 32-bit timestamp format */
unsigned int er_flags; /* event record characteristics */
int er_nparams; /* number of ea_param[] entries */
pmEventParameter er_param[1];
} pmEventRecord;
typedef struct {
unsigned int ea_len : 24; /* bytes for type/len + records */
unsigned int ea_type : 8; /* value type */
int ea_nrecords; /* number of ea_record entries */
pmEventRecord ea_record[1];
} pmEventArray;pmEventRecord structure is used to convey the number of events dropped - er_flags is used to indicate the presence of dropped events, and er_nparams is used to hold a count. Unsurprisingly, the parameters (er_param) will be empty in this situation.
pmEventParameter structure is as follows:
Example 3.9. pmEventParameter Structure
typedef struct {
pmID ep_pmid; /* parameter identifier */
unsigned int ep_type; /* value type */
int ep_len; /* bytes for type/len + vbuf */
/* actual value (vbuf) here */
} pmEventParameter;pmUnpackEventRecords function for monitor tools. This function is passed a pointer to a pmValueSet associated with an event metric (within a pmResult) from a pmFetch(3). For a given instance of that event metric, it returns an array of "unpacked" pmResult structures for each event.
event.flags and event.missed. Note that these metrics will only exist after the first call to pmUnpackEventRecords.
pmValueSet”:
Example 3.10. Unpacking Event Records from an Event Metric pmValueSet
enum { event_flags = 0, event_missed = 1 };
static char *metadata[] = { "event.flags", "event.missed" };
static pmID metapmid[2];
void dump_event(pmValueSet *vsp, int idx)
{
pmResult **res;
int r, sts, nrecords;
nrecords = pmUnpackEventRecords(vsp, idx, &res);
if (nrecords < 0)
fprintf(stderr, " pmUnpackEventRecords: %s\n", pmErrStr(nrecords));
else
printf(" %d event records\n", nrecords);
if ((sts = pmLookupName(2, &metadata, &metapmid)) < 0) {
fprintf(stderr, "Event metadata error: %s\n", pmErrStr(sts));
exit(1);
}
for (r = 0; r < nrecords; r++)
dump_event_record(res, r);
if (nrecords >= 0)
pmFreeEventResult(res);
}
void dump_event_record(pmResult *res, int r)
{
int p;
__pmPrintStamp(stdout, &res[r]->timestamp);
if (res[r]->numpmid == 0)
printf(" ==> No parameters\n");
for (p = 0; p < res[r]->numpmid; p++) {
pmValueSet *vsp = res[r]->vset[p];
if (vsp->numval < 0) {
int error = vsp->numval;
printf("%s: %s\n", pmIDStr(vsp->pmid), pmErrStr(error));
} else if (vsp->pmid == metapmid[event_flags]) {
int flags = vsp->vlist[0].value.lval;
printf(" flags 0x%x (%s)\n", flags, pmEventFlagsStr(flags));
} else if (vsp->pmid == metapmid[event_missed]) {
int count = vsp->vlist[0].value.lval;
printf(" ==> %d missed event records\n", count);
} else {
dump_event_record_parameters(vsp);
}
}
}
void dump_event_record_parameters(pmValueSet *vsp)
{
pmDesc desc;
char *name;
int sts, j;
if ((sts = pmLookupDesc(vsp->pmid, &desc)) < 0) {
fprintf(stderr, "pmLookupDesc: %s\n", pmErrStr(sts));
} else
if ((sts = pmNameID(vsp->pmid, &name)) < 0) {
fprintf(stderr, "pmNameID: %s\n", pmErrStr(sts));
} else {
printf("parameter %s", name);
for (j = 0; j < vsp->numval; j++) {
pmValue *vp = &vsp->vlist[j];
if (vsp->numval > 1) {
printf("[%d]", vp->inst);
pmPrintValue(stdout, vsp->valfmt, desc.type, vp, 1);
putchar('\n');
}
}
free(name);
}
}Note
pcp_pmda library to assist, however - refer to the man page entries for pmdaEventNewQueue(3) and pmdaEventNewClient(3) for further details.
pmEventArray structure, introduced in Example 3.8, “ pmEventArray and pmEventRecord Structures”. This can be done directly, or using the helper routine pmdaEventNewArray(3). If the latter, simpler model is chosen, the closely related routines pmdaEventAddRecord, pmdaEventAddParam and pmdaEventAddMissedRecord would also usually be used.
pmStore(3) mechanism to allow monitor tools to send a filter to the PMDA - using either a special control metric for the store operation, or the event metric itself. The filter sent will depend on the event metric, but it might be a regular expression, or a tracing script, or something else.
varargs approach and sentinel-terminated lists. Where the size of a result is known at the time of a call, it is the caller's responsibility to allocate (and possibly free) the storage, and the called function assumes that the resulting argument is of an appropriate size.
pmGetChildren, pmGetInDom, pmNameInDom, pmNameID, pmLookupText, and pmFetch), the underlying implementation uses dynamic allocation through malloc in the called function, with the caller responsible for subsequently calling free to release the storage when no longer required. In the case of the result from pmFetch, there is a function (pmFreeResult) to release the storage, due to the complexity of the data structure and the need to make multiple calls to free in the correct sequence. As a general rule, if the called function returns an error status, then no allocation is done, the pointer to the variable sized result is undefined, and free or pmFreeResult should not be called.
import cpmapi as c_api
from pcp import pmapi
from pcp.pmapi import pmErr
from pcp import ppmgui
context = pmapi.pmContext()or by automated processing of the command line arguments (refer to the
pmGetOptions man page for greater detail).
options = pmapi.pmOptions(...)
context = pmapi.pmContext.fromOptions(options, sys.argv)
Most PMAPI C functions have python equivalents with similar, although not identical, call signatures. Some of the python functions do not return native python types, but instead return native C types wrapped by the ctypes library. In most cases these types are opaque, or nearly so; for example pmid:
pmid = context.pmLookupName("mem.freemem")
desc = context.pmLookupDescs(pmid)
result = context.pmFetch(pmid)
...
See the comparison of a standalone C and python client application in Example 3.23, “PMAPI Error Handling”.
int. Values greater than or equal to zero indicate no error, and perhaps some positive status: for example, the number of items processed.
strerror is provided to translate error conditions into error messages; see the pmErrStr(3) and pmErrStr_r(3) man pages. The error condition is returned as the function value from a previous PMAPI call; there is no global error indicator (unlike errno). This is to accommodate multi-threaded performance tools.
pmerr -l
pmGetChildren Functionint pmGetChildren(const char*name, char***offspring)
Python:
[name1, name2...] = pmGetChildren(name)"" but not NULL or (char *)0), the immediate descendents of the root node in the PMNS are returned.
pmGetChildren returns the number of descendent names discovered, or a value less than zero for an error. The value zero indicates that the name is valid, and associated with a leaf node in the PMNS.
pmGetChildren with a single call to malloc, and it is the responsibility of the caller to issue a free(offspring) system call to release the space when it is no longer required. When the result of pmGetChildren is less than one, offspring is undefined (no space is allocated, and so calling free is counterproductive).
pmGetChildrenStatus Functionint pmGetChildrenStatus(const char *name, char ***offspring, int **status)
Python:
([name1, name2...],[status1, status2...]) = pmGetChildrenStatus(name)pmGetChildrenStatus function is an extension of pmGetChildren that optionally returns status information about each of the descendent names.
pmGetChildrenStatus returns by means of offspring a list of the relative names of all of the immediate descendent nodes of name in the current PMNS. If name is the empty string (””), it returns the immediate descendents of the root node in the PMNS.
pmGetChildrenStatus also returns the status of each child by means of status. This refers to either a leaf node (with value PMNS_LEAF_STATUS) or a non-leaf node (with value PMNS_NONLEAF_STATUS).
pmGetChildrenStatus returns the number of descendent names discovered, or else a value less than zero to indicate an error. The value zero indicates that name is a valid metric name, being associated with a leaf node in the PMNS.
pmGetChildrenStatus with a single call to malloc, and it is the responsibility of the caller to free(offspring) to release the space when it is no longer required. The same holds true for the status array.
pmGetPMNSLocation Functionint pmGetPMNSLocation(void)
Python:
int loc = pmGetPMNSLocation()pmGetPMNSLocation returns whether it is an archive (PMNS_ARCHIVE), a local PMNS file (PMNS_LOCAL), or a remote PMCD (PMNS_REMOTE). This information may be useful in determining an appropriate error message depending on PMNS location.
pmLoadNameSpace Functionint pmLoadNameSpace(const char *filename)
Python:
int status = pmLoadNameSpace(filename)pmLoadNameSpace.
PM_NS_DEFAULT may be used for filename, to force a default local PMNS to be established. Externally, a PMNS is stored in an ASCII format.
Note
pmLoadNameSpace. Without this call, the default PMNS is the one at the source of the performance metrics (PMCD or an archive).
pmLookupName Functionint pmLookupName(int numpmid, char *namelist[], pmID pmidlist[])
Python:
c_uint pmid [] = pmLookupName("MetricName")
c_uint pmid [] = pmLookupName(("MetricName1", "MetricName2", ...))pmLookupName returns the list of associated PMIDs through the pmidlist parameter. Invalid metrics names are translated to the error PMID value of PM_ID_NULL.
pmLookupName is the number of names translated in the absence of errors, or an error indication. Note that argument definition and the error protocol guarantee a one-to-one relationship between the elements of namelist and pmidlist; both lists contain exactly numpmid elements.
pmLookupDescs and pmFetch.
pmNameAll Functionint pmNameAll(pmID pmid, char ***nameset)
Python:
[name1, name2...] = pmNameAll(pmid)pmNameAll determines all the corresponding metric names, if any, in the PMNS, and returns these through nameset.
pmNameAll with a single call to malloc. It is the caller's responsibility to call free and release the space when it is no longer required.
pmNameAll returns the number of names in nameset.
pmNameID provides a simpler interface in the absence of duplicate names for a particular PMID.
pmNameID Functionint pmNameID(pmID pmid, char **name)
Python:
"metric name" = pmNameID(pmid)pmNameID determines the corresponding metric name, if any, in the current PMNS, and returns this through name.
pmNameID returns zero. The name argument is a null byte terminated string, allocated by pmNameID using malloc. It is the caller's responsibility to call free and release the space when it is no longer required.
pmTraversePMNS Functionint pmTraversePMNS(const char *name, void (*dometric)(const char *))Python:int status = pmTraversePMNS(name, traverse_callback)
pmTraversePMNS may be used to perform a depth-first traversal of the PMNS. The traversal starts at the node identified by name --if name is an empty string, the traversal starts at the root of the PMNS. Usually, name would be the pathname of a non-leaf node in the PMNS.
dometric is called with the full pathname of that metric in the PMNS as the single argument; this argument is a null byte-terminated string, and is constructed from a buffer that is managed internally to pmTraversePMNS. Consequently, the value is valid only during the call to dometric--if the pathname needs to be retained, it should be copied using strdup before returning from dometric; see the strdup(3) man page.
pmUnloadNameSpace Functionint pmUnloadNameSpace(void)
Python:
pmUnLoadNameSpace("NameSpace")pmLoadNameSpace, calling pmUnloadNameSpace frees up the memory associated with the PMNS and force all subsequent Name Space functions to use the distributed PMNS. If pmUnloadNameSpace is called before calling pmLoadNameSpace, it has no effect.
pmLoadNameSpace Function” there are few if any situations where clients need to call this routine in modern versions of PCP.
pmLookupDesc Functionint pmLookupDesc(pmID pmid, pmDesc *desc)
Python:
pmDesc* pmdesc = pmLookupDesc(c_uint pmid)
(pmDesc* pmdesc)[] = pmLookupDescs(c_uint pmids[N])
(pmDesc* pmdesc)[] = pmLookupDescs(c_uint pmid)pmLookupDesc returns the associated pmDesc structure through the parameter desc from the current PMAPI context. For more information about pmDesc, see Section 3.4, “Performance Metric Descriptions”.
pmDesc corresponding to pmid. The returned pmdesc is passed to pmExtractValue and pmLookupInDom. The python bindings provide an entry pmLookupDescs that is similar to pmLookupDesc but does a metric description lookup for each element in a PMID array pmids.
pmLookupInDomText Functionint pmLookupInDomText(pmInDom indom, int level, char **buffer)
Python:
"metric description" = pmGetInDomText(pmDesc pmdesc)PM_TEXT_ONELINE for a one-line summary, or PM_TEXT_HELP for a more verbose description suited to a help dialogue. The space pointed to by buffer is allocated in pmLookupInDomText with malloc, and it is the responsibility of the caller to free unneeded space; see the malloc(3) and free(3) man pages.
pmLookupInDomText are often created using newhelp and accessed by the appropriate PMDA response to requests forwarded to the PMDA by PMCD. Further details may be found in Section 2.4.4, “PMDA Help Text”.
pmLookupText Functionint pmLookupText(pmID pmid, int level, char **buffer)
Python:
"metric description" = pmLookupText(c_uint pmid)PM_TEXT_ONELINE for a one-line summary, or PM_TEXT_HELP for a more verbose description, suited to a help dialogue.
pmLookupText with malloc, and it is the responsibility of the caller to free the space when it is no longer required; see the malloc(3) and free(3) man pages.
pmLookupText are created using newhelp and accessed by the appropriate PMDA in response to requests forwarded to the PMDA by PMCD. Further details may be found in Section 2.4.4, “PMDA Help Text”.
pmGetInDom Functionint pmGetInDom(pmInDom indom, int **instlist, char ***namelist)
Python:
([instance1, instance2...] [name1, name2...]) pmGetInDom(pmDesc pmdesc)pmGetInDom with two calls to malloc, and it is the responsibility of the caller to use free(instlist) and free(namelist) to release the space when it is no longer required. When the result of pmGetInDom is less than one, both instlist and namelist are undefined (no space is allocated, and so calling free is a bad idea); see the malloc(3) and free(3) man pages.
pmLookupInDom Functionint pmLookupInDom(pmInDom indom, const char *name)
Python:
int instid = pmLookupInDom(pmDesc pmdesc, "Instance")pmNameInDom Functionint pmNameInDom(pmInDom indom, int inst, char **name)
Python:
"instance id" = pmNameInDom(pmDesc pmdesc, c_uint instid)pmNameInDom with malloc, and it is the responsibility of the caller to free the space when it is no longer required; see the malloc(3) and free(3) man pages.
Table 3.1. Context Components of PMAPI Functions
|
Function Name
|
Metrics Source
|
Instance Profile
|
Collection Time
|
Notes
|
|---|---|---|---|---|
|
Yes
|
Yes
| | | |
|
Yes
|
Yes
| | | |
|
Yes
|
Yes
|
Yes
| | |
|
Yes
|
Yes
|
Yes
| | |
|
Yes
| |
Yes
|
(1)
| |
|
Yes
| | |
(1)
| |
|
Yes
| | |
(1)
| |
|
Yes
| | | | |
|
Yes
| | | | |
|
Yes
| | | | |
|
Yes
| | | | |
|
Yes
| |
Yes
|
(2)
| |
|
Yes
| | |
(1)
| |
|
Yes
| | |
(3)
| |
|
Yes
| |
Yes
|
(2)
| |
|
Yes
| | |
(1,2)
| |
|
Yes
| | |
(4)
| |
|
Yes
| | | | |
|
Yes
| | |
(4)
| |
pmNameAll
|
Yes
| | | |
|
Yes
| | | | |
|
Yes
| |
Yes
|
(2)
| |
|
Yes
| | |
(1,2)
| |
|
Yes
| |
Yes
| | |
|
Yes
| | |
(5)
| |
|
Yes
| | | |
pmNewContext Functionint pmNewContext(int type, const char *name)
pmNewContext function may be used to establish a new PMAPI context. The source of metrics is identified by name, and may be a host specification (type is PM_CONTEXT_HOST) or a comma-separated list of names referring to a set of archive logs (type is PM_CONTEXT_ARCHIVE). Each element of the list may either be the base name common to all of the physical files of an archive log or the name of a directory containing archive logs.
pmproxy host. Various other connection attributes, such as authentication information (user name, password, authentication method, and so on) can also be specified. Further details can be found in the PCPIntro(3) man page, and the companion Performance Co-Pilot Tutorials and Case Studies document.
PM_CONTEXT_ARCHIVE, there are some restrictions on the archives within the specified set:
PM_CONTEXT_LOCAL, name is ignored, and the context uses a stand-alone connection to the PMDA methods used by PMCD. When this type of context is in effect, the range of accessible performance metrics is constrained to DSO PMDAs listed in the pmcd configuration file ${PCP_PMCDCONF_PATH}. The reason this is done, as opposed to all of the DSO PMDAs found below ${PCP_PMDAS_DIR} for example, is that DSO PMDAs listed there are very likely to have their metric names reflected in the local Name Space file, which will be loaded for this class of context.
pmFetch.
pmUseContext. This new PMAPI context stays in effect for all subsequent context sensitive calls across the PMAPI until another call to pmNewContext is made, or the context is explicitly changed with a call to pmDupContext or pmUseContext.
pmDestroyContext Functionint pmDestroyContext(int handle)
pmNewContext or pmDupContext.
pmDestroyContext returns zero. If handle was the current PMAPI context, then the current context becomes undefined. This means the application must explicitly re-establish a valid PMAPI context with pmUseContext, or create a new context with pmNewContext or pmDupContext, before the next PMAPI operation requiring a PMAPI context.
pmDupContext Functionint pmDupContext(void)
pmUseContext Functionint pmUseContext(int handle)
pmUseContext causes the current PMAPI context to be set to the context identified by handle. The value of handle must be one returned from an earlier call to pmNewContext or pmDupContext.
pmUseContext restores the context to the state it was in the last time the context was used, not the state of the context when it was established.
pmWhichContext Functionint pmWhichContext(void)
Python:
int ctx_idx = pmWhichContext()pmAddProfile Functionint pmAddProfile(pmInDom indom, int numinst, int instlist[])
Python:
int status = pmAddProfile(pmDesc pmdesc, [c_uint instid])PM_INDOM_NULL, or numinst is zero, then all instance domains are selected. If instlist is NULL, then all instances are selected. To enable all available instances in all domains, use this syntax:
pmAddProfile(PM_INDOM_NULL, 0, NULL).
pmDelProfile Functionint pmDelProfile(pmInDom indom, int numinst, int instlist[])
Python:
int status = pmDelProfile(pmDesc pmdesc, c_uint instid)
int status = pmDelProfile(pmDesc pmdesc, [c_uint instid])
PM_INDOM_NULL, then all instance domains are selected for deletion. If instlist is NULL, then all instances in the selected domains are removed from the profile. To disable all available instances in all domains, use this syntax:
pmDelProfile(PM_INDOM_NULL, 0, NULL)
pmSetMode Functionint pmSetMode(int mode, const struct timeval *when, int delta)
Python:
int status = pmSetMode(mode, timeVal timeval, int delta)pmFetch, pmFetchArchive, pmLookupDesc, pmGetInDom, pmLookupInDom , and pmNameInDom.
pmSetMode function requires the current PMAPI context to be of type PM_CONTEXT_ARCHIVE.
PM_MODE_INTERP then, in the case of pmFetch, the underlying code uses an interpolation scheme to compute the values of the metrics from the values recorded for times in the proximity of the time origin.
PM_MODE_FORW, then, in the case of pmFetch, the collection of recorded metric values is scanned forward, until values for at least one of the requested metrics is located after the time origin. Then all requested metrics stored in the PCP archive at that time are returned with a corresponding time stamp. This is the default mode when an archive context is first established with pmNewContext.
PM_MODE_BACK, then the situation is the same as for PM_MODE_FORW, except a pmFetch is serviced by scanning the collection of recorded metrics backward for metrics before the time origin.
pmFetch, the time origin is reset to the time stamp returned through the pmResult.
pmSetMode parameter delta defines an additional number of time unit that should be used to adjust the time origin (forward or backward) after the new time origin from the pmResult has been determined. This is useful when moving through archives with a mode of PM_MODE_INTERP. The high-order bits of the mode parameter field is also used to optionally set the units of time for the delta field. To specify the units of time, use the PM_XTB_SET macro with one of the values PM_TIME_NSEC, PM_TIME_MSEC, PM_TIME_SEC, or so on as follows:
PM_MODE_INTERP | PM_XTB_SET(PM_TIME_XXXX)
pmSetMode and pmFetch.
my.metric.name:
Example 3.11. Dumping Values in Temporal Sequence
int sts;
pmID pmid;
char *name = “my.metric.name”;
sts = pmNewContext(PM_CONTEXT_ARCHIVE, “myarchive”);
sts = pmLookupName(1, &name, &pmid);
for ( ; ; ) {
sts = pmFetch(1, &pmid, &result);
if (sts < 0)
break;
/* dump value(s) from result->vset[0]->vlist[] */
pmFreeResult(result);
}Example 3.12. Replaying Interpolated Metrics
int sts;
pmID pmid;
char *name = “my.metric.name”;
struct timeval endtime;
sts = pmNewContext(PM_CONTEXT_ARCHIVE, “myarchive”);
sts = pmLookupName(1, &name, &pmid);
sts = pmGetArchiveEnd(&endtime);
sts = pmSetMode(PM_MODE_INTERP, &endtime, -10000);
while (pmFetch(1, &pmid, &result) != PM_ERR_EOL) {
/*
* process interpolated metric values as of result->timestamp
*/
pmFreeResult(result);
}pmReconnectContext Functionint pmReconnectContext(int handle)
Python:
int status = pmReconnectContext()
pmReconnectContext allows an application to request that the PMAPI context identified by handle be re-established, provided the associated PMCD is accessible.
Note
pmReconnectContext attempts a reconnection only after a suitable delay from the previous attempt. This imposed restriction on the reconnect re-try time interval uses a default exponential back-off so that the initial delay is 5 seconds after the first unsuccessful attempt, then 10 seconds, then 20 seconds, then 40 seconds, and then 80 seconds thereafter. The intervals between reconnection attempts may be modified using the environment variable PMCD_RECONNECT_TIMEOUT and the time to wait before an attempted connection is deemed to have failed is controlled by the PMCD_CONNECT_TIMEOUT environment variable; see the PCPIntro(1) man page.
pmReconnectContext returns handle. Note that even in the case of a successful reconnection, pmReconnectContext does not change the current PMAPI context.
pmGetContextHostName Functionconst char *pmGetContextHostName(int id)
char *pmGetContextHostName_r(int id, char *buf, int buflen)
Python:
"hostname" = pmGetContextHostName()
pmNewContext or pmDupContext, the pmGetContextHostName function provides a possibility to retrieve a host name associated with a context regardless of the context type.
pmcd.hostname metric if it is available, and so is able to provide an accurate hostname in the presence of connection tunnelling and port forwarding.
pmGetContextHostName, the string value is held in a single static buffer, so concurrent calls may not produce the desired results. The pmGetContextHostName_r function allows a buffer and length to be passed in, into which the message is stored; this variant uses no shared storage and can be used in a thread-safe manner.
pmNewContextZone Functionint pmNewContextZone(void)
Python:
pmNewContextZone()
pmNewContextZone function uses the timezone from the archive label record in the first archive of the set to set the current reporting timezone. The current reporting timezone affects the timezone used by pmCtime and pmLocaltime.
pmNewContextZone executes a pmFetch to retrieve the value for the metric pmcd.timezone and uses that to set the current reporting timezone.
pmUseZone to restore this reporting timezone.
PM_ERR_NOCONTEXT indicates the current PMAPI context is not valid. A return value less than zero indicates a fatal error from a system call, most likely malloc.
pmNewZone Functionint pmNewZone(const char *tz)
Python:
int tz_handle = pmNewZone(int tz)pmNewZone function sets the current reporting timezone, and returns a value that may be used in a subsequent call to pmUseZone to restore this reporting timezone. The current reporting timezone affects the timezone used by pmCtime and pmLocaltime.
TZ environment variable. See the environ(7) man page.
malloc.
pmUseZone Functionint pmUseZone(const int tz_handle)
Python:
int status = pmUseZone(int tz_handle)
pmUseZone function, tz_handle identifies a reporting timezone as previously established by a call to pmNewZone or pmNewContextZone, and this becomes the current reporting timezone. The current reporting timezone effects the timezone used by pmCtime and pmLocaltime).
pmWhichZone Functionint pmWhichZone(char **tz)
Python:
"zone string" = pmWhichZone()pmWhichZone function returns the handle of the current timezone, as previously established by a call to pmNewZone or pmNewContextZone. If the call is successful (that is, there exists a current reporting timezone), a non-negative integer is returned and tz is set to point to a static buffer containing the timezone string itself. The current reporting timezone effects the timezone used by pmCtime and pmLocaltime.
pmFetch Functionint pmFetch(int numpmid, pmID pmidlist[], pmResult **result)
Python:
pmResult* pmresult = pmFetch(c_uint pmid[])pmFetch, specifying a list of PMIDs (for example, as constructed by pmLookupName) through pmidlist and numpmid. The call to pmFetch is executed in the context of a source of metrics, instance profile, and collection time, previously established by calls to the functions described in Section 3.8.4, “PMAPI Context Services”.
pmFetch is returned as a tree structured result, described in the Section 3.5, “Performance Metrics Values”.
pmValue structure in the result. If there are no available values for a metric, then numval is zero and the associated pmValue[] instance is empty; valfmt is undefined in these circumstances, but pmid is correctly set to the PMID of the metric with no values.
pmerr(1) and pmErrStr(3) man pages. Since all error codes are negative, values for a requested metric are unavailable if numval is less than or equal to zero.
pmValueSet in the result, and that the PMIDs appear in exactly the same sequence in both pmidlist and result. This makes the number and order of entries in result completely deterministic, and greatly simplifies the application programming logic after the call to pmFetch.
pmFetch is dynamically allocated using one or more calls to malloc and specialized allocation strategies, and should be released when no longer required by calling pmFreeResult. Under no circumstances should free be called directly to release this space.
malloc failure, and so on) would cause an error value to be returned by pmFetch. Otherwise, the value returned by the pmFetch function is zero.
pmValue structure) of selected performance metrics once every 10 seconds:
Example 3.13. PMAPI Metrics Services
int i, j, sts;
pmID pmidlist[10];
pmResult *result;
time_t now;
/* set up PMAPI context, numpmid and pmidlist[] ... */
while ((sts = pmFetch(10, pmidlist, &result)) >= 0) {
now = (time_t)result->timestamp.tv_sec;
printf("\n@ %s", ctime(&now));
for (i = 0; i < result->numpmid; i++) {
printf("PMID: %s", pmIDStr(result->vset[i]->pmid));
for (j = 0; j < result->vset[i]->numval; j++) {
printf(" 0x%x", result->vset[i]->vlist[j].value.lval);
putchar('\n');
}
}
pmFreeResult(result);
sleep(10);
}Note
pmFetch times out and returns PM_ERR_TIMEOUT. This is most likely to occur when the PMAPI client and PMCD are communicating over a slow network connection, but may also occur when one of the hosts is extremely busy. The time out period may be modified using the PMCD_REQUEST_TIMEOUT environment variable; see the PCPIntro(1) man page.
pmLookupName. The returned pmresult is passed to pmExtractValue.
pmFreeResult Functionvoid pmFreeResult(pmResult *result)
Python:
pmFreeResult(pmResult* pmresult)pmFetch.
pmStore Functionint pmStore(const pmResult *request)
Python:
pmResult* pmresult = pmStore(pmResult* pmresult)pmStore function is a lightweight inverse of pmFetch. The caller must build the pmResult data structure (which could have been returned from an earlier pmFetch call) and then call pmStore. It is an error to pass a request to pmStore in which the numval field within any of the pmValueSet structure has a value less than one.
pmStore cannot be used to make retrospective changes to information in a PCP archive log.
pmAtomValue pointer is given. Then, at each subsequent fetchgroup-fetch operation, all metrics are fetched, decoded/converted, and deposited in the desired field of the destination pmAtomValues. See Example 3.16, “ pmAtomValue Structure” for more on that data type. Similarly, a per-metric-instance status value is optionally available for detailed diagnostics reflecting fetch/conversion.
pmfetchgroup(3) man pages give detailed information on the C API; we only list some common cases here. The simplified Python binding to the same API is summarized below. One difference is that runtime errors in C are represented by status integers, but in Python are mapped to pmErr exceptions. Another is that supplying metric type codes are mandatory in the C API but optional in Python, since the latter language supports dynamic typing. Another difference is Python's wrapping of output metric values in callable "holder" objects. We demonstrate all of these below.
pmCreateFetchGroup function is used, with parameters similar to pmNewContext (see Section 3.8.4.1, “ pmNewContext Function”).
int sts;
pmFG fg;
sts = pmCreateFetchGroup(& fg, PM_CONTEXT_ARCHIVE, "./foo.meta");
assert(sts == 0);
Python
fg = pmapi.fetchgroup(c_api.PM_CONTEXT_ARCHIVE, './foo.meta')
int ctx = pmGetFetchGroupContext(fg);
sts = pmUseContext(ctx);
assert(sts == 0);
sts = pmSetMode(...);
Python
ctx = fg.get_context()
ctx.pmSetMode(...)
pmDesc Structure”). In C, the value destination is specified by pointer. In Python, a value-holder is returned.
static pmAtomValue ncpu, loadavg, idle;
sts = pmExtendFetchGroup_item(fg, "hinv.ncpu", NULL, NULL,
& ncpu, PM_TYPE_32, NULL);
assert (sts == 0);
sts = pmExtendFetchGroup_item(fg, "kernel.all.load", "5 minute", NULL,
& loadavg, PM_TYPE_DOUBLE, NULL);
assert (sts == 0);
sts = pmExtendFetchGroup_item(fg, "kernel.all.cpu.idle", NULL, "s/100s",
& idle, PM_TYPE_STRING, NULL);
assert (sts == 0);
Python
ncpu = fg.extend_item('hinv.cpu')
loadavg = fg.extend_item('kernel.all.load', instance='5 minute')
idle = fg.extend_item('kernel.all.cpu.idle, scale='s/100s')
Registering metrics with whole instance domains are also possible; these result in a vector of pmAtomValue instances, instance names and codes, and status codes, so the fetchgroup functions take more optional parameters. In Python, a value-holder-iterator object is returned.
enum { max_disks = 100 };
static unsigned num_disks;
static pmAtomValue disk_reads[max_disks];
static int disk_read_stss[max_disks];
static char *disk_names[max_disks];
sts = pmExtendFetchGroup_indom(fg, "disk.dm.read", NULL,
NULL, disk_names, disk_reads, PM_TYPE_32,
disk_read_stss, max_disks, & num_disks,
NULL);
Python
values = fg.extend_indom('disk.dm.read')
Registering interest in the future fetch-operation timestamp is also possible. In python, a datetime-holder object is returned.
struct timeval tv;
sts = pmExtendFetchGroup_timestamp(fg, & tv);
Python
tv = fg.extend_timestamp()
pmAtomValue union - whichever was requested with the PM_TYPE_* code. In the Python API, each metric value is accessed by calling the value-holder objects.
sts = pmFetchGroup(fg);
assert (sts == 0);
printf("%s", ctime(& tv.tv_sec));
printf("#cpus: %d, loadavg: %g, idle: %s\n", ncpu.l, loadavg.d, idle.cp);
for (i=0; i<num_disks; i++)
if (disk_read_stss[i] == 0)
printf("disk %s reads %d\n", disk_names[i], disk_reads[i].l);
Python
fg.fetch()
print(tv())
print("#cpus: %d, loadavg: %g, idle: %d\n" % (ncpu(), loadavg(), idle()))
for icode, iname, value in values():
print('disk %s reads %d' % (iname, value()))
The program may fetch and process the values only once, or in a loop. The program need not - must not - modify or free any of the output values/pointers supplied by the fetchgroup functions.
sts = pmDestroyFetchGroup(fg);
Python
del fg # or nothing
pmlogger co-processes, which they create and control for the purposes of recording live performance data from (possibly) multiple hosts. Since pmlogger records for one host only, these services can administer a group of loggers, and set up archive folios to track the logs. Tools like pmafm can subsequently use those folios to replay recorded data with the initiating tool. pmchart uses these concepts when providing its Record mode functionality.
pmRecordAddHost Functionint pmRecordAddHost(const char *host, int isdefault, pmRecordHost **rhp)
Python:
(int status, pmRecordHost* rhp) = pmRecordAddHost("host string", 1, "configure string")pmRecordAddHost function adds hosts once pmRecordSetup has established a new recording session. The pmRecordAddHost function along with the pmRecordSetup and pmRecordControl functions are used to create a PCP archive.
pmRecordAddHost is called for each host that is to be included in the recording session. A new pmRecordHost structure is returned via rhp. It is assumed that PMCD is running on the host as this is how pmlogger retrieves the required performance metrics.
pmlogger configuration onto the stdio stream returned via the f_config field in the pmRecordHost structure.
pmRecordAddHost returns 0 on success and a value less than 0 suitable for decoding with pmErrStr on failure. The value EINVAL has the same interpretation as errno being set to EINVAL.
pmRecordControl Functionint pmRecordControl(pmRecordHost *rhp, int request, const char *options)
Python:
int status = pmRecordControl("host string", 1, "configure string")pmlogger by calling the pmRecordControl function with a request of PM_REC_SETARG. The pmRecordControl along with the pmRecordSetup and pmRecordAddHost functions are used to create a PCP archive.
pmRecordControl is required for each distinct argument. An argument may be added for a particular pmlogger instance identified by rhp. If the rhp argument is NULL, the argument is added for all pmlogger instances that are launched in the current recording session.
pmRecordControl with a request of PM_REC_SETARG, each pmlogger instance is automatically launched with the following arguments: -c, -h, -l, -x, and the basename for the PCP archive log.
pmRecordControl with a request of PM_REC_ON, and rhp must be NULL. This launches one pmlogger process for each host in the recording session and initializes the fd_ipc, logfile, pid, and status fields in the associated pmRecordHost structure(s).
pmlogger instance identified by rhp, call pmRecordControl with a request of PM_REC_OFF. If the rhp argument to pmRecordControl is NULL, the termination request is broadcast to all pmlogger processes in the current recording session. An informative dialogue is generated directly by each pmlogger process.
pmlogger instance identified by rhp, call pmRecordControl with a request of PM_REC_STATUS. If the rhp argument to pmRecordControl is NULL, the status request is broadcast to all pmlogger processes in the current recording session. The display is generated directly by each pmlogger process.
pmlogger instance identified by rhp, allow it to continue independent of the application that launched the recording session and call pmRecordControl with a request of PM_REC_DETACH. If the rhp argument to pmRecordControl is NULL, the detach request is broadcast to all pmlogger processes in the current recording session.
pmRecordControl returns 0 on success and a value less than 0 suitable for decoding with pmErrStr on failure. The value EINVAL has the same interpretation as errno being set to EINVAL.
pmRecordControl returns PM_ERR_IPC if the associated pmlogger process has already exited.
pmRecordSetup FunctionFILE *pmRecordSetup(const char *folio, const char *creator, int replay)
Python:
int status = pmRecordSetup("folio string", "creator string", int replay)pmRecordSetup function along with the pmRecordAddHost and pmRecordControl functions may be used to create a PCP archive on the fly to support record-mode services for PMAPI client applications.
pmlogger process, with an overall Archive Folio format as understood by the pmafm command, to name and collect all of the archive logs associated with a single recording session.
pmRecordHost structure is used to maintain state information between the creator of the recording session and the associated pmlogger process(es). The structure, shown in Example 3.14, “ pmRecordHost Structure”, is defined as:
Example 3.14. pmRecordHost Structure
typedef struct {
FILE *f_config; /* caller writes pmlogger configuration here */
int fd_ipc; /* IPC channel to pmlogger */
char *logfile; /* full pathname for pmlogger error logfile */
pid_t pid; /* process id for pmlogger */
int status; /* exit status, -1 if unknown */
} pmRecordHost;Procedure 3.1. Creating a Recording Session
pmRecordSetup to establish a new recording session. A new Archive Folio is created using the name folio. If the folio file or directory already exists, or if it cannot be created, this is an error. The application that is creating the session is identified by creator (most often this would be the same as the global PMAPI application name, pmProgname). If the application knows how to create its own configuration file to replay the recorded session, replay should be nonzero. The pmRecordSetup function returns a stdio stream onto which the application writes the text of any required replay configuration file.
pmRecordAddHost. A new pmRecordHost structure is returned via rhp. It is assumed that PMCD is running on the host as this is how pmlogger retrieves the required performance metrics. See Section 3.8.8.1, “ pmRecordAddHost Function” for more information.
pmlogger by calling pmRecordControl with a request of PM_REC_SETARG. The argument is passed via options and one call to pmRecordControl is required for each distinct argument. See Section 3.8.8.2, “ pmRecordControl Function” for more information.
pmRecordControl with a request of PM_REC_ON, and rhp must be NULL.
pmlogger instance identified by rhp, call pmRecordControl with a request of PM_REC_OFF.
pmlogger instance identified by rhp, call pmRecordControl with a request of PM_REC_STATUS.
pmlogger instance identified by rhp, allow it to continue independent of the application that launched the recording session, call pmRecordControl with a request of PM_REC_DETACH.
pmRecordControl performs this task when recording is commenced.
pmlogger has been started for a recording session, pmlogger assumes responsibility for any dialogue with the user in the event that the application that launched the recording session should exit, particularly without terminating the recording session.
pmlogger is displayed using pmconfirm. This default is based on the assumption that most applications launching a recording session are GUI-based. In the event that pmconfirm fails to display the information (for example, because the DISPLAY environment variable is not set), pmlogger writes on its own stderr stream (not the stderr stream of the launching process). The output is assigned to the xxxxxx.host.log file. For convenience, the full pathname to this file is provided via the logfile field in the pmRecordHost structure.
pmRecordControl is not NULL, this string may be used to pass additional arguments to pmconfirm in those cases where a dialogue is to be displayed. One use of this capability is to provide a -geometry string to control the placement of the dialogue.
pmlogger process may be determined using the pmRecordHost structure, by calling select on the fd_ipc field or polling the status field that will contain the termination status from waitpid if known, or -1.
pmRecordSetup. In all cases, the xxxxxx component is the result of calling mkstemp.
pmafm command.
xxxxxx.host.config file is the pmlogger configuration for each host. If the same host is used in different calls to pmRecordAddHost within the same recording session, one of the letters 'a' through 'z' is appended to the xxxxxx part of all associated file names to ensure uniqueness.
xxxxxx.host.log is stdout and stderr for the pmlogger instance for each host.
xxxxxx.host.{0,meta,index} files comprise a single PCP archive for each host.
pmRecordSetup may return NULL in the event of an error. Check errno for the real cause. The value EINVAL typically means that the order of calls to these functions is not correct; that is, there is an obvious state associated with the current recording session that is maintained across calls to the functions.
pmRecordControl before calling pmRecordAddHost at least once, or calling pmRecordAddHost before calling pmRecordSetup would produce an EINVAL error.
pmGetArchiveLabel Functionint pmGetArchiveLabel(pmLogLabel *lp)
Python:
pmLogLabel loglabel = pmGetArchiveLabel()pmGetArchiveLabel function may be used to fetch the label record from the first archive in the set of archives. The structure returned through lp is as shown in Example 3.15, “ pmLogLabel Structure”:
Example 3.15. pmLogLabel Structure
/*
* Label Record at the start of every log file - as exported above the PMAPI ...
*/
#define PM_TZ_MAXLEN 40
#define PM_LOG_MAXHOSTLEN 64
#define PM_LOG_MAGIC 0x50052600
#define PM_LOG_VERS01 0x1
#define PM_LOG_VERS02 0x2
#define PM_LOG_VOL_TI -2 /* temporal index */
#define PM_LOG_VOL_META -1 /* meta data */
typedef struct {
int ll_magic; /* PM_LOG_MAGIC | log format version no. */
pid_t ll_pid; /* PID of logger */
struct timeval ll_start; /* start of this log */
char ll_hostname[PM_LOG_MAXHOSTLEN]; /* name of collection host */
char ll_tz[PM_TZ_MAXLEN]; /* $TZ at collection host */
} pmLogLabel;pmGetArchiveEnd Functionint pmGetArchiveEnd(struct timeval *tvp)
Python:
timeval tv = status = pmGetArchiveEnd()pmGetArchiveEnd finds the logical end of the last archive file in the set (after the last complete record in the archive), and returns the last recorded time stamp with tvp. This timestamp may be passed to pmSetMode to reliably position the context at the last valid log record, for example, in preparation for subsequent reading in reverse chronological order.
pmlogger at the same time that an application is trying to read the archive, the logical end of file may be before the physical end of file due to write buffering that is not aligned with the logical record boundaries.
pmGetInDomArchive Functionint pmGetInDomArchive(pmInDom indom, int **instlist, char ***namelist )
Python:
((instance1, instance2...) (name1, name2...)) pmGetInDom(pmDesc pmdesc)pmGetInDomArchive scans the metadata to generate the union of all instances for the instance domain indom that can be found in the set of archive logs, and returns through instlist the internal instance identifiers, and through namelist the full external identifiers.
pmGetInDomArchive with two calls to malloc, and it is the responsibility of the caller to use free(instlist) and free(namelist) to release the space when it is no longer required; see the malloc(3) and free(3) man pages.
pmGetInDomArchive is less than one, both instlist and namelist are undefined (no space is allocated; so calling free is a singularly bad idea).
pmLookupInDomArchive Functionint pmLookupInDomArchive(pmInDom indom, const char *name)
Python:
c_uint instid = pmLookupInDomArchive(pmDesc pmdesc, "Instance")pmLookupInDomArchive scans the metadata for the instance domain indom, locates the first instance with the external identification given by name, and returns the internal instance identifier.
pmLookupInDom.
pmLookupInDomArchive function returns a positive instance identifier on success.
pmNameInDomArchive Functionint pmNameInDomArchive(pmInDom indom, int inst, char **name)
Python:
"instance id" = pmNameInDomArchive(pmDesc pmdesc, c_uint instid)pmNameInDomArchive scans the metadata for the instance domain indom, locates the first instance with the internal instance identifier given by inst, and returns the full external instance identification through name. This function is a specialized version of the more general PMAPI function pmNameInDom.
pmNameInDomArchive with malloc, and it is the responsibility of the caller to free the space when it is no longer required; see the malloc(3) andfree(3) man pages.
pmFetchArchive Functionint pmFetchArchive(pmResult **result)
Python:
pmResult* pmresult = pmFetchArchive()pmFetch that may be used only when the current PMAPI context is associated with a set of PCP archive logs. The result is instantiated with all of the metrics (and instances) from the next archive record; consequently, there is no notion of a list of desired metrics, and the instance profile is ignored.
pmFetchArchive would be used to create utilities that scan archive logs (for example, pmdumplog and pmlogsummary), and the more common access to the archives would be through the pmFetch interface.
pmtime(1) man page.
pmchart, pmgadgets, pmstat, and pmval use the time control services to connect to an instance of the time control server process, pmtime, which provides a uniform graphical user interface to the time control services.
Table 3.2. Time Control Functions in PMAPI
|
Man Page
|
Synopsis of Time Control Function
|
|---|---|
pmCtime(3)
|
Formats the date and time for a reporting timezone.
|
pmLocaltime(3)
|
Converts the date and time for a reporting timezone.
|
pmParseTimeWindow(3)
|
Parses time window command line arguments.
|
pmTimeConnect(3)
|
Connects to a time control server via a command socket.
|
pmTimeDisconnect(3)
|
Closes the command socket to the time control server.
|
pmTimeGetPort(3)
|
Obtains the port name of the current time control server.
|
pmTimeRecv(3)
|
Blocks until the time control server sends a command message.
|
pmTimeSendAck(3)
|
Acknowledges completion of the step command.
|
pmTimeSendBounds(3)
|
Specifies beginning and end of archive time period.
|
pmTimeSendMode(3)
|
Requests time control server to change to a new VCR mode.
|
pmTimeSendPosition(3)
|
Requests time control server to change position or update intervals.
|
pmTimeSendTimezone(3)
|
Requests time control server to change timezones.
|
pmTimeShowDialog(3)
|
Changes the visibility of the time control dialogue.
|
pmTimeGetStatePixmap(3)
|
Returns array of pixmaps representing supplied time control state.
|
pmGetConfig Functionchar *pmGetConfig(const char *variable)
Python:
"env variable value = pmGetConfig("env variable")pmGetConfig function searches for a variable first in the environment and then, if one is not found, in the PCP configuration file and returns the string result. If a variable is not already in the environment, it is added with a call to the putenv function before returning.
/etc/pcp.conf, but this location may be changed by setting PCP_CONF in the environment to a new location, as described in the pcp.conf(5) man page.
PCP_CONF is not set in the environment), then a fatal error message is printed and the process will exit. Although this sounds drastic, it is the only course of action available because the PCP configuration or installation is fatally flawed.
getenv function (which should probably be a const char *), changing the content of the string is not recommended.
pmErrStr Functionconst char *pmErrStr(int code)
char *pmErrStr_r(int code, char *buf, int buflen);
Python:
"error string text" = pmErrStr(int error_code)errno.h, and the strings returned are according to strerror. The large, negative error codes are PMAPI error conditions, and pmErrStr returns an appropriate PMAPI error string, as determined by code.
pmErrStr, the string value is held in a single static buffer, so concurrent calls may not produce the desired results. The pmErrStr_r function allows a buffer and length to be passed in, into which the message is stored; this variant uses no shared storage and can be used in a thread-safe manner.
pmExtractValue Functionint pmExtractValue(int valfmt, const pmValue *ival, int itype,
pmAtomValue *oval, int otype)
Python:
pmAtomValue atomval = pmExtractValue(int valfmt, const pmValue * ival,
int itype,
pmAtomValue *oval,
int otype)pmValue structure is embedded within the pmResult structure, which is used to return one or more performance metrics; see the pmFetch man page.
pmAtomValue union, defined in Example 3.16, “ pmAtomValue Structure”:
Example 3.16. pmAtomValue Structure
/* Generic Union for Value-Type conversions */
typedef union {
__int32_t l; /* 32-bit signed */
__uint32_t ul; /* 32-bit unsigned */
__int64_t ll; /* 64-bit signed */
__uint64_t ull; /* 64-bit unsigned */
float f; /* 32-bit floating point */
double d; /* 64-bit floating point */
char *cp; /* char ptr */
void *vp; /* void ptr */
} pmAtomValue;pmExtractValue function provides a convenient mechanism for extracting values from the pmValue part of a pmResult structure, optionally converting the data type, and making the result available to the application programmer.
pmFetch man page). The otype argument defines the data type of the result to be placed in oval. The value for itype is typically extracted from a pmDesc structure, following a call to pmLookupDesc for a particular performance metric.
Note
PM_TYPE_STRING and PM_TYPE_AGGREGATE types are indeed possible, but are marked N; the rationale is that pmExtractValue should not attempt to duplicate functionality already available in the C library through sscanf and sprintf. No conversion involving the type PM_TYPE_EVENT is supported.
Table 3.3. PMAPI Type Conversion
|
TYPE
|
32
|
U32
|
64
|
U64
|
FLOAT
|
DBLE
|
STRING
|
AGGR
|
EVENT
|
|---|---|---|---|---|---|---|---|---|---|
32
|
Y
|
S
|
Y
|
S
|
P
|
P
|
N
|
N
|
N
|
U32
|
T
|
Y
|
Y
|
Y
|
P
|
P
|
N
|
N
|
N
|
64
|
T
|
T,S
|
Y
|
S
|
P
|
P
|
N
|
N
|
N
|
u64
|
T
|
T
|
T
|
Y
|
P
|
P
|
N
|
N
|
N
|
FLOAT
|
P, T
|
P, T, S
|
P, T
|
P, T, S
|
Y
|
Y
|
N
|
N
|
N
|
DBLE
|
P, T
|
P, T, S
|
P, T
|
P, T, S
|
P
|
Y
|
N
|
N
|
N
|
STRING
|
N
|
N
|
N
|
N
|
N
|
N
|
Y
|
N
|
N
|
AGGR
|
N
|
N
|
N
|
N
|
N
|
N
|
N
|
Y
|
N
|
EVENT
|
N
|
N
|
N
|
N
|
N
|
N
|
N
|
N
|
N
|
PM_TYPE_STRING or PM_TYPE_AGGREGATE, then the value buffer is allocated by pmExtractValue using malloc, and it is the caller's responsibility to free the space when it is no longer required; see the malloc(3) and free(3) man pages.
type field in a pmDesc structure, into a canonical type for local processing.
pmFetch), and convert its type from itype to otype.
pmConvScale Functionint
pmConvScale(int type, const pmAtomValue *ival, const pmUnits *iunit,
pmAtomValue *oval, pmUnits *ounit)
Python:
pmAtomValue atomval = pmConvScale(int itype, pmAtomValue value,
pmDesc* pmdesc , int descidx, int otype)pmDesc structure, following a call to pmLookupDesc for a particular performance metric.
pmConvScale is most useful when values returned through pmFetch (and possibly extracted using pmExtractValue) need to be normalized into some canonical scale and units for the purposes of computation.
pmUnitsStr Functionconst char *pmUnitsStr(const pmUnits *pu)
char *pmUnitsStr_r(const pmUnits *pu, char *buf, int buflen)
Python:
"units string" = pmUnitsStr(pmUnits pmunits)pmUnitsStr takes a dimension and scale specification as per pu, and returns the corresponding text string.
pmDesc structure, for example, as returned by pmLookupDesc.
{1, -2, 0, PM_SPACE_MBYTE, PM_TIME_MSEC, 0}, then the result string would be Mbyte/sec^2.
pmUnitsStr, the string value is held in a single static buffer; so concurrent calls may not produce the desired results. The pmUnitsStr_r function allows a buffer and length to be passed in, into which the units are stored; this variant uses no shared storage and can be used in a thread-safe manner.
pmIDStr Functionconst char *pmIDStr(pmID pmid)
char *pmIDStr_r(pmID pmid, char *buf, int buflen)
Python:
"ID string" = pmIDStr(int pmID)domain, cluster, and item subfields appearing as decimal numbers, separated by periods.
pmIDStr, the string value is held in a single static buffer; so concurrent calls may not produce the desired results. The pmIDStr_r function allows a buffer and length to be passed in, into which the identifier is stored; this variant uses no shared storage and can be used in a thread-safe manner.
pmInDomStr Functionconst char *pmInDomStr(pmInDom indom)
char *pmInDomStr_r(pmInDom indom, char *buf, int buflen)
Python:
"indom" = pmGetInDom(pmDesc pmdesc)domain and serial subfields appearing as decimal numbers, separated by periods.
pmInDomStrr, the string value is held in a single static buffer; so concurrent calls may not produce the desired results. The pmInDomStr_r function allows a buffer and length to be passed in, into which the identifier is stored; this variant uses no shared storage and can be used in a thread-safe manner.
pmTypeStr Functionconst char *pmTypeStr(int type)
char *pmTypeStr_r(int type, char *buf, int buflen)
Python:
"type" = pmTypeStr(int type)PM_TYPE_32), “U64” (for PM_TYPE_U64), “AGGREGATE” (for PM_TYPE_AGGREGATE), and so on.
pmTypeStr, the string value is held in a single static buffer; so concurrent calls may not produce the desired results. The pmTypeStr_r function allows a buffer and length to be passed in, into which the identifier is stored; this variant uses no shared storage and can be used in a thread-safe manner.
pmAtomStr Functionconst char *pmAtomStr(const pmAtomValue *avp, int type)
char *pmAtomStr_r(const pmAtomValue *avp, int typechar *buf, int buflen)
Python:
"value" = pmAtomStr(atom, type)pmAtomValue identified by avp, and a performance metric type, generate the corresponding metric value as a string, suitable for diagnostic or report output.
pmAtomStr, the string value is held in a single static buffer; so concurrent calls may not produce the desired results. The pmAtomStr_r function allows a buffer and length to be passed in, into which the identifier is stored; this variant uses no shared storage and can be used in a thread-safe manner.
pmNumberStr Functionconst char *pmNumberStr(double value) char *pmNumberStr_r(double value, char *buf, int buflen)
pmNumberStr function returns the address of a static 8-byte buffer that holds a null-byte terminated representation of value suitable for output with fixed-width fields.
pmPrintValue Functionvoid pmPrintValue(FILE *f, int valfmt, int type, const pmValue *val,
int minwidth)
Python:
pmPrintValue(FILE* file, pmResult pmresult, pmdesc, vset_index, vlist_index, min_width)pmValueSet within a pmResult) and the generic description of the metric's type from a pmDesc structure, passed in through.
pmPrintValue to Print Values” illustrates using pmPrintValue to print the values from a pmResult structure returned via pmFetch:
Example 3.17. Using pmPrintValue to Print Values
int numpmid, i, j, sts;
pmID pmidlist[10];
pmDesc desc[10];
pmResult *result;
/* set up PMAPI context, numpmid and pmidlist[] ... */
/* get metric descriptors */
for (i = 0; i < numpmid; i++) {
if ((sts = pmLookupDesc(pmidlist[i], &desc[i])) < 0) {
printf("pmLookupDesc(pmid=%s): %s\n",
pmIDStr(pmidlist[i]), pmErrStr(sts));
exit(1);
}
}
if ((sts = pmFetch(numpmid, pmidlist, &result)) >= 0) {
/* once per metric */
for (i = 0; i < result->numpmid; i++) {
printf("PMID: %s", pmIDStr(result->vset[i]->pmid));
/* once per instance for this metric */
for (j = 0; j < result->vset[i]->numval; j++) {
printf(" [%d]", result->vset[i]->vlist[j].inst);
pmPrintValue(stdout, result->vset[i]->valfmt,
desc[i].type,
&result->vset[i]->vlist[j],
8);
}
putchar('\n');
}
pmFreeResult(result);
}
else
printf("pmFetch: %s\n", pmErrStr(sts));pmflush Functionint pmflush(void);
Python:
int status = pmflush()pmflush function causes the internal buffer which is shared with pmprintf to be either displayed in a window, printed on standard error, or flushed to a file and the internal buffer to be cleared.
PCP_STDERR environment variable controls the output technique used by pmflush:
PCP_STDERR is unset, the text is written onto the stderr stream of the caller.
PCP_STDERR is set to the literal reserved word DISPLAY, then the text is displayed as a GUI dialogue using pmconfirm.
pmflush function returns a value of zero on successful completion. A negative value is returned if an error was encountered, and this can be passed to pmErrStr to obtain the associated error message.
pmprintf Functionint pmprintf(const char *fmt, ... /*args*/);
Python:
pmprintf("fmt", ... /*args*/);pmprintf function appends the formatted message string to an internal buffer shared by the pmprintf and pmflush functions, without actually producing any output. The fmt argument is used to control the conversion, formatting, and printing of the variable length args list.
pmprintf function uses the mkstemp function to securely create a pcp-prefixed temporary file in ${PCP_TMP_DIR}. This temporary file is deleted when pmflush is called.
pmprintf returns the number of characters transmitted. A negative value is returned if an error was encountered, and this can be passed to pmErrStr to obtain the associated error message.
pmSortInstances Functionvoid pmSortInstances(pmResult *result)
Python:
pmSortInstances (pmResult* pmresult)pmSortInstances function may be used to guarantee that for each performance metric in the result from pmFetch, the instances are in ascending internal instance identifier sequence. This is useful when trying to compute rates from two consecutive pmFetch results, where the underlying instance domain or metric availability is not static.
pmParseInterval Functionint pmParseInterval(const char *string, struct timeval *rslt, char **errmsg)
Python:
(struct timeval, "error message") = pmParseInterval("time string")pmParseInterval function parses the argument string specifying an interval of time and fills in the tv_sec and tv_usec components of the rslt structure to represent that interval. The input string is most commonly the argument following a -t command line option to a PCP application, and the syntax is fully described in the PCPIntro(1) man page.
pmParseInterval returns 0 and errmsg is undefined if the parsing is successful. If the given string does not conform to the required syntax, the function returns -1 and a dynamically allocated error message string in errmsg.
4minutes 30mumble
^ -- unexpected valuefree to release the space allocated for errmsg.
pmParseMetricSpec Functionint pmParseMetricSpec(const char *string, int isarch, char *source,
pmMetricSpec **rsltp, char **errmsg)
Python:
(pmMetricSpec metricspec, "error message") =
pmParseMetricSpec("metric specification", isarch, source)pmParseMetricSpec function accepts a string specifying the name of a PCP performance metric, and optionally the source (either a hostname, a set of PCP archive logs, or a local context) and instances for that metric. The syntax is described in the PCPIntro(1) man page.
isarch and source arguments are used to fill in the returned pmMetricSpec structure. In Example 3.18, “ pmMetricSpec Structure”, the pmMetricSpec structure, which is returned via rsltp, represents the parsed string.
Example 3.18. pmMetricSpec Structure
typedef struct {
int isarch; /* source type: 0 -> host, 1 -> archive, 2 -> local context */
char *source; /* name of source host or archive */
char *metric; /* name of metric */
int ninst; /* number of instances, 0 -> all */
char *inst[1]; /* array of instance names */
} pmMetricSpec;pmParseMetricSpec function returns 0 if the given string was successfully parsed. In this case, all the storage allocated by pmParseMetricSpec can be released by a single call to the free function by using the address returned from pmMetricSpec via rsltp. The convenience macro pmFreeMetricSpec is a thinly disguised wrapper for free.
pmParseMetricSpec function returns 0 if the given string was successfully parsed. It returns PM_ERR_GENERIC and a dynamically allocated error message string in errmsg if the given string does not parse. In this situation, the error message string can be released with the free function.
pmclient) using the PMAPI is shipped as part of the PCP package. See the pmclient(1) man page, and the source code, located in ${PCP_DEMOS_DIR}/pmclient.
LOADAV macro with the name kernel.all.load, and then be able to use LOADAV to get at the value of the corresponding metric.
pmLookupName and the PMIDs returned by this function, and the one-to-one association between the PMIDs input to pmFetch and the values returned by this function provide the basis for an automated solution.
pmgenmap takes the specification of a list of metric names and symbolic tags, in the order they should be passed to pmLookupName and pmFetch. For example, pmclient:
cat ${PCP_DEMOS_DIR}/pmclient/pmnsmap.spec
pmclient_init {
hinv.ncpu NUMCPU
}
pmclient_sample {
kernel.all.load LOADAV
kernel.percpu.cpu.user CPU_USR
kernel.percpu.cpu.sys CPU_SYS
mem.freemem FREEMEM
disk.all.total DKIOPS
}pmgenmap input produces the C code in Example 3.19, “C Code Produced by pmgenmap Input”. It is suitable for including with the #include statement:
Example 3.19. C Code Produced by pmgenmap Input
/*
* Performance Metrics Name Space Map
* Built by runme.sh from the file
* pmnsmap.spec
* on Thu Jan 9 14:13:49 EST 2014
*
* Do not edit this file!
*/
char *pmclient_init[] = {
#define NUMCPU 0
"hinv.ncpu",
};
char *pmclient_sample[] = {
#define LOADAV 0
"kernel.all.load",
#define CPU_USR 1
"kernel.percpu.cpu.user",
#define CPU_SYS 2
"kernel.percpu.cpu.sys",
#define FREEMEM 3
"mem.freemem",
#define DKIOPS 4
"disk.all.total",
};pmgenmap, you are now able to easily initialize the application's metric specifications as shown in Example 3.20, “Initializing Metric Specifications”:
Example 3.20. Initializing Metric Specifications
/* C code fragment from pmclient.c */
numpmid = sizeof(pmclient_sample) / sizeof(char *);
if ((pmidlist = (pmID *)malloc(numpmid * sizeof(pmidlist[0]))) == NULL) {...}
if ((sts = pmLookupName(numpmid, pmclient_sample, pmidlist)) < 0) {...}
# The equivalent python code would be
pmclient_sample = ("kernel.all.load", "kernel.percpu.cpu.user",
"kernel.percpu.cpu.sys", "mem.freemem", "disk.all.total")
pmidlist = context.pmLookupName(pmclient_sample)
pmidlist contains the PMID for the five metrics of interest.
Example 3.21. Iterative Processing
/* censored C code fragment from pmclient.c */
while (samples == -1 || samples-- > 0) {
if ((sts = pmFetch(numpmid, pmidlist, &crp)) < 0) { ... }
for (i = 0; i < numpmid; i++)
if ((sts = pmLookupDesc(pmidlist[i], &desclist[i])) < 0) { ... }
...
pmExtractValue(crp->vset[FREEMEM]->valfmt, crp->vset[FREEMEM]->vlist,
desclist[FREEMEM].type, &tmp, PM_TYPE_FLOAT);
pmConvScale(PM_TYPE_FLOAT, &tmp, &desclist[FREEMEM].units,
&atom, &mbyte_scale);
ip->freemem = atom.f;
...
__pmtimevalSleep(delta);
}# The equivalent python code would be
FREEMEM = 3
desclist = context.pmLookupDescs(metric_names)
while (samples > 0):
crp = context.pmFetch(metric_names)
val = context.pmExtractValue(crp.contents.get_valfmt(FREEMEM),
crp.contents.get_vlist(FREEMEM, 0),
desclist[FREEMEM].contents.type,
c_api.PM_TYPE_FLOAT)
atom = ctx.pmConvScale(c_api.PM_TYPE_FLOAT, val, desclist, FREEMEM,
c_api.PM_SPACE_MBYTE)
(tvdelta, errmsg) = c_api.pmParseInterval(delta)
c_api.pmtimevalSleep(delta)
pmgenmap utility is demonstrated by Example 3.22, “Adding a Metric”. Consider the requirement for reporting a third metric mem.physmem. This example shows how to add the line to the specification file:
Example 3.22. Adding a Metric
mem.freemem PHYSMEM
#include file, and augment pmclient.c:
pmExtractValue(crp->vset[PHYSMEM]->valfmt, crp->vset[PHYSMEM]->vlist,
desclist[PHYSMEM].type, &tmp, PM_TYPE_FLOAT);
pmConvScale(PM_TYPE_FLOAT, &tmp, &desclist[PHYSMEM].units,
&atom, &mbyte_scale);
# The equivalent python code would be: val = context.pmExtractValue(crp.contents.get_valfmt(PHYSMEM), crp.contents.get_vlist(PHYSMEM, 0), desclist[PHYSMEM].contents.type, c_api.PM_TYPE_FLOAT);
pmErr exception. For simplicity, no command line argument processing is shown here - in practice most tools use the pmGetOptions helper interface to assist with initial context creation and setup.
Example 3.23. PMAPI Error Handling
#include <pcp/pmapi.h>
int
main(int argc, char* argv[])
{
int sts = 0;
char *host = "local:";
char *metric = "mem.freemem";
pmID pmid;
pmDesc desc;
pmResult *result;
sts = pmNewContext(PM_CONTEXT_HOST, host);
if (sts < 0) {
fprintf(stderr, "Error connecting to pmcd on %s: %s\n",
host, pmErrStr(sts));
exit(1);
}
sts = pmLookupName(1, &metric, &pmid);
if (sts < 0) {
fprintf(stderr, "Error looking up %s: %s\n", metric,
pmErrStr(sts));
exit(1);
}
sts = pmLookupDesc(pmid, &desc);
if (sts < 0) {
fprintf(stderr, "Error getting descriptor for %s:%s: %s\n",
host, metric, pmErrStr(sts));
exit(1);
}
sts = pmFetch(1, &pmid, &result);
if (sts < 0) {
fprintf(stderr, "Error fetching %s:%s: %s\n", host, metric,
pmErrStr(sts));
exit(1);
}
sts = result->vset[0]->numval;
if (sts < 0) {
fprintf(stderr, "Error fetching %s:%s: %s\n", host, metric,
pmErrStr(sts));
exit(1);
}
fprintf(stdout, "%s:%s = ", host, metric);
if (sts == 0)
puts("(no value)");
else {
pmValueSet *vsp = result->vset[0];
pmPrintValue(stdout, vsp->valfmt, desc.type,
&vsp->vlist[0], 5);
printf(" %s\n", pmUnitsStr(&desc.units));
}
return 0;
}# The equivalent python code would be:
import sys
import traceback
from pcp import pmapi
from cpmapi import PM_TYPE_U32
try:
context = pmapi.pmContext()
pmid = context.pmLookupName("mem.freemem")
desc = context.pmLookupDescs(pmid)
result = context.pmFetch(pmid)
freemem = context.pmExtractValue(result.contents.get_valfmt(0),
result.contents.get_vlist(0, 0),
desc[0].contents.type,
PM_TYPE_U32)
print "freemem is " + str(int(freemem.ul))
except pmapi.pmErr, error:
print "%s: %s" % (sys.argv[0], error.message())
except Exception, error:
sys.stderr.write(str(error) + "\n")
sys.stderr.write(traceback.format_exc() + "\n")
#include <pcp/pmapi.h>
<pcp/impl.h> and <pcp/pmda.h>.
libpcp library; so to link a generic PMAPI application requires something akin to the following command:
cc mycode.c -lpcpNote
libpcp_mmv) for instrumenting an application. The example programs are installed in ${PCP_DEMOS_DIR}/mmv.
libpcp_trace) for instrumenting applications. The example programs are installed in ${PCP_DEMOS_DIR}/trace.
Warning
pcp_mmv and pcp_trace instrumentation libraries, the MMV and Trace PMDAs, and the rest of the PCP infrastructure is shown in Figure 4.1, “Application and PCP Relationship”:
pcp_mmv library provides function calls to assist with extracing important performance metrics from a program into a shared, in-memory location such that the MMV PMDA can examine and serve that information on behalf of PCP client tool requests. The pcp_mmv library is described in the mmv_stats_init(3), mmv_lookup_value_desc(3), mmv_inc_value(3) man pages. Additionally, the format of the shared memory mappings is described in detail in mmv(5).
same memory that the application is updating, when current values are sampled on behalf of PMAPI client tools. This relationship, and a simplified MMV API, are shown in Figure 4.2, “Memory Mapped Page Sharing”.
pcp_mmv library initialisation API, subsequent interactions with the library are not intrusive to the instrumented application. At the points where values are updated, the only cost involved is the memory mapping update, which is a single memory store operation. There is no need to explicitly transfer control to the MMV PMDA, nor allocate memory, nor make system or library calls. The PMDA will only sample the values at times driven by PMAPI client tools, and this places no overhead on the instrumented application.
libpcp_mmv Application Programming Interface (API) can be called from C, C++, Perl and Python (a separate project, Parfait, services the needs of Java applications). Each language has access to the complete set of functionality offered by libpcp_mmv. In most cases, the calling conventions differ only slightly between languages - in the case of Java and Parfait, they differ significantly however.
mmv_stats_init, and ended with a call to mmv_stats_stop. These calls manipulate global state shared by the library and application. These are the only calls requiring synchonization and a single call to each is typically performed early and late in the life of the application (although they can be used to reset the library state as well, at any time). As such, the choice of synchonization primitive is left to the application, and none is currently performed by the library.
void *mmv_stats_init(const char *name, int cluster, mmv_stats_flags_t flags,
const mmv_metric_t *stats, int nstats,
const mmv_indom_t *indoms, int nindoms)mmv.acme.products.count will be created by the MMV PMDA. With the MMV_FLAG_NOPREFIX flag set, that would instead become mmv.products.count. It is recommended to not disable the prefix - doing so requires the applications to ensure naming conflicts do not arise in the MMV PMDA metric names.
__pmID_int Structure”, for all MMV PMDA metrics.
mmv_stats_init define the metrics and instance domains that exist within the application. These are somewhat analagous to the final parameters of pmdaInit(3), and are best explained using Example 4.1, “Memory Mapped Value Instance Structures” and Example 4.2, “Memory Mapped Value Metrics Structures”. As mentioned earlier, the full source code for this example instrumented application can be found in ${PCP_DEMOS_DIR}/mmv.
Example 4.1. Memory Mapped Value Instance Structures
#include <pcp/pmapi.h>
#include <pcp/mmv_stats.h>
static mmv_instances_t products[] = {
{ .internal = 0, .external = "Anvils" },
{ .internal = 1, .external = "Rockets" },
{ .internal = 2, .external = "Giant_Rubber_Bands" },
};
#define ACME_PRODUCTS_INDOM 61
#define ACME_PRODUCTS_COUNT (sizeof(products)/sizeof(products[0]))
static mmv_indom_t indoms[] = {
{ .serial = ACME_PRODUCTS_INDOM,
.count = ACME_PRODUCTS_COUNT,
.instances = products,
.shorttext = "Acme products",
.helptext = "Most popular products produced by the Acme Corporation",
},
};pmdaInstid Structure”
pmdaIndom Structure” - the major difference is the addition of oneline and long help text, the purpose of which should be self-explanatory at this stage.
__pmInDom_int Structure”
mmv.acme.products metrics, and they reflect the rates at which products are built by the machines in the factory, how long these builds take for each product, and how long each product type spends queued (while waiting for factory capacity to become available).
Example 4.2. Memory Mapped Value Metrics Structures
static mmv_metric_t metrics[] = {
{ .name = "products.count",
.item = 7,
.type = MMV_TYPE_U64,
.semantics = MMV_SEM_COUNTER,
.dimension = MMV_UNITS(0,0,1,0,0,PM_COUNT_ONE),
.indom = ACME_PRODUCTS_INDOM,
.shorttext = "Acme factory product throughput",
.helptext =
"Monotonic increasing counter of products produced in the Acme Corporation\n"
"factory since starting the Acme production application. Quality guaranteed.",
},
{ .name = "products.time",
.item = 8,
.type = MMV_TYPE_U64,
.semantics = MMV_SEM_COUNTER,
.dimension = MMV_UNITS(0,1,0,0,PM_TIME_USEC,0),
.indom = ACME_PRODUCTS_INDOM,
.shorttext = "Machine time spent producing Acme products",
.helptext =
"Machine time spent producing Acme Corporation products. Does not include\n"
"time in queues waiting for production machinery.",
},
{ .name = "products.queuetime",
.item = 10,
.type = MMV_TYPE_U64,
.semantics = MMV_SEM_COUNTER,
.dimension = MMV_UNITS(0,1,0,0,PM_TIME_USEC,0),
.indom = ACME_PRODUCTS_INDOM,
.shorttext = "Queued time while producing Acme products",
.helptext =
"Time spent in the queue waiting to build Acme Corporation products,\n"
"while some other Acme product was being built instead of this one.",
},
};
#define INDOM_COUNT (sizeof(indoms)/sizeof(indoms[0]))
#define METRIC_COUNT (sizeof(metrics)/sizeof(metrics[0]))pmDesc Structure”
pmUnits and pmDesc Structures”
__pmID_int Structure”
mmv_lookup_value_desc function, as shown in Example 4.3, “Memory Mapped Value Handles”.
Example 4.3. Memory Mapped Value Handles
#define ACME_CLUSTER 321 /* PMID cluster identifier */
int
main(int argc, char * argv[])
{
void *base;
pmAtomValue *count[ACME_PRODUCTS_COUNT];
pmAtomValue *machine[ACME_PRODUCTS_COUNT];
pmAtomValue *inqueue[ACME_PRODUCTS_COUNT];
unsigned int working;
unsigned int product;
unsigned int i;
base = mmv_stats_init("acme", ACME_CLUSTER, 0,
metrics, METRIC_COUNT, indoms, INDOM_COUNT);
if (!base) {
perror("mmv_stats_init");
return 1;
}
for (i = 0; i < ACME_PRODUCTS_COUNT; i++) {
count[i] = mmv_lookup_value_desc(base,
"products.count", products[i].external);
machine[i] = mmv_lookup_value_desc(base,
"products.time", products[i].external);
inqueue[i] = mmv_lookup_value_desc(base,
"products.queuetime", products[i].external);
}mmv_stats_init function) - that is, space for each and every metric, and each value (instance) of each metric when an instance domain is used. To find the handle to the space set aside for one individual value requires the tuple of base memory address of the mapping, metric name, and instance name. In the case of metrics with no instance domain, the final instance name parameter should be either NULL or the empty string.
pmAtomValue pointer, as defined in Example 3.16, “ pmAtomValue Structure”, which is a union providing sufficient space to hold any single value.
pcp_mmv API, such as the mmv_stats_inc and mmv_stats_set functions.
Example 4.4. Memory Mapped Value Updates
while (1) {
/* choose a random number between 0-N -> product */
product = rand() % ACME_PRODUCTS_COUNT;
/* assign a time spent "working" on this product */
working = rand() % 50000;
/* pretend to "work" so process doesn't burn CPU */
usleep(working);
/* update the memory mapped values for this one: */
/* one more product produced and work time spent */
mmv_inc_value(base, machine[product], working); /* API */
count[product]->ull += 1; /* or direct mmap update */
/* all other products are "queued" for this time */
for (i = 0; i < ACME_PRODUCTS_COUNT; i++)
if (i != product)
mmv_inc_value(base, inqueue[i], working);
}${PCP_DEMOS_DIR}/mmv/acme.c. There is an associated Makefile to build it, in the same directory. Running the acme binary creates the instrumentation shown in Example 4.5, “Memory Mapped Value Reports”, with live values letting us explore simple queueing effects in products being created on the ACME factory floor.
Example 4.5. Memory Mapped Value Reports
pminfo -m mmv.acme
mmv.acme.products.queuetime PMID: 70.321.10
mmv.acme.products.time PMID: 70.321.8
mmv.acme.products.count PMID: 70.321.7
pmval -f2 -s3 mmv.acme.products.time
metric: mmv.acme.products.time
host: localhost
semantics: cumulative counter (converting to rate)
units: microsec (converting to time utilization)
samples: 3
interval: 1.00 sec
Anvils Rockets Giant_Rubber_Bands
0.37 0.12 0.50
0.35 0.25 0.38
0.57 0.20 0.23pcp_mmv library is providing timing information for long-running operations. For instrumenting long-running operations, like uploading downloading a file, the overall operation may be broken into smaller, discrete units of work which can be easily instrumented in terms of operations and througput measures. In other cases, there are no divisible units for long-running operations (for example a black-box library call) and instrumenting these operations presents a challenge. Sometimes the best that can be done is adding the instrumentation point at the completion of the operation, and simply accept the "bursty" nature of this approach. In these problematic cases, the work completed in one sampling-interval may have begun several intervals before, from the point of view of the monitoring tool, which can lead to misleading results.
mmv_stats_interval_start instrumentation function. Then, with help from the MMV PMDA which recognizes this type, the act of sampling the metric value causes an interim timestamp to be taken (by the MMV PMDA, not the application) and combined with the initial timestamp to form a more accurate reflection of time spent within the timed section, which effectively smooths out the bursty nature of the instrumentation.
mmv_stats_interval_end which signifies to the MMV PMDA that the operation is not active, and no extra "in-progress" time should be applied to the exported value. At that time, the elapsed time for the entire operation is calculated and accounted toward metrics value.
pcp_trace library provides function calls for identifying sections of a program as transactions or events for examination by the trace PMDA, a user command called pmdatrace. The pcp_trace library is described in the pmdatrace(3) man page
pmtracebegin call. Time is recorded from there to the corresponding pmtraceend call (with matching tag identifier). A transaction in progress can be cancelled by calling pmtraceabort.
pmtracepoint function. This is a simpler form of monitoring that exports only the number of times a particular point in a program is passed. The pmtraceobs and pmtracecount functions have similar semantics, but the former allows an arbitrary numeric value to be passed to the trace PMDA.
libpcp_trace library make calls through the trace Application Programming Interface (API). These calls result in interprocess communication of trace data between the application and the trace PMDA. This data consists of an identification tag and the performance data associated with that particular tag. The trace PMDA aggregates the incoming information and periodically updates the exported summary information to describe activity in the recent past.
Example 4.6. Rolling-Window Sampling Technique
trace.observe.rate trace.counter.rate trace.point.rate trace.transact.ave_time trace.transact.max_time trace.transact.min_time trace.transact.rate
trace.transact.rate. So if in the last 10 seconds, 8 transactions took place, the transaction rate would be 8/10 or 0.8 transactions per second.
Table 4.1. Selected Command-Line Options
libpcp_trace Application Programming Interface (API) is called from C, C++, Fortran, and Java. Each language has access to the complete set of functionality offered by libpcp_trace. In some cases, the calling conventions differ slightly between languages. This section presents an overview of each of the different tracing mechanisms offered by the API, as well as an explanation of their mappings to the actual performance metrics exported by the trace PMDA.
pmtracebegin and pmtraceend API functions result in transaction data being sent to the trace PMDA with a measure of the time interval between the two calls. This interval is the transaction service time. Using the pmtraceabort call causes data for that particular transaction to be discarded. The trace PMDA exports transaction data through the following trace.transact metrics listed in Table 4.2, “ trace.transact Metrics”:
Table 4.2. trace.transact Metrics
|
Metric
|
Description
|
|---|---|
|
The average service time per transaction type. This time is calculated over the last sample duration.
| |
|
The running count for each transaction type seen since the trace PMDA started.
| |
|
The maximum service time per transaction type within the last sample duration.
|
|
The minimum service time per transaction type within the last sample duration.
| |
|
|
The average rate at which each transaction type is completed. The rate is calculated over the last sample duration.
|
|
The cumulative time spent processing each transaction since the trace PMDA started running.
|
pmtracepoint function is most useful when start and end points are not well defined. For example, this function is useful when the code branches in such a way that a transaction cannot be clearly identified, or when processing does not follow a transactional model, or when the desired instrumentation is akin to event rates rather than event service times. This data is exported through the trace.point metrics listed in Table 4.3, “trace.point Metrics”:
Table 4.3. trace.point Metrics
pmtraceobs and pmtracecount functions have similar semantics to pmtracepoint, but also allow an arbitrary numeric value to be passed to the trace PMDA. The most recent value for each tag is then immediately available from the PMDA. Observation data is exported through the trace.observe metrics listed in Table 4.4, “ trace.observe Metrics”:
Table 4.4. trace.observe Metrics
|
Metric
|
Description
|
|---|---|
trace.observe.count
|
Running count of observations seen since the trace PMDA started.
|
trace.observe.rate
|
The average rate at which observations for each tag occur. This rate is calculated over the last sample duration.
|
trace.observe.value
|
The numeric value associated with the observation last seen by the trace PMDA.
|
trace.counter
|
Counter data is exported through the
trace.counter metrics. The only difference between trace.counter and trace.observe metrics is that the numeric value of trace.counter must be a monotonic increasing count.
|
Table 4.5. Environment Variables
|
Name
|
Description
|
|---|---|
|
The name of the host where the trace PMDA is running.
| |
|
The number of seconds to wait until assuming that the initial connection is not going to be made, and timeout will occur. The default is three seconds.
| |
|
The number of seconds to allow before timing out on awaiting acknowledgment from the trace PMDA after trace data has been sent to it. This variable has no effect in the asynchronous trace protocol (refer to Table 4.6, “State Flags”).
| |
|
A list of values which represents the backoff approach that the
libpcp_trace library routines take when attempting to reconnect to the trace PMDA after a connection has been lost. The list of values should be a positive number of seconds for the application to delay before making the next reconnection attempt. When the final value in the list is reached, that value is used for all subsequent reconnection attempts.
|
libpcp_trace routines. These are registered through the pmtracestate call, and they can be set either individually or together.
Table 4.6. State Flags
|
Flag
|
Description
|
|---|---|
|
High-level diagnostics. This flag simply displays entry into each of the API routines.
| |
|
Diagnostic messages related to establishing and maintaining the communication channel between application and PMDA.
| |
|
The full contents of the PDU buffers are dumped as PDUs are transmitted and received.
| |
Table A.1. Performance Co-Pilot Acronyms and Their Meanings
|
Acronym
|
Meaning
|
|---|---|
|
API
|
Application Programming Interface
|
|
DBMS
|
Database Management System
|
|
Domain Name Service
| |
|
Dynamic Shared Object
| |
|
I/O
|
Input/Output
|
|
IPC
|
Interprocess Communication
|
|
Performance Co-Pilot
| |
|
Protocol data unit
| |
|
Performance Metrics Application Programming Interface
| |
|
Performance Metrics Collection Daemon
| |
|
Performance Metrics Domain Agent
| |
|
Performance Metric Identifier
| |
|
Performance Metrics Name Space
| |
|
Transmission Control Protocol/Internet Protocol
|
pmGetArchiveEnd FunctionpmGetInDomArchive FunctionpmGetArchiveLabel FunctionpmNewContext Function pmWhichContext Functiondbpmda Debug UtilitypmTraversePMNS FunctionpmReconnectContext FunctionpmLookupInDomText FunctionpmLookupText FunctionpmLookupInDomText Function pmAddProfile Function pmGetInDomArchive FunctionpmGetInDom FunctionpmGetInDom Function pmAddProfile FunctionpmSetMode FunctionpmTraversePMNS FunctionpmLookupDesc FunctionpmLookupDesc FunctionpmFetch FunctionpmNewContext FunctionpmNewContext FunctionpmExtractValue Functionsimple_store in the Simple PMDAsimple_store in the Simple PMDApmExtractValue FunctionpmFetch FunctionpmExtractValue FunctionpmAddProfile FunctionpmDelProfile FunctionpmExtractValue FunctionpmAddProfile FunctionpmGetArchiveLabel FunctionpmLookupDesc FunctionpmGetInDom FunctionpmFetch FunctionpmGetChildren FunctionpmRecordAddHost FunctionpmNewContextZone FunctionpmAtomStr FunctionpmReconnectContext FunctionpmReconnectContext FunctionpmFetch FunctionpmConvScale Functionsimple_store in the Simple PMDApmDelProfile FunctionpmDestroyContext FunctionpmDupContext FunctionpmErrStr FunctionpmExtractValue Function pmConvScale FunctionpmNewContext Function pmSetMode Function pmFetch Function pmFetch Function pmFreeResult Function pmFetchArchive Function pmPrintValue Function pmSortInstances Function Symbolic Association between a Metric's Name and ValuepmSetMode Function pmFetchArchive Functionpmflush FunctionpmFetch Function pmFreeResult FunctionpmGetArchiveEnd FunctionpmGetArchiveLabel FunctionpmGetChildren Function pmGetChildrenStatus Function PMAPI Context ServicespmGetInDom Function PMAPI Context Services pmSetMode Function pmGetInDomArchive FunctionpmGetInDomArchive FunctionpmGetPMNSLocation Function PMAPI Context ServicespmIDStr FunctionpmInDomStr FunctionpmLoadNameSpace FunctionpmLookupDesc Function PMAPI Context Services pmSetMode Function pmExtractValue Function pmConvScale FunctionpmLookupInDom Function PMAPI Context Services pmSetMode FunctionpmLookupInDomArchive FunctionpmLookupInDomText Function PMAPI Context ServicespmLookupName Function PMAPI Context Services Symbolic Association between a Metric's Name and ValuepmLookupText Function PMAPI Context ServicespmNameAll FunctionpmNameID Function PMAPI Context ServicespmNameInDom Function PMAPI Context Services pmSetMode FunctionpmNameInDomArchive FunctionpmNewContext FunctionpmNewContextZone FunctionpmNewZone FunctionpmNumberStr FunctionpmParseInterval FunctionpmParseMetricSpec Functionpmprintf FunctionpmPrintValue FunctionpmReconnectContext FunctionpmRecordAddHost FunctionpmRecordControl FunctionpmRecordSetup FunctionpmSetMode Function pmGetArchiveEnd FunctionpmSortInstances Functionsimple_store in the Simple PMDA Debugging Information Performance Metrics Values PMAPI Context Services pmStore Function pmStore FunctionpmTraversePMNS Function PMAPI Context ServicespmTypeStr FunctionpmUnitsStr FunctionpmUnloadNameSpace FunctionpmNewContext Function pmUseContext FunctionpmUseZone FunctionpmWhichContext FunctionpmWhichZone FunctionpmRecordAddHost Functionsimple_store in the Simple PMDApmNewContextZone Function