PMREGISTERDERIVED
Section: C Library Functions (3)
Updated:
Index
Return to Main Contents
NAME
pmRegisterDerived
,
pmRegisterDerivedMetric
 register a derived metric name and definition
C SYNOPSIS
#include <
pcp/pmapi.h
>
char *pmRegisterDerived(char *
name
, char *
expr
);
int pmRegisterDerivedMetric(char *
name
, char *
expr
, char **
errmsg
);
cc ... lpcp
DESCRIPTION
Derived metrics provide a way of extending the Performance Metrics
Name Space (PMNS) with new metrics defined at the PCP clientside using
expressions over the existing performance metrics.
Typical uses would be to aggregate a number of similar metrics to provide
a higherlevel summary metric or to support the ``delta V over delta V''
class of metrics that are not possible in the base data semantics of PCP.
An example of the latter class would be the average I/O size, defined
as
delta(disk.dev.total_bytes) / delta(disk.dev.total)
where both of the
disk.dev
metrics are counters, and what is required
is to to sample both metrics, compute the difference between the current
and previous values and then calculate the ratio of these differences.
The arguments to
pmRegisterDerived
are the
name
of the new derived metric and
expr
is an expression defining how the values of
name
should be computed.
pmRegisterDerivedMetric
is the exact functional equivalent to
pmRegisterDerived
except that it provides a simplified model of error handling, where
a formatted message is returned via the
errmsg
parameter.
Syntactic checking is performed at the time
pmRegisterDerived
is called, but semantic checking is deferred until each new PMAPI context
is created with
pmNewContext
(3)
or reestablished with
pmReconnectContext
(3),
at which time the PMNS and metadata is available to
allow semantic checking and the metadata of the derived metrics
to be determined.
This means
pmRegisterDerived
does not apply retrospectively to any open PMAPI contexts, nor
to any PMAPI contexts already open at the time
pmRegisterDerived
is called, so the normal
use would be to make all calls to
pmRegisterDerived
(possibly via
pmLoadDerivedConfig
(3))
or
pmRegisterDerivedMetric
and then call
pmNewContext
(3).
name
should follow the syntactic rules for the names of performance metrics,
namely one or more components separated with a dot (``.''), and each
component must begin with an alphabetic followed by zero or more characters
drawn from the alphabetics, numerics and underscore (``_'').
For more details, refer to
PCPIntro
(1)
and
pmns
(5).
name
must be unique across all derived metrics and should
not
match the
name of any regular metric in the PMNS. It is acceptable for
name
to share some part of its prefix with an existing subtree of the PMNS,
e.g. the average I/O size metric above could be named
disk.dev.avgsz
which would place it amongst the other
disk.dev
metrics in the PMNS.
Alternatively, derived metrics could populate their own subtree
of the PMNS,
e.g. the average I/O size metric above could be named
my.summary.disk.avgsz
.
The expression
expr
follows these syntactic rules:

*

Terminal elements are either names of existing metrics or numeric constants.
Recursive definitions are not allowed, so only the names of regular
metrics (not other derived metrics) may be used. Numeric constants are
either integers constrained to the precision of 32bit unsigned integers
or double precision floating point numbers.

*

The usual binary arithmetic operators are supported, namely addition (``+''),
subtraction (``''), multiplication (``*'') and division (``/'') with
the normal precedence rules where multiplication and division have
higher precedence than addition and subtraction, so
a+b*c
is evaluated as
a+(b*c)
.

*

Unary negation may be used, e.g.
3*some.metric
.

*

Cstyle relational operators are supported, namely ``<'', ``<='', ``=='',
``>='', ``>'' and ``!=''.
Relational expresssions return a value as a 32bit unsigned number being
0 for false and 1 for true.
The expected operator precedence rules
apply, so arithmetic operators have higher precedence than relational operators,
and
ab>c+d
is evaluated as
(ab)>(c+d)
.
All the relational operators have equal precedence, so the (slightly odd)
expression involving consecutive relational operators
a>b!=c
is evaluated as
(a>b)!=c
.

*

Cstyle boolean operators are supported, namely and (``&&'') and or (``'').
Boolean expresssions return a value as a 32bit unsigned number being
0 for false and 1 for true.
The expected operator precedence rules
apply, so relational operators have higher precedence than boolean operators,
and
a>b*c&&d<=e+f
is evaluated as
(a>(b*c))&&(d<=(e+f))
.
Both the boolean operators have equal precedence, so the
expression involving consecutive boolean operators
a>=bb>c&&d!=ef>g
is evaluated as
(((a>=b)(b>c))&&(d!=e))(f>g)
.

*

Additionally, the ``!'' operator may be used to negate a boolean or
relational expression, returning a value as a 32bit unsigned number being
0 for false and 1 for true.
The expected operator precedence rules
apply, so boolean (and relational) operators have higher precedence
than boolean negation,
and
!a>bc<d
is evaluated as
!((a>b)(c<d))
,
while
!a<b+c
is evaluated as
!(a<(b+c))
.

*

Cstyle ternary conditional expressions are supported. In general terms
the expression
check ? foo : bar
is evaluated as
foo
(the ``true'' operand) if
check
(the ``guard'') is true, else the
expression evaluates to
bar
(the ``false'' operand).
Some special semantic rules apply to the ``guard'' expression and
the other two operand expressions:



(a)

Each expression may involve a singular value or a set of values (when
the expression involves one or more metrics with an instance domain).

(b)

All
expressions with a set of values must be defined over the
same
instance domain.

(c)

Both operand expressions must have the
same
metadata, so the same metric type, semantics and units (dimension and scale).

(d)

The ``guard'' expression must have an aritmetic or relational or boolean value,
so that it can be evaluated as 0 for false, else true.

(e)

If the ``guard'' expression has a singular value and one or more of
the other operand expressions involves an instance domain, the ``guard''
applies to all instances.

(f)

If the ``guard'' expression has a set of values and one or more of
the other operand expressions involves an instance domain, the ``guard''
is evaluated once for each instance (there must be one instance
domain as per rule (b) above).

(g)

If one of the operand expressions has a singular value and the other has
a set of values, and the singular value is selected based on the
evaluation of the ``guard'', then the result is a set of values
(all the same) with instance enumeration being taken from the other
operand expression. For example in the expression:
foo ? scalar : set
, if
foo
is true, then the result is a set of values
(all having the same value,
scalar
) over
the instance domain of
set
.

*

Selection of a single instance can be specified by the
construct ``[
instance_name
]'' which may be appended to
a metric name or a parenthesized expression.
For example:
fw.bytes = network.interface.in.bytes[eth1] + \
network.interface.out.bytes[eth1]
or (equivalently):
fw.bytes = (network.interface.in.bytes + \
network.interface.out.bytes)[eth1]


All characters between the ``['' and ``]'' are considered to
be part of the (external) instance name, so be careful to avoid
any spurious white space.
A backslash may be used as an escape prefix
in the (unlikely) event that the external instance name contains a ``]''.

*

Numeric constants can also be specified using the
mkconst()
constructor which takes a number of arguments: the first is a numeric
constant (either integer or floating point), then follow one or more
parameters of the form
tag=value
or
tag=
where the allowed values of
tag
and
value
are as follows:
tag

value


type

one of the numeric metric types from <
pcp/pmapi.h
>, stripped of the
PM_TYPE_
prefix, so
32
,
U32
,
64
,
U64
,
FLOAT
or
DOUBLE
.


semantics

one of the semantic types from <
pcp/pmapi.h
>, stripped of the
PM_SEM_
prefix, so
COUNTER
,
INSTANT
or
DISCRETE
.


units

a specification of dimension and scale (together forming the units),
in the syntax accepted by
pmParseUnitsStr
(3).




The
value
may optionally be enclosed in double quotes, and may appear in any
mix of upper and/or lower case.
The
tag
must be in lower case as shown in the table above.
This is most useful when the expression semantics require matching
type and/or semantics and/or units for operands, e.g.
idle = mem.util.free > mkconst(10485760, units=Kbyte)
avg_io_size = delta(disk.dev.total) == 0 ? \
mkconst(1.0, semantics=instant, units="kbyte / count") : \
delta(disk.dev.total_bytes) / delta(disk.dev.total)

*

Expressions may be rescaled using the
rescale
function that takes two arguments. The first is an
arithmetic expression to be
rescaled, and the second is the desired units after rescaling that
is a string value in the syntax accepted by
pmParseUnitsStr
(3).
For example:
rescale(network.interface.total.bytes, "Mbytes/hour")


The expression and the desired units must both have the same dimension,
e.g Space=1, Time=1 and Count=0 in the example above.

*

The following unary functions operate on a single performance metric
and return one or more values.
For all functions (except
count()
,
defined()
and
instant()
), the type of the operand metric must be arithmetic
(integer of various sizes and signedness, float or
double).
Function

Value


avg(x)

A singular instance being the average value across all instances for the metric x.


count(x)

A singular instance being the count of the number of instances for the metric x.
As a special case, if fetching the metric x returns an error, then
count(x)
will be 0.


defined(x)

A boolean value that is true (``1'') if the metric
x
is defined in the PMNS, else false (``0'').
The function is evaluated when a new PMAPI context
is created with
pmNewContext
(3)
or reestablished with
pmReconnectContext
(3).
So any subsequent changes to the PMNS after the PMAPI
context has been established will not change the value
of this function in the expression evaluation.


delta(x)

Returns the difference in values for the metric x between
one call to
pmFetch
(3)
and the next. There is one value in the result
for each instance that appears in both the current and the previous
sample.


rate(x)

Returns the difference in values for the metric x between
one call to
pmFetch
(3)
and the next divided by the elapsed time between the calls to
pmFetch
(3).
The semantics of the derived metric are based on the semantics of the
operand (x) with the dimension in the
time
domain decreased by one and scaling if required in the time utilization case
where the operand is in units of time, and the derived metric is unitless.
This mimics the rate conversion applied to counter metrics by tools
such as
pmval
(1),
pmie
(1)
and
pmchart
(1).
There is one value in the result
for each instance that appears in both the current and the previous
sample.


instant(x)

Returns the current value of the metric x, even it has
the semantics of a counter, i.e. PM_SEM_COUNTER.
The semantics of the derived metric are based on the semantics of the
operand (x); if x has semantics PM_SEM_COUNTER, the semantics of
instant(x) is PM_SEM_INSTANT, otherwise the semantics of the derived metric
is the same as the semantics of the metric x.


max(x)

A singular instance being the maximum value across all instances for the metric x.


min(x)

A singular instance being the minimum value across all instances for the metric x.


sum(x)

A singular instance being the sum of the values across all instances for the metric x.



*

The
matchinst
function may be used to select a subset of the instances from
an instance domain for a metric or expression.
The function takes two arguments:



(a)

A instance filter that consists of an optional negation operator ``!'' followed
by a regular expression delimited by ``/'' characters.
The regular expression follows the
POSIX Extended Regular Expression syntax as described in
regex
(3).
Backslashes may be used as escape prefixes, but double backslash is required to
escape any regular expression special characters, e.g. for the (extremely unlikely)
case of wanting to match instance names like ``some*text/other[text]'' a
regular expression of the form
/some\\*text\/other\\[text]/
would be required.
If present, the negation operator reverses the sense of the filtering,
so all instances
not
matching the regular expression will be selected.

(b)

A metric or expression that must be defined over an instance domain.
For example, the following expression will have values for the
metric
network.interface.in.bytes
for all network
interfaces
except
the loopback and virtual bridge devices:
matchinst(!/^(lo)(vbir)/, network.interface.in.bytes)

*

The
scalar
function may be used convert a metric or expression
defined over an instance domain into a scalar value that can be used
in other expressions.
For example:
net.in.bytes = scalar(network.interface.in.bytes[eth0]) + \
scalar(network.interface.in.bytes[eth1])


The instance domain is removed from the
metadata for the result and the instance identifier is removed from
the value during fetching.
If the metric or expression involves more than one instance then the
result is formed by picking the first instance  this is arbitrary
and implies the
scalar
function should only be used for metrics
or expressions that are expected to contain zero or one instances, e.g.
the construct ``[
instance_name
]'' or the
matchinst
function
with a pattern that matches at most one instance.

*

Parenthesis may be used for explicit grouping.

*

Lines beginning with ``#'' are treated as comments and ignored.

*

White space is ignored.
SEMANTIC CHECKS AND RULES
There are a number of conversions required to determine the
metadata for a derived metric and to ensure the semantics of
the expressions are sound.
In an arithmetic expression or a relational expression, if the semantics of both operands is not
a counter (i.e. PM_SEM_INSTANT or PM_SEM_DISCRETE) then the result
will have semantics PM_SEM_INSTANT unless both operands are
PM_SEM_DISCRETE in which case the result is also PM_SEM_DISCRETE.
For an arithmetic expression, the dimension of each operand must be the same.
For a relational expression, the dimension of each operand must be the same,
except that numeric constants (with no dimension) are allowed, e.g. in the expression
network.interface.in.drops > 0
.
To prevent arbitrary and nonsensical combinations
some restrictions apply to expressions that combine metrics with
counter semantics to produce a result with counter semantics.
For an arithmetic expression, if both operands have the semantics of
a counter, then only addition or subraction is allowed, or if the
left operand is a counter and the right operand is not, then only
multiplication or division are allowed, or if the left operand is
not a counter and the right operand is a counter, then only multiplication
is allowed.
Because relational expressions use the current value only and produce
a result that is not a counter, either or both operands of a relational expression
may be counters.
The mapping of the pmUnits of the metadata uses the following rules:

*

If both operands have a dimension of Count and the scales are not
the same, use the larger scale and convert the values of the operand
with the smaller scale.

*

If both operands have a dimension of Time and the scales are not
the same, use the larger scale and convert the values of the operand
with the smaller scale.

*

If both operands have a dimension of Space and the scales are not
the same, use the larger scale and convert the values of the operand
with the smaller scale.

*

For addition and subtraction all dimensions for each of the operands
and result are identical.

*

For multiplication, the dimensions of the result are the sum of the
dimensions of the operands.

*

For division, the dimensions of the result are the difference of the
dimensions of the operands.
Scale conversion involves division if the dimension is positive else
multiplication if the dimension is negative. If scale conversion is
applied to either of the operands, the result is promoted to type
PM_TYPE_DOUBLE.
Putting all of this together in an example, consider the derived
metric defined as follows:
x = network.interface.speed  delta(network.interface.in.bytes) / delta(sample.milliseconds)
The type, dimension and scale settings would propagate up the expression
tree as follows.
Expression

Type

Dimension & Scale

Scale Factor(s)


sample.milliseconds

DOUBLE

millisec


delta(...)

DOUBLE

millisec


network...bytes

U64

byte


delta(...)

U64

byte


delta(...) / delta(...)

DOUBLE

byte/millisec

/1048576 and *1000

network...speed

FLOAT

Mbyte/sec


x

DOUBLE

Mbyte/sec



Expressions involving single instance selection or the
matchinst
function must be associated with underlying
metrics that have an instance domain. These constructors
make no sense for singular metrics.
Because semantic checking cannot be done at the time
pmRegisterDerived
is called, errors found during semantic checking (when
any subsequent calls to
pmNewContext
(3)
or
pmReconnectContext
(3)
succeed) are reported using
pmprintf
(3).
These include:

Error: derived metric <name1>: operand: <name2>: <reason>

There was a problem calling
pmLookupName
(3)
to identify the operand metric <name2> used in the definition
of the derived metric <name1>.

Error: derived metric <name1>: operand (<name2> [<pmid2>]): <reason>

There was a problem calling
pmLookupDesc
(3)
to identify the operand metric <name2> with PMID <pmid2>
used in the definition of the derived metric <name1>.

Semantic error: derived metric <name>: <operand> : <operand> Different <metadata> for ternary operands

For a ternary expression, the ``true'' operand and the ``false'' operand
must have exactly the same metadata, so type, semantics, instance domain,
and units (dimension and scale).

Semantic error: derived metric <name>: <operand> <op> <operand>: Dimensions are not the same

Operands must have the same units (dimension and scale) for each of
addition, subtraction, the relational operators and the boolean ``and''
or ``or'' operators.

Semantic error: derived metric <name>: <operand> <op> <operand>: Illegal operator for counter and noncounter

Only multiplication or division are allowed if the left operand has the
semantics of a counter and the right operand is
not
a counter.

Semantic error: derived metric <name>: <operand> <op> <operand>: Illegal operator for counters

If both operands have the semantics of counter, only addition or subtraction
make sense, so multiplication and division are not allowed.

Semantic error: derived metric <name>: <operand> <op> <operand>: Illegal operator for noncounter and counter

Only multiplication is allowed if the right operand has the
semantics of a counter and the left operand is
not
a counter.

Semantic error: derived metric <metric> <expr> RESCALE <units>: Incompatible dimensions

The parameters <expr> and <units> to the
rescale
function must have the same dimension along the axes of Time,
Space and Count.

Semantic error: derived metric <name>: Incorrect time dimension for operand

Rate conversion using the
rate
()
function is only possible for operand metrics with a Time dimension of 0 or 1
(see
pmLookupDesc
(3)).
If the operand metric's Time dimension is 0, then
the derived metrics has a value "per second" (Time dimension of 1).
If the operand metric's Time dimension is 1, then
the derived metrics has a value of time utilization (Time dimension of 0).

Semantic error: derived metric <name>: <function>(<operand>): Nonarithmetic operand for function

The unary functions are only defined if the operand has arithmetic type.
Similarly the first argument to the
rescale
function must be of arithmetic type.

Semantic error: derived metric <name>: <expr> ? ...: Nonarithmetic operand for ternary guard

The first expression for a ternary operator must have an arithmetic type.

Semantic error: derived metric <name>: ...  ...: Nonarithmetic operand for unary negation

Unary negation only makes sense if the following expression
has an arithmetic type.

Semantic error: derived metric <name>: <operand> <op> <operand>: Nonarithmetic type for <leftorright> operand

The binary arithmetic operators are only allowed with operands with an
arithmetic type (integer of various sizes and signedness, float or
double).

Semantic error: derived metric <name>: <operand> <op> <operand>: Noncounter and not dimensionless <leftorright> operand

For multiplication or division or any of the relational operators, if
one of the operands has the semantics of a counter and the other
has the semantics of a noncounter (instantaneous or discrete) then
the noncounter operand must have no units (dimension and scale).

Semantic error: derived metric <name>: <expr> ? <expr> : <expr>: Nonscalar ternary guard with scalar expressions

If the ``true'' and ``false'' operands of a ternary expression have
a scalar value, then the ``guard'' expression must also have a
scalar value.

Semantic error: derived metric <name>: <expr> <op> <expr>: Operands should have the same instance domain

For all of the binary operators (arithmetic and relational), if both
operands have nonscalar values, then they must be defined over the
same instance domain.
EXPRESSION EVALUATION
For the binary arithmetic operators,
if either operand must be scaled (e.g. convert bytes to Kbytes) then the
result is promoted to PM_TYPE_DOUBLE.
Otherwise the type of the result is determined
by the types of the operands, as per the following table which is evaluated
from top to bottom until a match is found.
Operand Types

Operator

Result Type


either is PM_TYPE_DOUBLE

any

PM_TYPE_DOUBLE


any

division

PM_TYPE_DOUBLE


either is PM_TYPE_FLOAT

any

PM_TYPE_FLOAT


either is PM_TYPE_U64

any

PM_TYPE_U64


either is PM_TYPE_64

any

PM_TYPE_64


either is PM_TYPE_U32

any

PM_TYPE_U32


otherwise (both are PM_TYPE_32)

any

PM_TYPE_32


CAVEATS
Derived metrics are not available when using
pmFetchArchive
(3)
as this routine does not use a target list of PMIDs that could be
remapped (as is done for
pmFetch
(3)).
There is no
pmUnregisterDerived
method, so once registered a derived metric persists for the life
of the application.
DIAGNOSTICS
On success,
pmRegisterDerived
returns NULL.
If a syntactic error is found at the time of registration, the
value returned by
pmRegisterDerived
is a pointer into
expr
indicating
where
the error was found. To identify
what
the error was, the application should call
pmDerivedErrStr
(3)
to retrieve the corresponding parser error message.
pmRegisterDerivedMetric
returns 0 and
errmsg
is undefined if the parsing is successful.
If the given
expr
does not conform to the required syntax
pmRegisterDerivedMetric
returns 1 and a dynamically allocated error message string in
errmsg
.
The error message is terminated with a newline
and includes both the input
name
and
expr
,
along with an indicator of the position at which the
error was detected.
e.g.
Error: pmRegisterDerivedMetric("my.disk.rates", ...) syntax error
4rat(disk.dev.read)
^
The position indicator line may be followed by an additional
diagnostic line describing the nature of the error, when available.
In the case of an error, the caller is responsible for calling
free
(3)
to release the space allocated for
errmsg
.
SEE ALSO
PCPIntro
(1),
free
(3),
PMAPI
(3),
pmDerivedErrStr
(3),
pmFetch
(3),
pmLoadDerivedConfig
(3),
pmNewContext
(3),
pmprintf
(3)
and
pmReconnectContext
(3).
Index

NAME


C SYNOPSIS


DESCRIPTION


SEMANTIC CHECKS AND RULES


EXPRESSION EVALUATION


CAVEATS


DIAGNOSTICS


SEE ALSO
