Secure Client Connections


This chapter of the Performance Co-Pilot tutorial covers setting up secure connections between PCP collector and monitor components. We discuss setting up certificates on both the collector and monitor hosts. The certificate on the collector ensures that the connection is encrypted. The certificate on the client confirms to the collector that a valid connection is being made.

For an explanation of Performance Co-Pilot terms and acronyms, consult the PCP glossary .


The Performance Co-Pilot includes facilities for establishing secure connections between remote collector and monitoring components.

All connections made to the PCP metrics collector daemon ( pmcd ) are made using the PCP protocol, which is TCP/IP based. Traditionally, no functionality was available to secure connections between PCP collectors and monitors. However, as PCP evolved to be able to export sensitive information (event trace parameters and detailed per-process statistics, for example), it became necessary to provide safeguards against malicious behaviour.

The cryptographic services used to augment the PCP protocol are provided by Network Security Services (NSS), a library implementing Transport Layer Security (TLS) and the Secure Sockets Layer (SSL) standards, and base cryptographic functions. NSS includes a software-based cryptographic token which is FIPS 140-2 certified.

Both the pmcd and pmproxy daemons are capable of simultaneous TLS/SSL and non-SSL communications. This means that you do not have to choose between TLS/SSL or non-SSL communications for your PCP Collector systems; both can be used at the same time.

In addition, pmcd can be instructed to force connecting clients to provide certificates verifying their trust to the collector. This is with a new "-Q" option. Also, pmproxy can forward this requirement to clients.

Check local PCP collector installation (requires the pcp-verify utility):
$ pcp verify --secure

Enabling Client and Server TLS/SSL: Steps Involved

Before the PCP Collector system can be requested to communicate with TLS/SSL, certificates must be properly configured on the Collector Server and Client Monitor hosts.

This typically involves:

  1. Obtain and install certificates for your PCP Collector systems, and configure each system to trust the certification authority's (CA's) certificate. This tutorial shows how to setup your own local CA to generate certificates.
  2. Enable secure connections in the pmcd and pmproxy daemons by configuring the system certificate database with the PCP Collector certificate.
  3. Creating a certificate that is trusted by the collector and installing this on the client.
  4. Configuring the collector to reject non-local connections without a trusted client certificate.
  5. Ensure that each user monitoring a PCP Collector system obtains and installs a personal certificate for the tools that will communicate with that collector.
    This can be done by manually updating a monitor-side certificate database, or automatically by reviewing and accepting the certificate delivered to the monitor tools during the first attempt to access the PCP Collector system.

CA Setup

We will be using a local CA to generate the locally trusted certificates. At a high-level: a certificate request (CR) must be generated, then sent to the certificate authority (CA) you will be using. The CA will generate a new trusted certificate and send it to you. Once this certificate has been received, it will be installed on the client.

Create the local CA:

              $ openssl genrsa -out rootCA.key 2048
              $ openssl req -new -x509 -extensions v3_ca -key ca.key -out rootCA.crt -days 3650
              Common Name (eg, your name or your server's hostname) []:myCA

Create the client certificate:

              $ openssl genrsa -out pmclient.key 2048
              $ openssl req -new -key pmclient.key -out pmclient.csr
              Common Name (eg, your name or your server's hostname) []:pmclient
              $ openssl x509 -req -in pmclient.csr -CA rootCA.crt -CAkey rootCA.key -CAcreateserial -out pmclient.crt -days 500 -sha256
              $ openssl pkcs12 -export -out pmclient_pkcs12.key -inkey pmclient.key  -in pmclient.crt -certfile rootCA.crt

Create the server certificate:

              $ openssl genrsa -out pmserver.key 2048
              $ openssl req -new -key pmserver.key -out pmserver.csr
              Common Name (eg, your name or your server's hostname) []:pmserver
              $ openssl x509 -req -in pmserver.csr -CA rootCA.crt -CAkey rootCA.key -CAcreateserial -out pmserver.crt -days 500 -sha256
              $ openssl pkcs12 -export -out pmserver_pkcs12.key -inkey pmserver.key  -in pmserver.crt -certfile rootCA.crt

Collector Setup

All PCP Collector systems must have a valid certificate in order to participate in secure PCP protocol exchanges. We will use the server certificate created above and install it into a pcp specific directory that can be used by the pcp server components. This directory should exists with current PCP versions.

Install the CA and server certificate (As the PCP user) :

              $ echo > /tmp/empty
              $ certutil -d sql:/etc/pcp/nssdb  -N -f /tmp/empty
              $ certutil -d sql:/etc/pcp/nssdb -A -t "CT,," -n "Root CA" -i rootCA.crt
              $ certutil -d sql:/etc/pcp/nssdb -A -t "P,," -n "pmserver_cert" -i pmserver.crt
              $ pk12util -i pmserver_pkcs12.key -d sql:/etc/pcp/nssdb
              $ certutil -d sql:/etc/pcp/nssdb -L
              Certificate Nickname                                         Trust Attributes
              Root CA                                                      CT,, 
              pmserver_cert                                                P,,  

Configure pmcd to use our certificate and enforce client certificates:

              $ cat /etc/pcp/pmcd/pmcd.options
              -C sql:/etc/pcp/nssdb
              -M pmserver_cert

At this stage, attempts to restart the PCP Collector infrastructure will begin to take notice of the new contents of the certificate database.

Monitor Setup

Attempts to connect to the server wirthout a certificate will now fail:

Test remote connection:

              $ pminfo -h
              pminfo: Cannot connect to PMCD on host "": PMCD requires a client certificate

In this configuration, PCP Monitoring (client) tools require 2 certificates. A client certificate that can be sent to the server for authentication, and a trusted certificate to validate the server in a TLS/SSL connection. The first certificate was generated above and will be installed manually:

Install the CA and client certificate (As local user) :

              $ echo > /tmp/empty
              $ mkdir -p -m 0755 $HOME/.pki/nssdb
              $ certutil -d sql:$HOME/.pki/nssdb -N -f /tmp/empty
              $ certutil -d sql:$HOME/.pki/nssdb -A -t "CT,," -n "Root CA" -i ./rootCA.crt
              $ certutil -d sql:$HOME/.pki/nssdb -A -t "P,," -n "pmclient_cert" -i pmclient.crt
              $ pk12util -i pmclient_pkcs12.key -d sql:$HOME/.pki/nssdb

The second certificate can be installed beforehand or can be delivered via the TLS/SSL connection exchange. In the latter situation, the user is prompted as to whether the certificate is to be trusted (see example below).

Once certificates are in place, we are ready to attempt to establish secure connections between remote PCP Monitor and Collector hosts. This can be achieved by specifically requesting a secure connection for individual host connections. Alternatively, an environment variable can be set to request that all client connections within that shell environment be made securely. This environment variable should have the value enforce meaning "all connections must be secure, fail if this cannot be achieved".

Using the approach of certificate delivery via the TLS/SSL protocol, the database and certificate will be automatically setup in the correct location on your behalf. You can also set some environment variables if you are using self signed certs or if the domainname in the cert does not match. Without these, you will be interactively prompted to approve the certificate.

To establish a secure connection, in a shell enter:

              $ export PCP_SECURE_SOCKETS=enforce
              $ export PCP_ALLOW_BAD_CERT_DOMAIN=1
              $ export PCP_ALLOW_SERVER_SELF_CERT=1
              $ pminfo -h -f kernel.uname.nodename
                  value ""