The Performance Co-Pilot provides a cross-platform API for accessing live and historical performance data from the local or remote hosts. HTOP is a performance analysis utility reporting per-process metrics with good, customizable visualization of that data. This project will involve implementing a HTOP "platform" for PCP.
Expected results: A (C language) implementation of an HTOP backend using the PCP PMAPI, with accompanying tests and documentation. The student will extend their C programming skills, as well as learn about the ncurses library. Students can also expect to gain insight into the meaning of the kernel data reported by these tools, and the sampling techniques used in building the tools.
Prerequisite knowledge: C programming, ncurses library knowledge a plus.
Skill level: Intermediate
Mentors: Mark Goodwin , Ryan Doyle
Interested students so far: 3
The Performance Co-Pilot provides a cross-platform API for accessing live and historical performance data from the local or remote hosts. An initial PCP implementation (pcp-collectl) of the Collectl utility provides some initial support for the same style of reports. This project will extend that initial code to complete the reporting coverage provided by the initial collectl utility. Additionally, the pcp-collectl utility will be updated to use modern PCP API extensions, improve the test coverage and documentation.
Expected results: The student will extend their Python programming skills, and can also expect to gain insight into the meaning of the kernel data reported by this tool, and the sampling techniques it uses.
Prerequisite knowledge: Python programming.
Skill level: Intermediate
Mentors: Nathan Scott , Mark Goodwin
Interested students so far: 2
Processes can become blocked from running as they wait for resources. This may be processor time, waiting for I/O, blocked waiting for some other process, etc. Understanding these delays can be critical for a performance analyst optimizing elapsed time in a workflow.
In last years Google Summer of Code, a student created the pcp-pidstat utility. This year, that tool will be extended to detect and report on blocked processes - showing when, why and by whom processes become blocked and unblocked.
Expected results: The student will extend their Python programming skills, and can also expect to gain insight into the meaning of the kernel data reported by this tool, and the sampling techniques it uses.
Prerequisite knowledge: Python programming.
Skill level: Intermediate
Mentors: Mark Goodwin , Nathan Scott
Interested students so far: 2
The Linux kernel device mapper statistics framework (dmstats) provides flexible IO statistics for device-mapper block devices, including logical volumes, multipath and RAID devices, as well as thinly-provisioned and cache volumes. A command line tool, dmstats, allows administrators to configure and use the facility directly. This project will use the dmstats API to allow dmstats performance data to be made available to PCP users via a new PCP PMDA. The primary API for dmstats is C but an experimental Python API exists and may be suitable for prototyping. The project will explore not only exporting simple counter and metric values, but will also evaluate the feasibility of including support for advanced features including dmstats latency and IO distribution histograms.
Expected results: A PCP PMDA for the dmstats subsystem will be designed, implemented, tested and documented. The PMDA need not implement all the available dmstats functionality immediately but it should be designed in such a way as to permit future extension and enhancement.
Prerequisite knowledge: C, Python (optional), some knowledge of logical volume management and software defined storage is a bonus.
Skill level: Intermediate-Advanced
Mentors: Bryn M. Reeves , Mark Goodwin
Interested students so far: 1
eBPF (extended Berkeley Packet Filtering) is the latest tracing capability added to the Linux kernel, which is an enhancement over BPF . It is an in-kernel bytecode machine that can be used for tracing, instrumentation and other capabilities. It can be used to create custom metrics by either instrumenting the kernel or a userspace program.
Tools like bcc have an elegant API and scripts (in python) to use the eBPF interface to instrument the kernel/processes.
Expected results: A PCP PMDA (using bcc) for the bpf subsystem will be designed, implemented, tested and documented. The PMDA can implement a few basic metrics and it should be extensible, so that it can be enhanced. The student is expected to gain insights on Linux kernel and user-space instrumentation.
Prerequisite knowledge: Python, C, Linux kernel.
Skill level: Intermediate-Advanced
Mentors: Hemant Kumar , Mark Goodwin , Nathan Scott
Interested students so far: 1
The PCP pmlogger records statistics from a running machine. This project will develop a Python class library to read in PCP archives with multidimensional indexing for data-science style analysis and performance data mining. It will also feed pmlogsummary statistical data into Python's extensive data-science class librariess (e.g. numpy ) for analysis and graphical visualization.
Expected results: Construction of a class library to assist with exporting PCP data to scientific libraries, as well as tools to visualize that data.
Prerequisite knowledge: Python programming, PyQt knowledge a plus.
Skill level: Intermediate
Mentors: Mark Goodwin , Nathan Scott
Interested students so far: 1
Flame Graphs are a visualization of profiled software, allowing frequently travelled code paths to be identified quickly and accurately. This project will implement an efficient interface between the Vector web application, the Linux kernel eBPF and/or perf facilities, and a PCP agent (PMDA) like pmdapipe to trigger and relay the output to Vector
Expected results: A capability for rendering Flame Graphs on-demand over a given time interval is available from the Vector web application.
Prerequisite knowledge: C programming and Linux kernel knowledge, JavaScript, Angular is a plus.
Skill level: Advanced
Mentors: Martin Spier , Mark Goodwin , Lukas Berk
Interested students so far: 0
The Vector web application produces graphs and performance dashboards for live analysis. The student will design and build a new Vector widget for creating ad-hoc graphs, for arbitrary PCP metrics.
Expected results: The simple example shows creation of a custom Vector widget by-hand, it gives an idea of the sorts of configurable widget that we aim to generate.
Prerequisite knowledge: JavaScript, Angular is a plus.
Skill level: Intermediate
Mentors: Martin Spier , Mark Goodwin , Henry Chang
Interested students so far: 3
PCP has been ported to many platforms, including Windows. This project involves customizing and extending the popular git for Windows installer with PCP packages. This installer is a particularly good match for PCP, as it has many of the same sorts of needs, such as the need to install several POSIX command line utilities as well.
Expected results: Demonstrated monitoring of Windows servers with latest PCP, after a clean installation experience.
Prerequisite knowledge: Knowledge of Windows and POSIX systems, Win32 API experience a plus.
Skill level: Introductory-Intermediate
Mentors: Nathan Scott , Ken McDonell
Interested students so far: 1
PCP has been ported to many platforms, including Mac OS X. This project involves building a new PCP installer for the popular Mac OS X platform, based on technologies available in the latest versions of the operating system, switching from the (now deprecated) technologies previously used.
In addition, the student will transition PCP to using launchd for daemon process management.
Expected results: Improved installation and runtime experience for PCP on Mac OS X.
Prerequisite knowledge: Knowledge of Mac OS X and POSIX systems
Skill level: Introductory-Intermediate
Mentors: Nathan Scott , Paul Smith
Interested students so far: 0
PCP supports a mechanism for developers to expose metrics inside their own applications. There are three existing implementations - one for each of C, Java and Go. This project expands that to include Ruby.
Expected results: The project will be implemented in three stages:
Prerequisite knowledge: Ruby and C programming
Skill level: Intermediate-Advanced
Further reading:
Mentors: Ryan Doyle , Nathan Scott
Interested students so far: 0
PCP supports a mechanism for developers to expose metrics inside their own applications. There are three existing implementations - one for each of C, Java and Go. This project expands that to include Rust.
Expected results: The project will be implemented in three stages:
Prerequisite knowledge: Rust and C programming
Skill level: Intermediate-Advanced
Further reading:
Mentors: Ryan Doyle , Mark Goodwin , Suyash
Interested students so far: 1
The Parfait project provides a java-agent for JVM applications exporting metrics into PCP. Typically these metrics are accessed via JMX initially.
The issue we face is the lack of metadata associated with JMX beans - PCP needs to export the units, semantics and other metadata for these metrics. Currently we encode this information via Spring XML configuration in the parfait-agent directly. This project will implement a tool to aid automation of the process of tracking JMX metrics and categorising their metadata, building on the existing Spring-based Parfait agent code and support the addition of new metrics more easily.
Expected results: A Java utility will be designed, implemented, tested and documented that reports on the available JMX metrics from a running application, and the performance metric metadata associated with each. When new JMX beans are found, the utility will drive the process of applying performance-related metadata (such as units) before exporting values to PCP for recording, reporting, visualizing, inference and so on.
Prerequisite knowledge: Java
Skill level: Intermediate
Mentors: Nathan Scott , Jie Kang , Werner Keil , Paul Smith
Interested students so far: 1
The Hawkular open source monitoring solution provides a performance metric store for metrics from arbitrary sources. This project will use the PCP API to implement a bridge between the low-level platform metrics (kernel, hardware, databases, and so on) available from PCP and a Hawkular instance. The project will explore not only exporting metric values, but also other metric metadata available from PCP (such as units, value type, semantics and labels) into Hawkular for a rich integration.
Expected results: A Python utility will be designed, implemented, tested and documented that uses the PCP APIs to export performance metrics to Hawkular, via the Hawkular metrics REST interface.
Prerequisite knowledge: Python, Java knowledge a plus
Skill level: Intermediate
Mentors: Lukas Berk , Matthew Wringe
Interested students so far: 1
The Prometheus open source monitoring solution provides a performance metric store for metrics from arbitrary sources. This project will use the PCP API to implement a bridge between the low-level platform metrics (kernel, hardware, databases, and so on) available from PCP and a Prometheus instance. The project will explore not only exporting metric values, but also other metric metadata available from PCP (such as units, value type, semantics and labels) into Prometheus for a rich integration.
Expected results: A Python utility will be designed, implemented, tested and documented that uses the PCP APIs to export performance metrics to Prometheus, via the Prometheus exporter Python API.
Prerequisite knowledge: Python, Golang knowledge a plus
Skill level: Intermediate
Mentors: Nathan Scott , Lukas Berk
Interested students so far: 1