hardware: speed up reading PDU hwmon data
We found out that reading properties of our PDU (and PSU) is really slow.
A NETCONF GET for /ietf-hardware:hardware tree can take up 600 ms which
is (IMHO) unacceptable.
We traced the issue to kernel's pmbus driver for fsp-3y [1]. The driver
sleeps 20-30 ms when page switching [1, line 100] and our code was
(according to perf's off-cpu profiling [2]) invoking the sleep several
times.
By looking at the pmbus_driver_info for our PDU [1, line 187] we can see
what pmbus page is needed when accessing hwmon file. The trick is to
minimize the page switching so we should query all the files that trigger
reading data from the first page, then the second page, and so on.
This commit groups the sysfs (hwmon) file accesses according to the
location of the data in PDU's pmbus pages to minimize page switches.
The device has three pages so this order of reading invokes the sleep
at most 3 times.
The performance is still not ideal but we managed to cut some time by
reducing 17 sleeps when reading both PSU and PDU data to 11.
More changes are about to come.
[1] https://elixir.bootlin.com/linux/v5.16/source/drivers/hwmon/pmbus/fsp-3y.c
[2] https://github.com/KDAB/hotspot/blob/7f4ff4dd8f7d524dfa981aa9147f8dd824b9de6b/README.md#off-cpu-profiling
Change-Id: Ib4b05336ed71e798d3e0be94a63793713213cfc4
diff --git a/src/ietf-hardware/Factory.cpp b/src/ietf-hardware/Factory.cpp
index 9c8f169..df0a47f 100644
--- a/src/ietf-hardware/Factory.cpp
+++ b/src/ietf-hardware/Factory.cpp
@@ -13,19 +13,28 @@
void createPower(std::shared_ptr<velia::ietf_hardware::IETFHardware> ietfHardware)
{
+ /*
+ * The order of reading hwmon files of the PDU is important.
+ * Reading properties from hwmon can trigger page change in the device which can take more than 20ms.
+ * We have therefore grouped the properties based on their page location to minimize the page changes.
+ * See linux/drivers/hwmon/pmbus/fsp-3y.c
+ */
auto pdu = std::make_shared<velia::ietf_hardware::sysfs::HWMon>("/sys/bus/i2c/devices/2-0025/hwmon");
ietfHardware->registerDataReader(StaticData("ne:pdu", "ne", {{"class", "iana-hardware:power-supply"}}));
+
ietfHardware->registerDataReader(SysfsValue<SensorType::VoltageDC>("ne:pdu:voltage-12V", "ne:pdu", pdu, 1));
- ietfHardware->registerDataReader(SysfsValue<SensorType::VoltageDC>("ne:pdu:voltage-5V", "ne:pdu", pdu, 2));
- ietfHardware->registerDataReader(SysfsValue<SensorType::VoltageDC>("ne:pdu:voltage-3V3", "ne:pdu", pdu, 3));
ietfHardware->registerDataReader(SysfsValue<SensorType::Current>("ne:pdu:current-12V", "ne:pdu", pdu, 1));
- ietfHardware->registerDataReader(SysfsValue<SensorType::Current>("ne:pdu:current-5V", "ne:pdu", pdu, 2));
- ietfHardware->registerDataReader(SysfsValue<SensorType::Current>("ne:pdu:current-3V3", "ne:pdu", pdu, 3));
+ ietfHardware->registerDataReader(SysfsValue<SensorType::Power>("ne:pdu:power-12V", "ne:pdu", pdu, 1));
ietfHardware->registerDataReader(SysfsValue<SensorType::Temperature>("ne:pdu:temperature-1", "ne:pdu", pdu, 1));
ietfHardware->registerDataReader(SysfsValue<SensorType::Temperature>("ne:pdu:temperature-2", "ne:pdu", pdu, 2));
ietfHardware->registerDataReader(SysfsValue<SensorType::Temperature>("ne:pdu:temperature-3", "ne:pdu", pdu, 3));
- ietfHardware->registerDataReader(SysfsValue<SensorType::Power>("ne:pdu:power-12V", "ne:pdu", pdu, 1));
+
+ ietfHardware->registerDataReader(SysfsValue<SensorType::VoltageDC>("ne:pdu:voltage-5V", "ne:pdu", pdu, 2));
+ ietfHardware->registerDataReader(SysfsValue<SensorType::Current>("ne:pdu:current-5V", "ne:pdu", pdu, 2));
ietfHardware->registerDataReader(SysfsValue<SensorType::Power>("ne:pdu:power-5V", "ne:pdu", pdu, 2));
+
+ ietfHardware->registerDataReader(SysfsValue<SensorType::VoltageDC>("ne:pdu:voltage-3V3", "ne:pdu", pdu, 3));
+ ietfHardware->registerDataReader(SysfsValue<SensorType::Current>("ne:pdu:current-3V3", "ne:pdu", pdu, 3));
ietfHardware->registerDataReader(SysfsValue<SensorType::Power>("ne:pdu:power-3V3", "ne:pdu", pdu, 3));
ietfHardware->registerDataReader([psu =