Fan PWM monitoring and control

Matthew Barth msbarth at linux.ibm.com
Wed Sep 25 00:55:37 AEST 2019 


On 9/23/19 11:25 PM, rgrs wrote:
> Thanks Matt :) It works now.
> 
> I have one small issue on configuration of ADT7462.
> After AC power cycle, it defaults to "automatic" fan control.
> We need to change this "manual" so that phosphor-fan control register writes are not ignored by ADT7462.
> 
> I came across https://github.com/openbmc/docs/blob/master/sensor-architecture.md
> 
> 
>> ENABLE
>>    Will write a value to a pwmN_enable file on startup if present.
>>
>>   ENABLE_fan1 = 2 #Write a 2 to pwm1_enable
> 
> But this doesn't seem to work. We need it to be initialized to '1'.
> (It always defaults to 2 after AC cycle)
This sounds like the correct hwmon config to use to set it to a '1' upon 
hwmon starting after an AC cycle. First ensure your hwmon config has 
these `ENABLE_fan[#]` entries set to a corresponding `pwm[#]_enable` 
file in sysfs.

Is the `pwm[#]_enable` sysfs file set to a '1' at any point after the AC 
cycle (i.e. after a poweron?)? If you think the configs are set 
correctly, is the hwmon service for the pwm devices running after the AC 
cycle while at BMC standby?
> 
> Is host-power-on script a good place to "echo 1 > pwmN_enable"?
> (planning to add a script to initialize before host power on)
> 
> Thanks,
> Raj
> 
> 
> ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
> On Friday, September 20, 2019 9:49 PM, Matthew Barth <msbarth at linux.ibm.com> wrote:
> 
>> On 9/20/19 4:55 AM, rgrs wrote:
>>
>>> Hi Matt,
>>> I'm able to control fan speed via busctl.
>>> root at saber:/sys/class/hwmon/hwmon0# busctl set-property xyz.openbmc_project.Hwmon-2429843755.Hwmon1 /xyz/openbmc_project/sensors/fan_tach/FAN0 xyz.openbmc_project.Control.FanPwm Target t '250'
>>> root at saber:/sys/class/hwmon/hwmon0# cat fan*_input
>>> 12498
>>> 0
>>
>> Great! that's progress.
>>
>>> I need a way to verify if the sensor associations are working properly that fans will ramp up when associated temperature sensor's value increases.
>>
>> Be sure that your fans.yaml contains the correct sensor name ("FAN0",
>> etc..) for each fan that contains the Target property on the FanPwm
>> interface.
>>
>> You can use busctl to monitor the target pwm sensor changes based on
>> what you configured within events.yaml for speed increases. This would
>> show that fan control is requesting updated targets to hwmon. Then hwmon
>> would write these values out to the pwm sysfs file for your fan
>> controller device.
>>
>>> Thanks,
>>> Raj
>>> ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
>>> On Friday, September 20, 2019 2:57 PM, rgrs rgrs at protonmail.com wrote:
>>>
>>>> Hi Matt,
>>>>
>>>>>> Not sure I follow what you mean by "no pwm writes", is there an
>>>>
>>>> application writing to the fan sensor's pwm interface target property?
>>>> I thought phoshor-fan control writes to pwm to increase fan speed based on temperature sensors mentioned in events.yaml
>>>> (recipes-phosphor/fans/phosphor-fan-control-events-config/events.yaml)
>>>> Is it not the case?
>>>> Thanks,
>>>> Raj
>>>> ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
>>>> On Thursday, September 19, 2019 8:26 PM, Matthew Barth msbarth at linux.ibm.com wrote:
>>>>
>>>>> On 9/19/19 5:16 AM, rgrs wrote:
>>>>>
>>>>>> Hi Patrick, Lei,
>>>>>> Thank you, I made some changes according your comments
>>>>>>
>>>>>> -   added Mutability::Read to fan yaml.
>>>>>> -   I didn't see ipmi-sensor-inventory.yaml of romulus having fan sensors, so removed them from IPMI.
>>>>>>
>>>>>> Now,"busctl monitor" displays Fan tach reads, no pwm writes.
>>>>>
>>>>> Not sure I follow what you mean by "no pwm writes", is there an
>>>>> application writing to the fan sensor's pwm interface target property?
>>>>>
>>>>>> Is there any way to manually set PWM value and test this?
>>>>>> How do I verify PWM values are being written to ADT7462?
>>>>>
>>>>> If no other application are writing to the PWM sysfs entries, you should
>>>>> be able to `echo` a value to each fan's PWM sysfs file to alter the fan
>>>>> speeds.
>>>>>
>>>>>> Another observation is Inventory.Item seems to have invalid(stale) entries.
>>>>>
>>>>> I believe inventory is persisted by default so that could explain why
>>>>> these entries are still showing up. These are persisted under
>>>>> `/var/lib/phosphor-inventory-manager/..`.
>>>>>
>>>>>> IPMI sensor names appear in inventory even after removing from IPMI sensor yaml files (exported JSON in attachment).
>>>>>> Logs:
>>>>>>
>>>>>> busctl monitor | grep -i fan
>>>>>>
>>>>>> =============================
>>>>>>
>>>>>> Monitoring bus message stream.
>>>>>> Sender=:1.45 Path=/xyz/openbmc_project/sensors/fan_tach/FAN0 Interface=org.freedesktop.DBus.Properties Member=PropertiesChanged
>>>>>> Sender=:1.45 Path=/xyz/openbmc_project/sensors/fan_tach/FAN1 Interface=org.freedesktop.DBus.Properties Member=PropertiesChanged
>>>>>> Sender=:1.45 Path=/xyz/openbmc_project/sensors/fan_tach/FAN2 Interface=org.freedesktop.DBus.Properties Member=PropertiesChanged
>>>>>> [...]
>>>>>> Sender=:1.45 Path=/xyz/openbmc_project/sensors/fan_tach/FAN1 Interface=org.freedesktop.DBus.Properties Member=PropertiesChanged
>>>>>> Sender=:1.45 Path=/xyz/openbmc_project/sensors/fan_tach/FAN3 Interface=org.freedesktop.DBus.Properties Member=PropertiesChanged
>>>>>> ^C
>>>>>> busctl get-property xyz.openbmc_project.Hwmon-2429843755.Hwmon1 /xyz/openbmc_project/sensors/fan_tach/FAN0 xyz.openbmc_project.Control.FanPwm Target
>>>>>>
>>>>>> =============================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================
>>>>>>
>>>>>> t 0
>>>>>> busctl set-property xyz.openbmc_project.Hwmon-2429843755.Hwmon1 /xyz/openbmc_project/sensors/fan_tach/FAN0 xyz.openbmc_project.Control.FanPwm Target 8000
>>>>>>
>>>>>> ==============================================================================================================================================================
>>>>>
>>>>> This is not a valid call to `busctl set-property...`, its missing the
>>>>> property's signature type (refer to `busctl help`). Also, a target PWM
>>>>> of 8000 is invalid, PWM is 0 - 255.
>>>>>
>>>>>> Failed to create bus message: Invalid argument
>>>>>> busctl introspect --no-pager xyz.openbmc_project.Hwmon-2429843755.Hwmon1 /xyz/openbmc_project/sensors/fan_tach/FAN1
>>>>>>
>>>>>> ===================================================================================================================================================================
>>>>>>
>>>>>> NAME TYPE SIGNATURE RESULT/VALUE FLAGS
>>>>>> org.freedesktop.DBus.Introspectable interface - - -
>>>>>> .Introspect method - s -
>>>>>> org.freedesktop.DBus.Peer interface - - -
>>>>>> .GetMachineId method - s -
>>>>>> .Ping method - - -
>>>>>> org.freedesktop.DBus.Properties interface - - -
>>>>>> .Get method ss v -
>>>>>> .GetAll method s a{sv} -
>>>>>> .Set method ssv - -
>>>>>> .PropertiesChanged signal sa{sv}as - -
>>>>>> xyz.openbmc_project.Control.FanPwm interface - - -
>>>>>> .Target property t 0 emits-change writable
>>>>>> xyz.openbmc_project.Sensor.Value interface - - -
>>>>>> .MaxValue property x 0 emits-change writable
>>>>>> .MinValue property x 0 emits-change writable
>>>>>> .Scale property x 0 emits-change writable
>>>>>> .Unit property s "xyz.openbmc_project.Sensor.Value.Unit.… emits-change writable
>>>>>> .Value property x 2983 emits-change writable
>>>>>> xyz.openbmc_project.State.Decorator.OperationalStatus interface - - -
>>>>>> .Functional property b true emits-change writable
>>>>>> Thanks,
>>>>>> Raj
>>>>>> ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
>>>>>> On Wednesday, September 18, 2019 9:35 PM, Patrick Venture venture at google.com wrote:
>>>>>>
>>>>>>> On Wed, Sep 18, 2019 at 2:30 AM rgrs rgrs at protonmail.com wrote:
>>>>>>>
>>>>>>>> Hi Patrick, Matt,
>>>>>>>> I tried instructions from this wiki: https://developer.ibm.com/linuxonpower/2019/01/07/how-to-port-openbmc/
>>>>>>>> There is no GPIO to detect presence of fans, so tach readings is used instead.
>>>>>>>> Here's what I did while trying to enable Fan control:
>>>>>>>>
>>>>>>>> 1.  Added Fan tach sensor in hwmon (adt7462.conf) is in the form LABEL=FANx_RPM
>>>>>>>> 2.  "ipmi-sensor-inventory.yaml" has below config:
>>>>>>>>
>>>>>>>> 60: &fan_tach
>>>>>>>> sensorType: 0x04
>>>>>>>> path: /xyz/openbmc_project/sensors/fan_tach/FAN0_RPM
>>>>>>>> sensorReadingType: 0x01
>>>>>>>> multiplierM: 100
>>>>>>>> offsetB: 0
>>>>>>>> bExp: 0
>>>>>>>> unit: xyz.openbmc_project.Sensor.Value.Unit.RPMS
>>>>>>>> scale: 0
>>>>>>>> rExp: 0
>>>>>>>> serviceInterface: org.freedesktop.DBus.Properties
>>>>>>>> readingType: readingData
>>>>>>>> interfaces:
>>>>>>>> xyz.openbmc_project.Sensor.Value:
>>>>>>>> Value:
>>>>>>>> Offsets:
>>>>>>>> 0xFF:
>>>>>>>> type: int64_t
>>>>>>>> and FAN PWM as
>>>>>>>> 89: &fanpwm
>>>>>>>> sensorType: 0x04
>>>>>>>> path: /xyz/openbmc_project/sensors/fan_tach/FAN0_RPM
>>>>>>>> sensorReadingType: 0x01
>>>>>>>> multiplierM: 1
>>>>>>>> offsetB: 0
>>>>>>>> bExp: 0
>>>>>>>> mutability: Mutability::Write
>>>>>>>
>>>>>>> You may need to add ::Read here too, I'm not sure anymore, but you can
>>>>>>> check the script that builds this:
>>>>>>> https://github.com/openbmc/phosphor-host-ipmid/blob/master/scripts/writesensor.mako.cpp#L69
>>>>>>> It'll default to read, but if you've only set write, I don't know.
>>>>>>>
>>>>>>>>        serviceInterface: org.freedesktop.DBus.Properties
>>>>>>>>        readingType: readingData
>>>>>>>>        interfaces:
>>>>>>>>            xyz.openbmc_project.Control.FanPwm:
>>>>>>>>                Target:
>>>>>>>>                    Offsets:
>>>>>>>>                        0xFF:
>>>>>>>>                            type: uint64_t
>>>>>>>>
>>>>>>>>
>>>>>>>> 3.  I copied meta-romulus/recipes-phosphor/fans to my platform and changed FAN names from "fanx" to "FANx_RPM" in yaml files.
>>>>>>>>
>>>>>>>> This is what get in busctl tree related to Fans:
>>>>>>>> (I expected to see FanPwm entries in busctl and I'm not sure why there are multiple entries in inventory).
>>>>>>>> Service xyz.openbmc_project.Hwmon-2429843755.Hwmon1:
>>>>>>>> `-/xyz`-/xyz/openbmc_project
>>>>>>>> `-/xyz/openbmc_project/sensors |-/xyz/openbmc_project/sensors/fan_tach | |-/xyz/openbmc_project/sensors/fan_tach/FAN0_RPM | |-/xyz/openbmc_project/sensors/fan_tach/FAN1_RPM | |-/xyz/openbmc_project/sensors/fan_tach/FAN2_RPM |`-/xyz/openbmc_project/sensors/fan_tach/FAN3_RPM
>>>>>>>> `-/xyz/openbmc_project/sensors/temperature`-/xyz/openbmc_project/sensors/temperature/BACKPLANE_TEMP
>>>>>>>
>>>>>>> You don't get a separate entry for the PWM from phosphor-hwmon.
>>>>>>> It's a separate interface on the same service and path.
>>>>>>> I recommend you introspect on the path and see what interfaces are
>>>>>>> available (as well as adding "| read" above):
>>>>>>> busctl introspect --no-pager
>>>>>>> xyz.openbmc_project.Hwmon-2429843755.Hwmon1
>>>>>>> /xyz/openbmc_project/sensors/fan_tach/FAN0_RPM
>>>>>>>
>>>>>>>> Service xyz.openbmc_project.Inventory.Manager:
>>>>>>>> `-/xyz`-/xyz/openbmc_project
>>>>>>>> `-/xyz/openbmc_project/inventory`-/xyz/openbmc_project/inventory/system
>>>>>>>> `-/xyz/openbmc_project/inventory/system/chassis`-/xyz/openbmc_project/inventory/system/chassis/motherboard
>>>>>>>> |-/xyz/openbmc_project/inventory/system/chassis/motherboard/FAN0
>>>>>>>> | `-/xyz/openbmc_project/inventory/system/chassis/motherboard/FAN0/FAN0 |-/xyz/openbmc_project/inventory/system/chassis/motherboard/FAN0_RPM |`-/xyz/openbmc_project/inventory/system/chassis/motherboard/FAN0_RPM/FAN0_RPM
>>>>>>>> |-/xyz/openbmc_project/inventory/system/chassis/motherboard/FAN1
>>>>>>>> | `-/xyz/openbmc_project/inventory/system/chassis/motherboard/FAN1/FAN1 |-/xyz/openbmc_project/inventory/system/chassis/motherboard/FAN1_RPM |`-/xyz/openbmc_project/inventory/system/chassis/motherboard/FAN1_RPM/FAN1_RPM
>>>>>>>> |-/xyz/openbmc_project/inventory/system/chassis/motherboard/FAN2
>>>>>>>> | `-/xyz/openbmc_project/inventory/system/chassis/motherboard/FAN2/FAN2 |-/xyz/openbmc_project/inventory/system/chassis/motherboard/FAN2_RPM |`-/xyz/openbmc_project/inventory/system/chassis/motherboard/FAN2_RPM/FAN2_RPM
>>>>>>>> |-/xyz/openbmc_project/inventory/system/chassis/motherboard/FAN3
>>>>>>>> | `-/xyz/openbmc_project/inventory/system/chassis/motherboard/FAN3/FAN3 |-/xyz/openbmc_project/inventory/system/chassis/motherboard/FAN3_RPM |`-/xyz/openbmc_project/inventory/system/chassis/motherboard/FAN3_RPM/FAN3_RPM
>>>>>>>> |-/xyz/openbmc_project/inventory/system/chassis/motherboard/fan0
>>>>>>>> |-/xyz/openbmc_project/inventory/system/chassis/motherboard/fan1
>>>>>>>> |-/xyz/openbmc_project/inventory/system/chassis/motherboard/fan2
>>>>>>>> `-/xyz/openbmc_project/inventory/system/chassis/motherboard/fan3
>>>>>>>> Thanks,
>>>>>>>> Raj
>>>>>>>> ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
>>>>>>>> On Tuesday, September 17, 2019 9:18 PM, Patrick Venture venture at google.com wrote:
>>>>>>>>
>>>>>>>>> On Tue, Sep 17, 2019 at 8:32 AM Matthew Barth msbarth at linux.ibm.com wrote:
>>>>>>>>>
>>>>>>>>>> On 9/17/19 5:40 AM, rgrs wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi,
>>>>>>>>>>> I need some help with phosphor hwmon related to fan sensors.
>>>>>>>>>>> I'm trying to control Fans in a server motherboard using ADT7462
>>>>>>>>>>> controller. Hw mon porting is done and the sysfs interfaces are working
>>>>>>>>>>> as expected.
>>>>>>>>>>> While I am able to read RPM, i don't find pwm/target to control the fans
>>>>>>>>>>> exposed as dbus objects.
>>>>>>>>>>> Do you have the hwmon config set similar to this to denote the fan
>>>>>>>>>>> targets are of PWM type?
>>>>>>>>>>
>>>>>>>>>> https://github.com/openbmc/openbmc/blob/master/meta-ibm/meta-romulus/recipes-phosphor/sensors/phosphor-hwmon/obmc/hwmon/ahb/apb/pwm-tacho-controller%401e786000.conf
>>>>>>>>>>
>>>>>>>>>>> busctl --no-page tree xyz.openbmc_project.FanSensor
>>>>>>>>>>>
>>>>>>>>>>> ====================================================
>>>>>>>>>>>
>>>>>>>>>>> Failed to introspect object / of service xyz.openbmc_project.FanSensor:
>>>>>>>>>>> The name is not activatable
>>>>>>>>>>> No objects discovered.
>>>>>>>>>>> That does not look like a hwmon service name. If you are using
>>>>>>>>>>> phosphor-objmgr, you can use `# mapper get-service /xyz/openbmc_project/sensors/fan_tach` otherwise you'll need to find the
>>>>>>>>>>> hwmon service name associated with your fan controller device.
>>>>>>>>>
>>>>>>>>> running`busctl tree --no-pager` will just list everything, and you
>>>>>>>>> should see your fans in there. You can use the mapper too. I was
>>>>>>>>> curious, are you using entity-manager/dbus-sensors? When you mention
>>>>>>>>> FanSensor as the service, that's what it sounds like.
>>>>>>>>>
>>>>>>>>>>> I came across this post, I think it is similar to my issue.
>>>>>>>>>>> (https://gerrit.openbmc-project.xyz/c/openbmc/phosphor-hwmon/+/8353)
>>>>>>>>>>> Please someone point to latest documentation or reference Fan porting
>>>>>>>>>>> from openBMC repo?
>>>>>>>>>>> What are the exact fields in YAML that are used create FanPWM objects.
>>>>>>>>>>> The Romulus machine should be a good reference for PWM based fan control
>>>>>>>>>>> using phosphor-hwmon. It uses the
>>>>>>>>>>> phosphor-fan-presence/[control|monitor] apps for controlling and
>>>>>>>>>>> monitoring fans
>>>>>>>>>>
>>>>>>>>>> https://github.com/openbmc/openbmc/tree/master/meta-ibm/meta-romulus
>>>>>>>>>>
>>>>>>>>>>> (I'm using Feb release in my sandbox,
>>>>>>>>>>> https://github.com/openbmc/docs/blob/master/release/release-notes.md#26-feb-4-2019)
>>>>>>>>>>> Thanks,
>>>>>>>>>>> Raj
>>>>>>>>>>
>>>>>>>>>> Matt
> 
>

More information about the openbmc mailing list