[Skiboot] [PATCH v7 0/9] Initial OpenCAPI 3.0 Support for P9

[Frederic Barrat]

The series looks good to me.
If Andrew is ok with what I tried to explain in a reply to patch 6 (link 
training), then I think it could be merged. There's a useless header 
file included in patch 8 (zaius platform), but hopefully that can be 
resolved without a full respin.

   Fred


Le 28/02/2018 à 07:41, Andrew Donnellan a écrit :
> This series implements OpenCAPI support for P9.
> 
> The series is divided as follows:
> 
>   - Patches 1-3: general refactoring
> 
>     Add various structs and fields we'll need later.
> 
>   - Patch 4: setting up the NPU
> 
>     At present, we're doing NPU configuration separately from the existing
>     NVLink code. At a later point, we intend to rework this to make it
>     possible to support both NVLink and OpenCAPI links on the same NPU.
> 
>   - Patches 5-6: training links and creating OpenCAPI PHBs
> 
>     Unlike NVLink, which presents a single PHB to represent the entire NPU,
>     we present a single PHB per device/link. This is necessary due to
>     limitations in MMIO window allocation. Unfortunately this makes the
>     structures we share with NVLink a little more complex, oh well.
> 
>   - Patch 7: OPAL API calls
> 
>     We define three new API calls for handling the Shared Process Area and
>     setting OpenCAPI TL template capabilities.
> 
>   - Patch 8: platform support
> 
>     See below.
> 
>   - Patch 9: device tree documentation
> 
> Notable limitations:
> 
>   - We only support the zaius platform for now. We'll be adding ZZ after a bit
>     more testing booting ZZs with hostboot rather than BML. Witherspoon
>     support will come at a later point (our testing shows it's mostly
>     working now!)
> 
>   - No mixing of OpenCAPI and NVLink devices on the same NPU. This will come in a
>     later series. The only platform this currently impacts is Witherspoon.
> 
>   - No support for link ganging - there are also no OpenCAPI devices that support
>     link ganging yet, so this will come when we get to it...
> 
>   - Link information is hardcoded per platform, and we don't have any form of
>     presence detection apart from failing to train the link. Eventually, this
>     will be detected via I2C once Hostboot adds the relevant link info to HDAT.
> 
>   - No LPC memory - this will come in a later patch once we've done a bit more
>     testing internally.
> 
>   - No link reset functionality - this will come in a later patch.
> 
> This series has been tested on a Zaius. I've also tested it on a
> GPU-equipped Witherspoon to ensure it doesn't break NVLink.
> 
> Thanks to everyone who's helped us with this, especially to Alistair Popple for
> his advice on NVLink, and to the OpenCAPI hardware teams in Austin and
> Rochester who provided us with a lot of vital assistance.
> 
> Comments welcome!
> 
> ---
> 
> v1->v2:
> 
>    - Address various comments from Fred
> 
> v2->v3:
> 
>    - Address minor comments from Fred
>    - Silence ioda_reset warning in Linux
>    - Fix PHB device tree issues
>    - Mask FIRs to correctly handle Datalink Layer Stall and NoStall signals
>      (Fred)
>    - Add device tree binding information
>    - Reverse order of lane masks, per Alistair's patch at
>      https://patchwork.ozlabs.org/patch/865155/
> 
> v3->v4:
> 
>    - Address comments from Fred and Alistair
> 
> v4->v5:
> 
>    - I accidentally a --- line in patch 5, whoops
> 
> v5->v6:
> 
>    - Reduce link training retries
>    - Fix brick<->ODL mapping on OBUS3 (only relevant to witherspoon)
>    - Skip unnecessary phy training subprocedure that triggers FIR bit
>    - Add some comments
> 
> v6->v7:
> 
>    - Remove the p9-adu code to set PowerBus hotplug mode register.
>      Following clarification with hardware team, we believe this is all
>      correctly set up for us in Hostboot.
>    - Add some debug options (in NVRAM) for link training
>    - Use ibm,link-speed property in device tree to determine PHY speed
>      select field
>    - Disable fast reboot - it almost certainly won't work right now...
> 
> Andrew Donnellan (8):
>    npu2: Split out common helper functions into separate file
>    npu2: Rework NPU data structures for OpenCAPI
>    platform: Add fields for OpenCAPI platform data
>    npu2-opencapi: Configure NPU for OpenCAPI
>    npu2-hw-procedures: Add support for OpenCAPI PHY link training
>    npu2-opencapi: Train OpenCAPI links and setup devices
>    platforms: Add OpenCAPI platform data and device tree nodes
>    doc/device-tree: Add OpenCAPI device tree bindings
> 
> Frederic Barrat (1):
>    npu2-opencapi: Add OpenCAPI OPAL API calls
> 
>   core/init.c                                     |    2 +-
>   core/pci.c                                      |    3 +-
>   core/platform.c                                 |   11 +-
>   doc/device-tree/opencapi.rst                    |   95 +-
>   doc/opal-api/opal-npu2-opencapi-159-160-161.rst |  126 +-
>   hw/Makefile.inc                                 |    4 +-
>   hw/npu2-common.c                                |  114 +-
>   hw/npu2-hw-procedures.c                         |  134 +-
>   hw/npu2-opencapi.c                              | 1708 ++++++++++++++++-
>   hw/npu2.c                                       |  243 +--
>   include/npu2-regs.h                             |  156 +-
>   include/npu2.h                                  |   90 +-
>   include/opal-api.h                              |    6 +-
>   include/pci.h                                   |    1 +-
>   include/platform.h                              |   14 +-
>   include/skiboot.h                               |    1 +-
>   platforms/astbmc/zaius.c                        |   98 +-
>   platforms/ibm-fsp/zz.c                          |   11 +-
>   18 files changed, 2617 insertions(+), 200 deletions(-)
>   create mode 100644 doc/device-tree/opencapi.rst
>   create mode 100644 doc/opal-api/opal-npu2-opencapi-159-160-161.rst
>   create mode 100644 hw/npu2-common.c
>   create mode 100644 hw/npu2-opencapi.c
> 
> base-commit: 49c175d9a026648f5f902db018f1986365a72610
>