[Skiboot] DDR4 Calibration and address inversion

Tue Jul 2 01:09:23 AEST 2019

Great. I am available from 10am Central Time US ( -5 UTC ) to 3pm on
Tuesday July 2. I can set up a webex or you can use what works for you .

Edgar R. Cordero
Senior Technical Staff Member
IBM Systems
ecordero at us.ibm.com
(512) 286-5788  (t/l 363-5788)

From:	Alexandre Ghiti <aghiti at upmem.com>
To:	Edgar Cordero <ecordero at us.ibm.com>
Cc:	"Oliver O'Halloran" <oohall at gmail.com>, skiboot list
            <skiboot at lists.ozlabs.org>
Date:	06/28/2019 10:32 AM
Subject:	[EXTERNAL] Re: Re: [Skiboot] DDR4 Calibration and address
            inversion

Le ven. 28 juin 2019 à 16:58, Edgar Cordero <ecordero at us.ibm.com> a écrit :
  I have a few things you could look at at. Can you reach out to me? One
  thing I would like to know is the manufacturer of the buffers on your
  DIMMs as well as when you pulled the the firmware procedures. We had an
  issue we know about and I think you might have gotten caught in it.

Hi Edgar,

Our hostboot is a fork from raptorcs team: their remote is
https://scm.raptorcs.com/scm/git/talos-hostboot and the
commit e36ec63e58344f854317e79d060f2ab9108103c2, which is kind of outdated
(last commit is April 2018).
Our RDIMM RCD is an IDT 4RCD0124KC0ATG.

We can arrange an hangout call or whatever whenever you are available.

Thanks for your support,

Alex

  Edgar R. Cordero
  Senior Technical Staff Member
  IBM Systems
  ecordero at us.ibm.com
  (512) 286-5788 (t/l 363-5788)

  Inactive hide details for Alexandre Ghiti ---06/28/2019 12:25:33 AM---Le
  jeu. 27 juin 2019 à 20:05, Oliver O'Halloran <oohall at gAlexandre Ghiti
  ---06/28/2019 12:25:33 AM---Le jeu. 27 juin 2019 à 20:05, Oliver
  O'Halloran <oohall at gmail.com> a écrit : > On Thu, Jun 27, 2019

  From: Alexandre Ghiti <aghiti at upmem.com>
  To: "Oliver O'Halloran" <oohall at gmail.com>
  Cc: skiboot list <skiboot at lists.ozlabs.org>, ecordero at us.ibm.com
  Date: 06/28/2019 12:25 AM
  Subject: [EXTERNAL] Re: [Skiboot] DDR4 Calibration and address inversion

  Le jeu. 27 juin 2019 à 20:05, Oliver O'Halloran <oohall at gmail.com> a
  écrit :
        On Thu, Jun 27, 2019 at 10:00 PM Alexandre Ghiti <aghiti at upmem.com>
        wrote:
        >
        > Le jeu. 27 juin 2019 à 12:38, Oliver O'Halloran <oohall at gmail.com
        > a écrit :
        >>
        >> On Thu, Jun 27, 2019 at 7:54 PM Alexandre Ghiti <
        aghiti at upmem.com> wrote:
        >> >
        >> > Hi everyone,
        >> >
        >> > We are currently testing our RDIMM in a Talos server which
        runs custom hostboot/skiboot from RaptorCS team.
        >> > Our RDIMM (single-rank) is 'rejected' during calibration as
        half the DRAMs fail the DQS alignment step. We know, for some
        internal reasons, that if this step tries to access the DRAMs with
        BG1=1, it will fail, which will happen if address inversion is
        enabled (see RCD jedec JESD82-31 section 2.9): so half the failing
        DRAMs would represent the side-b.
        >> >
        >> > I did not succeed to disable address inversion, certainly
        because the piece of HW that does the actual calibration supposes
        the address inversion is activated and then invert the
        corresponding bits to prevent RCD inversion.
        >> >
        >> > My questions are:
        >> > - Is the CCS responsible for executing the calibrations ?
        >> > - Is there anyway to modify its behaviour ? (I don't think the
        calibration steps are purely HW...)
        >> >
        >> > Thanks for your answers, and if this is not the place to
        discuss such things, please do not hesitate to say so.
        >>
        >> We don't mind hearing about this sort of thing, but it's very
        outside
        >> our area of expertise. Memory training is handled by the
        hardware
        >> procedures in hostboot, but it sounds like you've already tried
        to
        >> fiddle with the stuff in
        src/import/chips/p9/procedures/hwp/memory
        >> without luck? I'll see if I can find someone who might have more
        of a
        >> clue.
        >
        >
        >
        > Yes, I know that the memory training happens in the function
        p9_mss_draminit_training.
        >
        > From my understanding, it is done by setting the register
        DDRPHY_PC_INIT_CAL_CONFIG0_P0 (Power9
        > Processor Registers Specification, Volume 3) with the calibration
        steps we want to do.
        >
        > Then, those steps are indeed executed by setting the field
        CCS_INST_ARR1_00_DDR_CALIBRATION_ENABLE
        > of MC01.MCBIST.CCS.CCS_INST_ARR1_00 register. So to me it seems
        those steps are entirely HW.
        >
        > Note that we do not have a Centaur platform, our processor is the
        Nimbus version.
        >
        > Thanks again for your quick answer,
        >
        > Alex

        I asked around and Edgar (+cc) from the memory team said he might
        be
        able to help. He does have some questions for you though.

  Great, whatever he needs :)
  Thanks Oliver,

  Alex

        Oliver

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ozlabs.org/pipermail/skiboot/attachments/20190701/b8ddec3e/attachment-0001.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: graycol.gif
Type: image/gif
Size: 105 bytes
Desc: not available
URL: <http://lists.ozlabs.org/pipermail/skiboot/attachments/20190701/b8ddec3e/attachment-0001.gif>