[Skiboot] DDR4 Calibration and address inversion

[Edgar Cordero]

I have a few things you could look at at. Can you reach out to me? One
thing I would like to know is the manufacturer of the buffers on your DIMMs
as well as when you pulled the the firmware procedures. We had an issue we
know about and I think you might have gotten caught in it.




Edgar R. Cordero
Senior Technical Staff Member
IBM Systems
ecordero at us.ibm.com
(512) 286-5788  (t/l 363-5788)




From:	Alexandre Ghiti <aghiti at upmem.com>
To:	"Oliver O'Halloran" <oohall at gmail.com>
Cc:	skiboot list <skiboot at lists.ozlabs.org>, ecordero at us.ibm.com
Date:	06/28/2019 12:25 AM
Subject:	[EXTERNAL] Re: [Skiboot] DDR4 Calibration and address inversion




Le jeu. 27 juin 2019 à 20:05, Oliver O'Halloran <oohall at gmail.com> a
écrit :
  On Thu, Jun 27, 2019 at 10:00 PM Alexandre Ghiti <aghiti at upmem.com>
  wrote:
  >
  > Le jeu. 27 juin 2019 à 12:38, Oliver O'Halloran <oohall at gmail.com> a
  écrit :
  >>
  >> On Thu, Jun 27, 2019 at 7:54 PM Alexandre Ghiti <aghiti at upmem.com>
  wrote:
  >> >
  >> > Hi everyone,
  >> >
  >> > We are currently testing our RDIMM in a Talos server which runs
  custom hostboot/skiboot from RaptorCS team.
  >> > Our RDIMM (single-rank) is 'rejected' during calibration as half the
  DRAMs fail the DQS alignment step. We know, for some internal reasons,
  that if this step tries to access the DRAMs with BG1=1, it will fail,
  which will happen if address inversion is enabled (see RCD jedec
  JESD82-31 section 2.9): so half the failing DRAMs would represent the
  side-b.
  >> >
  >> > I did not succeed to disable address inversion, certainly because
  the piece of HW that does the actual calibration supposes the address
  inversion is activated and then invert the corresponding bits to prevent
  RCD inversion.
  >> >
  >> > My questions are:
  >> > - Is the CCS responsible for executing the calibrations ?
  >> > - Is there anyway to modify its behaviour ? (I don't think the
  calibration steps are purely HW...)
  >> >
  >> > Thanks for your answers, and if this is not the place to discuss
  such things, please do not hesitate to say so.
  >>
  >> We don't mind hearing about this sort of thing, but it's very outside
  >> our area of expertise. Memory training is handled by the hardware
  >> procedures in hostboot, but it sounds like you've already tried to
  >> fiddle with the stuff in src/import/chips/p9/procedures/hwp/memory
  >> without luck? I'll see if I can find someone who might have more of a
  >> clue.
  >
  >
  >
  > Yes, I know that the memory training happens in the function
  p9_mss_draminit_training.
  >
  > From my understanding, it is done by setting the register
  DDRPHY_PC_INIT_CAL_CONFIG0_P0 (Power9
  > Processor Registers Specification, Volume 3) with the calibration steps
  we want to do.
  >
  > Then, those steps are indeed executed by setting the field
  CCS_INST_ARR1_00_DDR_CALIBRATION_ENABLE
  > of MC01.MCBIST.CCS.CCS_INST_ARR1_00 register. So to me it seems those
  steps are entirely HW.
  >
  > Note that we do not have a Centaur platform, our processor is the
  Nimbus version.
  >
  > Thanks again for your quick answer,
  >
  > Alex

  I asked around and Edgar (+cc) from the memory team said he might be
  able to help. He does have some questions for you though.

Great, whatever he needs :)
Thanks Oliver,

Alex


  Oliver

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ozlabs.org/pipermail/skiboot/attachments/20190628/6c525266/attachment.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: graycol.gif
Type: image/gif
Size: 105 bytes
Desc: not available
URL: <http://lists.ozlabs.org/pipermail/skiboot/attachments/20190628/6c525266/attachment.gif>