Antwort: RE: Migrating from 2.6.11 to 2.6.23 breaks pci-e

Tue Jul 22 08:34:44 EST 2008

Joachim,

No, we are stable now once we fixed the bus numbering issue.

Vince

  _____  

From: joachim.bader at diehl-aerospace.de
[mailto:joachim.bader at diehl-aerospace.de] 
Sent: Monday, July 21, 2008 3:15 AM
To: vasbridge at sanblaze.com
Cc: 'Dave Maruska'; linuxppc-embedded at ozlabs.org; 'Mitul Patel'; 'Geary
Sean-R60898'; 'Stephen Shirron'; 'Hynes, Stephen'
Subject: Antwort: RE: Migrating from 2.6.11 to 2.6.23 breaks pci-e 

Hello Vince, 

we ran into the similar problem using U-boot version 1.1.4 and Linux kernel
2.6.23 on a Freescale MPC8641D platform. 
The kernel did not scan all buses and did not recognize the connected pci-e
bridges and devices. 

Your mail was very helpfull and due to the information you sent we disabled
CONFIG_PCI in U-boot and now the kernel was able to detect all connected
bridges and devices. But during allocation of the pci-e resources the kernel
runs in an address access violation. We will continue to investigate this
behavior. 

Do you have some more information? regarding behavior of newer kernels? 

Thank you 
Joachim Bader 
-- 
Dipl.-Ing. Joachim Bader
Research & Technology
Cockpit and Display Systems

Diehl Aerospace GmbH
An der Sandelmuehle 13
D-60439 Frankfurt
Phone +49-69-5805-1270
Fax       +49-69-5805-1400
e-mail:  joachim.bader at diehl-aerospace.de
http://www.diehl-aerospace.de 

"Vince Asbridge" <vasbridge at sanblaze.com> 
Gesendet von:
linuxppc-embedded-bounces+joachim.bader=diehl-aerospace.de at ozlabs.org 

02.07.2008 23:40 

An
<linuxppc-embedded at ozlabs.org>, "'Stephen Shirron'" <sshirron at sanblaze.com>,
"'Mitul Patel'" <mpatel at sanblaze.com>, "'Dave Maruska'"
<dmaruska at sanblaze.com>, "'Geary Sean-R60898'" <sean.geary at freescale.com>,
"'Hynes, Stephen'" <Stephen.Hynes at lsi.com> 

Kopie

Thema
RE: Migrating from 2.6.11 to 2.6.23 breaks pci-e with LSI 1068 SAS
chip (solved!)

Solved!

Problem summary.

LSI logic devices (1068e, 1064e, fc949e) do not work with uBoot version
1.3.0 and Linux kernel 2.6.23, where they work perfectly fine in kernel
2.6.11.

Symptom is that they do not show up at all to Linux once booted (lspci shows
nothing).

We obtained a pci-e analyzer and found the following:

uBoot V1.3.0 now scans the pci configuration, and when it does it assigns
bus numbers to the pci-e devices it finds.  These numbers that are assigned
are different from those that are assigned by Linux when the system boots.

It is legal to change pci-e bus numbers "on the fly" but doing so requires a
config write cycle to the pci-e device, because the pci-e spec states that
the device must register a new bus number if a config write cycle is issues
with the new bus number.

When we boot the Freescale machine with uBoot 1.3.0 and kernel 2.6.23, the
bus number under uBoot gets assigned to 1, and the bus number under Linux
gets assigned to 4.  Between the change of bus numbers we do not see a
config write go to the LSI device with the new bus number, and therefore it
continues to respond on bus 1 and ignore config reads to bus 4.

We temporarily fixed the issue by defining the CONFIG_PCI_NOSCAN flag under
uBoot, which causes the LSI device not to get assigned a bus number at
uBoot, and therefore when Linux begins doing config reads at bus 4 to the
LSI device registers the bus number and responds correctly.

We will do a bit more investigation to see if this has already been fixed in
a newer kernel, but for now not scanning at uBoot fixes the issue.

It's a bit of a mystery why the LSI devices behave differently from any
other pci-e device we put in the system, but they seem to be adhering to the
letter of the specification.

Vince Asbridge

------- original post -------

All,

I'm new to this mailing list, but have not had any luck finding information
on this issue.

Please be kind if I break the forum rules on my first post.

We recently tried to upgrade our Freescale CDS 8548 look-alike module (code
name ATCA1000) from the 2.6.11 based BSP to the 2.6.23 based BSP.

The upgrade went fairly smoothly, until we tried using SOME pci-e devices
(some work fine, some don't show up to lspci).

LSI pci-e controllers no longer show up at all!

We see the ixgbe (intel 10G), SiliconImage SATA controller but do not see
LSI devices (Specifically 1068 SAS, FC949-E fibrechannel).

We're guessing it's a resource issue behind the bridge, because the LSI
devices try to allocate 1 - 3M behind the bridge, but we can't find the bug,
or where we would debug such an issue.

The devices seem to "train" correctly, because we have an LED on the pci-e
switch (PLX 8 port pci-e switch), and it's ON indicating pci-e link between
the bridge and the 1068 device).

We're totally at a loss as to why this always worked on the 2.6.11 kernel
but doesn't work on 2.6.23.

Using lspci, the LSI adapters do not show up in the list at all, as though
they are not plugged into the system.

Is there something that needs to be done with respect to PCI-E devices that
is new in the 2.6.23 based BSP that did not need to be done in the 2.6.11
based kit?  For example, are pci resources allocated by a different piece of
code, that may have some issue allocating resources for the LSI adapters?

I currently do not have access to a pci-e analyzer.

Thanks for any help,
Vince Asbridge

_______________________________________________
Linuxppc-embedded mailing list
Linuxppc-embedded at ozlabs.org
https://ozlabs.org/mailman/listinfo/linuxppc-embedded 

____________________________________________________________________________
___________________________________________
Der Inhalt dieser E-Mail ist fC
Informieren Sie uns bitte, wenn Sie diese E-Mail fC$lschlicherweise erhalten
haben (Fax: +49-69-5805-1399). Bitte lC6schen Sie in diesem Fall die
Nachricht. Jede Form der weiteren Benutzung ist untersagt.

The content of this e-mail is not legally binding upon the sender.
If this e-mail was transmitted to you by error, then please inform us
accordingly (Fax: +49-69-5805-1399). In such case you are requested to erase
the message. Any use of such e-mail message is strictly prohibited.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://ozlabs.org/pipermail/linuxppc-embedded/attachments/20080721/b96f53c6/attachment.htm>