BAR resizing broken in 6.18 (PPC only?)
Ilpo Järvinen
ilpo.jarvinen at linux.intel.com
Wed Oct 22 03:20:44 AEDT 2025
On Sun, 19 Oct 2025, Simon Richter wrote:
> On 10/17/25 00:45, Ilpo Järvinen wrote:
>
> > Is this stock 6.18-rc1?
>
> This is drm-tip, plus a few extras that actually allow me to load the GPU
> driver, but no PCIe changes.
Okay, it explains the BAR0 release then.
> > Was this resize attempted automatically by the xe driver during boot, not
> > triggered manually through sysfs, right? (Inferring that's likely the case
> > given the proximity of the "enabling device" message.)
>
> Yes, that's the xe driver doing that.
>
> > I need to know PCI resource allocations of the parents too. Preferrably
> > dmesg and /proc/iomem from both working and broken cases so it's easier
> > to find the differences.
>
> I've attached those.
>
> - dmesg.good/iomem.good: taken with 6.17 (drm-tip from last week)
> - dmesg.bad/iomem.bad: taken with 6.18rc1 (drm-tip after update)
> - dmesg.meh/iomem.meh: with the rebar patch 1/2 from Lucas. 2/2 is already
> merged. The GPU works at least, but has a 256MB aperture
> - dmesg.rev/iomem.rev: with 85796d20a690 and 3ab10f83e277 reverted
Could you please test if the patch below helps.
> > I'd very much want to do this too but I've higher priority items on my
> > list, one of which attempts to resolve these resizable BARs using other
> > way, that is, size the bridge windows considering the maximum size of the
> > BAR right from the start. That would remove the need to try to enlarge
> > BARs afterwards as there won't be more room for them anyway.
>
> The bridge windows are set up by firmware, but they should be large enough on
> POWER, given that each slot gets its own domain, and controllers are allocated
> 256G each.
>
> I also dimly remember these are be configured so that the first 2G of address
> space are on the PCIe bus, and the first 2G of memory are addressed at a
> higher address and only accessible to 64 bit capable cards, so 32 bit cards
> can still access 2 GB of PCIe memory for P2PDMA and 2 GB of RAM (so the root
> complex performs address translation).
>
> The result of this seems to be that smaller BARs end up in the 2 GB window, so
> only the VRAM aperture is inside the larger 64-bit-only window, so the window
> containing the bridge's BAR0 does not need to be changed.
There's indeed something messy and odd going on here with the resource and
window mappings, in the bad case there's also this line which doesn't make
much sense:
+pci 0030:01:00.0: bridge window [mem 0x6200000000000-0x6203fbff0ffff 64bit pref]: can't claim; address conflict with 0030:01:00.0 [mem 0x6200020000000-0x62000207fffff 64bit pref]
...but that conflicting resource was not assigned in between releasing
this bridge window and trying to claim it back so how did that
conflicting resource get there is totally mysterious to me. It doesn't
seem related directly to the the resize no longer working though.
> > > Also, why do we change the BAR assignment while mem decoding is active?
>
> > pci_resize_resource() does check for
> > pci_resize_is_memory_decoding_enabled(), do you think that is not enough?
>
> It's a bit weird that there is a log message that says "enabling device", then
> the BARs are reconfigured. I'd want the decoding logic to be inactive while
> addresses are assigned.
So no real issue here and only logging is not the way you'd want it?
--
From: =?UTF-8?q?Ilpo=20J=C3=A4rvinen?= <ilpo.jarvinen at linux.intel.com>
Date: Tue, 21 Oct 2025 19:13:31 +0300
Subject: [PATCH 1/1] PCI: Remove old_size limit from bridge window sizing
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
calculate_memsize() applies lower bound to the resource size before
aligning the resource size making it impossible to shrink bridge window
resources. I've not found any justification for this lower bound and
nothing indicated it was to work around some HW issue.
Prior to the commit 3baeae36039a ("PCI: Use pci_release_resource()
instead of release_resource()"), releasing a bridge window during BAR
resize resulted in clearing start and end address of the resource.
Clearing addresses destroys the resource size as a side-effect,
therefore nullifying the effect of the old size lower bound.
After the commit 3baeae36039a ("PCI: Use pci_release_resource() instead
of release_resource()"), BAR resize uses the aligned old size, which
results in exceeding what fits into the parent window in some cases:
xe 0030:03:00.0: [drm] Attempting to resize bar from 256MiB -> 16384MiB
xe 0030:03:00.0: BAR 0 [mem 0x620c000000000-0x620c000ffffff 64bit]: releasing
xe 0030:03:00.0: BAR 2 [mem 0x6200000000000-0x620000fffffff 64bit pref]: releasing
pci 0030:02:01.0: bridge window [mem 0x6200000000000-0x620001fffffff 64bit pref]: releasing
pci 0030:01:00.0: bridge window [mem 0x6200000000000-0x6203fbff0ffff 64bit pref]: releasing
pci 0030:00:00.0: bridge window [mem 0x6200000000000-0x6203fbff0ffff 64bit pref]: was not released (still contains assigned resources)
pci 0030:00:00.0: Assigned bridge window [mem 0x6200000000000-0x6203fbff0ffff 64bit pref] to [bus 01-04] free space at [mem 0x6200400000000-0x62007ffffffff 64bit pref]
pci 0030:00:00.0: Assigned bridge window [mem 0x6200000000000-0x6203fbff0ffff 64bit pref] to [bus 01-04] cannot fit 0x4000000000 required for 0030:01:00.0 bridging to [bus 02-04]
The old size of 0x6200000000000-0x6203fbff0ffff resource was used as
the lower bound which results in 0x4000000000 size request due to
alignment. That exceed what can fit into the parent window.
Since the lower bound never even was enforced fully because the
resource addresses were cleared when the bridge window is released,
remove the old_size lower bound entirely and trust the calculated
bridge window size is enough.
This same problem may occur on io window side but seems less likely to
cause issues due to general difference in alignment. Removing the lower
bound may have other unforeseen consequences in case of io window so
it's better to do leave as -next material if no problem is reported
related to io window sizing (BAR resize shouldn't touch io windows
anyway).
Reported-by: Simon Richter <Simon.Richter at hogyros.de>
Fixes: 3baeae36039a ("PCI: Use pci_release_resource() instead of release_resource()")
Signed-off-by: Ilpo Järvinen <ilpo.jarvinen at linux.intel.com>
---
drivers/pci/setup-bus.c | 15 +++++----------
1 file changed, 5 insertions(+), 10 deletions(-)
diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c
index 362ad108794d..e842feb85be0 100644
--- a/drivers/pci/setup-bus.c
+++ b/drivers/pci/setup-bus.c
@@ -1066,16 +1066,13 @@ static resource_size_t calculate_memsize(resource_size_t size,
resource_size_t min_size,
resource_size_t add_size,
resource_size_t children_add_size,
- resource_size_t old_size,
resource_size_t align)
{
if (size < min_size)
size = min_size;
- if (old_size == 1)
- old_size = 0;
size = max(size, add_size) + children_add_size;
- return ALIGN(max(size, old_size), align);
+ return ALIGN(size, align);
}
resource_size_t __weak pcibios_window_alignment(struct pci_bus *bus,
@@ -1317,7 +1314,6 @@ static void pbus_size_mem(struct pci_bus *bus, unsigned long type,
resource_size_t children_add_align = 0;
resource_size_t add_align = 0;
resource_size_t relaxed_align;
- resource_size_t old_size;
if (!b_res)
return;
@@ -1388,11 +1384,10 @@ static void pbus_size_mem(struct pci_bus *bus, unsigned long type,
}
}
- old_size = resource_size(b_res);
win_align = window_alignment(bus, b_res->flags);
min_align = calculate_mem_align(aligns, max_order);
min_align = max(min_align, win_align);
- size0 = calculate_memsize(size, min_size, 0, 0, old_size, min_align);
+ size0 = calculate_memsize(size, min_size, 0, 0, min_align);
if (size0) {
resource_set_range(b_res, min_align, size0);
@@ -1404,7 +1399,7 @@ static void pbus_size_mem(struct pci_bus *bus, unsigned long type,
relaxed_align = 1ULL << (max_order + __ffs(SZ_1M));
relaxed_align = max(relaxed_align, win_align);
min_align = min(min_align, relaxed_align);
- size0 = calculate_memsize(size, min_size, 0, 0, old_size, win_align);
+ size0 = calculate_memsize(size, min_size, 0, 0, win_align);
resource_set_range(b_res, min_align, size0);
pci_info(bus->self, "bridge window %pR to %pR requires relaxed alignment rules\n",
b_res, &bus->busn_res);
@@ -1413,7 +1408,7 @@ static void pbus_size_mem(struct pci_bus *bus, unsigned long type,
if (realloc_head && (add_size > 0 || children_add_size > 0)) {
add_align = max(min_align, add_align);
size1 = calculate_memsize(size, min_size, add_size, children_add_size,
- old_size, add_align);
+ add_align);
if (bus->self && size1 &&
!pbus_upstream_space_available(bus, b_res, size1, add_align)) {
@@ -1421,7 +1416,7 @@ static void pbus_size_mem(struct pci_bus *bus, unsigned long type,
relaxed_align = max(relaxed_align, win_align);
min_align = min(min_align, relaxed_align);
size1 = calculate_memsize(size, min_size, add_size, children_add_size,
- old_size, win_align);
+ win_align);
pci_info(bus->self,
"bridge window %pR to %pR requires relaxed alignment rules\n",
b_res, &bus->busn_res);
base-commit: 2f2c7254931f41b5736e3ba12aaa9ac1bbeeeb92
--
2.39.5
More information about the Linuxppc-dev
mailing list