<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<HTML><HEAD>
<META http-equiv=Content-Type content="text/html; charset=gb2312">
<META content="MSHTML 6.00.2900.3462" name=GENERATOR>
<STYLE>@font-face {
font-family: ËÎÌå;
}
@font-face {
font-family: Verdana;
}
@font-face {
font-family: @ËÎÌå;
}
@page Section1 {size: 595.3pt 841.9pt; margin: 72.0pt 90.0pt 72.0pt 90.0pt; layout-grid: 15.6pt; }
P.MsoNormal {
TEXT-JUSTIFY: inter-ideograph; FONT-SIZE: 10.5pt; MARGIN: 0cm 0cm 0pt; FONT-FAMILY: "Times New Roman"; TEXT-ALIGN: justify
}
LI.MsoNormal {
TEXT-JUSTIFY: inter-ideograph; FONT-SIZE: 10.5pt; MARGIN: 0cm 0cm 0pt; FONT-FAMILY: "Times New Roman"; TEXT-ALIGN: justify
}
DIV.MsoNormal {
TEXT-JUSTIFY: inter-ideograph; FONT-SIZE: 10.5pt; MARGIN: 0cm 0cm 0pt; FONT-FAMILY: "Times New Roman"; TEXT-ALIGN: justify
}
A:link {
COLOR: blue; TEXT-DECORATION: underline
}
SPAN.MsoHyperlink {
COLOR: blue; TEXT-DECORATION: underline
}
A:visited {
COLOR: purple; TEXT-DECORATION: underline
}
SPAN.MsoHyperlinkFollowed {
COLOR: purple; TEXT-DECORATION: underline
}
SPAN.EmailStyle17 {
FONT-WEIGHT: normal; COLOR: windowtext; FONT-STYLE: normal; FONT-FAMILY: Verdana; TEXT-DECORATION: none; mso-style-type: personal-compose
}
DIV.Section1 {
page: Section1
}
UNKNOWN {
FONT-SIZE: 10pt
}
BLOCKQUOTE {
MARGIN-TOP: 0px; MARGIN-BOTTOM: 0px; MARGIN-LEFT: 2em
}
OL {
MARGIN-TOP: 0px; MARGIN-BOTTOM: 0px
}
UL {
MARGIN-TOP: 0px; MARGIN-BOTTOM: 0px
}
</STYLE>
</HEAD>
<BODY style="FONT-SIZE: 10pt; FONT-FAMILY: verdana">
<DIV><FONT face=Verdana color=#c0c0c0 size=2></FONT></DIV>
<DIV><FONT face=Verdana color=#000080 size=2>
<H3 class=r>Thanks for all the suggestions and Comments!</H3>
<DIV class=r></FONT><FONT face=Verdana color=#000080 size=4>In the system,
the total memory size is less than 4GB. I want to
know how to map the 3GB pci address to the kernel ,</FONT></DIV>
<DIV class=r><FONT color=#000080 size=4>and how can my driver access all the pci
device. Usually we have only 1GB kernel address, minus the 896MB</FONT></DIV>
<DIV class=r><FONT color=#000080><FONT size=4>for memory map, we can only use
the 128M for ioremap.</FONT> <FONT size=4>I can adjust the PAGE_OFFSET to a
lower value, but it's not enough.</FONT></FONT></DIV>
<DIV class=r><FONT color=#000080 size=4></FONT> </DIV>
<DIV class=r>
<DIV>>One can mmap() a PCI BAR from userspace, in which case the mapping comes</DIV>
<DIV>>out of the "max userspace size" pool instead of the "all ioremap()s" pool. </DIV>
<DIV>>The userspace pool is per processes. So while having four kernel drivers</DIV>
<DIV>>each call ioremap(..., 1GB) will never work, it is possible to have four</DIV>
<DIV>>userspace processes each call mmap("/sys/bus/pci.../resource", 1GB) and</DIV>
<DIV>>have it work.</DIV></DIV>
<DIV class=r><FONT color=#000080 size=4></FONT><FONT face=Verdana color=#000080
size=2></FONT> </DIV>
<DIV class=r><FONT face=Verdana><FONT color=#000080 size=4>There are many pci
devices in the system and every pci device has only several tens of MB, so how
can I call the
mmap("/sys/bus/pci.../resource", 1GB) </FONT></FONT></DIV>
<DIV class=r><FONT color=#000080 size=4>and how can I use it by my drivers
?</FONT></DIV>
<DIV class=r><FONT color=#000080 size=4></FONT> </DIV>
<DIV class=r><FONT color=#000080 size=4>Thanks again for all your
help!</FONT></DIV>
<DIV class=r><FONT face=Verdana color=#000080 size=2> </DIV></DIV>
<DIV>
<HR>
</DIV></FONT>
<DIV>
<DIV><FONT face=Verdana size=2></FONT></DIV><FONT face=Verdana size=2></FONT>
</DIV>
<DIV><FONT face=Verdana size=2>
<DIV>On Fri, 12 Dec 2008, Kumar Gala wrote:</DIV>
<DIV>> On Dec 12, 2008, at 3:04 AM, Trent Piepho wrote:</DIV>
<DIV>>
> On Thu, 11 Dec 2008, Kumar Gala wrote:</DIV>
<DIV>> >
> On Dec 11, 2008, at 10:07 PM, Trent Piepho wrote:</DIV>
<DIV>> > >
> On Thu, 11 Dec 2008, Kumar Gala wrote:</DIV>
<DIV>> > > >
> The 36-bit support is current (in tree) in complete. Work is in </DIV>
<DIV>> > > >
> add swiotlb support to PPC which will generically enable what you </DIV>
<DIV>> > > > </DIV>
<DIV>> > >
> Don't the ATMU windows in the pcie controller serve as a IOMMU, making</DIV>
<DIV>> > > > swiotlb</DIV>
<DIV>> > >
> unnecessary and wasteful?</DIV>
<DIV>> > > </DIV>
<DIV>> >
> Nope. You have no way to tell when to switch a window as you have no </DIV>
<DIV>> > > idea</DIV>
<DIV>> >
> when a device might DMA data.</DIV>
<DIV>> > </DIV>
<DIV>>
> Isn't that what dma_alloc_coherent() and dma_map_single() are for?</DIV>
<DIV>></DIV>
<DIV>> Nope. How would manipulate the PCI ATMU?</DIV>
<DIV> </DIV>
<DIV>Umm, out_be32()? Why would it be any different than other iommu</DIV>
<DIV>implementations, like the pseries one for example?</DIV>
<DIV> </DIV>
<DIV>Just define set a of fsl dma ops that use an inbound ATMU window if they</DIV>
<DIV>need to. The only issue would be if you have a 32-bit device with multiple</DIV>
<DIV>concurrent DMA buffers scattered over
> 32 bits of address space and run</DIV>
<DIV>out of ATMU windows. But other iommu implementations have that same</DIV>
<DIV>limitation. You just have to try harder to allocate GFP_DMA memory that</DIV>
<DIV>doesn't need an ATMU window or create larger contiguous bounce buffer to</DIV>
<DIV>replace scattered smaller buffers.</DIV>
<DIV> </DIV>
<DIV>>
> It sounded like the original poster was talking about having 3GB of PCI</DIV>
<DIV>>
> BARs. How does swiotlb even enter the picture for that?</DIV>
<DIV>></DIV>
<DIV>> It wasn't clear how much system memory they wanted. If they can fit their </DIV>
<DIV>> entire memory map for PCI addresses in 4G of address space (this includes all </DIV>
<DIV>> of system DRAM) than they don't need anything special.</DIV>
<DIV> </DIV>
<DIV>Why the need to fit the entire PCI memory map into the lower 4G? What</DIV>
<DIV>issue is there with mapping a PCI BAR above 4G if you have 36-bit support?</DIV>
<DIV> </DIV>
<DIV>Putting system memory below 4GB is only an issue if you're talking about</DIV>
<DIV>DMA. For mapping a PCI BAR, what does it doesn't matter?</DIV>
<DIV> </DIV>
<DIV>The problem I see with having large PCI BARs, is that the max userspace</DIV>
<DIV>process size plus low memory plus all ioremap()s must be less than 4GB. If</DIV>
<DIV>one wants to call ioremap(..., 3GB), then only 1 GB is left for userspace</DIV>
<DIV>plus low memory. That's not very much.</DIV>
<DIV> </DIV>
<DIV>One can mmap() a PCI BAR from userspace, in which case the mapping comes</DIV>
<DIV>out of the "max userspace size" pool instead of the "all ioremap()s" pool. </DIV>
<DIV>The userspace pool is per processes. So while having four kernel drivers</DIV>
<DIV>each call ioremap(..., 1GB) will never work, it is possible to have four</DIV>
<DIV>userspace processes each call mmap("/sys/bus/pci.../resource", 1GB) and</DIV>
<DIV>have it work.</DIV>
<DIV> </DIV>
<DIV>> >
>From what I've read about swiotlb, it is a hack that allows one to do DMA</DIV>
<DIV>>
> with 32-bit PCI devices on 64-bit systems that lack an IOMMU. It reserves</DIV>
<DIV>>
> a large block of RAM under 32-bits (technically it uses GFP_DMA) and doles</DIV>
<DIV>>
> this out to drivers that allocate DMA memory.</DIV>
<DIV>></DIV>
<DIV>> correct. It bounce buffers the DMAs to a 32-bit dma'ble region and copies </DIV>
<DIV>> to/from the
>32-bit address.</DIV></FONT></DIV></BODY></HTML>