<div dir="ltr">On Thu, Jan 7, 2021 at 2:48 AM Florian Fainelli <<a href="mailto:f.fainelli@gmail.com">f.fainelli@gmail.com</a>> wrote:<br>><br>> Hi,<br>><br>> First of all let me say that I am glad that someone is working on a<br>> upstream solution for this issue, would appreciate if you could CC and<br>> Jim Quinlan on subsequent submissions.<br><br>Sure!<br><br>><br>><br>> On 1/5/21 7:41 PM, Claire Chang wrote:<br>> > This series implements mitigations for lack of DMA access control on<br>> > systems without an IOMMU, which could result in the DMA accessing the<br>> > system memory at unexpected times and/or unexpected addresses, possibly<br>> > leading to data leakage or corruption.<br>> ><br>> > For example, we plan to use the PCI-e bus for Wi-Fi and that PCI-e bus is<br>> > not behind an IOMMU. As PCI-e, by design, gives the device full access to<br>> > system memory, a vulnerability in the Wi-Fi firmware could easily escalate<br>> > to a full system exploit (remote wifi exploits: [1a], [1b] that shows a<br>> > full chain of exploits; [2], [3]).<br>> ><br>> > To mitigate the security concerns, we introduce restricted DMA. Restricted<br>> > DMA utilizes the existing swiotlb to bounce streaming DMA in and out of a<br>> > specially allocated region and does memory allocation from the same region.<br>> > The feature on its own provides a basic level of protection against the DMA<br>> > overwriting buffer contents at unexpected times. However, to protect<br>> > against general data leakage and system memory corruption, the system needs<br>> > to provide a way to restrict the DMA to a predefined memory region (this is<br>> > usually done at firmware level, e.g. in ATF on some ARM platforms).<br>><br>> Can you explain how ATF gets involved and to what extent it does help,<br>> besides enforcing a secure region from the ARM CPU's perpsective? Does<br>> the PCIe root complex not have an IOMMU but can somehow be denied access<br>> to a region that is marked NS=0 in the ARM CPU's MMU? If so, that is<br>> still some sort of basic protection that the HW enforces, right?<br><font face="arial, sans-serif"><br>We need the ATF support for memory MPU (memory protection unit).<br>Restricted DMA (with reserved-memory in dts) makes sure the predefined memory</font><div><font face="arial, sans-serif">region is for PCIe DMA only, but we still need MPU to locks down PCIe access to</font></div><div><font face="arial, sans-serif">that specific regions.<br><br>><br>> On Broadcom STB SoCs we have had something similar for a while however<br>> and while we don't have an IOMMU for the PCIe bridge, we do have a a<br>> basic protection mechanism whereby we can configure a region in DRAM to<br>> be PCIe read/write and CPU read/write which then gets used as the PCIe<br>> inbound region for the PCIe EP. By default the PCIe bridge is not<br>> allowed access to DRAM so we must call into a security agent to allow<br>> the PCIe bridge to access the designated DRAM region.<br>><br>> We have done this using a private CMA area region assigned via Device<br>> Tree, assigned with a and requiring the PCIe EP driver to use<br>> dma_alloc_from_contiguous() in order to allocate from this device<br>> private CMA area. The only drawback with that approach is that it<br>> requires knowing how much memory you need up front for buffers and DMA<br>> descriptors that the PCIe EP will need to process. The problem is that<br>> it requires driver modifications and that does not scale over the number<br>> of PCIe EP drivers, some we absolutely do not control, but there is no<br>> need to bounce buffer. Your approach scales better across PCIe EP<br>> drivers however it does require bounce buffering which could be a<br>> performance hit.<br><br>Only the streaming DMA (map/unmap) needs bounce buffering.<br>I also added alloc/free support in this series<br>(<a href="https://lore.kernel.org/patchwork/patch/1360995/">https://lore.kernel.org/patchwork/patch/1360995/</a>), so dma_direct_alloc() will</font></div><div><font face="arial, sans-serif">try to allocate memory from the predefined memory region.<br><br>As for the performance hit, it should be similar to the default swiotlb.<br>Here are my experiment results. Both SoCs lack IOMMU for PCIe.<br><br>PCIe wifi vht80 throughput -<br><br> MTK SoC tcp_tx tcp_rx udp_tx udp_rx<br> w/o Restricted DMA 244.1 134.66 312.56 350.79<br> w/ Restricted DMA 246.95 136.59 363.21 351.99<br><br> Rockchip SoC tcp_tx tcp_rx udp_tx udp_rx<br> w/o Restricted DMA 237.87 133.86 288.28 361.88<br> w/ Restricted DMA 256.01 130.95 292.28 353.19<br><br>The CPU usage doesn't increase too much either.<br>Although I didn't measure the CPU usage very precisely, it's ~3% with a single</font></div><div><font face="arial, sans-serif">big core (Cortex-A72) and ~5% with a single small core (Cortex-A53).<br><br>Thanks!<br>><br>> Thanks!</font></div></div><br><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Thu, Jan 7, 2021 at 2:48 AM Florian Fainelli <<a href="mailto:f.fainelli@gmail.com">f.fainelli@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Hi,<br>
<br>
First of all let me say that I am glad that someone is working on a<br>
upstream solution for this issue, would appreciate if you could CC and<br>
Jim Quinlan on subsequent submissions.<br>
<br>
On 1/5/21 7:41 PM, Claire Chang wrote:<br>
> This series implements mitigations for lack of DMA access control on<br>
> systems without an IOMMU, which could result in the DMA accessing the<br>
> system memory at unexpected times and/or unexpected addresses, possibly<br>
> leading to data leakage or corruption.<br>
> <br>
> For example, we plan to use the PCI-e bus for Wi-Fi and that PCI-e bus is<br>
> not behind an IOMMU. As PCI-e, by design, gives the device full access to<br>
> system memory, a vulnerability in the Wi-Fi firmware could easily escalate<br>
> to a full system exploit (remote wifi exploits: [1a], [1b] that shows a<br>
> full chain of exploits; [2], [3]).<br>
> <br>
> To mitigate the security concerns, we introduce restricted DMA. Restricted<br>
> DMA utilizes the existing swiotlb to bounce streaming DMA in and out of a<br>
> specially allocated region and does memory allocation from the same region.<br>
> The feature on its own provides a basic level of protection against the DMA<br>
> overwriting buffer contents at unexpected times. However, to protect<br>
> against general data leakage and system memory corruption, the system needs<br>
> to provide a way to restrict the DMA to a predefined memory region (this is<br>
> usually done at firmware level, e.g. in ATF on some ARM platforms).<br>
<br>
Can you explain how ATF gets involved and to what extent it does help,<br>
besides enforcing a secure region from the ARM CPU's perpsective? Does<br>
the PCIe root complex not have an IOMMU but can somehow be denied access<br>
to a region that is marked NS=0 in the ARM CPU's MMU? If so, that is<br>
still some sort of basic protection that the HW enforces, right?<br>
<br>
On Broadcom STB SoCs we have had something similar for a while however<br>
and while we don't have an IOMMU for the PCIe bridge, we do have a a<br>
basic protection mechanism whereby we can configure a region in DRAM to<br>
be PCIe read/write and CPU read/write which then gets used as the PCIe<br>
inbound region for the PCIe EP. By default the PCIe bridge is not<br>
allowed access to DRAM so we must call into a security agent to allow<br>
the PCIe bridge to access the designated DRAM region.<br>
<br>
We have done this using a private CMA area region assigned via Device<br>
Tree, assigned with a and requiring the PCIe EP driver to use<br>
dma_alloc_from_contiguous() in order to allocate from this device<br>
private CMA area. The only drawback with that approach is that it<br>
requires knowing how much memory you need up front for buffers and DMA<br>
descriptors that the PCIe EP will need to process. The problem is that<br>
it requires driver modifications and that does not scale over the number<br>
of PCIe EP drivers, some we absolutely do not control, but there is no<br>
need to bounce buffer. Your approach scales better across PCIe EP<br>
drivers however it does require bounce buffering which could be a<br>
performance hit.<br>
<br>
Thanks!<br>
-- <br>
Florian<br>
</blockquote></div>