[RFCv2 PATCH 0/7] A General Accelerator Framework, WarpDrive

Kenneth Lee liguozhu at hisilicon.com
Fri Sep 14 23:05:35 AEST 2018

On Fri, Sep 14, 2018 at 06:50:55AM +0000, Tian, Kevin wrote:
> Date: Fri, 14 Sep 2018 06:50:55 +0000
> From: "Tian, Kevin" <kevin.tian at intel.com>
> To: Jerome Glisse <jglisse at redhat.com>, Kenneth Lee <liguozhu at hisilicon.com>
> CC: Kenneth Lee <nek.in.cn at gmail.com>, Alex Williamson
>  <alex.williamson at redhat.com>, Herbert Xu <herbert at gondor.apana.org.au>,
>  "kvm at vger.kernel.org" <kvm at vger.kernel.org>, Jonathan Corbet
>  <corbet at lwn.net>, Greg Kroah-Hartman <gregkh at linuxfoundation.org>, Zaibo
>  Xu <xuzaibo at huawei.com>, "linux-doc at vger.kernel.org"
>  <linux-doc at vger.kernel.org>, "Kumar, Sanjay K" <sanjay.k.kumar at intel.com>,
>  Hao Fang <fanghao11 at huawei.com>, "linux-kernel at vger.kernel.org"
>  <linux-kernel at vger.kernel.org>, "linuxarm at huawei.com"
>  <linuxarm at huawei.com>, "iommu at lists.linux-foundation.org"
>  <iommu at lists.linux-foundation.org>, "David S . Miller"
>  <davem at davemloft.net>, "linux-crypto at vger.kernel.org"
>  <linux-crypto at vger.kernel.org>, Zhou Wang <wangzhou1 at hisilicon.com>,
>  Philippe Ombredanne <pombredanne at nexb.com>, Thomas Gleixner
>  <tglx at linutronix.de>, Joerg Roedel <joro at 8bytes.org>,
>  "linux-accelerators at lists.ozlabs.org"
>  <linux-accelerators at lists.ozlabs.org>, Lu Baolu <baolu.lu at linux.intel.com>
> Subject: RE: [RFCv2 PATCH 0/7] A General Accelerator Framework, WarpDrive
> Message-ID: <AADFC41AFE54684AB9EE6CBC0274A5D191303A7F at SHSMSX101.ccr.corp.intel.com>
> > From: Jerome Glisse
> > Sent: Thursday, September 13, 2018 10:52 PM
> >
> [...]
>  > AFAIK, on x86 and PPC at least, all PCIE devices are in the same group
> > by default at boot or at least all devices behind the same bridge.
> the group thing reflects physical hierarchy limitation, not changed
> cross boot. Please note iommu group defines the minimal isolation
> boundary - all devices within same group must be attached to the
> same iommu domain or address space, because physically IOMMU
> cannot differentiate DMAs out of those devices. devices behind
> legacy PCI-X bridge is one example. other examples include devices
> behind a PCIe switch port which doesn't support ACS thus cannot
> route p2p transaction to IOMMU. If talking about typical PCIe 
> endpoint (with upstreaming ports all supporting ACS), you'll get
> one device per group.
> One iommu group today is attached to only one iommu domain.
> In the future one group may attach to multiple domains, as the
> aux domain concept being discussed in another thread.
> > 
> > Maybe they are kernel option to avoid that and userspace init program
> > can definitly re-arrange that base on sysadmin policy).
> I don't think there is such option, as it may break isolation model
> enabled by IOMMU.
> [...]
> > > > That is why i am being pedantic :) on making sure there is good reasons
> > > > to do what you do inside VFIO. I do believe that we want a common
> > frame-
> > > > work like the one you are proposing but i do not believe it should be
> > > > part of VFIO given the baggages it comes with and that are not relevant
> > > > to the use cases for this kind of devices.
> > >
> The purpose of VFIO is clear - the kernel portal for granting generic 
> device resource (mmio, irq, etc.) to user space. VFIO doesn't care
> what exactly a resource is used for (queue, cmd reg, etc.). If really
> pursuing VFIO path is necessary, maybe such common framework
> should lay down in user space, which gets all granted resource from
> kernel driver thru VFIO and then provides accelerator services to 
> other processes?

Yes. I think this is exactly what WarpDrive is now doing. This patch is just let
the type1 driver use parent IOMMU for mdev.

> Thanks
> Kevin


This e-mail and its attachments contain confidential information from HUAWEI,
which is intended only for the person or entity whose address is listed above.
Any use of the 
information contained herein in any way (including, but not limited to, total or
partial disclosure, reproduction, or dissemination) by persons other than the
recipient(s) is prohibited. If you receive this e-mail in error, please notify
the sender by phone or email immediately and delete it!

More information about the Linux-accelerators mailing list