To extend the feature of vfio-mdev

[Kenneth Lee]

On Mon, Oct 23, 2017 at 06:37:29PM +0800, Kenneth Lee wrote:
> Date: Mon, 23 Oct 2017 18:37:29 +0800
> From: Kenneth Lee <liguozhu at hisilicon.com>
> To: Jean-Philippe Brucker <jean-philippe.brucker at arm.com>
> CC: Alex Williamson <alex.williamson at redhat.com>,
>  "kenneth-lee-2012 at foxmail.com" <kenneth-lee-2012 at foxmail.com>, Jon Masters
>  <jcm at jonmasters.org>, "jcm at redhat.com" <jcm at redhat.com>,
>  "xuzaibo at huawei.com" <xuzaibo at huawei.com>,
>  "linux-accelerators at lists.ozlabs.org"
>  <linux-accelerators at lists.ozlabs.org>
> Subject: Re: To extend the feature of vfio-mdev
> Message-ID: <20171023103729.GF211370 at Turing-Arch-b>
> 
> On Mon, Oct 23, 2017 at 09:31:25AM +0100, Jean-Philippe Brucker wrote:
> > Date: Mon, 23 Oct 2017 09:31:25 +0100
> > From: Jean-Philippe Brucker <jean-philippe.brucker at arm.com>
> > To: Kenneth Lee <liguozhu at hisilicon.com>, Alex Williamson
> >  <alex.williamson at redhat.com>
> > CC: "kenneth-lee-2012 at foxmail.com" <kenneth-lee-2012 at foxmail.com>, Jon
> >  Masters <jcm at jonmasters.org>, "jcm at redhat.com" <jcm at redhat.com>,
> >  "xuzaibo at huawei.com" <xuzaibo at huawei.com>,
> >  "linux-accelerators at lists.ozlabs.org"
> >  <linux-accelerators at lists.ozlabs.org>
> > Subject: Re: To extend the feature of vfio-mdev
> > User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101
> >  Thunderbird/52.4.0
> > Message-ID: <6fed6c27-4b38-7d65-ee5d-63d6bb283e95 at arm.com>
> > 
> > On 23/10/17 06:18, Kenneth Lee wrote:
> > >>>>> 3. Let iommu driver accept more than one iommu_domain for the same
> > >>>>>    device. The substream id or pasid should be support for that  
> > >>>>
> > >>>> You're really extending the definition of an iommu_domain to include
> > >>>> PASID to do this, I don't think it makes sense in the general case.  So
> > >>>> perhaps you're talking about a PASID management layer sitting on top of
> > >>>> an iommu_domain.  AIUI for PCIe, a device has a requester ID which is
> > >>>> used to find the context entry for that device.  The IOMMU may support
> > >>>> PASID, which would cause a first level lookup via those set of page
> > >>>> tables, or it might only support second level translation.  The
> > >>>> iommu_domain is a reflection of that initial, single requester ID.  
> > >>>
> > >>> Maybe I misunderstand this. But the IOMMU hardware, such as SMMU for ARM,
> > >>> support multiple page table and is referred by something like ASID. If we should
> > >>> support it in Linux, iommu_domain should be the best choice (no matter you call
> > >>> it cookie or id or something else). Or where you can get a object referring to it?
> > >>
> > >> For PASID, a PASID is unique only within the requester ID.  I don't
> > >> know of anything equivalent to your ASID within PCIe.
> > >>
> > > 
> > > As my understanding to the ARM Spec:
> > > 
> > > 	PCIE[PASID]=ARM_SMMU[ASID]=ARM_SMMU[SubstreamID],
> > > 	PCIE[RequestID]=ARM_SMMU[StreamID]
> > 
> > In the SMMUv3 spec, the PASID is dissociated from the ASID. We set the
> > ASID in the context descriptor indexed by PASID, which provides more
> > flexibility when the CPU's ASID capacity and endpoint's PASID capacity differ.
> 
> Yes. Sorry, I tried to make the concept simply. But it is wrong slightly.
> 
> > 
> > > For ARM SMMU, the Stream ID is used to index a Context Descriptors
> > > Table while and Sub-stream ID is used to index a Descriptor which refer
> > > to a general page table in the same format as MMU.
> > > 
> > > So both RequestID and PASID uniquely identify a address space. Then
> > > every iommu-enabled device can service more than one user process at the
> > > same time.
> > > 
> > > So the IOMMU should support more than one page table, which in turn
> > > should be added to somewhere in Linux. If the iommu_domain refer to one
> > > address space, the iommu driver should accept more than one
> > > iommu_domain.
> > 
> > The current design choice is to have multiple address space per
> > iommu_domain (and one PASID table per domain). It fits better with
> > existing IOMMU and VFIO APIs.
> > 
> 
> I am still reading the patchset. But if the PASID is used to index the address
> space table in iommu_domain, it will become a general concept in the whole
> system. But if we would bind the address space with iommu_domain, the
> iommu_domain become a handle for any process to control the device dma. It will
> be easier for one process with two or more domains to control the device for
> different purpose. And it is also easier for a process hand over the domain to
> another without change anything. And the process need not to know anything about
> pasid. A domain is a handle to control the device separately. I think this make
> thing simpler.
> 
> > Thanks,
> > Jean
> 
> Thanks for discussion.
> -- 
> 			-Kenneth(Hisilicon)

Hi, Jean,

I read your patchset on git://linux-arm.org/linux-jpb in branch
svm/rfc2.

I don't understand why you bind the multiple asid/pasid feature with
SVM. I think it is risky. The performance of SVM is still a risk to many
hardware. It rely on the workflow. In many cases, we just need to
create a page table particularly for the devices itself without
dedicate the whole process space to it. So why not just make them as two
features? binding a new address space to the iommu_process or dedicate
the whole process address space to it.

Best Regards

-- 
			-Kenneth Lee (Hisilicon)