Using VFIO vs. developing a kernel module

[Andrew Jeffery]

On Thu, 27 May 2021, at 22:53, sainath grandhi wrote:
> Hello,
> Our project has an FPGA connected to BMC as a PCIe endpoint. This
> endpoint provides a set of registers via MMIO and an interrupt for
> notifying completion of work. This endpoint also implements AER
> capability.
> 
> We have two options to enable this endpoint.
> 1) Write a new kernel module with a character device interface for
> user-space interaction.
> 2) Use VFIO infrastructure provided by Linux and write an user-space
> application.
> 
> I am reaching out to the community to check if there is any
> recommended option, using VFIO vs. implementing a new kernel module,
> or any previous experiences weighing in one option over the other.

I don't have any experience with VFIO, so take this with a grain of salt.

Generally you should write an in-kernel driver for it. The reason you 
might not want to do so is if the device's register interface changes 
frequently, as it's more pain to update the kernel than some userspace 
application, which slows iteration. But handling DMAs and interrupts 
make userspace more painful, so unless VFIO helps there (I assume it 
does), then that would push the implementation back towards the kernel.

Andrew