[PATCH] mpt2sas: Fall back to 64 bit coherent mask if 64 bit DMA / 32 bit coherent mask not supported

[Brian King]

On 05/12/2015 05:10 PM, Benjamin Herrenschmidt wrote:
> On Tue, 2015-05-12 at 17:07 -0500, Brian King wrote:
>> The mpt2sas driver was changed late last year in that it now requests a 64 bit DMA
>> mask, then requests a 32 bit coherent DMA mask, then later requests a 64 bit coherent
>> DMA mask. This was 5fb1bf8aaa832e1e9ca3198de7bbecb8eff7db9c. This breaks 64 bit DMA
>> support for mpt2sas on Power and we always fall back to using 32 bit DMA. Looking
>> at the commit log, it looks like this was an intentional change.
>>
>> Ben - you had a patch set you had posted to the list back in Feb of this year, but it
>> doesn't look like it got merged. 
> 
> Right, it was broken, we can't do that unfortunately. It's a hard
> problem to fix.
> 
>> https://lists.ozlabs.org/pipermail/linuxppc-dev/2015-February/125087.html
>>
>> This would fix the issue I'm seeing on mpt2sas. Do you plan to dust that patch set
>> off and upstream it? Were there issues with it that still need to be resolved?
> 
> No that patch doesn't actually work.
> 
> What we need is essentially a new set of DMA ops that can route an
> individual map request via the iommu or the bypass window depending on
> what mask applies.

Ok. For the interim, then, Sreekanth, would you consider a patch such as the one
below to allow mpt2sas to support 64 bit DMA on architectures that don't support
a 32 bit coherent mask and a 64 bit dma mask at the same time? Only compile tested
at this point. Will run it on a Power machine shortly to confirm it doesn't
break anything...


-- 
Brian King
Power Linux I/O
IBM Linux Technology Center


8<-------


Commit 5fb1bf8aaa832e1e9ca3198de7bbecb8eff7db9c broke 64 bit DMA for mpt2sas on Power.
That commit changed the sequence for setting up the DMA and coherent DMA masks so
that during initialization the driver requests a 64 bit DMA mask and a 32 bit consistent
DMA mask, then later requests a 64 bit consistent DMA mask. The Power architecture does
not currently support this, which results in always falling back to a 32 bit DMA window,
which has a negative impact on performance. Tweak this algorithm slightly so that
if requesting a 32 bit consistent mask fails after we've successfully set a 64 bit
DMA mask, just try to get a 64 bit consistent mask. This should preserve existing
behavior on platforms that support mixed mask setting and restore previous functionality
to those that do not.

Signed-off-by: Brian King <brking at linux.vnet.ibm.com>
---

 drivers/scsi/mpt2sas/mpt2sas_base.c |   14 ++++++++------
 1 file changed, 8 insertions(+), 6 deletions(-)

diff -puN drivers/scsi/mpt2sas/mpt2sas_base.c~mpt2sas_dma_mask drivers/scsi/mpt2sas/mpt2sas_base.c
--- linux/drivers/scsi/mpt2sas/mpt2sas_base.c~mpt2sas_dma_mask	2015-05-12 16:09:58.228687852 -0500
+++ linux-bjking1/drivers/scsi/mpt2sas/mpt2sas_base.c	2015-05-12 17:38:31.588060903 -0500
@@ -1200,12 +1200,14 @@ _base_config_dma_addressing(struct MPT2S
 		const uint64_t required_mask =
 		    dma_get_required_mask(&pdev->dev);
 		if ((required_mask > DMA_BIT_MASK(32)) &&
-		    !pci_set_dma_mask(pdev, DMA_BIT_MASK(64)) &&
-		    !pci_set_consistent_dma_mask(pdev, consistent_dma_mask)) {
-			ioc->base_add_sg_single = &_base_add_sg_single_64;
-			ioc->sge_size = sizeof(Mpi2SGESimple64_t);
-			ioc->dma_mask = 64;
-			goto out;
+		    !pci_set_dma_mask(pdev, DMA_BIT_MASK(64))) {
+			if (!pci_set_consistent_dma_mask(pdev, consistent_dma_mask) ||
+			    !pci_set_consistent_dma_mask(pdev, DMA_BIT_MASK(64))) {
+				ioc->base_add_sg_single = &_base_add_sg_single_64;
+				ioc->sge_size = sizeof(Mpi2SGESimple64_t);
+				ioc->dma_mask = 64;
+				goto out;
+			}
 		}
 	}
 
_