[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-Id: <20220927055337.22630-6-lizhijian@fujitsu.com>
Date: Tue, 27 Sep 2022 13:53:31 +0800
From: Li Zhijian <lizhijian@...itsu.com>
To: Bob Pearson <rpearsonhpe@...il.com>,
Leon Romanovsky <leon@...nel.org>,
Jason Gunthorpe <jgg@...pe.ca>, linux-rdma@...r.kernel.org
Cc: Zhu Yanjun <zyjzyj2000@...il.com>, yangx.jy@...itsu.com,
y-goto@...itsu.com, mbloch@...dia.com, liangwenpeng@...wei.com,
tom@...pey.com, tomasz.gromadzki@...el.com,
dan.j.williams@...el.com, linux-kernel@...r.kernel.org,
Li Zhijian <lizhijian@...itsu.com>
Subject: [for-next PATCH v5 05/11] RDMA/rxe: Allow registering persistent flag for pmem MR only
Memory region could support at most 2 flush access flags:
IB_ACCESS_FLUSH_PERSISTENT and IB_ACCESS_FLUSH_GLOBAL
But we only allow user to register persistent flush flags to the pmem MR
where it has the ability of persisting data across power cycles.
So register a persistent access flag to a non-pmem MR will be rejected.
CC: Dan Williams <dan.j.williams@...el.com>
Signed-off-by: Li Zhijian <lizhijian@...itsu.com>
---
V5: make sure the whole MR is pmem
V4: set is_pmem more simple
V2: new scheme check is_pmem # Dan
update commit message, get rid of confusing ib_check_flush_access_flags() # Tom
---
drivers/infiniband/sw/rxe/rxe_mr.c | 19 +++++++++++++++++++
1 file changed, 19 insertions(+)
diff --git a/drivers/infiniband/sw/rxe/rxe_mr.c b/drivers/infiniband/sw/rxe/rxe_mr.c
index 74a38d06332f..1da3ad5eba64 100644
--- a/drivers/infiniband/sw/rxe/rxe_mr.c
+++ b/drivers/infiniband/sw/rxe/rxe_mr.c
@@ -112,6 +112,13 @@ void rxe_mr_init_dma(int access, struct rxe_mr *mr)
mr->type = IB_MR_TYPE_DMA;
}
+static bool vaddr_in_pmem(char *vaddr)
+{
+ return REGION_INTERSECTS ==
+ region_intersects(virt_to_phys(vaddr), 1, IORESOURCE_MEM,
+ IORES_DESC_PERSISTENT_MEMORY);
+}
+
int rxe_mr_init_user(struct rxe_dev *rxe, u64 start, u64 length, u64 iova,
int access, struct rxe_mr *mr)
{
@@ -122,6 +129,7 @@ int rxe_mr_init_user(struct rxe_dev *rxe, u64 start, u64 length, u64 iova,
int num_buf;
void *vaddr;
int err;
+ bool is_pmem = false;
int i;
umem = ib_umem_get(&rxe->ib_dev, start, length, access);
@@ -149,6 +157,7 @@ int rxe_mr_init_user(struct rxe_dev *rxe, u64 start, u64 length, u64 iova,
num_buf = 0;
map = mr->map;
if (length > 0) {
+ is_pmem = true;
buf = map[0]->buf;
for_each_sgtable_page (&umem->sgt_append.sgt, &sg_iter, 0) {
@@ -166,6 +175,10 @@ int rxe_mr_init_user(struct rxe_dev *rxe, u64 start, u64 length, u64 iova,
goto err_cleanup_map;
}
+ /* True only if the *whole* MR is pmem */
+ if (is_pmem)
+ is_pmem = vaddr_in_pmem(vaddr);
+
buf->addr = (uintptr_t)vaddr;
buf->size = PAGE_SIZE;
num_buf++;
@@ -174,6 +187,12 @@ int rxe_mr_init_user(struct rxe_dev *rxe, u64 start, u64 length, u64 iova,
}
}
+ if (!is_pmem && access & IB_ACCESS_FLUSH_PERSISTENT) {
+ pr_warn("Cannot register IB_ACCESS_FLUSH_PERSISTENT for non-pmem memory\n");
+ err = -EINVAL;
+ goto err_release_umem;
+ }
+
mr->umem = umem;
mr->access = access;
mr->offset = ib_umem_offset(umem);
--
2.31.1
Powered by blists - more mailing lists