linux-kernel - [PATCH 3/3] drm: omapdrm: Do no allocate non-scanout GEMs through DMM/TILER

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:   Wed, 19 Jan 2022 12:23:11 +0200
From:   Ivaylo Dimitrov <ivo.g.dimitrov.75@...il.com>
To:     tomba@...nel.org, airlied@...ux.ie, daniel@...ll.ch
Cc:     dri-devel@...ts.freedesktop.org, linux-kernel@...r.kernel.org,
        linux-omap@...r.kernel.org, merlijn@...zup.org, tony@...mide.com,
        Ivaylo Dimitrov <ivo.g.dimitrov.75@...il.com>
Subject: [PATCH 3/3] drm: omapdrm: Do no allocate non-scanout GEMs through DMM/TILER

On devices with DMM, all allocations are done through either DMM or TILER.
DMM/TILER being a limited resource means that such allocations will start
to fail before actual free memory is exhausted. What is even worse is that
with time DMM/TILER space gets fragmented to the point that even if we have
enough free DMM/TILER space and free memory, allocation fails because there
is no big enough free block in DMM/TILER space.

Such failures can be easily observed with OMAP xorg DDX, for example -
starting few GUI applications (so buffers for their windows are allocated)
and then rotating landscape<->portrait while closing and opening new
windows soon results in allocation failures.

Fix that by mapping buffers through DMM/TILER only when really needed,
like, for scanout buffers.

Signed-off-by: Ivaylo Dimitrov <ivo.g.dimitrov.75@...il.com>
---
 drivers/gpu/drm/omapdrm/omap_gem.c | 12 ++++++++----
 1 file changed, 8 insertions(+), 4 deletions(-)

diff --git a/drivers/gpu/drm/omapdrm/omap_gem.c b/drivers/gpu/drm/omapdrm/omap_gem.c
index 41c1a6d..cf57179 100644
--- a/drivers/gpu/drm/omapdrm/omap_gem.c
+++ b/drivers/gpu/drm/omapdrm/omap_gem.c
@@ -821,10 +821,12 @@ int omap_gem_pin(struct drm_gem_object *obj, dma_addr_t *dma_addr)
 			if (ret)
 				goto fail;

-			if (priv->has_dmm) {
-				ret = omap_gem_pin_tiler(obj);
-				if (ret)
-					goto fail;
+			if (omap_obj->flags & OMAP_BO_SCANOUT) {
+				if (priv->has_dmm) {
+					ret = omap_gem_pin_tiler(obj);
+					if (ret)
+						goto fail;
+				}
 			}
 		} else {
 			refcount_inc(&omap_obj->pin_cnt);
@@ -861,6 +863,8 @@ static void omap_gem_unpin_locked(struct drm_gem_object *obj)
 			kfree(omap_obj->sgt);
 			omap_obj->sgt = NULL;
 		}
+		if (!(omap_obj->flags & OMAP_BO_SCANOUT))
+			return;
 		if (priv->has_dmm) {
 			ret = tiler_unpin(omap_obj->block);
 			if (ret) {
-- 
1.9.1