| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Message-ID: <f0957d1d-3d90-7851-ee40-a53fb6e8df8e@amd.com>
Date: Wed, 20 Sep 2023 15:06:14 +0200
From: Christian König <christian.koenig@....com>
To: Thomas Hellström
<thomas.hellstrom@...ux.intel.com>,
Danilo Krummrich <dakr@...hat.com>
Cc: airlied@...il.com, daniel@...ll.ch, matthew.brost@...el.com,
sarah.walker@...tec.com, donald.robson@...tec.com,
boris.brezillon@...labora.com, faith.ekstrand@...labora.com,
dri-devel@...ts.freedesktop.org, nouveau@...ts.freedesktop.org,
linux-kernel@...r.kernel.org
Subject: Re: [PATCH drm-misc-next v3 6/7] drm/gpuvm: generalize
dma_resv/extobj handling and GEM validation
Am 20.09.23 um 14:06 schrieb Thomas Hellström:
>
> On 9/20/23 12:51, Christian König wrote:
>> Am 20.09.23 um 09:44 schrieb Thomas Hellström:
>>> Hi,
>>>
>>> On 9/20/23 07:37, Christian König wrote:
>>>> Am 19.09.23 um 17:23 schrieb Thomas Hellström:
>>>>>
>>>>> On 9/19/23 17:16, Danilo Krummrich wrote:
>>>>>> On 9/19/23 14:21, Thomas Hellström wrote:
>>>>>>> Hi Christian
>>>>>>>
>>>>>>> On 9/19/23 14:07, Christian König wrote:
>>>>>>>> Am 13.09.23 um 17:46 schrieb Danilo Krummrich:
>>>>>>>>> On 9/13/23 17:33, Christian König wrote:
>>>>>>>>>> Am 13.09.23 um 17:15 schrieb Danilo Krummrich:
>>>>>>>>>>> On 9/13/23 16:26, Christian König wrote:
>>>>>>>>>>>> Am 13.09.23 um 14:16 schrieb Danilo Krummrich:
>>>>>>>>>>>>> As mentioned in a different mail thread, the reply is
>>>>>>>>>>>>> based on the assumption
>>>>>>>>>>>>> that we don't support anything else than GPUVM updates
>>>>>>>>>>>>> from the IOCTL.
>>>>>>>>>>>>
>>>>>>>>>>>> I think that this assumption is incorrect.
>>>>>>>>>>>
>>>>>>>>>>> Well, more precisely I should have said "don't support GPUVM
>>>>>>>>>>> updated from within
>>>>>>>>>>> fence signaling critical sections". And looking at the code,
>>>>>>>>>>> that doesn't seem what
>>>>>>>>>>> you're doing there.
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Vulkan is just once specific use case, but this here should
>>>>>>>>>>>> probably be able to handle other use cases as well.
>>>>>>>>>>>>
>>>>>>>>>>>> Especially with HMM you get the requirement that you need
>>>>>>>>>>>> to be able to invalidate GPUVM mappings without grabbing a
>>>>>>>>>>>> reservation lock.
>>>>>>>>>>>
>>>>>>>>>>> What do you mean with "invalidate GPUVM mappings" in this
>>>>>>>>>>> context? drm_gpuvm_bo_evict()
>>>>>>>>>>> should only be called from a ttm_device_funcs::move.
>>>>>>>>>>> callback, we should hold the dma-resv
>>>>>>>>>>> lock there.
>>>>>>>>>>
>>>>>>>>>> Well the question is which dma-resv lock do we hold?
>>>>>>>>>>
>>>>>>>>>> In the move callback we only hold the dma-resv lock of the BO
>>>>>>>>>> which is moved, but when that is a shared BO then that's not
>>>>>>>>>> the same as the one for the VM.
>>>>>>>>>
>>>>>>>>> Correct, Thomas' idea was to use the GEM's dma_resv lock to
>>>>>>>>> protect drm_gpuvm_bo::evicted
>>>>>>>>> and then actually move the drm_gpuvm_bo to the VM's evicted
>>>>>>>>> list once we grabbed all
>>>>>>>>> dma-resv locks when locking the VM's BOs using drm_exec. We
>>>>>>>>> can remove them from the evicted
>>>>>>>>> list on validate(). This way we never touch the evicted list
>>>>>>>>> without holding at least the VM's
>>>>>>>>> dma-resv lock.
>>>>>>>>>
>>>>>>>>> Do you have any concerns about that?
>>>>>>>>
>>>>>>>> Scratching my head a bit how that is supposed to work.
>>>>>>>>
>>>>>>>> This implies that you go over all the evicted BOs during
>>>>>>>> validation and not just the one mentioned in the CS.
>>>>>>>>
>>>>>>>> That might work for Vulkan, but is pretty much a no-go for OpenGL.
>>>>>>>>
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> See what the eviction lock in amdgpu is doing for example.
>>>>>>>>>>>
>>>>>>>>>>> The eviction_lock seems to protect a VM state "evicting" of
>>>>>>>>>>> whether any BO that
>>>>>>>>>>> is associated with the VM is currently evicting. At the same
>>>>>>>>>>> time amdgpu protects
>>>>>>>>>>> the eviceted list of the VM with a different lock. So this
>>>>>>>>>>> seems to be entirely
>>>>>>>>>>> unrelated. Tracking a "currently evicting" state is not part
>>>>>>>>>>> of the GPUVM
>>>>>>>>>>> implementation currently and hence nothing would change for
>>>>>>>>>>> amdgpu there.
>>>>>>>>>>
>>>>>>>>>> Sorry for the confusion we use different terminology in amdgpu.
>>>>>>>>>>
>>>>>>>>>> The eviction lock and evicted state is for the VM page
>>>>>>>>>> tables, e.g. if the whole VM is currently not used and
>>>>>>>>>> swapped out or even de-allocated.
>>>>>>>>>>
>>>>>>>>>> This is necessary because we have cases where we need to
>>>>>>>>>> access the VM data without holding the dma-resv lock of this
>>>>>>>>>> VM. Especially figuring out which parts of an address space
>>>>>>>>>> contain mappings and which doesn't.
>>>>>>>>>
>>>>>>>>> I think this is fine, this has nothing to do with lists of
>>>>>>>>> evicted GEM objects or external GEM
>>>>>>>>> objects, right? Marking mappings (drm_gpuva) as invalidated
>>>>>>>>> (DRM_GPUVA_INVALIDATED) or accessing
>>>>>>>>> the VA space does not require any dma-resv locks.
>>>>>>>>
>>>>>>>> I hope so, but I'm not 100% sure.
>>>>>>>>
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> This is a requirement which comes with HMM handling, you
>>>>>>>>>> won't see this with Vulkan (or OpenGL, VAAPI etc..).
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> The invalidation lock on the other hand is what in this
>>>>>>>>>> discussion is called eviction lock. This one is needed
>>>>>>>>>> because what I wrote above, during the move callback only the
>>>>>>>>>> dma-resv of the BO which is moved is locked, but not
>>>>>>>>>> necessarily the dma-resv of the VM.
>>>>>>>>>
>>>>>>>>> That's yet another thing, right? This is used to track whether
>>>>>>>>> *any* BO that belongs to the VM is
>>>>>>>>> currently being evicted, correct? As mentioned, as by now this
>>>>>>>>> is not supported in GPUVM and hence
>>>>>>>>> would be the same driver specific code with the same driver
>>>>>>>>> specifc lock.
>>>>>>>>
>>>>>>>> That is most likely a show stopper using this for OpenGL based
>>>>>>>> workloads as far as I can see. For those you need to able to
>>>>>>>> figure out which non-VM BOs have been evicted and which parts
>>>>>>>> of the VM needs updates.
>>>>>>>
>>>>>>> We identify those with a bool in the gpuvm_bo, and that bool is
>>>>>>> protected by the bo_resv. In essence, the "evicted" list must be
>>>>>>> made up-to-date with all relevant locks held before traversing
>>>>>>> in the next exec.
>>>>>>
>>>>>> What I still miss with this idea is how do we find all the
>>>>>> drm_gpuvm_bo structures with the evicted bool set to true? When
>>>>>> doing the drm_exec dance we come across all external ones and can
>>>>>> add them to the list if needed, but what about the BOs having the
>>>>>> VM's dma-resv?
>>>>>
>>>>> Oh, they can be added to the evict list directly (no bool needed)
>>>>> in the eviction code, like in v3. Since for those we indeed hold
>>>>> the VM's dma_resv since it's aliased with the object's dma-resv.
>>>>
>>>> Yeah, I wanted to note what Danilo seems to think about as well.
>>>> How do we figure out the non-VM BOs evicted?
>>>>
>>>> We can't walk over the list of all non-VM BOs on every submission,
>>>> that's to much overhead for cases with lots of non-VM BOs.
>>>>
>>>> And we can't rely on userspace sending all non-VM BOs as used list
>>>> down to the kernel with each submission.
>>>>
>>>> Regards,
>>>> Christian.
>>>
>>> No, that's not needed: Mechanism below.
>>>
>>> 1) We maintain an evicted list. Typically protected by the vm resv.
>>> 2) Each gpuvm_bo has a bool "evicted". Protected by the bo resv.
>>>
>>> a) Evicting a vm bo: The vm resv is held by the eviction code. Just
>>> put it on the evicted list.
>>> b) Evicting a shared/external bo: The bo resv is held by the
>>> eviction code. Set the "evicted" bool
>>> c) Validating the evicted list on exec:
>>
>>
>>> Loop through all *external/shared* bos.
>>
>> And this is what you can't do. For Vulkan it probably doesn't matter,
>> but for OpenGL and especially multimedia we have much more BOs on the
>> shared list than what's allocated for the VM.
>
> But you need to lock- and fence all those so you need to loop through
> them anyway, so we're still O(n_shared)? Or is there some clever
> optimization in amdgpu?
Why should I lock and fence them? Only the BOs in the relocation list
are locked and fenced.
Regards,
Christian.
>
> I think with some UMDs, xe might end up with similar large lists...
>
> /Thomas
>
>
>>
>> Regards,
>> Christian.
>>
>>> Lock them. After locking, check the "evicted" bool, if it's true.
>>> put the bo on the evicted list (we hold the VM resv at this point)
>>> and clear the "evicted" bool. Note that other vms will have their
>>> own gpuvm_bo which is marked evicted.
>>>
>>> I have this coded up in a patch for Xe and it seems to be working
>>> properly.
>>>
>>> /Thomas
>>>
>>>
>>>>
>>>>>
>>>>> /Thomas
>>>>>
>>>>>
>>>>>
>>>>>>
>>>>>>>
>>>>>>> If you mean that we need to unbind all vmas of all vms of
>>>>>>> evicted bos before evicting, We don't do that, at least not in
>>>>>>> Xe, since evicting we wait for VM idle, and it cant access
>>>>>>> anything through the stale vmas until they have been revalidated
>>>>>>> and rebound.
>>>>>>>
>>>>>>> /Thomas
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Regards,
>>>>>>>>>> Christian.
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Regards,
>>>>>>>>>>>> Christian.
>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Wed, Sep 13, 2023 at 11:14:46AM +0200, Thomas Hellström
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>> Hi!
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Wed, 2023-09-13 at 01:36 +0200, Danilo Krummrich wrote:
>>>>>>>>>>>>>>> On Tue, Sep 12, 2023 at 09:23:08PM +0200, Thomas
>>>>>>>>>>>>>>> Hellström wrote:
>>>>>>>>>>>>>>>> On 9/12/23 18:50, Danilo Krummrich wrote:
>>>>>>>>>>>>>>>>> On Tue, Sep 12, 2023 at 06:20:32PM +0200, Thomas
>>>>>>>>>>>>>>>>> Hellström wrote:
>>>>>>>>>>>>>>>>>> Hi, Danilo,
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> On 9/9/23 17:31, Danilo Krummrich wrote:
>>>>>>>>>>>>>>>>>>> So far the DRM GPUVA manager offers common
>>>>>>>>>>>>>>>>>>> infrastructure to
>>>>>>>>>>>>>>>>>>> track GPU VA
>>>>>>>>>>>>>>>>>>> allocations and mappings, generically connect GPU VA
>>>>>>>>>>>>>>>>>>> mappings
>>>>>>>>>>>>>>>>>>> to their
>>>>>>>>>>>>>>>>>>> backing buffers and perform more complex mapping
>>>>>>>>>>>>>>>>>>> operations
>>>>>>>>>>>>>>>>>>> on the GPU VA
>>>>>>>>>>>>>>>>>>> space.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> However, there are more design patterns commonly
>>>>>>>>>>>>>>>>>>> used by
>>>>>>>>>>>>>>>>>>> drivers, which
>>>>>>>>>>>>>>>>>>> can potentially be generalized in order to make the
>>>>>>>>>>>>>>>>>>> DRM GPUVA
>>>>>>>>>>>>>>>>>>> manager
>>>>>>>>>>>>>>>>>>> represent a basic GPU-VM implementation. In this
>>>>>>>>>>>>>>>>>>> context,
>>>>>>>>>>>>>>>>>>> this patch aims
>>>>>>>>>>>>>>>>>>> at generalizing the following elements.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> 1) Provide a common dma-resv for GEM objects not
>>>>>>>>>>>>>>>>>>> being used
>>>>>>>>>>>>>>>>>>> outside of
>>>>>>>>>>>>>>>>>>> this GPU-VM.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> 2) Provide tracking of external GEM objects (GEM
>>>>>>>>>>>>>>>>>>> objects
>>>>>>>>>>>>>>>>>>> which are
>>>>>>>>>>>>>>>>>>> shared with other GPU-VMs).
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> 3) Provide functions to efficiently lock all GEM
>>>>>>>>>>>>>>>>>>> objects dma-
>>>>>>>>>>>>>>>>>>> resv the
>>>>>>>>>>>>>>>>>>> GPU-VM contains mappings of.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> 4) Provide tracking of evicted GEM objects the GPU-VM
>>>>>>>>>>>>>>>>>>> contains mappings
>>>>>>>>>>>>>>>>>>> of, such that validation of evicted GEM
>>>>>>>>>>>>>>>>>>> objects is
>>>>>>>>>>>>>>>>>>> accelerated.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> 5) Provide some convinience functions for common
>>>>>>>>>>>>>>>>>>> patterns.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Rather than being designed as a "framework", the
>>>>>>>>>>>>>>>>>>> target is to
>>>>>>>>>>>>>>>>>>> make all
>>>>>>>>>>>>>>>>>>> features appear as a collection of optional helper
>>>>>>>>>>>>>>>>>>> functions,
>>>>>>>>>>>>>>>>>>> such that
>>>>>>>>>>>>>>>>>>> drivers are free to make use of the DRM GPUVA
>>>>>>>>>>>>>>>>>>> managers basic
>>>>>>>>>>>>>>>>>>> functionality and opt-in for other features without
>>>>>>>>>>>>>>>>>>> setting
>>>>>>>>>>>>>>>>>>> any feature
>>>>>>>>>>>>>>>>>>> flags, just by making use of the corresponding
>>>>>>>>>>>>>>>>>>> functions.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Big kudos to Boris Brezillon for his help to figure out
>>>>>>>>>>>>>>>>>>> locking for drivers
>>>>>>>>>>>>>>>>>>> updating the GPU VA space within the fence
>>>>>>>>>>>>>>>>>>> signalling path.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Suggested-by: Matthew Brost <matthew.brost@...el.com>
>>>>>>>>>>>>>>>>>>> Signed-off-by: Danilo Krummrich <dakr@...hat.com>
>>>>>>>>>>>>>>>>>>> ---
>>>>>>>>>>>>>>>>>>> drivers/gpu/drm/drm_gpuvm.c | 516
>>>>>>>>>>>>>>>>>>> ++++++++++++++++++++++++++++++++++++
>>>>>>>>>>>>>>>>>>> include/drm/drm_gpuvm.h | 197 ++++++++++++++
>>>>>>>>>>>>>>>>>>> 2 files changed, 713 insertions(+)
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> diff --git a/drivers/gpu/drm/drm_gpuvm.c
>>>>>>>>>>>>>>>>>>> b/drivers/gpu/drm/drm_gpuvm.c
>>>>>>>>>>>>>>>>>>> index f4411047dbb3..8e62a043f719 100644
>>>>>>>>>>>>>>>>>>> --- a/drivers/gpu/drm/drm_gpuvm.c
>>>>>>>>>>>>>>>>>>> +++ b/drivers/gpu/drm/drm_gpuvm.c
>>>>>>>>>>>>>>>>>>> @@ -73,6 +73,21 @@
>>>>>>>>>>>>>>>>>>> * &drm_gem_object list of &drm_gpuvm_bos for an
>>>>>>>>>>>>>>>>>>> existing
>>>>>>>>>>>>>>>>>>> instance of this
>>>>>>>>>>>>>>>>>>> * particular combination. If not existent a new
>>>>>>>>>>>>>>>>>>> instance
>>>>>>>>>>>>>>>>>>> is created and linked
>>>>>>>>>>>>>>>>>>> * to the &drm_gem_object.
>>>>>>>>>>>>>>>>>>> + *
>>>>>>>>>>>>>>>>>>> + * &drm_gpuvm_bo structures, since unique for a given
>>>>>>>>>>>>>>>>>>> &drm_gpuvm, are also used
>>>>>>>>>>>>>>>>>>> + * as entry for the &drm_gpuvm's lists of external and
>>>>>>>>>>>>>>>>>>> evicted objects. Those
>>>>>>>>>>>>>>>>>>> + * list are maintained in order to accelerate
>>>>>>>>>>>>>>>>>>> locking of
>>>>>>>>>>>>>>>>>>> dma-resv locks and
>>>>>>>>>>>>>>>>>>> + * validation of evicted objects bound in a
>>>>>>>>>>>>>>>>>>> &drm_gpuvm. For
>>>>>>>>>>>>>>>>>>> instance the all
>>>>>>>>>>>>>>>>>>> + * &drm_gem_object's &dma_resv of a given
>>>>>>>>>>>>>>>>>>> &drm_gpuvm can be
>>>>>>>>>>>>>>>>>>> locked by calling
>>>>>>>>>>>>>>>>>>> + * drm_gpuvm_exec_lock(). Once locked drivers can call
>>>>>>>>>>>>>>>>>>> drm_gpuvm_validate() in
>>>>>>>>>>>>>>>>>>> + * order to validate all evicted &drm_gem_objects.
>>>>>>>>>>>>>>>>>>> It is
>>>>>>>>>>>>>>>>>>> also possible to lock
>>>>>>>>>>>>>>>>>>> + * additional &drm_gem_objects by providing the
>>>>>>>>>>>>>>>>>>> corresponding parameters to
>>>>>>>>>>>>>>>>>>> + * drm_gpuvm_exec_lock() as well as open code the
>>>>>>>>>>>>>>>>>>> &drm_exec
>>>>>>>>>>>>>>>>>>> loop while making
>>>>>>>>>>>>>>>>>>> + * use of helper functions such as
>>>>>>>>>>>>>>>>>>> drm_gpuvm_prepare_range()
>>>>>>>>>>>>>>>>>>> or
>>>>>>>>>>>>>>>>>>> + * drm_gpuvm_prepare_objects().
>>>>>>>>>>>>>>>>>>> + *
>>>>>>>>>>>>>>>>>>> + * Every bound &drm_gem_object is treated as
>>>>>>>>>>>>>>>>>>> external object
>>>>>>>>>>>>>>>>>>> when its &dma_resv
>>>>>>>>>>>>>>>>>>> + * structure is different than the &drm_gpuvm's common
>>>>>>>>>>>>>>>>>>> &dma_resv structure.
>>>>>>>>>>>>>>>>>>> */
>>>>>>>>>>>>>>>>>>> /**
>>>>>>>>>>>>>>>>>>> @@ -420,6 +435,20 @@
>>>>>>>>>>>>>>>>>>> * Subsequent calls to drm_gpuvm_bo_obtain() for
>>>>>>>>>>>>>>>>>>> the same
>>>>>>>>>>>>>>>>>>> &drm_gpuvm and
>>>>>>>>>>>>>>>>>>> * &drm_gem_object must be able to observe previous
>>>>>>>>>>>>>>>>>>> creations and destructions
>>>>>>>>>>>>>>>>>>> * of &drm_gpuvm_bos in order to keep instances
>>>>>>>>>>>>>>>>>>> unique.
>>>>>>>>>>>>>>>>>>> + *
>>>>>>>>>>>>>>>>>>> + * The &drm_gpuvm's lists for keeping track of
>>>>>>>>>>>>>>>>>>> external and
>>>>>>>>>>>>>>>>>>> evicted objects are
>>>>>>>>>>>>>>>>>>> + * protected against concurrent insertion / removal
>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>> iteration internally.
>>>>>>>>>>>>>>>>>>> + *
>>>>>>>>>>>>>>>>>>> + * However, drivers still need ensure to protect
>>>>>>>>>>>>>>>>>>> concurrent
>>>>>>>>>>>>>>>>>>> calls to functions
>>>>>>>>>>>>>>>>>>> + * iterating those lists, such as
>>>>>>>>>>>>>>>>>>> drm_gpuvm_validate() and
>>>>>>>>>>>>>>>>>>> + * drm_gpuvm_prepare_objects(). Every such function
>>>>>>>>>>>>>>>>>>> contains
>>>>>>>>>>>>>>>>>>> a particular
>>>>>>>>>>>>>>>>>>> + * comment and lockdep checks if possible.
>>>>>>>>>>>>>>>>>>> + *
>>>>>>>>>>>>>>>>>>> + * Functions adding or removing entries from those
>>>>>>>>>>>>>>>>>>> lists,
>>>>>>>>>>>>>>>>>>> such as
>>>>>>>>>>>>>>>>>>> + * drm_gpuvm_bo_evict() or
>>>>>>>>>>>>>>>>>>> drm_gpuvm_bo_extobj_add() may be
>>>>>>>>>>>>>>>>>>> called with external
>>>>>>>>>>>>>>>>>>> + * locks being held, e.g. in order to avoid the
>>>>>>>>>>>>>>>>>>> corresponding list to be
>>>>>>>>>>>>>>>>>>> + * (safely) modified while potentially being
>>>>>>>>>>>>>>>>>>> iternated by
>>>>>>>>>>>>>>>>>>> other API functions.
>>>>>>>>>>>>>>>>>>> + * However, this is entirely optional.
>>>>>>>>>>>>>>>>>>> */
>>>>>>>>>>>>>>>>>>> /**
>>>>>>>>>>>>>>>>>>> @@ -632,6 +661,131 @@
>>>>>>>>>>>>>>>>>>> * }
>>>>>>>>>>>>>>>>>>> */
>>>>>>>>>>>>>>>>>>> +/**
>>>>>>>>>>>>>>>>>>> + * get_next_vm_bo_from_list() - get the next vm_bo
>>>>>>>>>>>>>>>>>>> element
>>>>>>>>>>>>>>>>>>> + * @__gpuvm: The GPU VM
>>>>>>>>>>>>>>>>>>> + * @__list_name: The name of the list we're
>>>>>>>>>>>>>>>>>>> iterating on
>>>>>>>>>>>>>>>>>>> + * @__local_list: A pointer to the local list used
>>>>>>>>>>>>>>>>>>> to store
>>>>>>>>>>>>>>>>>>> already iterated items
>>>>>>>>>>>>>>>>>>> + * @__prev_vm_bo: The previous element we got from
>>>>>>>>>>>>>>>>>>> drm_gpuvm_get_next_cached_vm_bo()
>>>>>>>>>>>>>>>>>>> + *
>>>>>>>>>>>>>>>>>>> + * This helper is here to provide lockless list
>>>>>>>>>>>>>>>>>>> iteration.
>>>>>>>>>>>>>>>>>>> Lockless as in, the
>>>>>>>>>>>>>>>>>>> + * iterator releases the lock immediately after
>>>>>>>>>>>>>>>>>>> picking the
>>>>>>>>>>>>>>>>>>> first element from
>>>>>>>>>>>>>>>>>>> + * the list, so list insertion deletion can happen
>>>>>>>>>>>>>>>>>>> concurrently.
>>>>>>>>>>>>>>>>>> Are the list spinlocks needed for that async state
>>>>>>>>>>>>>>>>>> update from
>>>>>>>>>>>>>>>>>> within the
>>>>>>>>>>>>>>>>>> dma-fence critical section we've discussed previously?
>>>>>>>>>>>>>>>>> Yes, but also for other reasons, see below.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Otherwise it should be sufficient to protect the
>>>>>>>>>>>>>>>>>> lists with the
>>>>>>>>>>>>>>>>>> gpuvm's resv
>>>>>>>>>>>>>>>>>> (or for the extobj list with an outer lock).
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> If those spinlocks are still needed in some
>>>>>>>>>>>>>>>>>> situations, perhaps
>>>>>>>>>>>>>>>>>> could we
>>>>>>>>>>>>>>>>>> have an option to set them to NULL (Like IIRC the
>>>>>>>>>>>>>>>>>> maple tree
>>>>>>>>>>>>>>>>>> allows for)?
>>>>>>>>>>>>>>>>> The evict spinlock is needed in any case, since in
>>>>>>>>>>>>>>>>> drm_gpuvm_bo_evict() we're
>>>>>>>>>>>>>>>>> holding only the dma-resv lock from the BO this
>>>>>>>>>>>>>>>>> function gets
>>>>>>>>>>>>>>>>> called for. Hence,
>>>>>>>>>>>>>>>>> the spinlock protects concurrent drm_gpuvm_bo_evict()
>>>>>>>>>>>>>>>>> calls with
>>>>>>>>>>>>>>>>> different BOs.
>>>>>>>>>>>>>>>> No. Only if you try to add external objects to the vm's
>>>>>>>>>>>>>>>> evict list
>>>>>>>>>>>>>>>> from
>>>>>>>>>>>>>>>> within the evict code. That's not necessary since you
>>>>>>>>>>>>>>>> loop through
>>>>>>>>>>>>>>>> all
>>>>>>>>>>>>>>>> external objects anyway when locking them so an
>>>>>>>>>>>>>>>> "evicted" bool in
>>>>>>>>>>>>>>>> the vm_bo,
>>>>>>>>>>>>>>>> protected by the bo resv would be sufficient. The
>>>>>>>>>>>>>>>> extobj locking
>>>>>>>>>>>>>>>> loop can
>>>>>>>>>>>>>>>> then add the bo to the evicted list.
>>>>>>>>>>>>>>> And validate() can remove it while still holding all
>>>>>>>>>>>>>>> dma-resv locks,
>>>>>>>>>>>>>>> neat!
>>>>>>>>>>>>>>> However, what if two tasks are trying to lock the VA space
>>>>>>>>>>>>>>> concurrently? What
>>>>>>>>>>>>>>> do we do when the drm_gpuvm_bo's refcount drops to zero in
>>>>>>>>>>>>>>> drm_gpuva_unlink()?
>>>>>>>>>>>>>>> Are we guaranteed that at this point of time the
>>>>>>>>>>>>>>> drm_gpuvm_bo is not
>>>>>>>>>>>>>>> on the
>>>>>>>>>>>>>>> evicted list? Because otherwise we would call
>>>>>>>>>>>>>>> drm_gpuvm_bo_destroy()
>>>>>>>>>>>>>>> with the
>>>>>>>>>>>>>>> dma-resv lock held, which wouldn't be allowed, since
>>>>>>>>>>>>>>> drm_gpuvm_bo_destroy()
>>>>>>>>>>>>>>> might drop the last reference to the drm_gem_object and
>>>>>>>>>>>>>>> hence we'd
>>>>>>>>>>>>>>> potentially
>>>>>>>>>>>>>>> free the dma-resv lock while holding it, at least if
>>>>>>>>>>>>>>> it's an external
>>>>>>>>>>>>>>> object.
>>>>>>>>>>>>>> Easiest way in this scheme is to think of the lists as
>>>>>>>>>>>>>> being protected
>>>>>>>>>>>>>> by the vm's resv lock. That means anybody calling
>>>>>>>>>>>>>> unlink() must also
>>>>>>>>>>>>>> hold the vm's resv lock. (Which is OK from an UAF point
>>>>>>>>>>>>>> of view, but
>>>>>>>>>>>>>> perhaps not from a locking inversion POW from an async
>>>>>>>>>>>>>> list update).
>>>>>>>>>>>>> This would mean that on unlink() we'd need to hold the
>>>>>>>>>>>>> VM's resv lock and the
>>>>>>>>>>>>> corresponding GEM's resv lock (in case they're not the
>>>>>>>>>>>>> same anyways) because the
>>>>>>>>>>>>> VM's resv lock would protect the external / evicted object
>>>>>>>>>>>>> lists and the GEM
>>>>>>>>>>>>> objects resv lock protects the GEM's list of drm_gpuvm_bos
>>>>>>>>>>>>> and the
>>>>>>>>>>>>> drm_gpuvm_bo's list of drm_gpuvas.
>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> For extobjs an outer lock would be enough in case of
>>>>>>>>>>>>>>>>> Xe, but I
>>>>>>>>>>>>>>>>> really would not
>>>>>>>>>>>>>>>>> like to add even more complexity just to get the
>>>>>>>>>>>>>>>>> spinlock out of
>>>>>>>>>>>>>>>>> the way in case
>>>>>>>>>>>>>>>>> the driver already has an outer lock protecting this
>>>>>>>>>>>>>>>>> path.
>>>>>>>>>>>>>>>> I must disagree here. These spinlocks and atomic
>>>>>>>>>>>>>>>> operations are
>>>>>>>>>>>>>>>> pretty
>>>>>>>>>>>>>>>> costly and as discussed earlier this type of locking
>>>>>>>>>>>>>>>> was the reason
>>>>>>>>>>>>>>>> (at
>>>>>>>>>>>>>>>> least according to the commit message) that made
>>>>>>>>>>>>>>>> Christian drop the
>>>>>>>>>>>>>>>> XArray
>>>>>>>>>>>>>>>> use in drm_exec for the same set of objects: "The
>>>>>>>>>>>>>>>> locking overhead
>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>> unecessary and measurable". IMHO the spinlock is the added
>>>>>>>>>>>>>>>> complexity and a
>>>>>>>>>>>>>>>> single wide lock following the drm locking guidelines
>>>>>>>>>>>>>>>> set out by
>>>>>>>>>>>>>>>> Daniel and
>>>>>>>>>>>>>>>> David should really be the default choice with an
>>>>>>>>>>>>>>>> opt-in for a
>>>>>>>>>>>>>>>> spinlock if
>>>>>>>>>>>>>>>> needed for async and pushing out to a wq is not an option.
>>>>>>>>>>>>>>> For the external object list an outer lock would work as
>>>>>>>>>>>>>>> long as it's
>>>>>>>>>>>>>>> not the
>>>>>>>>>>>>>>> dma-resv lock of the corresponding GEM object, since
>>>>>>>>>>>>>>> here we actually
>>>>>>>>>>>>>>> need to
>>>>>>>>>>>>>>> remove the list entry from the external object list on
>>>>>>>>>>>>>>> drm_gpuvm_bo_destroy().
>>>>>>>>>>>>>>> It's just a bit weird design wise that drivers would
>>>>>>>>>>>>>>> need to take
>>>>>>>>>>>>>>> this outer
>>>>>>>>>>>>>>> lock on:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> - drm_gpuvm_bo_extobj_add()
>>>>>>>>>>>>>>> - drm_gpuvm_bo_destroy() (and hence also
>>>>>>>>>>>>>>> drm_gpuvm_bo_put())
>>>>>>>>>>>>>>> - drm_gpuva_unlink() (because it needs to call
>>>>>>>>>>>>>>> drm_gpuvm_bo_put())
>>>>>>>>>>>>>>> - drm_gpuvm_exec_lock()
>>>>>>>>>>>>>>> - drm_gpuvm_exec_lock_array()
>>>>>>>>>>>>>>> - drm_gpuvm_prepare_range()
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Given that it seems reasonable to do all the required
>>>>>>>>>>>>>>> locking
>>>>>>>>>>>>>>> internally.
>>>>>>>>>>>>>> From a design POW, there has been a clear direction in
>>>>>>>>>>>>>> XE to make
>>>>>>>>>>>>>> things similar to mmap() / munmap(), so this outer lock,
>>>>>>>>>>>>>> which in Xe is
>>>>>>>>>>>>>> an rwsem, is used in a similar way as the mmap_lock. It's
>>>>>>>>>>>>>> protecting
>>>>>>>>>>>>>> the page-table structures and vma rb tree, the userptr
>>>>>>>>>>>>>> structures and
>>>>>>>>>>>>>> the extobj list. Basically it's taken early in the exec
>>>>>>>>>>>>>> IOCTL, the
>>>>>>>>>>>>>> VM_BIND ioctl, the compute rebind worker and the
>>>>>>>>>>>>>> pagefault handler, so
>>>>>>>>>>>>>> all of the above are just asserting that it is taken in
>>>>>>>>>>>>>> the correct
>>>>>>>>>>>>>> mode.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> But strictly with this scheme one could also use the vm's
>>>>>>>>>>>>>> dma_resv for
>>>>>>>>>>>>>> the extobj list since with drm_exec, it's locked before
>>>>>>>>>>>>>> traversing the
>>>>>>>>>>>>>> list.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> The whole point of this scheme is to rely on locks that
>>>>>>>>>>>>>> you already are
>>>>>>>>>>>>>> supposed to be holding for various reasons and is simple
>>>>>>>>>>>>>> to comprehend.
>>>>>>>>>>>>> I don't agree that we're supposed to hold the VM's resv
>>>>>>>>>>>>> lock anyways for
>>>>>>>>>>>>> functions like drm_gpuvm_bo_put() or drm_gpuva_unlink(),
>>>>>>>>>>>>> but I'm fine using it
>>>>>>>>>>>>> for that purpose nevertheless.
>>>>>>>>>>>>>
>>>>>>>>>>>>>>> In order to at least place lockdep checks, the driver
>>>>>>>>>>>>>>> would need to
>>>>>>>>>>>>>>> supply the
>>>>>>>>>>>>>>> corresponding lock's lockdep_map, because the GPUVM
>>>>>>>>>>>>>>> otherwise doesn't
>>>>>>>>>>>>>>> know about
>>>>>>>>>>>>>>> the lock.
>>>>>>>>>>>>>> Yes, that sounds reasonable. One lockdep map per list.
>>>>>>>>>>>>> I'd really like to avoid that, especially now that
>>>>>>>>>>>>> everything got simpler. We
>>>>>>>>>>>>> should define the actual locks to take instead.
>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Out of curiosity, what is the overhead of a spin_lock()
>>>>>>>>>>>>>>> that doesn't
>>>>>>>>>>>>>>> need to
>>>>>>>>>>>>>>> spin?
>>>>>>>>>>>>>> I guess it's hard to tell exactly, but it is much lower
>>>>>>>>>>>>>> on modern x86
>>>>>>>>>>>>>> than what it used to be. Not sure about ARM, which is the
>>>>>>>>>>>>>> other
>>>>>>>>>>>>>> architecture important to us. I figure if there is little
>>>>>>>>>>>>>> cache-line
>>>>>>>>>>>>>> bouncing the main overhead comes from the implied barriers.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> A pretty simple way that would not add much code would be
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> static void gpuvm_cond_spin_lock(const struct drm_gpuvm
>>>>>>>>>>>>>>>> *gpuvm,
>>>>>>>>>>>>>>>> spinlock_t
>>>>>>>>>>>>>>>> *lock)
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> {
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> if (!gpuvm->resv_protected_lists)
>>>>>>>>>>>>>>>> spin_lock(lock);
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> }
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> For such drivers, that would require anybody calling
>>>>>>>>>>>>>>>>>> unlink to
>>>>>>>>>>>>>>>>>> hold the vm's
>>>>>>>>>>>>>>>>>> resv, though.
>>>>>>>>>>>>>>>>> In V4 I want to go back to having a dedicated lock for
>>>>>>>>>>>>>>>>> the GEMs
>>>>>>>>>>>>>>>>> gpuva list (or
>>>>>>>>>>>>>>>>> VM_BO list to be more precise). We can't just use the
>>>>>>>>>>>>>>>>> dma-resv
>>>>>>>>>>>>>>>>> lock for that
>>>>>>>>>>>>>>>>> with VM_BO abstractions, because on destruction of a
>>>>>>>>>>>>>>>>> VM_BO we
>>>>>>>>>>>>>>>>> otherwise wouldn't
>>>>>>>>>>>>>>>>> be allowed to already hold the dma-resv lock. That's
>>>>>>>>>>>>>>>>> the fix I
>>>>>>>>>>>>>>>>> was referring to
>>>>>>>>>>>>>>>>> earlier.
>>>>>>>>>>>>>>>> Yeah, I can see the need for a dedicated lock for the
>>>>>>>>>>>>>>>> GEM's gpuva
>>>>>>>>>>>>>>>> list, but
>>>>>>>>>>>>>>>> holding the vm's dma-resv lock across the unlink
>>>>>>>>>>>>>>>> shouldn't be a
>>>>>>>>>>>>>>>> problem. We
>>>>>>>>>>>>>>>> may free the object and a pointer to the vm's resv
>>>>>>>>>>>>>>>> during unlink
>>>>>>>>>>>>>>>> but we
>>>>>>>>>>>>>>>> don't free the vm's resv. It'd be a matter of ensuring
>>>>>>>>>>>>>>>> that any
>>>>>>>>>>>>>>>> calls to
>>>>>>>>>>>>>>>> unlink from *within* drm_gpuvm allows it to be held.
>>>>>>>>>>>>>>> Drivers calling unlink() from the fence signaling path
>>>>>>>>>>>>>>> can't use the
>>>>>>>>>>>>>>> VM's
>>>>>>>>>>>>>>> dma-resv lock.
>>>>>>>>>>>>>> Yes, that made me a bit curious because in the current
>>>>>>>>>>>>>> version the code
>>>>>>>>>>>>>> required the object's dma_resv for unlink() which can't
>>>>>>>>>>>>>> be grabbed
>>>>>>>>>>>>>> either from the fence signaling path. So are there any
>>>>>>>>>>>>>> drivers actually
>>>>>>>>>>>>>> wanting to do that? If so, they will either need to
>>>>>>>>>>>>>> resort to the
>>>>>>>>>>>>>> current spinlock solution or they will need to call
>>>>>>>>>>>>>> unlink from a
>>>>>>>>>>>>>> workqueue item.
>>>>>>>>>>>>> As Boris already mentioned we have the dma-resv lock by
>>>>>>>>>>>>> default or a driver
>>>>>>>>>>>>> specific GEM gpuva lock as opt-in. Now, we can get rid of
>>>>>>>>>>>>> the latter.
>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Also, what if the object is an external object? We can't
>>>>>>>>>>>>>>> use the VM's
>>>>>>>>>>>>>>> dma-resv
>>>>>>>>>>>>>>> lock here.
>>>>>>>>>>>>>> Why? Typically (sync) unlink is only ever called from an
>>>>>>>>>>>>>> unbind-like
>>>>>>>>>>>>>> operation where it should be trivial to grab the vm's
>>>>>>>>>>>>>> resv. Or, for
>>>>>>>>>>>>>> that matter any outer lock protecting the extobj list.
>>>>>>>>>>>>>> Rule would be
>>>>>>>>>>>>>> the drm_gpuvm_bo::entry::extobj and
>>>>>>>>>>>>>> drm_gpuvm_bo::entry::evict would
>>>>>>>>>>>>>> be protected by either the vm's dma_resv (or possibly an
>>>>>>>>>>>>>> outer lock in
>>>>>>>>>>>>>> the case of the extobj list).
>>>>>>>>>>>>> Outer lock wouldn't have been working for updates in the
>>>>>>>>>>>>> async path, but
>>>>>>>>>>>>> shouldn't be relevant anymore. We could use the VM's resv
>>>>>>>>>>>>> for that.
>>>>>>>>>>>>>
>>>>>>>>>>>>>>> And we can't have the GEM objs dma-resv lock held when
>>>>>>>>>>>>>>> calling
>>>>>>>>>>>>>>> unlink(), since unlink() calls drm_gpuvm_bo_put(), which
>>>>>>>>>>>>>>> if the
>>>>>>>>>>>>>>> refcount drops
>>>>>>>>>>>>>>> to zero calls drm_gpuvm_bo_destroy() and
>>>>>>>>>>>>>>> drm_gpuvm_bo_destroy() might
>>>>>>>>>>>>>>> drop the
>>>>>>>>>>>>>>> last reference of the GEM object.
>>>>>>>>>>>>>> Yes, but this is a different problem as to what exactly
>>>>>>>>>>>>>> protects
>>>>>>>>>>>>>> drm_gpuvm_bo::entry::gem. Either as you suggest an
>>>>>>>>>>>>>> internal per bo list
>>>>>>>>>>>>>> lock, or if we want to keep the bo's dma_resv we need to
>>>>>>>>>>>>>> ensure that
>>>>>>>>>>>>>> the caller of dma_resv_unlock(obj->resv) actually
>>>>>>>>>>>>>> refcounts its obj
>>>>>>>>>>>>>> pointer, and doesn't implicitly rely on the gpuvm_bo's
>>>>>>>>>>>>>> refcount (I know
>>>>>>>>>>>>>> Boris didn't like that, but requiring an explicit
>>>>>>>>>>>>>> refcount for a
>>>>>>>>>>>>>> pointer you dereference unless you're under a lock that
>>>>>>>>>>>>>> ensures keeping
>>>>>>>>>>>>>> the object alive is pretty much required?) But anyway for
>>>>>>>>>>>>>> the
>>>>>>>>>>>>>> drm_gpuvm_bo::entry::gem list protection (bo resv or
>>>>>>>>>>>>>> internal spinlock)
>>>>>>>>>>>>>> I don't have a strong preference.
>>>>>>>>>>>>> We can keep the GEM objects dma-resv lock, however as
>>>>>>>>>>>>> mentioned above
>>>>>>>>>>>>> drm_gpuva_unlink() and drm_gpuvm_bo_put() then requires
>>>>>>>>>>>>> both the VM's resv lock
>>>>>>>>>>>>> and the GEM's resv lock in case they differ.
>>>>>>>>>>>>>
>>>>>>>>>>>>>>> All those problems go away with a dedicated
>>>>>>>>>>>>>>> GEM gpuva list lock.
>>>>>>>>>>>>>> I don't think these are real problems.
>>>>>>>>>>>>>> With the excepton of the eviction list "trick" where we
>>>>>>>>>>>>>> currently have
>>>>>>>>>>>>>> slightly different approach to collect external bos
>>>>>>>>>>>>>> needing rebinding,
>>>>>>>>>>>>>> we have this working fine.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> TBH I think pretty much the only situation where the
>>>>>>>>>>>>>> spinlock is needed
>>>>>>>>>>>>>> is for async updates of these lists, unless a wq item can
>>>>>>>>>>>>>> be used for
>>>>>>>>>>>>>> that, but it doesn't really seem like the current code
>>>>>>>>>>>>>> allows for such
>>>>>>>>>>>>>> updates anyway? It complicates the code a lot, adds
>>>>>>>>>>>>>> overhead and also
>>>>>>>>>>>>>> adds the requirement for refcounting during list traversal.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> /Thomas
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> /Thomas
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> It seems that with that also the refcount could be
>>>>>>>>>>>>>>>>>> make non-
>>>>>>>>>>>>>>>>>> atomic.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> All in the spirit of the drm locking guidelines "use
>>>>>>>>>>>>>>>>>> big locks
>>>>>>>>>>>>>>>>>> when
>>>>>>>>>>>>>>>>>> possible".
>>>>>>>>>>>>>>>>>> Lower level locks only when necessary for performance or
>>>>>>>>>>>>>>>>>> locking inversion?
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> /Thomas
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> + *
>>>>>>>>>>>>>>>>>>> + * Elements popped from the original list are kept
>>>>>>>>>>>>>>>>>>> in a
>>>>>>>>>>>>>>>>>>> local list, so removal
>>>>>>>>>>>>>>>>>>> + * and is_empty checks can still happen while we're
>>>>>>>>>>>>>>>>>>> iterating the list.
>>>>>>>>>>>>>>>>>>> + */
>>>>>>>>>>>>>>>>>>> +#define get_next_vm_bo_from_list(__gpuvm, __list_name,
>>>>>>>>>>>>>>>>>>> __local_list, __prev_vm_bo) \
>>>>>>>>>>>>>>>>>>> + ({
>>>>>>>>>>>>>>>>>>> \
>>>>>>>>>>>>>>>>>>> + struct drm_gpuvm_bo
>>>>>>>>>>>>>>>>>>> *__vm_bo; \
>>>>>>>>>>>>>>>>>>> +
>>>>>>>>>>>>>>>>>>> \
>>>>>>>>>>>>>>>>>>> + drm_gpuvm_bo_put(__prev_vm_bo);
>>>>>>>>>>>>>>>>>>> \
>>>>>>>>>>>>>>>>>>> +
>>>>>>>>>>>>>>>>>>> \
>>>>>>>>>>>>>>>>>>> + spin_lock(&(__gpuvm)-
>>>>>>>>>>>>>>>>>>>> __list_name.lock); \
>>>>>>>>>>>>>>>>>>> + while (!list_empty(&(__gpuvm)-
>>>>>>>>>>>>>>>>>>>> __list_name.list)) { \
>>>>>>>>>>>>>>>>>>> + __vm_bo =
>>>>>>>>>>>>>>>>>>> list_first_entry(&(__gpuvm)->__list_name.list, \
>>>>>>>>>>>>>>>>>>> + struct
>>>>>>>>>>>>>>>>>>> drm_gpuvm_bo, \
>>>>>>>>>>>>>>>>>>> +
>>>>>>>>>>>>>>>>>>> list.entry.__list_name); \
>>>>>>>>>>>>>>>>>>> + if
>>>>>>>>>>>>>>>>>>> (drm_gpuvm_bo_get_unless_zero(__vm_bo))
>>>>>>>>>>>>>>>>>>> { \
>>>>>>>>>>>>>>>>>>> + list_move_tail(&(__vm_bo)-
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> list.entry.__list_name, \
>>>>>>>>>>>>>>>>>>> +
>>>>>>>>>>>>>>>>>>> __local_list); \
>>>>>>>>>>>>>>>>>>> + break;
>>>>>>>>>>>>>>>>>>> \
>>>>>>>>>>>>>>>>>>> + } else
>>>>>>>>>>>>>>>>>>> { \
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> + list_del_init(&(__vm_bo)-
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> list.entry.__list_name); \
>>>>>>>>>>>>>>>>>>> + __vm_bo =
>>>>>>>>>>>>>>>>>>> NULL; \
>>>>>>>>>>>>>>>>>>> + }
>>>>>>>>>>>>>>>>>>> \
>>>>>>>>>>>>>>>>>>> + }
>>>>>>>>>>>>>>>>>>> \
>>>>>>>>>>>>>>>>>>> + spin_unlock(&(__gpuvm)-
>>>>>>>>>>>>>>>>>>>> __list_name.lock); \
>>>>>>>>>>>>>>>>>>> +
>>>>>>>>>>>>>>>>>>> \
>>>>>>>>>>>>>>>>>>> + __vm_bo;
>>>>>>>>>>>>>>>>>>> \
>>>>>>>>>>>>>>>>>>> + })
>>>>>>>>>>>>>>>>>>> +
>>>>>>>>>>>>>>>>>>> +/**
>>>>>>>>>>>>>>>>>>> + * for_each_vm_bo_in_list() - internal vm_bo list
>>>>>>>>>>>>>>>>>>> iterator
>>>>>>>>>>>>>>>>>>> + *
>>>>>>>>>>>>>>>>>>> + * This helper is here to provide lockless list
>>>>>>>>>>>>>>>>>>> iteration.
>>>>>>>>>>>>>>>>>>> Lockless as in, the
>>>>>>>>>>>>>>>>>>> + * iterator releases the lock immediately after
>>>>>>>>>>>>>>>>>>> picking the
>>>>>>>>>>>>>>>>>>> first element from the
>>>>>>>>>>>>>>>>>>> + * list, so list insertion and deletion can happen
>>>>>>>>>>>>>>>>>>> concurrently.
>>>>>>>>>>>>>>>>>>> + *
>>>>>>>>>>>>>>>>>>> + * Typical use:
>>>>>>>>>>>>>>>>>>> + *
>>>>>>>>>>>>>>>>>>> + * struct drm_gpuvm_bo *vm_bo;
>>>>>>>>>>>>>>>>>>> + * LIST_HEAD(my_local_list);
>>>>>>>>>>>>>>>>>>> + *
>>>>>>>>>>>>>>>>>>> + * ret = 0;
>>>>>>>>>>>>>>>>>>> + * drm_gpuvm_for_each_vm_bo(gpuvm, <list_name>,
>>>>>>>>>>>>>>>>>>> &my_local_list, vm_bo) {
>>>>>>>>>>>>>>>>>>> + * ret = do_something_with_vm_bo(...,
>>>>>>>>>>>>>>>>>>> vm_bo);
>>>>>>>>>>>>>>>>>>> + * if (ret)
>>>>>>>>>>>>>>>>>>> + * break;
>>>>>>>>>>>>>>>>>>> + * }
>>>>>>>>>>>>>>>>>>> + * drm_gpuvm_bo_put(vm_bo);
>>>>>>>>>>>>>>>>>>> + * drm_gpuvm_restore_vm_bo_list(gpuvm,
>>>>>>>>>>>>>>>>>>> <list_name>,
>>>>>>>>>>>>>>>>>>> &my_local_list);
>>>>>>>>>>>>>>>>>>> + *
>>>>>>>>>>>>>>>>>>> + *
>>>>>>>>>>>>>>>>>>> + * Only used for internal list iterations, not
>>>>>>>>>>>>>>>>>>> meant to be
>>>>>>>>>>>>>>>>>>> exposed to the outside
>>>>>>>>>>>>>>>>>>> + * world.
>>>>>>>>>>>>>>>>>>> + */
>>>>>>>>>>>>>>>>>>> +#define for_each_vm_bo_in_list(__gpuvm, __list_name,
>>>>>>>>>>>>>>>>>>> __local_list, __vm_bo) \
>>>>>>>>>>>>>>>>>>> + for (__vm_bo =
>>>>>>>>>>>>>>>>>>> get_next_vm_bo_from_list(__gpuvm,
>>>>>>>>>>>>>>>>>>> __list_name, \
>>>>>>>>>>>>>>>>>>> + __local_list,
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> NULL); \
>>>>>>>>>>>>>>>>>>> +
>>>>>>>>>>>>>>>>>>> __vm_bo;
>>>>>>>>>>>>>>>>>>> \
>>>>>>>>>>>>>>>>>>> + __vm_bo =
>>>>>>>>>>>>>>>>>>> get_next_vm_bo_from_list(__gpuvm,
>>>>>>>>>>>>>>>>>>> __list_name, \
>>>>>>>>>>>>>>>>>>> + __local_list,
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> __vm_bo)) \
>>>>>>>>>>>>>>>>>>> +
>>>>>>>>>>>>>>>>>>> +/**
>>>>>>>>>>>>>>>>>>> + * restore_vm_bo_list() - move vm_bo elements back
>>>>>>>>>>>>>>>>>>> to their
>>>>>>>>>>>>>>>>>>> original list
>>>>>>>>>>>>>>>>>>> + * @__gpuvm: The GPU VM
>>>>>>>>>>>>>>>>>>> + * @__list_name: The name of the list we're
>>>>>>>>>>>>>>>>>>> iterating on
>>>>>>>>>>>>>>>>>>> + * @__local_list: A pointer to the local list used
>>>>>>>>>>>>>>>>>>> to store
>>>>>>>>>>>>>>>>>>> already iterated items
>>>>>>>>>>>>>>>>>>> + *
>>>>>>>>>>>>>>>>>>> + * When we're done iterating a vm_bo list, we
>>>>>>>>>>>>>>>>>>> should call
>>>>>>>>>>>>>>>>>>> restore_vm_bo_list()
>>>>>>>>>>>>>>>>>>> + * to restore the original state and let new
>>>>>>>>>>>>>>>>>>> iterations take
>>>>>>>>>>>>>>>>>>> place.
>>>>>>>>>>>>>>>>>>> + */
>>>>>>>>>>>>>>>>>>> +#define restore_vm_bo_list(__gpuvm, __list_name,
>>>>>>>>>>>>>>>>>>> __local_list) \
>>>>>>>>>>>>>>>>>>> + do
>>>>>>>>>>>>>>>>>>> {
>>>>>>>>>>>>>>>>>>> \
>>>>>>>>>>>>>>>>>>> + /* Merge back the two lists, moving
>>>>>>>>>>>>>>>>>>> local
>>>>>>>>>>>>>>>>>>> list elements to the \
>>>>>>>>>>>>>>>>>>> + * head to preserve previous
>>>>>>>>>>>>>>>>>>> ordering, in
>>>>>>>>>>>>>>>>>>> case it matters. \
>>>>>>>>>>>>>>>>>>> +
>>>>>>>>>>>>>>>>>>> */
>>>>>>>>>>>>>>>>>>> \
>>>>>>>>>>>>>>>>>>> + spin_lock(&(__gpuvm)-
>>>>>>>>>>>>>>>>>>>> __list_name.lock); \
>>>>>>>>>>>>>>>>>>> + list_splice(__local_list, &(__gpuvm)-
>>>>>>>>>>>>>>>>>>>> __list_name.list); \
>>>>>>>>>>>>>>>>>>> + spin_unlock(&(__gpuvm)-
>>>>>>>>>>>>>>>>>>>> __list_name.lock); \
>>>>>>>>>>>>>>>>>>> + } while (0)
>>>>>>>>>>>>>>>>>>> +/**
>>>>>>>>>>>>>>>>>>> + * drm_gpuvm_bo_list_add() - insert a vm_bo into
>>>>>>>>>>>>>>>>>>> the given
>>>>>>>>>>>>>>>>>>> list
>>>>>>>>>>>>>>>>>>> + * @__vm_bo: the &drm_gpuvm_bo
>>>>>>>>>>>>>>>>>>> + * @__list_name: the name of the list to insert into
>>>>>>>>>>>>>>>>>>> + *
>>>>>>>>>>>>>>>>>>> + * Inserts the given @__vm_bo into the list
>>>>>>>>>>>>>>>>>>> specified by
>>>>>>>>>>>>>>>>>>> @__list_name and
>>>>>>>>>>>>>>>>>>> + * increases the vm_bo's reference count.
>>>>>>>>>>>>>>>>>>> + */
>>>>>>>>>>>>>>>>>>> +#define drm_gpuvm_bo_list_add(__vm_bo,
>>>>>>>>>>>>>>>>>>> __list_name) �� \
>>>>>>>>>>>>>>>>>>> + do
>>>>>>>>>>>>>>>>>>> {
>>>>>>>>>>>>>>>>>>> \
>>>>>>>>>>>>>>>>>>> + spin_lock(&(__vm_bo)->vm-
>>>>>>>>>>>>>>>>>>>> __list_name.lock); \
>>>>>>>>>>>>>>>>>>> + if (list_empty(&(__vm_bo)-
>>>>>>>>>>>>>>>>>>>> list.entry.__list_name)) \
>>>>>>>>>>>>>>>>>>> + list_add_tail(&(__vm_bo)-
>>>>>>>>>>>>>>>>>>>> list.entry.__list_name, \
>>>>>>>>>>>>>>>>>>> + &(__vm_bo)->vm-
>>>>>>>>>>>>>>>>>>>> __list_name.list); \
>>>>>>>>>>>>>>>>>>> + spin_unlock(&(__vm_bo)->vm-
>>>>>>>>>>>>>>>>>>>> __list_name.lock); \
>>>>>>>>>>>>>>>>>>> + } while (0)
>>>>>>>>>>>>>>>>>>> +
>>>>>>>>>>>>>>>>>>> +/**
>>>>>>>>>>>>>>>>>>> + * drm_gpuvm_bo_list_del() - remove a vm_bo from
>>>>>>>>>>>>>>>>>>> the given
>>>>>>>>>>>>>>>>>>> list
>>>>>>>>>>>>>>>>>>> + * @__vm_bo: the &drm_gpuvm_bo
>>>>>>>>>>>>>>>>>>> + * @__list_name: the name of the list to insert into
>>>>>>>>>>>>>>>>>>> + *
>>>>>>>>>>>>>>>>>>> + * Removes the given @__vm_bo from the list
>>>>>>>>>>>>>>>>>>> specified by
>>>>>>>>>>>>>>>>>>> @__list_name and
>>>>>>>>>>>>>>>>>>> + * decreases the vm_bo's reference count.
>>>>>>>>>>>>>>>>>>> + */
>>>>>>>>>>>>>>>>>>> +#define drm_gpuvm_bo_list_del(__vm_bo,
>>>>>>>>>>>>>>>>>>> __list_name) \
>>>>>>>>>>>>>>>>>>> + do
>>>>>>>>>>>>>>>>>>> {
>>>>>>>>>>>>>>>>>>> \
>>>>>>>>>>>>>>>>>>> + spin_lock(&(__vm_bo)->vm-
>>>>>>>>>>>>>>>>>>>> __list_name.lock); \
>>>>>>>>>>>>>>>>>>> + if (!list_empty(&(__vm_bo)-
>>>>>>>>>>>>>>>>>>>> list.entry.__list_name)) \
>>>>>>>>>>>>>>>>>>> + list_del_init(&(__vm_bo)-
>>>>>>>>>>>>>>>>>>>> list.entry.__list_name); \
>>>>>>>>>>>>>>>>>>> + spin_unlock(&(__vm_bo)->vm-
>>>>>>>>>>>>>>>>>>>> __list_name.lock); \
>>>>>>>>>>>>>>>>>>> + } while (0)
>>>>>>>>>>>>>>>>>>> +
>>>>>>>>>>>>>>>>>>> +static int __must_check
>>>>>>>>>>>>>>>>>>> +drm_gpuvm_bo_get_unless_zero(struct drm_gpuvm_bo
>>>>>>>>>>>>>>>>>>> *vm_bo);
>>>>>>>>>>>>>>>>>>> +
>>>>>>>>>>>>>>>>>>> #define
>>>>>>>>>>>>>>>>>>> to_drm_gpuva(__node) container_of((__node), struct
>>>>>>>>>>>>>>>>>>> drm_gpuva, rb.node)
>>>>>>>>>>>>>>>>>>> #define GPUVA_START(node) ((node)->va.addr)
>>>>>>>>>>>>>>>>>>> @@ -713,6 +867,12 @@ drm_gpuvm_init(struct drm_gpuvm
>>>>>>>>>>>>>>>>>>> *gpuvm,
>>>>>>>>>>>>>>>>>>> struct drm_device *drm,
>>>>>>>>>>>>>>>>>>> gpuvm->rb.tree = RB_ROOT_CACHED;
>>>>>>>>>>>>>>>>>>> INIT_LIST_HEAD(&gpuvm->rb.list);
>>>>>>>>>>>>>>>>>>> + INIT_LIST_HEAD(&gpuvm->extobj.list);
>>>>>>>>>>>>>>>>>>> + spin_lock_init(&gpuvm->extobj.lock);
>>>>>>>>>>>>>>>>>>> +
>>>>>>>>>>>>>>>>>>> + INIT_LIST_HEAD(&gpuvm->evict.list);
>>>>>>>>>>>>>>>>>>> + spin_lock_init(&gpuvm->evict.lock);
>>>>>>>>>>>>>>>>>>> +
>>>>>>>>>>>>>>>>>>> ��drm_gpuva_check_overflow(start_offset,
>>>>>>>>>>>>>>>>>>> range);
>>>>>>>>>>>>>>>>>>> gpuvm->mm_start = start_offset;
>>>>>>>>>>>>>>>>>>> gpuvm->mm_range = range;
>>>>>>>>>>>>>>>>>>> @@ -754,10 +914,302 @@ drm_gpuvm_destroy(struct
>>>>>>>>>>>>>>>>>>> drm_gpuvm
>>>>>>>>>>>>>>>>>>> *gpuvm)
>>>>>>>>>>>>>>>>>>> WARN(!RB_EMPTY_ROOT(&gpuvm->rb.tree.rb_root),
>>>>>>>>>>>>>>>>>>> "GPUVA tree is not empty, potentially
>>>>>>>>>>>>>>>>>>> leaking
>>>>>>>>>>>>>>>>>>> memory.\n");
>>>>>>>>>>>>>>>>>>> + WARN(!list_empty(&gpuvm->extobj.list),
>>>>>>>>>>>>>>>>>>> "Extobj list
>>>>>>>>>>>>>>>>>>> should be empty.\n");
>>>>>>>>>>>>>>>>>>> + WARN(!list_empty(&gpuvm->evict.list), "Evict
>>>>>>>>>>>>>>>>>>> list
>>>>>>>>>>>>>>>>>>> should be empty.\n");
>>>>>>>>>>>>>>>>>>> +
>>>>>>>>>>>>>>>>>>> drm_gem_private_object_fini(&gpuvm->d_obj);
>>>>>>>>>>>>>>>>>>> }
>>>>>>>>>>>>>>>>>>> EXPORT_SYMBOL_GPL(drm_gpuvm_destroy);
>>>>>>>>>>>>>>>>>>> +/**
>>>>>>>>>>>>>>>>>>> + * drm_gpuvm_prepare_objects() - prepare all
>>>>>>>>>>>>>>>>>>> assoiciated BOs
>>>>>>>>>>>>>>>>>>> + * @gpuvm: the &drm_gpuvm
>>>>>>>>>>>>>>>>>>> + * @exec: the &drm_exec locking context
>>>>>>>>>>>>>>>>>>> + * @num_fences: the amount of &dma_fences to reserve
>>>>>>>>>>>>>>>>>>> + *
>>>>>>>>>>>>>>>>>>> + * Calls drm_exec_prepare_obj() for all
>>>>>>>>>>>>>>>>>>> &drm_gem_objects the
>>>>>>>>>>>>>>>>>>> given
>>>>>>>>>>>>>>>>>>> + * &drm_gpuvm contains mappings of.
>>>>>>>>>>>>>>>>>>> + *
>>>>>>>>>>>>>>>>>>> + * Using this function directly, it is the drivers
>>>>>>>>>>>>>>>>>>> responsibility to call
>>>>>>>>>>>>>>>>>>> + * drm_exec_init() and drm_exec_fini() accordingly.
>>>>>>>>>>>>>>>>>>> + *
>>>>>>>>>>>>>>>>>>> + * Note: This function is safe against concurrent
>>>>>>>>>>>>>>>>>>> insertion
>>>>>>>>>>>>>>>>>>> and removal of
>>>>>>>>>>>>>>>>>>> + * external objects, however it is not safe against
>>>>>>>>>>>>>>>>>>> concurrent usage itself.
>>>>>>>>>>>>>>>>>>> + *
>>>>>>>>>>>>>>>>>>> + * Drivers need to make sure to protect this case with
>>>>>>>>>>>>>>>>>>> either an outer VM lock
>>>>>>>>>>>>>>>>>>> + * or by calling drm_gpuvm_prepare_vm() before this
>>>>>>>>>>>>>>>>>>> function
>>>>>>>>>>>>>>>>>>> within the
>>>>>>>>>>>>>>>>>>> + * drm_exec_until_all_locked() loop, such that the
>>>>>>>>>>>>>>>>>>> GPUVM's
>>>>>>>>>>>>>>>>>>> dma-resv lock ensures
>>>>>>>>>>>>>>>>>>> + * mutual exclusion.
>>>>>>>>>>>>>>>>>>> + *
>>>>>>>>>>>>>>>>>>> + * Returns: 0 on success, negative error code on
>>>>>>>>>>>>>>>>>>> failure.
>>>>>>>>>>>>>>>>>>> + */
>>>>>>>>>>>>>>>>>>> +int
>>>>>>>>>>>>>>>>>>> +drm_gpuvm_prepare_objects(struct drm_gpuvm *gpuvm,
>>>>>>>>>>>>>>>>>>> + struct drm_exec *exec,
>>>>>>>>>>>>>>>>>>> + unsigned int num_fences)
>>>>>>>>>>>>>>>>>>> +{
>>>>>>>>>>>>>>>>>>> + struct drm_gpuvm_bo *vm_bo;
>>>>>>>>>>>>>>>>>>> + LIST_HEAD(extobjs);
>>>>>>>>>>>>>>>>>>> + int ret = 0;
>>>>>>>>>>>>>>>>>>> +
>>>>>>>>>>>>>>>>>>> + for_each_vm_bo_in_list(gpuvm, extobj, &extobjs,
>>>>>>>>>>>>>>>>>>> vm_bo) {
>>>>>>>>>>>>>>>>>>> + ret = drm_exec_prepare_obj(exec,
>>>>>>>>>>>>>>>>>>> vm_bo->obj,
>>>>>>>>>>>>>>>>>>> num_fences);
>>>>>>>>>>>>>>>>>>> + if (ret)
>>>>>>>>>>>>>>>>>>> + break;
>>>>>>>>>>>>>>>>>>> + }
>>>>>>>>>>>>>>>>>>> + /* Drop ref in case we break out of the
>>>>>>>>>>>>>>>>>>> loop. */
>>>>>>>>>>>>>>>>>>> + drm_gpuvm_bo_put(vm_bo);
>>>>>>>>>>>>>>>>>>> + restore_vm_bo_list(gpuvm, extobj, &extobjs);
>>>>>>>>>>>>>>>>>>> +
>>>>>>>>>>>>>>>>>>> + return ret;
>>>>>>>>>>>>>>>>>>> +}
>>>>>>>>>>>>>>>>>>> +EXPORT_SYMBOL_GPL(drm_gpuvm_prepare_objects);
>>>>>>>>>>>>>>>>>>> +
>>>>>>>>>>>>>>>>>>> +/**
>>>>>>>>>>>>>>>>>>> + * drm_gpuvm_prepare_range() - prepare all BOs
>>>>>>>>>>>>>>>>>>> mapped within
>>>>>>>>>>>>>>>>>>> a given range
>>>>>>>>>>>>>>>>>>> + * @gpuvm: the &drm_gpuvm
>>>>>>>>>>>>>>>>>>> + * @exec: the &drm_exec locking context
>>>>>>>>>>>>>>>>>>> + * @addr: the start address within the VA space
>>>>>>>>>>>>>>>>>>> + * @range: the range to iterate within the VA space
>>>>>>>>>>>>>>>>>>> + * @num_fences: the amount of &dma_fences to reserve
>>>>>>>>>>>>>>>>>>> + *
>>>>>>>>>>>>>>>>>>> + * Calls drm_exec_prepare_obj() for all
>>>>>>>>>>>>>>>>>>> &drm_gem_objects
>>>>>>>>>>>>>>>>>>> mapped between @addr
>>>>>>>>>>>>>>>>>>> + * and @addr + @range.
>>>>>>>>>>>>>>>>>>> + *
>>>>>>>>>>>>>>>>>>> + * Returns: 0 on success, negative error code on
>>>>>>>>>>>>>>>>>>> failure.
>>>>>>>>>>>>>>>>>>> + */
>>>>>>>>>>>>>>>>>>> +int
>>>>>>>>>>>>>>>>>>> +drm_gpuvm_prepare_range(struct drm_gpuvm *gpuvm,
>>>>>>>>>>>>>>>>>>> struct
>>>>>>>>>>>>>>>>>>> drm_exec *exec,
>>>>>>>>>>>>>>>>>>> + u64 addr, u64 range,
>>>>>>>>>>>>>>>>>>> unsigned int
>>>>>>>>>>>>>>>>>>> num_fences)
>>>>>>>>>>>>>>>>>>> +{
>>>>>>>>>>>>>>>>>>> + struct drm_gpuva *va;
>>>>>>>>>>>>>>>>>>> + u64 end = addr + range;
>>>>>>>>>>>>>>>>>>> + int ret;
>>>>>>>>>>>>>>>>>>> +
>>>>>>>>>>>>>>>>>>> + drm_gpuvm_for_each_va_range(va, gpuvm, addr,
>>>>>>>>>>>>>>>>>>> end) {
>>>>>>>>>>>>>>>>>>> + struct drm_gem_object *obj =
>>>>>>>>>>>>>>>>>>> va->gem.obj;
>>>>>>>>>>>>>>>>>>> +
>>>>>>>>>>>>>>>>>>> + ret = drm_exec_prepare_obj(exec, obj,
>>>>>>>>>>>>>>>>>>> num_fences);
>>>>>>>>>>>>>>>>>>> + if (ret)
>>>>>>>>>>>>>>>>>>> + return ret;
>>>>>>>>>>>>>>>>>>> + }
>>>>>>>>>>>>>>>>>>> +
>>>>>>>>>>>>>>>>>>> + return 0;
>>>>>>>>>>>>>>>>>>> +}
>>>>>>>>>>>>>>>>>>> +EXPORT_SYMBOL_GPL(drm_gpuvm_prepare_range);
>>>>>>>>>>>>>>>>>>> +
>>>>>>>>>>>>>>>>>>> +/**
>>>>>>>>>>>>>>>>>>> + * drm_gpuvm_exec_lock() - lock all dma-resv of all
>>>>>>>>>>>>>>>>>>> assoiciated BOs
>>>>>>>>>>>>>>>>>>> + * @vm_exec: the &drm_gpuvm_exec abstraction
>>>>>>>>>>>>>>>>>>> + * @num_fences: the amount of &dma_fences to reserve
>>>>>>>>>>>>>>>>>>> + * @interruptible: sleep interruptible if waiting
>>>>>>>>>>>>>>>>>>> + *
>>>>>>>>>>>>>>>>>>> + * Acquires all dma-resv locks of all
>>>>>>>>>>>>>>>>>>> &drm_gem_objects the
>>>>>>>>>>>>>>>>>>> given
>>>>>>>>>>>>>>>>>>> + * &drm_gpuvm contains mappings of.
>>>>>>>>>>>>>>>>>>> + *
>>>>>>>>>>>>>>>>>>> + * Addionally, when calling this function with struct
>>>>>>>>>>>>>>>>>>> drm_gpuvm_exec::extra
>>>>>>>>>>>>>>>>>>> + * being set the driver receives the given @fn
>>>>>>>>>>>>>>>>>>> callback to
>>>>>>>>>>>>>>>>>>> lock additional
>>>>>>>>>>>>>>>>>>> + * dma-resv in the context of the &drm_gpuvm_exec
>>>>>>>>>>>>>>>>>>> instance.
>>>>>>>>>>>>>>>>>>> Typically, drivers
>>>>>>>>>>>>>>>>>>> + * would call drm_exec_prepare_obj() from within this
>>>>>>>>>>>>>>>>>>> callback.
>>>>>>>>>>>>>>>>>>> + *
>>>>>>>>>>>>>>>>>>> + * Returns: 0 on success, negative error code on
>>>>>>>>>>>>>>>>>>> failure.
>>>>>>>>>>>>>>>>>>> + */
>>>>>>>>>>>>>>>>>>> +int
>>>>>>>>>>>>>>>>>>> +drm_gpuvm_exec_lock(struct drm_gpuvm_exec *vm_exec,
>>>>>>>>>>>>>>>>>>> + unsigned int num_fences,
>>>>>>>>>>>>>>>>>>> + bool interruptible)
>>>>>>>>>>>>>>>>>>> +{
>>>>>>>>>>>>>>>>>>> + struct drm_gpuvm *gpuvm = vm_exec->vm;
>>>>>>>>>>>>>>>>>>> + struct drm_exec *exec = &vm_exec->exec;
>>>>>>>>>>>>>>>>>>> + uint32_t flags;
>>>>>>>>>>>>>>>>>>> + int ret;
>>>>>>>>>>>>>>>>>>> +
>>>>>>>>>>>>>>>>>>> + flags = interruptible ?
>>>>>>>>>>>>>>>>>>> DRM_EXEC_INTERRUPTIBLE_WAIT :
>>>>>>>>>>>>>>>>>>> 0 |
>>>>>>>>>>>>>>>>>>> + DRM_EXEC_IGNORE_DUPLICATES;
>>>>>>>>>>>>>>>>>>> +
>>>>>>>>>>>>>>>>>>> + drm_exec_init(exec, flags);
>>>>>>>>>>>>>>>>>>> +
>>>>>>>>>>>>>>>>>>> + drm_exec_until_all_locked(exec) {
>>>>>>>>>>>>>>>>>>> + ret = drm_gpuvm_prepare_vm(gpuvm, exec,
>>>>>>>>>>>>>>>>>>> num_fences);
>>>>>>>>>>>>>>>>>>> + drm_exec_retry_on_contention(exec);
>>>>>>>>>>>>>>>>>>> + if (ret)
>>>>>>>>>>>>>>>>>>> + goto err;
>>>>>>>>>>>>>>>>>>> +
>>>>>>>>>>>>>>>>>>> + ret =
>>>>>>>>>>>>>>>>>>> drm_gpuvm_prepare_objects(gpuvm, exec,
>>>>>>>>>>>>>>>>>>> num_fences);
>>>>>>>>>>>>>>>>>>> + drm_exec_retry_on_contention(exec);
>>>>>>>>>>>>>>>>>>> + if (ret)
>>>>>>>>>>>>>>>>>>> + goto err;
>>>>>>>>>>>>>>>>>>> +
>>>>>>>>>>>>>>>>>>> + if (vm_exec->extra.fn) {
>>>>>>>>>>>>>>>>>>> + ret =
>>>>>>>>>>>>>>>>>>> vm_exec->extra.fn(vm_exec,
>>>>>>>>>>>>>>>>>>> num_fences);
>>>>>>>>>>>>>>>>>>> + drm_exec_retry_on_contention(exec);
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> + if (ret)
>>>>>>>>>>>>>>>>>>> + goto err;
>>>>>>>>>>>>>>>>>>> + }
>>>>>>>>>>>>>>>>>>> + }
>>>>>>>>>>>>>>>>>>> +
>>>>>>>>>>>>>>>>>>> + return 0;
>>>>>>>>>>>>>>>>>>> +
>>>>>>>>>>>>>>>>>>> +err:
>>>>>>>>>>>>>>>>>>> + drm_exec_fini(exec);
>>>>>>>>>>>>>>>>>>> + return ret;
>>>>>>>>>>>>>>>>>>> +}
>>>>>>>>>>>>>>>>>>> +EXPORT_SYMBOL_GPL(drm_gpuvm_exec_lock);
>>>>>>>>>>>>>>>>>>> +
>>>>>>>>>>>>>>>>>>> +static int
>>>>>>>>>>>>>>>>>>> +fn_lock_array(struct drm_gpuvm_exec *vm_exec,
>>>>>>>>>>>>>>>>>>> unsigned int
>>>>>>>>>>>>>>>>>>> num_fences)
>>>>>>>>>>>>>>>>>>> +{
>>>>>>>>>>>>>>>>>>> + struct {
>>>>>>>>>>>>>>>>>>> + struct drm_gem_object **objs;
>>>>>>>>>>>>>>>>>>> + unsigned int num_objs;
>>>>>>>>>>>>>>>>>>> + } *args = vm_exec->extra.priv;
>>>>>>>>>>>>>>>>>>> +
>>>>>>>>>>>>>>>>>>> + return
>>>>>>>>>>>>>>>>>>> drm_exec_prepare_array(&vm_exec->exec, args-
>>>>>>>>>>>>>>>>>>>> objs,
>>>>>>>>>>>>>>>>>>> + args->num_objs,
>>>>>>>>>>>>>>>>>>> num_fences);
>>>>>>>>>>>>>>>>>>> +}
>>>>>>>>>>>>>>>>>>> +
>>>>>>>>>>>>>>>>>>> +/**
>>>>>>>>>>>>>>>>>>> + * drm_gpuvm_exec_lock_array() - lock all dma-resv
>>>>>>>>>>>>>>>>>>> of all
>>>>>>>>>>>>>>>>>>> assoiciated BOs
>>>>>>>>>>>>>>>>>>> + * @vm_exec: the &drm_gpuvm_exec abstraction
>>>>>>>>>>>>>>>>>>> + * @objs: additional &drm_gem_objects to lock
>>>>>>>>>>>>>>>>>>> + * @num_objs: the number of additional
>>>>>>>>>>>>>>>>>>> &drm_gem_objects to
>>>>>>>>>>>>>>>>>>> lock
>>>>>>>>>>>>>>>>>>> + * @num_fences: the amount of &dma_fences to reserve
>>>>>>>>>>>>>>>>>>> + * @interruptible: sleep interruptible if waiting
>>>>>>>>>>>>>>>>>>> + *
>>>>>>>>>>>>>>>>>>> + * Acquires all dma-resv locks of all
>>>>>>>>>>>>>>>>>>> &drm_gem_objects the
>>>>>>>>>>>>>>>>>>> given &drm_gpuvm
>>>>>>>>>>>>>>>>>>> + * contains mappings of, plus the ones given
>>>>>>>>>>>>>>>>>>> through @objs.
>>>>>>>>>>>>>>>>>>> + *
>>>>>>>>>>>>>>>>>>> + * Returns: 0 on success, negative error code on
>>>>>>>>>>>>>>>>>>> failure.
>>>>>>>>>>>>>>>>>>> + */
>>>>>>>>>>>>>>>>>>> +int
>>>>>>>>>>>>>>>>>>> +drm_gpuvm_exec_lock_array(struct drm_gpuvm_exec
>>>>>>>>>>>>>>>>>>> *vm_exec,
>>>>>>>>>>>>>>>>>>> + struct drm_gem_object **objs,
>>>>>>>>>>>>>>>>>>> + unsigned int num_objs,
>>>>>>>>>>>>>>>>>>> + unsigned int num_fences,
>>>>>>>>>>>>>>>>>>> + bool interruptible)
>>>>>>>>>>>>>>>>>>> +{
>>>>>>>>>>>>>>>>>>> + struct {
>>>>>>>>>>>>>>>>>>> + struct drm_gem_object **objs;
>>>>>>>>>>>>>>>>>>> + unsigned int num_objs;
>>>>>>>>>>>>>>>>>>> + } args;
>>>>>>>>>>>>>>>>>>> +
>>>>>>>>>>>>>>>>>>> + args.objs = objs;
>>>>>>>>>>>>>>>>>>> + args.num_objs = num_objs;
>>>>>>>>>>>>>>>>>>> +
>>>>>>>>>>>>>>>>>>> + vm_exec->extra.fn = fn_lock_array;
>>>>>>>>>>>>>>>>>>> + vm_exec->extra.priv = &args;
>>>>>>>>>>>>>>>>>>> +
>>>>>>>>>>>>>>>>>>> + return drm_gpuvm_exec_lock(vm_exec, num_fences,
>>>>>>>>>>>>>>>>>>> interruptible);
>>>>>>>>>>>>>>>>>>> +}
>>>>>>>>>>>>>>>>>>> +EXPORT_SYMBOL_GPL(drm_gpuvm_exec_lock_array);
>>>>>>>>>>>>>>>>>>> +
>>>>>>>>>>>>>>>>>>> +/**
>>>>>>>>>>>>>>>>>>> + * drm_gpuvm_exec_lock_range() - prepare all BOs
>>>>>>>>>>>>>>>>>>> mapped
>>>>>>>>>>>>>>>>>>> within a given range
>>>>>>>>>>>>>>>>>>> + * @vm_exec: the &drm_gpuvm_exec abstraction
>>>>>>>>>>>>>>>>>>> + * @addr: the start address within the VA space
>>>>>>>>>>>>>>>>>>> + * @range: the range to iterate within the VA space
>>>>>>>>>>>>>>>>>>> + * @num_fences: the amount of &dma_fences to reserve
>>>>>>>>>>>>>>>>>>> + * @interruptible: sleep interruptible if waiting
>>>>>>>>>>>>>>>>>>> + *
>>>>>>>>>>>>>>>>>>> + * Acquires all dma-resv locks of all &drm_gem_objects
>>>>>>>>>>>>>>>>>>> mapped between @addr and
>>>>>>>>>>>>>>>>>>> + * @addr + @range.
>>>>>>>>>>>>>>>>>>> + *
>>>>>>>>>>>>>>>>>>> + * Returns: 0 on success, negative error code on
>>>>>>>>>>>>>>>>>>> failure.
>>>>>>>>>>>>>>>>>>> + */
>>>>>>>>>>>>>>>>>>> +int
>>>>>>>>>>>>>>>>>>> +drm_gpuvm_exec_lock_range(struct drm_gpuvm_exec
>>>>>>>>>>>>>>>>>>> *vm_exec,
>>>>>>>>>>>>>>>>>>> + u64 addr, u64 range,
>>>>>>>>>>>>>>>>>>> + unsigned int num_fences,
>>>>>>>>>>>>>>>>>>> + bool interruptible)
>>>>>>>>>>>>>>>>>>> +{
>>>>>>>>>>>>>>>>>>> + struct drm_gpuvm *gpuvm = vm_exec->vm;
>>>>>>>>>>>>>>>>>>> + struct drm_exec *exec = &vm_exec->exec;
>>>>>>>>>>>>>>>>>>> + uint32_t flags;
>>>>>>>>>>>>>>>>>>> + int ret;
>>>>>>>>>>>>>>>>>>> +
>>>>>>>>>>>>>>>>>>> + flags = interruptible ?
>>>>>>>>>>>>>>>>>>> DRM_EXEC_INTERRUPTIBLE_WAIT :
>>>>>>>>>>>>>>>>>>> 0 |
>>>>>>>>>>>>>>>>>>> + DRM_EXEC_IGNORE_DUPLICATES;
>>>>>>>>>>>>>>>>>>> +
>>>>>>>>>>>>>>>>>>> + drm_exec_init(exec, flags);
>>>>>>>>>>>>>>>>>>> +
>>>>>>>>>>>>>>>>>>> + drm_exec_until_all_locked(exec) {
>>>>>>>>>>>>>>>>>>> + ret = drm_gpuvm_prepare_range(gpuvm,
>>>>>>>>>>>>>>>>>>> exec,
>>>>>>>>>>>>>>>>>>> addr, range,
>>>>>>>>>>>>>>>>>>> + num_fences);
>>>>>>>>>>>>>>>>>>> + drm_exec_retry_on_contention(exec);
>>>>>>>>>>>>>>>>>>> + if (ret)
>>>>>>>>>>>>>>>>>>> + goto err;
>>>>>>>>>>>>>>>>>>> + }
>>>>>>>>>>>>>>>>>>> +
>>>>>>>>>>>>>>>>>>> + return ret;
>>>>>>>>>>>>>>>>>>> +
>>>>>>>>>>>>>>>>>>> +err:
>>>>>>>>>>>>>>>>>>> + drm_exec_fini(exec);
>>>>>>>>>>>>>>>>>>> + return ret;
>>>>>>>>>>>>>>>>>>> +}
>>>>>>>>>>>>>>>>>>> +EXPORT_SYMBOL_GPL(drm_gpuvm_exec_lock_range);
>>>>>>>>>>>>>>>>>>> +
>>>>>>>>>>>>>>>>>>> +/**
>>>>>>>>>>>>>>>>>>> + * drm_gpuvm_validate() - validate all BOs marked
>>>>>>>>>>>>>>>>>>> as evicted
>>>>>>>>>>>>>>>>>>> + * @gpuvm: the &drm_gpuvm to validate evicted BOs
>>>>>>>>>>>>>>>>>>> + *
>>>>>>>>>>>>>>>>>>> + * Calls the &drm_gpuvm_ops.bo_validate callback
>>>>>>>>>>>>>>>>>>> for all
>>>>>>>>>>>>>>>>>>> evicted buffer
>>>>>>>>>>>>>>>>>>> + * objects being mapped in the given &drm_gpuvm.
>>>>>>>>>>>>>>>>>>> + *
>>>>>>>>>>>>>>>>>>> + * Returns: 0 on success, negative error code on
>>>>>>>>>>>>>>>>>>> failure.
>>>>>>>>>>>>>>>>>>> + */
>>>>>>>>>>>>>>>>>>> +int
>>>>>>>>>>>>>>>>>>> +drm_gpuvm_validate(struct drm_gpuvm *gpuvm)
>>>>>>>>>>>>>>>>>>> +{
>>>>>>>>>>>>>>>>>>> + const struct drm_gpuvm_ops *ops = gpuvm->ops;
>>>>>>>>>>>>>>>>>>> + struct drm_gpuvm_bo *vm_bo;
>>>>>>>>>>>>>>>>>>> + LIST_HEAD(evict);
>>>>>>>>>>>>>>>>>>> + int ret = 0;
>>>>>>>>>>>>>>>>>>> +
>>>>>>>>>>>>>>>>>>> + if (unlikely(!ops || !ops->bo_validate))
>>>>>>>>>>>>>>>>>>> + return -ENOTSUPP;
>>>>>>>>>>>>>>>>>>> +
>>>>>>>>>>>>>>>>>>> + for_each_vm_bo_in_list(gpuvm, evict, &evict,
>>>>>>>>>>>>>>>>>>> vm_bo) {
>>>>>>>>>>>>>>>>>>> + dma_resv_assert_held(vm_bo->obj->resv);
>>>>>>>>>>>>>>>>>>> + ret = ops->bo_validate(vm_bo->obj);
>>>>>>>>>>>>>>>>>>> + if (ret)
>>>>>>>>>>>>>>>>>>> + break;
>>>>>>>>>>>>>>>>>>> + }
>>>>>>>>>>>>>>>>>>> + /* Drop ref in case we break out of the
>>>>>>>>>>>>>>>>>>> loop. */
>>>>>>>>>>>>>>>>>>> + drm_gpuvm_bo_put(vm_bo);
>>>>>>>>>>>>>>>>>>> + restore_vm_bo_list(gpuvm, evict, &evict);
>>>>>>>>>>>>>>>>>>> +
>>>>>>>>>>>>>>>>>>> + return ret;
>>>>>>>>>>>>>>>>>>> +}
>>>>>>>>>>>>>>>>>>> +EXPORT_SYMBOL_GPL(drm_gpuvm_validate);
>>>>>>>>>>>>>>>>>>> +
>>>>>>>>>>>>>>>>>>> +/**
>>>>>>>>>>>>>>>>>>> + * drm_gpuvm_resv_add_fence - add fence to private
>>>>>>>>>>>>>>>>>>> and all
>>>>>>>>>>>>>>>>>>> extobj
>>>>>>>>>>>>>>>>>>> + * dma-resv
>>>>>>>>>>>>>>>>>>> + * @gpuvm: the &drm_gpuvm to add a fence to
>>>>>>>>>>>>>>>>>>> + * @exec: the &drm_exec locking context
>>>>>>>>>>>>>>>>>>> + * @fence: fence to add
>>>>>>>>>>>>>>>>>>> + * @private_usage: private dma-resv usage
>>>>>>>>>>>>>>>>>>> + * @extobj_usage: extobj dma-resv usage
>>>>>>>>>>>>>>>>>>> + */
>>>>>>>>>>>>>>>>>>> +void
>>>>>>>>>>>>>>>>>>> +drm_gpuvm_resv_add_fence(struct drm_gpuvm *gpuvm,
>>>>>>>>>>>>>>>>>>> + struct drm_exec *exec,
>>>>>>>>>>>>>>>>>>> + struct dma_fence *fence,
>>>>>>>>>>>>>>>>>>> + enum dma_resv_usage
>>>>>>>>>>>>>>>>>>> private_usage,
>>>>>>>>>>>>>>>>>>> + enum dma_resv_usage
>>>>>>>>>>>>>>>>>>> extobj_usage)
>>>>>>>>>>>>>>>>>>> +{
>>>>>>>>>>>>>>>>>>> + struct drm_gem_object *obj;
>>>>>>>>>>>>>>>>>>> + unsigned long index;
>>>>>>>>>>>>>>>>>>> +
>>>>>>>>>>>>>>>>>>> + drm_exec_for_each_locked_object(exec, index,
>>>>>>>>>>>>>>>>>>> obj) {
>>>>>>>>>>>>>>>>>>> + dma_resv_assert_held(obj->resv);
>>>>>>>>>>>>>>>>>>> + dma_resv_add_fence(obj->resv, fence,
>>>>>>>>>>>>>>>>>>> + drm_gpuvm_is_extobj(gpuvm,
>>>>>>>>>>>>>>>>>>> obj) ?
>>>>>>>>>>>>>>>>>>> + private_usage :
>>>>>>>>>>>>>>>>>>> extobj_usage);
>>>>>>>>>>>>>>>>>>> + }
>>>>>>>>>>>>>>>>>>> +}
>>>>>>>>>>>>>>>>>>> +EXPORT_SYMBOL_GPL(drm_gpuvm_resv_add_fence);
>>>>>>>>>>>>>>>>>>> +
>>>>>>>>>>>>>>>>>>> /**
>>>>>>>>>>>>>>>>>>> * drm_gpuvm_bo_create() - create a new instance
>>>>>>>>>>>>>>>>>>> of struct
>>>>>>>>>>>>>>>>>>> drm_gpuvm_bo
>>>>>>>>>>>>>>>>>>> * @gpuvm: The &drm_gpuvm the @obj is mapped in.
>>>>>>>>>>>>>>>>>>> @@ -790,6 +1242,9 @@ drm_gpuvm_bo_create(struct
>>>>>>>>>>>>>>>>>>> drm_gpuvm
>>>>>>>>>>>>>>>>>>> *gpuvm,
>>>>>>>>>>>>>>>>>>> INIT_LIST_HEAD(&vm_bo->list.gpuva);
>>>>>>>>>>>>>>>>>>> INIT_LIST_HEAD(&vm_bo->list.entry.gem);
>>>>>>>>>>>>>>>>>>> + INIT_LIST_HEAD(&vm_bo->list.entry.extobj);
>>>>>>>>>>>>>>>>>>> + INIT_LIST_HEAD(&vm_bo->list.entry.evict);
>>>>>>>>>>>>>>>>>>> +
>>>>>>>>>>>>>>>>>>> drm_gem_object_get(obj);
>>>>>>>>>>>>>>>>>>> return vm_bo;
>>>>>>>>>>>>>>>>>>> @@ -807,6 +1262,14 @@ drm_gpuvm_bo_destroy(struct
>>>>>>>>>>>>>>>>>>> kref *kref)
>>>>>>>>>>>>>>>>>>> drm_gem_gpuva_assert_lock_held(vm_bo->obj);
>>>>>>>>>>>>>>>>>>> + spin_lock(&gpuvm->extobj.lock);
>>>>>>>>>>>>>>>>>>> + list_del(&vm_bo->list.entry.extobj);
>>>>>>>>>>>>>>>>>>> + spin_unlock(&gpuvm->extobj.lock);
>>>>>>>>>>>>>>>>>>> +
>>>>>>>>>>>>>>>>>>> + spin_lock(&gpuvm->evict.lock);
>>>>>>>>>>>>>>>>>>> + list_del(&vm_bo->list.entry.evict);
>>>>>>>>>>>>>>>>>>> + spin_unlock(&gpuvm->evict.lock);
>>>>>>>>>>>>>>>>>>> +
>>>>>>>>>>>>>>>>>>> list_del(&vm_bo->list.entry.gem);
>>>>>>>>>>>>>>>>>>> drm_gem_object_put(obj);
>>>>>>>>>>>>>>>>>>> @@ -822,6 +1285,11 @@ drm_gpuvm_bo_destroy(struct
>>>>>>>>>>>>>>>>>>> kref *kref)
>>>>>>>>>>>>>>>>>>> * @vm_bo: the &drm_gpuvm_bo to release the
>>>>>>>>>>>>>>>>>>> reference of
>>>>>>>>>>>>>>>>>>> *
>>>>>>>>>>>>>>>>>>> * This releases a reference to @vm_bo.
>>>>>>>>>>>>>>>>>>> + *
>>>>>>>>>>>>>>>>>>> + * If the reference count drops to zero, the
>>>>>>>>>>>>>>>>>>> &gpuvm_bo is
>>>>>>>>>>>>>>>>>>> destroyed, which
>>>>>>>>>>>>>>>>>>> + * includes removing it from the GEMs gpuva list.
>>>>>>>>>>>>>>>>>>> Hence, if
>>>>>>>>>>>>>>>>>>> a call to this
>>>>>>>>>>>>>>>>>>> + * function can potentially let the reference count
>>>>>>>>>>>>>>>>>>> to zero
>>>>>>>>>>>>>>>>>>> the caller must
>>>>>>>>>>>>>>>>>>> + * hold the dma-resv or driver specific GEM gpuva
>>>>>>>>>>>>>>>>>>> lock.
>>>>>>>>>>>>>>>>>>> */
>>>>>>>>>>>>>>>>>>> void
>>>>>>>>>>>>>>>>>>> drm_gpuvm_bo_put(struct drm_gpuvm_bo *vm_bo)
>>>>>>>>>>>>>>>>>>> @@ -831,6 +1299,12 @@ drm_gpuvm_bo_put(struct
>>>>>>>>>>>>>>>>>>> drm_gpuvm_bo
>>>>>>>>>>>>>>>>>>> *vm_bo)
>>>>>>>>>>>>>>>>>>> }
>>>>>>>>>>>>>>>>>>> EXPORT_SYMBOL_GPL(drm_gpuvm_bo_put);
>>>>>>>>>>>>>>>>>>> +static int __must_check
>>>>>>>>>>>>>>>>>>> +drm_gpuvm_bo_get_unless_zero(struct drm_gpuvm_bo
>>>>>>>>>>>>>>>>>>> *vm_bo)
>>>>>>>>>>>>>>>>>>> +{
>>>>>>>>>>>>>>>>>>> + return kref_get_unless_zero(&vm_bo->kref);
>>>>>>>>>>>>>>>>>>> +}
>>>>>>>>>>>>>>>>>>> +
>>>>>>>>>>>>>>>>>>> static struct drm_gpuvm_bo *
>>>>>>>>>>>>>>>>>>> __drm_gpuvm_bo_find(struct drm_gpuvm *gpuvm,
>>>>>>>>>>>>>>>>>>> struct drm_gem_object *obj)
>>>>>>>>>>>>>>>>>>> @@ -938,6 +1412,48 @@
>>>>>>>>>>>>>>>>>>> drm_gpuvm_bo_obtain_prealloc(struct
>>>>>>>>>>>>>>>>>>> drm_gpuvm_bo *__vm_bo)
>>>>>>>>>>>>>>>>>>> }
>>>>>>>>>>>>>>>>>>> EXPORT_SYMBOL_GPL(drm_gpuvm_bo_obtain_prealloc);
>>>>>>>>>>>>>>>>>>> +/**
>>>>>>>>>>>>>>>>>>> + * drm_gpuvm_bo_extobj_add() - adds the
>>>>>>>>>>>>>>>>>>> &drm_gpuvm_bo to its
>>>>>>>>>>>>>>>>>>> &drm_gpuvm's
>>>>>>>>>>>>>>>>>>> + * extobj list
>>>>>>>>>>>>>>>>>>> + * @vm_bo: The &drm_gpuvm_bo to add to its
>>>>>>>>>>>>>>>>>>> &drm_gpuvm's the
>>>>>>>>>>>>>>>>>>> extobj list.
>>>>>>>>>>>>>>>>>>> + *
>>>>>>>>>>>>>>>>>>> + * Adds the given @vm_bo to its &drm_gpuvm's extobj
>>>>>>>>>>>>>>>>>>> list if
>>>>>>>>>>>>>>>>>>> not on the list
>>>>>>>>>>>>>>>>>>> + * already and if the corresponding &drm_gem_object
>>>>>>>>>>>>>>>>>>> is an
>>>>>>>>>>>>>>>>>>> external object,
>>>>>>>>>>>>>>>>>>> + * actually.
>>>>>>>>>>>>>>>>>>> + */
>>>>>>>>>>>>>>>>>>> +void
>>>>>>>>>>>>>>>>>>> +drm_gpuvm_bo_extobj_add(struct drm_gpuvm_bo *vm_bo)
>>>>>>>>>>>>>>>>>>> +{
>>>>>>>>>>>>>>>>>>> + struct drm_gpuvm *gpuvm = vm_bo->vm;
>>>>>>>>>>>>>>>>>>> +
>>>>>>>>>>>>>>>>>>> + if (drm_gpuvm_is_extobj(gpuvm, vm_bo->obj))
>>>>>>>>>>>>>>>>>>> + drm_gpuvm_bo_list_add(vm_bo, extobj);
>>>>>>>>>>>>>>>>>>> +}
>>>>>>>>>>>>>>>>>>> +EXPORT_SYMBOL_GPL(drm_gpuvm_bo_extobj_add);
>>>>>>>>>>>>>>>>>>> +
>>>>>>>>>>>>>>>>>>> +/**
>>>>>>>>>>>>>>>>>>> + * drm_gpuvm_bo_evict() - add / remove a
>>>>>>>>>>>>>>>>>>> &drm_gem_object to
>>>>>>>>>>>>>>>>>>> / from a
>>>>>>>>>>>>>>>>>>> + * &drm_gpuvms evicted list
>>>>>>>>>>>>>>>>>>> + * @obj: the &drm_gem_object to add or remove
>>>>>>>>>>>>>>>>>>> + * @evict: indicates whether the object is evicted
>>>>>>>>>>>>>>>>>>> + *
>>>>>>>>>>>>>>>>>>> + * Adds a &drm_gem_object to or removes it from all
>>>>>>>>>>>>>>>>>>> &drm_gpuvms evicted
>>>>>>>>>>>>>>>>>>> + * list containing a mapping of this &drm_gem_object.
>>>>>>>>>>>>>>>>>>> + */
>>>>>>>>>>>>>>>>>>> +void
>>>>>>>>>>>>>>>>>>> +drm_gpuvm_bo_evict(struct drm_gem_object *obj, bool
>>>>>>>>>>>>>>>>>>> evict)
>>>>>>>>>>>>>>>>>>> +{
>>>>>>>>>>>>>>>>>>> + struct drm_gpuvm_bo *vm_bo;
>>>>>>>>>>>>>>>>>>> +
>>>>>>>>>>>>>>>>>>> + drm_gem_for_each_gpuvm_bo(vm_bo, obj) {
>>>>>>>>>>>>>>>>>>> + if (evict)
>>>>>>>>>>>>>>>>>>> + drm_gpuvm_bo_list_add(vm_bo,
>>>>>>>>>>>>>>>>>>> evict);
>>>>>>>>>>>>>>>>>>> + else
>>>>>>>>>>>>>>>>>>> + drm_gpuvm_bo_list_del(vm_bo,
>>>>>>>>>>>>>>>>>>> evict);
>>>>>>>>>>>>>>>>>>> + }
>>>>>>>>>>>>>>>>>>> +}
>>>>>>>>>>>>>>>>>>> +EXPORT_SYMBOL_GPL(drm_gpuvm_bo_evict);
>>>>>>>>>>>>>>>>>>> +
>>>>>>>>>>>>>>>>>>> static int
>>>>>>>>>>>>>>>>>>> __drm_gpuva_insert(struct drm_gpuvm *gpuvm,
>>>>>>>>>>>>>>>>>>> struct drm_gpuva *va)
>>>>>>>>>>>>>>>>>>> diff --git a/include/drm/drm_gpuvm.h
>>>>>>>>>>>>>>>>>>> b/include/drm/drm_gpuvm.h
>>>>>>>>>>>>>>>>>>> index afa50b9059a2..834bb6d6617e 100644
>>>>>>>>>>>>>>>>>>> --- a/include/drm/drm_gpuvm.h
>>>>>>>>>>>>>>>>>>> +++ b/include/drm/drm_gpuvm.h
>>>>>>>>>>>>>>>>>>> @@ -26,10 +26,12 @@
>>>>>>>>>>>>>>>>>>> */
>>>>>>>>>>>>>>>>>>> #include <linux/list.h>
>>>>>>>>>>>>>>>>>>> +#include <linux/dma-resv.h>
>>>>>>>>>>>>>>>>>>> #include <linux/rbtree.h>
>>>>>>>>>>>>>>>>>>> #include <linux/types.h>
>>>>>>>>>>>>>>>>>>> #include <drm/drm_gem.h>
>>>>>>>>>>>>>>>>>>> +#include <drm/drm_exec.h>
>>>>>>>>>>>>>>>>>>> struct drm_gpuvm;
>>>>>>>>>>>>>>>>>>> struct drm_gpuvm_bo;
>>>>>>>>>>>>>>>>>>> @@ -259,6 +261,38 @@ struct drm_gpuvm {
>>>>>>>>>>>>>>>>>>> * space
>>>>>>>>>>>>>>>>>>> */
>>>>>>>>>>>>>>>>>>> struct dma_resv *resv;
>>>>>>>>>>>>>>>>>>> +
>>>>>>>>>>>>>>>>>>> + /**
>>>>>>>>>>>>>>>>>>> + * @extobj: structure holding the extobj list
>>>>>>>>>>>>>>>>>>> + */
>>>>>>>>>>>>>>>>>>> + struct {
>>>>>>>>>>>>>>>>>>> + /**
>>>>>>>>>>>>>>>>>>> + * @list: &list_head storing
>>>>>>>>>>>>>>>>>>> &drm_gpuvm_bos
>>>>>>>>>>>>>>>>>>> serving as
>>>>>>>>>>>>>>>>>>> + * external object
>>>>>>>>>>>>>>>>>>> + */
>>>>>>>>>>>>>>>>>>> + struct list_head list;
>>>>>>>>>>>>>>>>>>> +
>>>>>>>>>>>>>>>>>>> + /**
>>>>>>>>>>>>>>>>>>> + * @lock: spinlock to protect the
>>>>>>>>>>>>>>>>>>> extobj list
>>>>>>>>>>>>>>>>>>> + */
>>>>>>>>>>>>>>>>>>> + spinlock_t lock;
>>>>>>>>>>>>>>>>>>> + } extobj;
>>>>>>>>>>>>>>>>>>> +
>>>>>>>>>>>>>>>>>>> + /**
>>>>>>>>>>>>>>>>>>> + * @evict: structure holding the evict list
>>>>>>>>>>>>>>>>>>> and evict
>>>>>>>>>>>>>>>>>>> list lock
>>>>>>>>>>>>>>>>>>> + */
>>>>>>>>>>>>>>>>>>> + struct {
>>>>>>>>>>>>>>>>>>> + /**
>>>>>>>>>>>>>>>>>>> + * @list: &list_head storing
>>>>>>>>>>>>>>>>>>> &drm_gpuvm_bos
>>>>>>>>>>>>>>>>>>> currently being
>>>>>>>>>>>>>>>>>>> + * evicted
>>>>>>>>>>>>>>>>>>> + */
>>>>>>>>>>>>>>>>>>> + struct list_head list;
>>>>>>>>>>>>>>>>>>> +
>>>>>>>>>>>>>>>>>>> + /**
>>>>>>>>>>>>>>>>>>> + * @lock: spinlock to protect the
>>>>>>>>>>>>>>>>>>> evict list
>>>>>>>>>>>>>>>>>>> + */
>>>>>>>>>>>>>>>>>>> + spinlock_t lock;
>>>>>>>>>>>>>>>>>>> + } evict;
>>>>>>>>>>>>>>>>>>> };
>>>>>>>>>>>>>>>>>>> void drm_gpuvm_init(struct drm_gpuvm *gpuvm, struct
>>>>>>>>>>>>>>>>>>> drm_device *drm,
>>>>>>>>>>>>>>>>>>> @@ -268,6 +302,21 @@ void drm_gpuvm_init(struct
>>>>>>>>>>>>>>>>>>> drm_gpuvm
>>>>>>>>>>>>>>>>>>> *gpuvm, struct drm_device *drm,
>>>>>>>>>>>>>>>>>>> const struct drm_gpuvm_ops *ops);
>>>>>>>>>>>>>>>>>>> void drm_gpuvm_destroy(struct drm_gpuvm *gpuvm);
>>>>>>>>>>>>>>>>>>> +/**
>>>>>>>>>>>>>>>>>>> + * drm_gpuvm_is_extobj() - indicates whether the given
>>>>>>>>>>>>>>>>>>> &drm_gem_object is an
>>>>>>>>>>>>>>>>>>> + * external object
>>>>>>>>>>>>>>>>>>> + * @gpuvm: the &drm_gpuvm to check
>>>>>>>>>>>>>>>>>>> + * @obj: the &drm_gem_object to check
>>>>>>>>>>>>>>>>>>> + *
>>>>>>>>>>>>>>>>>>> + * Returns: true if the &drm_gem_object &dma_resv
>>>>>>>>>>>>>>>>>>> differs
>>>>>>>>>>>>>>>>>>> from the
>>>>>>>>>>>>>>>>>>> + * &drm_gpuvms &dma_resv, false otherwise
>>>>>>>>>>>>>>>>>>> + */
>>>>>>>>>>>>>>>>>>> +static inline bool drm_gpuvm_is_extobj(struct
>>>>>>>>>>>>>>>>>>> drm_gpuvm
>>>>>>>>>>>>>>>>>>> *gpuvm,
>>>>>>>>>>>>>>>>>>> + struct drm_gem_object
>>>>>>>>>>>>>>>>>>> *obj)
>>>>>>>>>>>>>>>>>>> +{
>>>>>>>>>>>>>>>>>>> + return obj && obj->resv != gpuvm->resv;
>>>>>>>>>>>>>>>>>>> +}
>>>>>>>>>>>>>>>>>>> +
>>>>>>>>>>>>>>>>>>> static inline struct drm_gpuva *
>>>>>>>>>>>>>>>>>>> __drm_gpuva_next(struct drm_gpuva *va)
>>>>>>>>>>>>>>>>>>> {
>>>>>>>>>>>>>>>>>>> @@ -346,6 +395,128 @@ __drm_gpuva_next(struct
>>>>>>>>>>>>>>>>>>> drm_gpuva *va)
>>>>>>>>>>>>>>>>>>> #define drm_gpuvm_for_each_va_safe(va__, next__,
>>>>>>>>>>>>>>>>>>> gpuvm__)
>>>>>>>>>>>>>>>>>>> \
>>>>>>>>>>>>>>>>>>> list_for_each_entry_safe(va__, next__,
>>>>>>>>>>>>>>>>>>> &(gpuvm__)-
>>>>>>>>>>>>>>>>>>>> rb.list, rb.entry)
>>>>>>>>>>>>>>>>>>> +/**
>>>>>>>>>>>>>>>>>>> + * struct drm_gpuvm_exec - &drm_gpuvm abstraction of
>>>>>>>>>>>>>>>>>>> &drm_exec
>>>>>>>>>>>>>>>>>>> + *
>>>>>>>>>>>>>>>>>>> + * This structure should be created on the stack as
>>>>>>>>>>>>>>>>>>> &drm_exec should be.
>>>>>>>>>>>>>>>>>>> + *
>>>>>>>>>>>>>>>>>>> + * Optionally, @extra can be set in order to lock
>>>>>>>>>>>>>>>>>>> additional
>>>>>>>>>>>>>>>>>>> &drm_gem_objects.
>>>>>>>>>>>>>>>>>>> + */
>>>>>>>>>>>>>>>>>>> +struct drm_gpuvm_exec {
>>>>>>>>>>>>>>>>>>> + /**
>>>>>>>>>>>>>>>>>>> + * @exec: the &drm_exec structure
>>>>>>>>>>>>>>>>>>> + */
>>>>>>>>>>>>>>>>>>> + struct drm_exec exec;
>>>>>>>>>>>>>>>>>>> +
>>>>>>>>>>>>>>>>>>> + /**
>>>>>>>>>>>>>>>>>>> + * @vm: the &drm_gpuvm to lock its DMA
>>>>>>>>>>>>>>>>>>> reservations
>>>>>>>>>>>>>>>>>>> + */
>>>>>>>>>>>>>>>>>>> + struct drm_gpuvm *vm;
>>>>>>>>>>>>>>>>>>> +
>>>>>>>>>>>>>>>>>>> + /**
>>>>>>>>>>>>>>>>>>> + * @extra: Callback and corresponding
>>>>>>>>>>>>>>>>>>> private data
>>>>>>>>>>>>>>>>>>> for the driver to
>>>>>>>>>>>>>>>>>>> + * lock arbitrary additional &drm_gem_objects.
>>>>>>>>>>>>>>>>>>> + */
>>>>>>>>>>>>>>>>>>> + struct {
>>>>>>>>>>>>>>>>>>> + /**
>>>>>>>>>>>>>>>>>>> + * @fn: The driver callback to lock
>>>>>>>>>>>>>>>>>>> additional &drm_gem_objects.
>>>>>>>>>>>>>>>>>>> + */
>>>>>>>>>>>>>>>>>>> + int (*fn)(struct drm_gpuvm_exec
>>>>>>>>>>>>>>>>>>> *vm_exec,
>>>>>>>>>>>>>>>>>>> + unsigned int num_fences);
>>>>>>>>>>>>>>>>>>> +
>>>>>>>>>>>>>>>>>>> + /**
>>>>>>>>>>>>>>>>>>> + * @priv: driver private data for
>>>>>>>>>>>>>>>>>>> the @fn
>>>>>>>>>>>>>>>>>>> callback
>>>>>>>>>>>>>>>>>>> + */
>>>>>>>>>>>>>>>>>>> + void *priv;
>>>>>>>>>>>>>>>>>>> + } extra;
>>>>>>>>>>>>>>>>>>> +};
>>>>>>>>>>>>>>>>>>> +
>>>>>>>>>>>>>>>>>>> +/**
>>>>>>>>>>>>>>>>>>> + * drm_gpuvm_prepare_vm() - prepare the GPUVMs
>>>>>>>>>>>>>>>>>>> common dma-
>>>>>>>>>>>>>>>>>>> resv
>>>>>>>>>>>>>>>>>>> + * @gpuvm: the &drm_gpuvm
>>>>>>>>>>>>>>>>>>> + * @exec: the &drm_exec context
>>>>>>>>>>>>>>>>>>> + * @num_fences: the amount of &dma_fences to reserve
>>>>>>>>>>>>>>>>>>> + *
>>>>>>>>>>>>>>>>>>> + * Calls drm_exec_prepare_obj() for the GPUVMs dummy
>>>>>>>>>>>>>>>>>>> &drm_gem_object.
>>>>>>>>>>>>>>>>>>> + *
>>>>>>>>>>>>>>>>>>> + * Using this function directly, it is the drivers
>>>>>>>>>>>>>>>>>>> responsibility to call
>>>>>>>>>>>>>>>>>>> + * drm_exec_init() and drm_exec_fini() accordingly.
>>>>>>>>>>>>>>>>>>> + *
>>>>>>>>>>>>>>>>>>> + * Returns: 0 on success, negative error code on
>>>>>>>>>>>>>>>>>>> failure.
>>>>>>>>>>>>>>>>>>> + */
>>>>>>>>>>>>>>>>>>> +static inline int
>>>>>>>>>>>>>>>>>>> +drm_gpuvm_prepare_vm(struct drm_gpuvm *gpuvm,
>>>>>>>>>>>>>>>>>>> + struct drm_exec *exec,
>>>>>>>>>>>>>>>>>>> + unsigned int num_fences)
>>>>>>>>>>>>>>>>>>> +{
>>>>>>>>>>>>>>>>>>> + return drm_exec_prepare_obj(exec,
>>>>>>>>>>>>>>>>>>> &gpuvm->d_obj,
>>>>>>>>>>>>>>>>>>> num_fences);
>>>>>>>>>>>>>>>>>>> +}
>>>>>>>>>>>>>>>>>>> +
>>>>>>>>>>>>>>>>>>> +int drm_gpuvm_prepare_objects(struct drm_gpuvm *gpuvm,
>>>>>>>>>>>>>>>>>>> + struct drm_exec *exec,
>>>>>>>>>>>>>>>>>>> + unsigned int num_fences);
>>>>>>>>>>>>>>>>>>> +
>>>>>>>>>>>>>>>>>>> +int drm_gpuvm_prepare_range(struct drm_gpuvm *gpuvm,
>>>>>>>>>>>>>>>>>>> + struct drm_exec *exec,
>>>>>>>>>>>>>>>>>>> + u64 addr, u64 range,
>>>>>>>>>>>>>>>>>>> + unsigned int num_fences);
>>>>>>>>>>>>>>>>>>> +
>>>>>>>>>>>>>>>>>>> +int drm_gpuvm_exec_lock(struct drm_gpuvm_exec
>>>>>>>>>>>>>>>>>>> *vm_exec,
>>>>>>>>>>>>>>>>>>> + unsigned int num_fences,
>>>>>>>>>>>>>>>>>>> + bool interruptible);
>>>>>>>>>>>>>>>>>>> +
>>>>>>>>>>>>>>>>>>> +int drm_gpuvm_exec_lock_array(struct drm_gpuvm_exec
>>>>>>>>>>>>>>>>>>> *vm_exec,
>>>>>>>>>>>>>>>>>>> + struct drm_gem_object **objs,
>>>>>>>>>>>>>>>>>>> + unsigned int num_objs,
>>>>>>>>>>>>>>>>>>> + unsigned int num_fences,
>>>>>>>>>>>>>>>>>>> + bool interruptible);
>>>>>>>>>>>>>>>>>>> +
>>>>>>>>>>>>>>>>>>> +int drm_gpuvm_exec_lock_range(struct drm_gpuvm_exec
>>>>>>>>>>>>>>>>>>> *vm_exec,
>>>>>>>>>>>>>>>>>>> + u64 addr, u64 range,
>>>>>>>>>>>>>>>>>>> + unsigned int num_fences,
>>>>>>>>>>>>>>>>>>> + bool interruptible);
>>>>>>>>>>>>>>>>>>> +
>>>>>>>>>>>>>>>>>>> +/**
>>>>>>>>>>>>>>>>>>> + * drm_gpuvm_lock() - lock all dma-resv of all
>>>>>>>>>>>>>>>>>>> assoiciated
>>>>>>>>>>>>>>>>>>> BOs
>>>>>>>>>>>>>>>>>>> + * @gpuvm: the &drm_gpuvm
>>>>>>>>>>>>>>>>>>> + *
>>>>>>>>>>>>>>>>>>> + * Releases all dma-resv locks of all &drm_gem_objects
>>>>>>>>>>>>>>>>>>> previously acquired
>>>>>>>>>>>>>>>>>>> + * through drm_gpuvm_lock() or its variants.
>>>>>>>>>>>>>>>>>>> + *
>>>>>>>>>>>>>>>>>>> + * Returns: 0 on success, negative error code on
>>>>>>>>>>>>>>>>>>> failure.
>>>>>>>>>>>>>>>>>>> + */
>>>>>>>>>>>>>>>>>>> +static inline void
>>>>>>>>>>>>>>>>>>> +drm_gpuvm_exec_unlock(struct drm_gpuvm_exec *vm_exec)
>>>>>>>>>>>>>>>>>>> +{
>>>>>>>>>>>>>>>>>>> + drm_exec_fini(&vm_exec->exec);
>>>>>>>>>>>>>>>>>>> +}
>>>>>>>>>>>>>>>>>>> +
>>>>>>>>>>>>>>>>>>> +int drm_gpuvm_validate(struct drm_gpuvm *gpuvm);
>>>>>>>>>>>>>>>>>>> +void drm_gpuvm_resv_add_fence(struct drm_gpuvm *gpuvm,
>>>>>>>>>>>>>>>>>>> + struct drm_exec *exec,
>>>>>>>>>>>>>>>>>>> + struct dma_fence *fence,
>>>>>>>>>>>>>>>>>>> + enum dma_resv_usage
>>>>>>>>>>>>>>>>>>> private_usage,
>>>>>>>>>>>>>>>>>>> + enum dma_resv_usage
>>>>>>>>>>>>>>>>>>> extobj_usage);
>>>>>>>>>>>>>>>>>>> +
>>>>>>>>>>>>>>>>>>> +/**
>>>>>>>>>>>>>>>>>>> + * drm_gpuvm_exec_resv_add_fence()
>>>>>>>>>>>>>>>>>>> + * @vm_exec: the &drm_gpuvm_exec abstraction
>>>>>>>>>>>>>>>>>>> + * @fence: fence to add
>>>>>>>>>>>>>>>>>>> + * @private_usage: private dma-resv usage
>>>>>>>>>>>>>>>>>>> + * @extobj_usage: extobj dma-resv usage
>>>>>>>>>>>>>>>>>>> + *
>>>>>>>>>>>>>>>>>>> + * See drm_gpuvm_resv_add_fence().
>>>>>>>>>>>>>>>>>>> + */
>>>>>>>>>>>>>>>>>>> +static inline void
>>>>>>>>>>>>>>>>>>> +drm_gpuvm_exec_resv_add_fence(struct drm_gpuvm_exec
>>>>>>>>>>>>>>>>>>> *vm_exec,
>>>>>>>>>>>>>>>>>>> + struct dma_fence *fence,
>>>>>>>>>>>>>>>>>>> + enum dma_resv_usage
>>>>>>>>>>>>>>>>>>> private_usage,
>>>>>>>>>>>>>>>>>>> + enum dma_resv_usage
>>>>>>>>>>>>>>>>>>> extobj_usage)
>>>>>>>>>>>>>>>>>>> +{
>>>>>>>>>>>>>>>>>>> + drm_gpuvm_resv_add_fence(vm_exec->vm,
>>>>>>>>>>>>>>>>>>> &vm_exec->exec,
>>>>>>>>>>>>>>>>>>> fence,
>>>>>>>>>>>>>>>>>>> + private_usage,
>>>>>>>>>>>>>>>>>>> extobj_usage);
>>>>>>>>>>>>>>>>>>> +}
>>>>>>>>>>>>>>>>>>> +
>>>>>>>>>>>>>>>>>>> /**
>>>>>>>>>>>>>>>>>>> * struct drm_gpuvm_bo - structure representing a
>>>>>>>>>>>>>>>>>>> &drm_gpuvm and
>>>>>>>>>>>>>>>>>>> * &drm_gem_object combination
>>>>>>>>>>>>>>>>>>> @@ -398,6 +569,18 @@ struct drm_gpuvm_bo {
>>>>>>>>>>>>>>>>>>> * gpuva list.
>>>>>>>>>>>>>>>>>>> */
>>>>>>>>>>>>>>>>>>> struct list_head gem;
>>>>>>>>>>>>>>>>>>> +
>>>>>>>>>>>>>>>>>>> + /**
>>>>>>>>>>>>>>>>>>> + * @evict: List entry to
>>>>>>>>>>>>>>>>>>> attach to
>>>>>>>>>>>>>>>>>>> the &drm_gpuvms
>>>>>>>>>>>>>>>>>>> + * extobj list.
>>>>>>>>>>>>>>>>>>> + */
>>>>>>>>>>>>>>>>>>> + struct list_head extobj;
>>>>>>>>>>>>>>>>>>> +
>>>>>>>>>>>>>>>>>>> + /**
>>>>>>>>>>>>>>>>>>> + * @evict: List entry to
>>>>>>>>>>>>>>>>>>> attach to
>>>>>>>>>>>>>>>>>>> the &drm_gpuvms evict
>>>>>>>>>>>>>>>>>>> + * list.
>>>>>>>>>>>>>>>>>>> + */
>>>>>>>>>>>>>>>>>>> + struct list_head evict;
>>>>>>>>>>>>>>>>>>> } entry;
>>>>>>>>>>>>>>>>>>> } list;
>>>>>>>>>>>>>>>>>>> };
>>>>>>>>>>>>>>>>>>> @@ -432,6 +615,9 @@ struct drm_gpuvm_bo *
>>>>>>>>>>>>>>>>>>> drm_gpuvm_bo_find(struct drm_gpuvm *gpuvm,
>>>>>>>>>>>>>>>>>>> struct drm_gem_object *obj);
>>>>>>>>>>>>>>>>>>> +void drm_gpuvm_bo_evict(struct drm_gem_object *obj,
>>>>>>>>>>>>>>>>>>> bool
>>>>>>>>>>>>>>>>>>> evict);
>>>>>>>>>>>>>>>>>>> +void drm_gpuvm_bo_extobj_add(struct drm_gpuvm_bo
>>>>>>>>>>>>>>>>>>> *vm_bo);
>>>>>>>>>>>>>>>>>>> +
>>>>>>>>>>>>>>>>>>> /**
>>>>>>>>>>>>>>>>>>> * drm_gpuvm_bo_for_each_va() - iterator to walk
>>>>>>>>>>>>>>>>>>> over a
>>>>>>>>>>>>>>>>>>> list of &drm_gpuva
>>>>>>>>>>>>>>>>>>> * @va__: &drm_gpuva structure to assign to in each
>>>>>>>>>>>>>>>>>>> iteration step
>>>>>>>>>>>>>>>>>>> @@ -837,6 +1023,17 @@ struct drm_gpuvm_ops {
>>>>>>>>>>>>>>>>>>> * used.
>>>>>>>>>>>>>>>>>>> */
>>>>>>>>>>>>>>>>>>> int (*sm_step_unmap)(struct drm_gpuva_op
>>>>>>>>>>>>>>>>>>> *op, void
>>>>>>>>>>>>>>>>>>> *priv);
>>>>>>>>>>>>>>>>>>> +
>>>>>>>>>>>>>>>>>>> + /**
>>>>>>>>>>>>>>>>>>> + * @bo_validate: called from
>>>>>>>>>>>>>>>>>>> drm_gpuvm_validate()
>>>>>>>>>>>>>>>>>>> + *
>>>>>>>>>>>>>>>>>>> + * Drivers receive this callback for every
>>>>>>>>>>>>>>>>>>> evicted
>>>>>>>>>>>>>>>>>>> &drm_gem_object being
>>>>>>>>>>>>>>>>>>> + * mapped in the corresponding &drm_gpuvm.
>>>>>>>>>>>>>>>>>>> + *
>>>>>>>>>>>>>>>>>>> + * Typically, drivers would call their driver
>>>>>>>>>>>>>>>>>>> specific variant of
>>>>>>>>>>>>>>>>>>> + * ttm_bo_validate() from within this callback.
>>>>>>>>>>>>>>>>>>> + */
>>>>>>>>>>>>>>>>>>> + int (*bo_validate)(struct drm_gem_object *obj);
>>>>>>>>>>>>>>>>>>> };
>>>>>>>>>>>>>>>>>>> int drm_gpuvm_sm_map(struct drm_gpuvm *gpuvm,
>>>>>>>>>>>>>>>>>>> void *priv,
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>
>>
Powered by blists - more mailing lists