linux-kernel - Re: [PATCH] arm64: head: Fix cache inconsistency of the identity-mapped region

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <f1b32237-f178-0656-b3ee-814eec4edb71@nvidia.com>
Date:   Wed, 20 Apr 2022 10:02:13 -0500
From:   Shanker R Donthineni <sdonthineni@...dia.com>
To:     Will Deacon <will@...nel.org>
Cc:     Marc Zyngier <maz@...nel.org>,
        Catalin Marinas <catalin.marinas@....com>,
        Mark Rutland <mark.rutland@....com>,
        linux-arm-kernel@...ts.infradead.org, linux-kernel@...r.kernel.org,
        Ard Biesheuvel <ardb@...nel.org>,
        Vikram Sethi <vsethi@...dia.com>,
        Thierry Reding <treding@...dia.com>,
        Anshuman Khandual <anshuman.khandual@....com>
Subject: Re: [PATCH] arm64: head: Fix cache inconsistency of the
 identity-mapped region

Hi Will,

On 4/20/22 5:08 AM, Will Deacon wrote:
> Non-coherent? You mean non-cacheable, right? At this stage, we only
>>> have a single CPU, so I'm not sure coherency is the problem here. When
>>> you say 'drop', is that an eviction? Replaced by what? By the previous
>>> version of the cache line, containing the stale value?
>> Yes,non-cacheable. The cache line corresponding to function enter_vhe() was
>> marked with shareable & WB-cache as a result of write to RELA, the same cache
>> line is being fetched with non-shareable & non-cacheable. The eviction is
>> not pushed to PoC and got dropped because of cache-line attributes mismatch.
> I'm really struggling to understand this. Why is the instruction fetch
> non-shareable? I'm trying to align your observations with the rules about
> mismatched aliases in the architecture and I'm yet to satisfy myself that
> the CPU is allowed to drop a dirty line on the floor in response to an
> unexpected hit.
>
> My mental model (which seems to align with Marc) is something like:
>
>
> 1. The boot CPU fetches the line via a cacheable mapping and dirties it
>    in its L1 as part of applying the relocations.

ARM-CPU core is sending ReadNotSharedDirty CHI command to LLC (last-level-cache).
This cache-line is marked as checked-out in LLC, would be used to keep track
of coherency.

> 2. The boot CPU then enters EL2 with the MMU off and fetches the same
>    line on the I-side. AFAICT, the architecture says this is done with
>    outer-shareable, non-cacheable attributes.

ARM-CPU core is sending ReadNoSnoop CHI command when MMU disabled. The
marked cache-line from the step 1) become invalid in LLC. As per the
ARM-ARM specification, CMO is recommended whenever memory attributes
are changed for a given memory region.

With my understating the CPU core must generate coherent accesses for
Shared+Cacheable memory but not clear for OSH+non-cacheable regions
in the spec.

Are you expecting "OSH+non-cacheable" must generate coherent accesses?

> 3. !!! Somehow the instruction fetch results in the _dirty_ line from (1)
>        being discarded !!!

The ARM-CPU is sending WritebackFull CHI command to LLC for the cache line
which was marked as invalid from step 2). The write CMD is ignored/dropped.

> 4. A secondary CPU later on fetches the line via a cacheable mapping and
>    explodes because the relocation hasn't been applied.
>
The modified data was dropped from step 3.

> Is that what you're seeing? If so, we really need more insight into what
> is going on at step (3) because it feels like it could have a much more
> significant impact than the issue we're trying to fix here.

The actual problem happens from step 2 when CPU executes 'b enter_vhe'.

>  How is the line
> dropped? Is it due to back invalidation from a shared cache? Is it due to
> IDC snooping? Does the line actually stick around on the D-side, but somehow
> the I-side is shared between CPUs?
>
> Many questions...
>
> Will