Message-ID:
 <AM9PR04MB86047231B5320C01759BFCAE95CCA@AM9PR04MB8604.eurprd04.prod.outlook.com>
Date: Wed, 12 Nov 2025 11:38:10 +0000
From: Pankaj Gupta <pankaj.gupta@....com>
To: Marco Felsch <m.felsch@...gutronix.de>
CC: Jonathan Corbet <corbet@....net>, Rob Herring <robh@...nel.org>, Krzysztof
 Kozlowski <krzk+dt@...nel.org>, Conor Dooley <conor+dt@...nel.org>, Shawn Guo
	<shawnguo@...nel.org>, Sascha Hauer <s.hauer@...gutronix.de>, Pengutronix
 Kernel Team <kernel@...gutronix.de>, Fabio Estevam <festevam@...il.com>,
	"devicetree@...r.kernel.org" <devicetree@...r.kernel.org>,
	"imx@...ts.linux.dev" <imx@...ts.linux.dev>, "linux-doc@...r.kernel.org"
	<linux-doc@...r.kernel.org>, Frank Li <frank.li@....com>,
	"linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
	"linux-arm-kernel@...ts.infradead.org" <linux-arm-kernel@...ts.infradead.org>
Subject: RE: [EXT] Re: [PATCH v19 0/7] firmware: imx: driver for NXP
 secure-enclave

> On 25-11-07, Pankaj Gupta wrote:
> > >> On 25-10-27, Pankaj Gupta wrote:
> > >>> On 25-09-27, Pankaj Gupta wrote:
> 
> ...
> 
> > > > It is not yet up-streamed to OPTEE-OS repo.
> > >
> > > My intention of adding the above OP-TEE discussion link was to point
> > > out that an ELE-FW bug exists which needs to be fixed.
> 
> ...
> 
> > > This adapts the timeout value to 100ms and seems more like an
> workaround.
> > >
> > There are additional fixes in OPTEE-OS, that will be part of LF Q4'25.
> 
> Thanks for this info.
> 
> > > However, can NXP confirm that the ELE concurrent access is possible
> > > without a previous ELE FW update?
> >
> > Fix in the ELE FW, released as part of LF Q3 2025,  is a must to
> > include. OPTEE fixes are also needed.  OPTEE fixes will be up-streamed
> > soon.
> 
> Okay, so there are ELE-FW fixes too, thanks.
> 
> ...
> 
> > > Does this mean that all i.MX9x, i.MX10x and so on do have the the
> > > secure and non-secure MU setup? Or is it based on the SoC release date?
> > > Because regarding the datasheet the i.MX8ULP is newer than the
> > > i.MX93, therefore I assumed that the i.MX8ULP has two MUs as well.
> >
> > From i.MX93 and onward, there is only one RoT. Hence , it is designed
> > to have dedicated MU for each world.
> 
> Okay.
> 
> > > I checked the the NXP OP-TEE source code and found the following
> commit:
> > >
> > > 8<----------------
> > > commit 44388d37e68000ee50a9b1d656e0a60ae6614977
> > > Author: Sahil Malhotra <sahil.malhotra@....com>
> > > Date:   Tue Apr 1 20:04:44 2025 +0200
> > >
> > >     core: imx: disable ELE on i.MX8ULP by default
> > >
> > >     On i.MX8ULP, there is only one MU to communicate with ELE,
> > >     which cannot be dedicated on OP-TEE side all the time.
> > >     There may be ELE services running on Linux side, which can
> > >     cause conflict with OP-TEE, So disabling ELE by default.
> > >     Moreover i.MX8ULP also has CAAM, so HUK and Random number
> > >     are coming from CAAM.
> > >
> > >     Signed-off-by: Sahil Malhotra <sahil.malhotra@....com>
> > >     Acked-by: Jens Wiklander <jens.wiklander@...aro.org>
> > > 8<----------------
> > >
> > > So it's possible to configure the XRDC (configured in the TF-A) in a
> > > way to map the ELE access to the secure world. If I got the TF-A and OP-TEE
> commits right.
> > >
> > > To me this sound more like a NXP design decision to move the ELE to
> > > the non- secure and the CAAM to the secure world.
> >
> > As per the i.MX8ULP boot-up sequence and ELE's initial role in
> > boot-up, with CAAM co-exists, ELE is logical to be with Linux.
> >
> > Another point here that CAAM has 4 JR(s) and hence CAAM can be shared
> > between Linux and OPTEE-OS.
> 
> Please see my answer below where you explained the ELE functions.
> 
> ...
> 
> > > i.MX8ULP is getting really interesting, though.
> > >
> > > May I ask what RoT is used by this SoC if there are two?
> > >
> > ELE is Root of Trust during secure boot.
> > For OPTEE-OS, CAAM is RoT.
> 
> The i.MX8ULP is very interesting, since you also need to handle the SoC
> secure-state twice, right? However, this topic alone is worth it a standalone
> discussion thread, therefore no further comments.
> 
> ...
> 
> > > How does the i.MX8ULP fuse flow work, after the LOCK_DOWN fuse is
> blown?
> > There is no such issue on i.MX8ULP
> > >
> > > This was one of my main concers why having OP-TEE required in the
> > > first place, because the i.MX93 requires the that the fuse-request
> > > comes from the secure-world if the device is in LOCK_DOWN state.
> > >
> > > Is this also the case for the i.MX8ULP?
> > >
> >
> > No, this is not a valid case for i.MX8ULP.
> 
> Thanks for the input.
> 
> > > > > Also according your IOCTL docuementation you want to expose the
> > > > > whole device to the user-space?
> > > >
> > > > > | What:          /dev/<se>_mu[0-9]+_ch[0-9]+
> > > > > | Date:          Mar 2025
> > > > > | KernelVersion: 6.8
> > > > > | Contact:       linux-imx@....com, pankaj.gupta@....com
> > > > > | Description:
> > > > > |                NXP offers multiple hardware IP(s) for secure
> > > > > | enclaves like
> > > EdgeLock-
> > > > > |                Enclave(ELE), SECO. The character device file descriptors
> > > > > |                /dev/<se>_mu*_ch* are the interface between
> > > > > | userspace NXP's
> > > secure-
> > > > > |                enclave shared library and the kernel driver.
> > > > > |
> > > > > |                The ioctl(2)-based ABI is defined and documented in
> > > > > |                [include]<linux/firmware/imx/ele_mu_ioctl.h>.
> > > > > |                ioctl(s) are used primarily for:
> > > > > |                        - shared memory management
> > > > > |                        - allocation of I/O buffers
> > > > > |                        - getting mu info
> > > > > |                        - setting a dev-ctx as receiver to
> > > > > | receive all the commands
> > > from FW
> > > > > |                        - getting SoC info
> > > > > |                        - send command and receive command
> > > > > | response
> > > >                                 ^
> > > > > This is a rather uncommon approach. The kernel has interfaces to
> > > > > abstract hardware. You completely bypass this if you expose
> > > > > everything to the userspace.
> > > >
> > > > It is in-correct.
> > > > Not everything, just exposed file-operation. and ioctl(s) for
> > > > DMA(eable)
> > > buffer allocation from reserved lower memory region.
> > > > Things managed by Kernel:
> > > > * Send/receive path to ELE, is managed by Kernel.
> > > > * Receive/send patch to the ELE's slave NVM-manager, is managed by
> kernel.
> > > > * Low power management handled by kernel driver. In case of
> > > > low-power
> > > state, ELE driver re-init the V2X IP as part of resume.
> > > > * Other kernel management layers like NVMEM, kernel HWRNG, will
> > > > use the
> > > api(s) exposed by this driver.
> > >
> > > But you also expose an uAPI which allows the user to bypass
> > > everything via sending arbitrary commands, right?
> >
> > Yes. But it's not unusual at all. The pattern of userspace sending
> > commands directly to the kernel, is quite common
> 
> Please see below.
> 
> ...
> 
> > > Some features require the device to be in LOCK_DOWN mode, which
> > > requires secure-world eFuse write path only afterwards. But it seems
> > > like NXP really wants to maintain two write paths.
> > >
> > > > * Low power management at Linux driver.
> > >
> > > The power-modes are selected via the ELE?
> >
> >
> > Voltage regulation for i.MX93 in Linux kernel, is done by ELE.
> >
> > During Linux suspend-resume, Secure-enclave (V2X on i.MX95) part of
> > wake-up domain, will be managed by secure-enclaves(ELE) part of
> > always-on domain.
> 
> So to sum-up, please correct me if I got it wrong:
> 
>  - NXP puts the ELE into the non-secure world, in case only one MU
>    exists. The reason for this is that the ELE is also used to handle
>    power-management.
For NXP SoCs with multi-MU(s) too, NXP proposes to put ELE driver into non-secure world.

> 
>  - NXP exposes an uAPI which can be used to send arbitrary commands from
>    userspace to the ELE. (no filtering done yet)
It is not correct to say that no filtering is done.
Before sending as well as after receiving the message, the message header of the buffers
are parsed to check:
- TX-buffer with Command-tag is allowed to be sent, RX-buffer with response-tag is allowed to be received, without logging errors.
- TX buffer size & Rx-Buffer size should match the size mentioned in the buffer.
- FW version from the header is checked, if required secondary or runtime FW Is loaded, before forwarding the request to ELE.
- In certain cases especially for exceptions, the message IDs are also compared.

> 
>  --> Sounds to me that the userpace can influence the system behavior
>      very badly.
> 
Messages created and sent by User-space library(https://github.com/nxp-imx/imx-secure-enclave), are scrutinized as stated above in the kernel driver.

Moreover,
As part of this library, message creation, send-receive, IOCTLS etc. kernel interface implementation logic, is not exposed to users of this library.
With the help of secure-boot and IMA-EVM, rootfs can be restricted to not allow any new application or modified userspace library, to execute.
This way bad impact to the system behavior can be prevented.

> > > > * Linux HWRNG.
> > > > * Loading the secondary runtime fw.
> > >
> > > What is a secondary runtime-fw?
> > ELE FW size is larger than the size of ELE internal secure memory.
> > Hence FW is split into two.
> >
> > Primary FW, is the FW, that enables features that helps for SoC boot-up.
> > Secondary runtime FW, is the FW, that enables features like HSM.
> 
> Ah okay, thanks for the input.
> 
> > > To conclude this longly discussion:
> 
> ...
> 
> > > I still have mixed feeling about the fusing (including the 1-MU
> > > case), since it requires a secure-world OS in place once the LOCK_DOWN
> fuse was burned.
> > > It's fine by me if NXP wants to have and wants to maintain a multi-path
> here.
> >
> > Write fuse API will be added, to allow writing fuses from secure world
> > too.
> 
> This is a device life-cycle problem and if NXP decides to maintain multiple write
> paths, depending on the runtime-SoC state, this is fine by me.
> 
> What needs to be ensured is, that the fuse-issue doesn't exist for the 1-MU
> case (i.MX8ULP) as you said.
As said above "Write fuse API will be added, to allow writing fuses from secure world too."
This will be true for 1 MU or multi-MU.

> 
> > > Last but least, the uAPI which can be used to send arbitrary ELE
> > > commands seems unusual. But I don't know how secure-enclaves are
> > > abstracted within the kernel, so these are just my two cents.
> >
> > it's not unusual at all. The pattern of userspace sending commands
> > directly to the kernel via a queue is quite common like:
> >
> > GPUs: As you mentioned, userspace drivers (like those in Vulkan or
> > CUDA) often build command buffers and submit them directly to the
> > kernel or hardware.
> 
> That's right, but these drivers do at least some filtering on the OPs and check if
> they are allowed. According your patchset, you just write
> (se_if_fops_write()) the provided userspace buffer.

We are validating the buffer size against the size mentioned in the buffer header.
Refer above comments for more details.

> > Secure Enclaves: In systems like Intel SGX or AMD SEV, userspace
> > applications interact with enclaves via ioctl or mmap interfaces,
> > often sending structured commands or messages.
> 
> What I'm aware of is, that most secure-enclaves are switching to the standard
> TPM API.
In case of NXP SoC with ELE HW IP, ELE is considered as on-SoC TPM. No additional or external TPM is needed, if ELE is present on SoC.

> 
> Regards,
>   Marco
> 

Regards
Pankaj

Powered by blists - more mailing lists