linux-kernel - Re: [PATCH v2 1/3] kunit: tool: add subscripts for type annotations where appropriate

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-Id: <20230430181500.122664-1-sj@kernel.org>
Date:   Sun, 30 Apr 2023 18:15:00 +0000
From:   SeongJae Park <sj@...nel.org>
To:     Daniel Latypov <dlatypov@...gle.com>
Cc:     brendanhiggins@...gle.com, davidgow@...gle.com, rmoar@...gle.com,
        linux-kernel@...r.kernel.org, kunit-dev@...glegroups.com,
        linux-kselftest@...r.kernel.org, skhan@...uxfoundation.org,
        johannes@...solutions.net, Johannes Berg <johannes.berg@...el.com>,
        regressions@...ts.linux.dev
Subject: Re: [PATCH v2 1/3] kunit: tool: add subscripts for type annotations where appropriate

Hi Daniel,

On Thu, 16 Mar 2023 15:06:36 -0700 Daniel Latypov <dlatypov@...gle.com> wrote:

> E.g. for subprocess.Popen, it can be opened in `text=True` mode where it
> returns strings, or `text=False` where it returns bytes.
> To differentiate, you can annotate types as `Popen[str]` or
> `Popen[bytes]`.
> 
> This patch should add subscripts in all the places we were missing them.

I just found this patch is in the latest mainline tree, and it causes kunit
failure on my test machine like below.

    $ python3 --version
    Python 3.8.10
    $
    $ ./tools/testing/kunit/kunit.py run --build_dir ../kunit.out/
    Traceback (most recent call last):
      File "./tools/testing/kunit/kunit.py", line 24, in <module>
        import kunit_kernel
      File "/home/sjpark/linux/tools/testing/kunit/kunit_kernel.py", line 42, in <module>
        class LinuxSourceTreeOperations:
      File "/home/sjpark/linux/tools/testing/kunit/kunit_kernel.py", line 95, in LinuxSourceTreeOperations
        def start(self, params: List[str], build_dir: str) -> subprocess.Popen[str]:
    TypeError: 'type' object is not subscriptable
    $

I further confirmed reverting this patch makes it run again.  Do you have any
idea?


Thanks,
SJ

> 
> Reported-by: Johannes Berg <johannes.berg@...el.com>
> Link: https://lore.kernel.org/linux-kselftest/20230315105055.9b2be0153625.I7a2cb99b95dff216c0feed4604255275e0b156a7@changeid/
> Signed-off-by: Daniel Latypov <dlatypov@...gle.com>
> ---
> Note: this is unchanged, just added a 3rd patch to this series.
> ---
>  tools/testing/kunit/kunit_kernel.py  | 6 +++---
>  tools/testing/kunit/kunit_printer.py | 2 +-
>  tools/testing/kunit/run_checks.py    | 2 +-
>  3 files changed, 5 insertions(+), 5 deletions(-)
> 
> diff --git a/tools/testing/kunit/kunit_kernel.py b/tools/testing/kunit/kunit_kernel.py
> index 53e90c335834..e6fc8fcb071a 100644
> --- a/tools/testing/kunit/kunit_kernel.py
> +++ b/tools/testing/kunit/kunit_kernel.py
> @@ -92,7 +92,7 @@ class LinuxSourceTreeOperations:
>  		if stderr:  # likely only due to build warnings
>  			print(stderr.decode())
>  
> -	def start(self, params: List[str], build_dir: str) -> subprocess.Popen:
> +	def start(self, params: List[str], build_dir: str) -> subprocess.Popen[str]:
>  		raise RuntimeError('not implemented!')
>  
>  
> @@ -112,7 +112,7 @@ class LinuxSourceTreeOperationsQemu(LinuxSourceTreeOperations):
>  		kconfig.merge_in_entries(base_kunitconfig)
>  		return kconfig
>  
> -	def start(self, params: List[str], build_dir: str) -> subprocess.Popen:
> +	def start(self, params: List[str], build_dir: str) -> subprocess.Popen[str]:
>  		kernel_path = os.path.join(build_dir, self._kernel_path)
>  		qemu_command = ['qemu-system-' + self._qemu_arch,
>  				'-nodefaults',
> @@ -141,7 +141,7 @@ class LinuxSourceTreeOperationsUml(LinuxSourceTreeOperations):
>  		kconfig.merge_in_entries(base_kunitconfig)
>  		return kconfig
>  
> -	def start(self, params: List[str], build_dir: str) -> subprocess.Popen:
> +	def start(self, params: List[str], build_dir: str) -> subprocess.Popen[str]:
>  		"""Runs the Linux UML binary. Must be named 'linux'."""
>  		linux_bin = os.path.join(build_dir, 'linux')
>  		params.extend(['mem=1G', 'console=tty', 'kunit_shutdown=halt'])
> diff --git a/tools/testing/kunit/kunit_printer.py b/tools/testing/kunit/kunit_printer.py
> index 5f1cc55ecdf5..015adf87dc2c 100644
> --- a/tools/testing/kunit/kunit_printer.py
> +++ b/tools/testing/kunit/kunit_printer.py
> @@ -15,7 +15,7 @@ _RESET = '\033[0;0m'
>  class Printer:
>  	"""Wraps a file object, providing utilities for coloring output, etc."""
>  
> -	def __init__(self, output: typing.IO):
> +	def __init__(self, output: typing.IO[str]):
>  		self._output = output
>  		self._use_color = output.isatty()
>  
> diff --git a/tools/testing/kunit/run_checks.py b/tools/testing/kunit/run_checks.py
> index 066e6f938f6d..61cece1684df 100755
> --- a/tools/testing/kunit/run_checks.py
> +++ b/tools/testing/kunit/run_checks.py
> @@ -37,7 +37,7 @@ def main(argv: Sequence[str]) -> None:
>  	if argv:
>  		raise RuntimeError('This script takes no arguments')
>  
> -	future_to_name: Dict[futures.Future, str] = {}
> +	future_to_name: Dict[futures.Future[None], str] = {}
>  	executor = futures.ThreadPoolExecutor(max_workers=len(commands))
>  	for name, argv in commands.items():
>  		if name in necessary_deps and shutil.which(necessary_deps[name]) is None:
> 
> base-commit: 2c6a96dad5797e57b4cf04101d6c8d5c7a571603
> -- 
> 2.40.0.rc1.284.g88254d51c5-goog
>