| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Message-Id: <1519965121-12017-1-git-send-email-yamada.masahiro@socionext.com>
Date: Fri, 2 Mar 2018 13:31:50 +0900
From: Masahiro Yamada <yamada.masahiro@...ionext.com>
To: linux-kbuild@...r.kernel.org
Cc: Sam Ravnborg <sam@...nborg.org>,
Michal Marek <michal.lkml@...kovi.net>,
Ulf Magnusson <ulfalizer@...il.com>,
Randy Dunlap <rdunlap@...radead.org>,
"Luis R . Rodriguez" <mcgrof@...nel.org>,
Masahiro Yamada <yamada.masahiro@...ionext.com>,
Tony Luck <tony.luck@...el.com>, linux-ia64@...r.kernel.org,
linux-kernel@...r.kernel.org, Fenghua Yu <fenghua.yu@...el.com>
Subject: [PATCH v2 00/11] Add Kconfig unit tests
I am applying various cleanups to Kconfig these days.
However, I fear regressions. I have been thinking of unit-tests.
There are various cryptic parts in Kconfig and corner cases where
it is difficult to notice breakage. If unit-tests cover those,
I will be able to apply changes more confidently.
This series introduces a unit-tests. The framework is written based
on pytest. Also, this is written in Python 3. Python 2 will retire
in 2020. So, I believe new python tools should be written in Python 3.
This is my Python 3 and pytest versions.
$ python3 --version
Python 3.5.2
$ python3 -m pytest --version
This is pytest version 3.4.1, imported from /home/masahiro/.local/lib/python3.5/site-packages/pytest.py
How to use?
-----------
Please make sure Python3 and pytest are installed on your system.
Then, run "make testconfig"
The result looks like as follows:
$ make testconfig
python3 -B -m pytest ./scripts/kconfig/tests \
-o cache_dir=/home/masahiro/workspace/bsp/linux-yamada/scripts/kconfig/tests/.cache \
================================= test session starts =================================
platform linux -- Python 3.5.2, pytest-3.4.1, py-1.5.2, pluggy-0.6.0 -- /usr/bin/python3
cachedir: scripts/kconfig/tests/.cache
rootdir: /home/masahiro/workspace/bsp/linux-yamada/scripts/kconfig/tests, inifile: pytest.ini
collected 14 items
scripts/kconfig/tests/auto_submenu/__init__.py::test PASSED [ 7%]
scripts/kconfig/tests/choice/__init__.py::test_oldask0 PASSED [ 14%]
scripts/kconfig/tests/choice/__init__.py::test_oldask1 PASSED [ 21%]
scripts/kconfig/tests/choice/__init__.py::test_allyes PASSED [ 28%]
scripts/kconfig/tests/choice/__init__.py::test_allmod PASSED [ 35%]
scripts/kconfig/tests/choice/__init__.py::test_allno PASSED [ 42%]
scripts/kconfig/tests/choice/__init__.py::test_alldef PASSED [ 50%]
scripts/kconfig/tests/choice_value_with_m_dep/__init__.py::test PASSED [ 57%]
scripts/kconfig/tests/err_recursive_inc/__init__.py::test PASSED [ 64%]
scripts/kconfig/tests/inter_choice/__init__.py::test PASSED [ 71%]
scripts/kconfig/tests/new_choice_with_dep/__init__.py::test PASSED [ 78%]
scripts/kconfig/tests/no_write_if_dep_unmet/__init__.py::test PASSED [ 85%]
scripts/kconfig/tests/rand_nested_choice/__init__.py::test PASSED [ 92%]
scripts/kconfig/tests/warn_recursive_dep/__init__.py::test PASSED [100%]
Changes in v2:
- Add backward compatibility for PYTHON
- Add 'randconfig' support
- Add docstring
- Fix various style errors detected by 'pep8' and 'pep257'
- coding style clean-up based on PEP8, PEP257
- shorten directory name 'auto_submenu_creation' -> 'auto_submenu'
- coding style clean-up based on PEP8, PEP257
- coding style clean-up based on PEP8, PEP257
- coding style clean-up based on PEP8, PEP257
- coding style clean-up based on PEP8, PEP257
- Newly added
- Newly added
- Fix missing end quote
- coding style clean-up based on PEP8, PEP257
- coding style clean-up based on PEP8, PEP257
Masahiro Yamada (11):
kbuild: define PYTHON2 and PYTHON3 variables instead of PYTHON
kconfig: unittest: add framework for Kconfig unit testing
kconfig: unittest: add basic 'choice' tests
kconfig: unittest: test automatic submenu creation
kconfig: unittest: test if new symbols in choice are asked
kconfig: unittest: check unneeded "is not set" with unmet dependency
kconfig: unittest: check visibility of tri-choice values in y choice
kconfig: unittest: test defconfig when two choices interact
kconfig: unittest: test randconfig for choice in choice
kconfig: unittest: test if recursive dependencies are detected
kconfig: unittest: test if recursive inclusion is detected
Makefile | 5 +-
arch/ia64/Makefile | 2 +-
scripts/kconfig/Makefile | 8 +
scripts/kconfig/tests/auto_submenu/Kconfig | 50 ++++
scripts/kconfig/tests/auto_submenu/__init__.py | 12 +
scripts/kconfig/tests/auto_submenu/expected_stdout | 10 +
scripts/kconfig/tests/choice/Kconfig | 54 ++++
scripts/kconfig/tests/choice/__init__.py | 40 +++
.../kconfig/tests/choice/alldef_expected_config | 5 +
.../kconfig/tests/choice/allmod_expected_config | 9 +
scripts/kconfig/tests/choice/allno_expected_config | 5 +
.../kconfig/tests/choice/allyes_expected_config | 9 +
.../kconfig/tests/choice/oldask0_expected_stdout | 10 +
scripts/kconfig/tests/choice/oldask1_config | 2 +
.../kconfig/tests/choice/oldask1_expected_stdout | 15 ++
.../kconfig/tests/choice_value_with_m_dep/Kconfig | 20 ++
.../tests/choice_value_with_m_dep/__init__.py | 15 ++
.../kconfig/tests/choice_value_with_m_dep/config | 2 +
.../tests/choice_value_with_m_dep/expected_config | 3 +
.../tests/choice_value_with_m_dep/expected_stdout | 4 +
scripts/kconfig/tests/conftest.py | 291 +++++++++++++++++++++
scripts/kconfig/tests/err_recursive_inc/Kconfig | 1 +
.../kconfig/tests/err_recursive_inc/Kconfig.inc | 1 +
.../kconfig/tests/err_recursive_inc/__init__.py | 10 +
.../tests/err_recursive_inc/expected_stderr | 4 +
scripts/kconfig/tests/inter_choice/Kconfig | 24 ++
scripts/kconfig/tests/inter_choice/__init__.py | 14 +
scripts/kconfig/tests/inter_choice/defconfig | 1 +
scripts/kconfig/tests/inter_choice/expected_config | 4 +
scripts/kconfig/tests/new_choice_with_dep/Kconfig | 37 +++
.../kconfig/tests/new_choice_with_dep/__init__.py | 14 +
scripts/kconfig/tests/new_choice_with_dep/config | 3 +
.../tests/new_choice_with_dep/expected_stdout | 10 +
.../kconfig/tests/no_write_if_dep_unmet/Kconfig | 14 +
.../tests/no_write_if_dep_unmet/__init__.py | 19 ++
scripts/kconfig/tests/no_write_if_dep_unmet/config | 1 +
.../tests/no_write_if_dep_unmet/expected_config | 5 +
scripts/kconfig/tests/pytest.ini | 6 +
scripts/kconfig/tests/rand_nested_choice/Kconfig | 33 +++
.../kconfig/tests/rand_nested_choice/__init__.py | 16 ++
.../tests/rand_nested_choice/expected_stdout0 | 2 +
.../tests/rand_nested_choice/expected_stdout1 | 4 +
.../tests/rand_nested_choice/expected_stdout2 | 5 +
scripts/kconfig/tests/warn_recursive_dep/Kconfig | 62 +++++
.../kconfig/tests/warn_recursive_dep/__init__.py | 9 +
.../tests/warn_recursive_dep/expected_stderr | 30 +++
46 files changed, 897 insertions(+), 3 deletions(-)
create mode 100644 scripts/kconfig/tests/auto_submenu/Kconfig
create mode 100644 scripts/kconfig/tests/auto_submenu/__init__.py
create mode 100644 scripts/kconfig/tests/auto_submenu/expected_stdout
create mode 100644 scripts/kconfig/tests/choice/Kconfig
create mode 100644 scripts/kconfig/tests/choice/__init__.py
create mode 100644 scripts/kconfig/tests/choice/alldef_expected_config
create mode 100644 scripts/kconfig/tests/choice/allmod_expected_config
create mode 100644 scripts/kconfig/tests/choice/allno_expected_config
create mode 100644 scripts/kconfig/tests/choice/allyes_expected_config
create mode 100644 scripts/kconfig/tests/choice/oldask0_expected_stdout
create mode 100644 scripts/kconfig/tests/choice/oldask1_config
create mode 100644 scripts/kconfig/tests/choice/oldask1_expected_stdout
create mode 100644 scripts/kconfig/tests/choice_value_with_m_dep/Kconfig
create mode 100644 scripts/kconfig/tests/choice_value_with_m_dep/__init__.py
create mode 100644 scripts/kconfig/tests/choice_value_with_m_dep/config
create mode 100644 scripts/kconfig/tests/choice_value_with_m_dep/expected_config
create mode 100644 scripts/kconfig/tests/choice_value_with_m_dep/expected_stdout
create mode 100644 scripts/kconfig/tests/conftest.py
create mode 100644 scripts/kconfig/tests/err_recursive_inc/Kconfig
create mode 100644 scripts/kconfig/tests/err_recursive_inc/Kconfig.inc
create mode 100644 scripts/kconfig/tests/err_recursive_inc/__init__.py
create mode 100644 scripts/kconfig/tests/err_recursive_inc/expected_stderr
create mode 100644 scripts/kconfig/tests/inter_choice/Kconfig
create mode 100644 scripts/kconfig/tests/inter_choice/__init__.py
create mode 100644 scripts/kconfig/tests/inter_choice/defconfig
create mode 100644 scripts/kconfig/tests/inter_choice/expected_config
create mode 100644 scripts/kconfig/tests/new_choice_with_dep/Kconfig
create mode 100644 scripts/kconfig/tests/new_choice_with_dep/__init__.py
create mode 100644 scripts/kconfig/tests/new_choice_with_dep/config
create mode 100644 scripts/kconfig/tests/new_choice_with_dep/expected_stdout
create mode 100644 scripts/kconfig/tests/no_write_if_dep_unmet/Kconfig
create mode 100644 scripts/kconfig/tests/no_write_if_dep_unmet/__init__.py
create mode 100644 scripts/kconfig/tests/no_write_if_dep_unmet/config
create mode 100644 scripts/kconfig/tests/no_write_if_dep_unmet/expected_config
create mode 100644 scripts/kconfig/tests/pytest.ini
create mode 100644 scripts/kconfig/tests/rand_nested_choice/Kconfig
create mode 100644 scripts/kconfig/tests/rand_nested_choice/__init__.py
create mode 100644 scripts/kconfig/tests/rand_nested_choice/expected_stdout0
create mode 100644 scripts/kconfig/tests/rand_nested_choice/expected_stdout1
create mode 100644 scripts/kconfig/tests/rand_nested_choice/expected_stdout2
create mode 100644 scripts/kconfig/tests/warn_recursive_dep/Kconfig
create mode 100644 scripts/kconfig/tests/warn_recursive_dep/__init__.py
create mode 100644 scripts/kconfig/tests/warn_recursive_dep/expected_stderr
--
2.7.4
Powered by blists - more mailing lists