tests: Document automated mac80211_hwsim test setup
This provides guidance on how to build the test setup, how to run the tests, and how to add new test cases. Signed-hostap: Jouni Malinen <j@w1.fi>
This commit is contained in:
Jouni Malinen
parent
5f3eddac8e
commit
883ab1b9f1
3 changed files with 354 additions and 0 deletions
175
tests/hwsim/README
Normal file
175
tests/hwsim/README
Normal file
|
|
@ -0,0 +1,175 @@
|
||||||
|
Automated hostapd/wpa_supplicant testing with mac80211_hwsim
|
||||||
|
------------------------------------------------------------
|
||||||
|
|
||||||
|
This directory contains testing infrastructure and test cases to run
|
||||||
|
automated tests of full hostapd and wpa_supplicant functionality. This
|
||||||
|
testing is done with the help of mac80211_hwsim which is Linux kernel
|
||||||
|
driver that simulates IEEE 802.11 radios without requiring any
|
||||||
|
additional hardware. This setup most of the hostapd and wpa_supplicant
|
||||||
|
functionality (and large parts of the Linux cfg80211 and mac80211
|
||||||
|
functionality for that matter) to be tested.
|
||||||
|
|
||||||
|
mac80211_hwsim is loaded with five simulated radios to allow different
|
||||||
|
device combinations to be tested. wlantest is used analyze raw packets
|
||||||
|
captured through the hwsim0 monitor interface that capture all frames
|
||||||
|
sent on all channels. tcpdump is used to store the frames for
|
||||||
|
analysis. Three wpa_supplicant processed are used to control three
|
||||||
|
virtual radios and one hostapd process is used to dynamically control
|
||||||
|
the other two virtual radios. hwsim_test is used to verify that data
|
||||||
|
connection (both unicast and broadcast) works between two netdevs.
|
||||||
|
|
||||||
|
The python scripts and tools in this directory control test case
|
||||||
|
execution. They interact wpa_supplicant and hostapd through control
|
||||||
|
interfaces to perform the operations. In addition, wlantest_cli and
|
||||||
|
hwsim_test are used to verify that operations have been performed
|
||||||
|
correctly and that the network connection works in the expected way.
|
||||||
|
|
||||||
|
These test cases are run automatically against the hostap.git commits
|
||||||
|
for regression testing and to help in keeping the hostap.git master
|
||||||
|
branch in stable state. Results from these tests are available here:
|
||||||
|
http://buildbot.w1.fi:8010/waterfall
|
||||||
|
|
||||||
|
|
||||||
|
Building binaries for testing
|
||||||
|
-----------------------------
|
||||||
|
|
||||||
|
You will need to build (or use already built) components to be
|
||||||
|
tested. These are available in the hostap.git repository and can be
|
||||||
|
built for example as follows:
|
||||||
|
|
||||||
|
cd ../../wpa_supplicant
|
||||||
|
cp ../tests/hwsim/example-wpa_supplicant.config .config
|
||||||
|
make clean
|
||||||
|
make
|
||||||
|
cd ../hostapd
|
||||||
|
cp ../tests/hwsim/example-hostapd.config .config
|
||||||
|
make clean
|
||||||
|
make
|
||||||
|
cd ../wlantest
|
||||||
|
make clean
|
||||||
|
make
|
||||||
|
cd ../mac80211_hwsim/tools
|
||||||
|
make
|
||||||
|
|
||||||
|
The test scripts can find the binaries in the locations where they were
|
||||||
|
built. It is also possible to install hwsim_test and wlantest_cli
|
||||||
|
somewhat on the path to use pre-built tools.
|
||||||
|
|
||||||
|
|
||||||
|
wpaspy
|
||||||
|
------
|
||||||
|
|
||||||
|
The python scripts use wpaspy.py to interact with the wpa_supplicant
|
||||||
|
control interface. This needs to be installed in a way that python can
|
||||||
|
find it. For example, with this command:
|
||||||
|
|
||||||
|
sudo cp ../wpaspy/wpaspy.py /usr/local/lib/python2.7/dist-packages
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
mac80211_hwsim
|
||||||
|
--------------
|
||||||
|
|
||||||
|
mac80211_hwsim kernel module is available from the upstream Linux
|
||||||
|
kernel. Some Linux distributions enable it by default. If that's not the
|
||||||
|
case, you can either enable it in the kernel configuration
|
||||||
|
(CONFIG_MAC80211_HWSIM=m) and rebuild your kernel or use Backports with
|
||||||
|
CPTCFG_MAC80211_HWSIM=m to replace the wireless LAN components in the
|
||||||
|
base kernel.
|
||||||
|
|
||||||
|
|
||||||
|
sudo
|
||||||
|
----
|
||||||
|
|
||||||
|
Some parts of the testing process requires root privileges. The test
|
||||||
|
scripts are currently using sudo to achieve this. To be able to run the
|
||||||
|
tests, you'll probably want to enable sudo with a timeout to not expire
|
||||||
|
password entry very quickly. For example, use this in the sudoers file:
|
||||||
|
|
||||||
|
Defaults env_reset,timestamp_timeout=180
|
||||||
|
|
||||||
|
Or on a dedicated test system, you could even disable password prompting
|
||||||
|
with this in sudoers:
|
||||||
|
|
||||||
|
%sudo ALL=NOPASSWD: ALL
|
||||||
|
|
||||||
|
|
||||||
|
Other network interfaces
|
||||||
|
------------------------
|
||||||
|
|
||||||
|
Some of the test scripts are still using hardcoded interface names, so
|
||||||
|
the easiest way of making things work is to avoid using other network
|
||||||
|
devices that may use conflicting interface names. For example, unload
|
||||||
|
any wireless LAN driver before running the tests and make sure that
|
||||||
|
wlan0..4 gets assigned as the interface names for the mac80211_hwsim
|
||||||
|
radios. It may also be possible to rename the interface expectations in
|
||||||
|
run-tests.py to allow other names to be used.
|
||||||
|
|
||||||
|
|
||||||
|
Running tests
|
||||||
|
-------------
|
||||||
|
|
||||||
|
Simplest way to run a full set of the test cases is by running
|
||||||
|
run-all.sh in tests/hwsim directory. This will use start.sh to load the
|
||||||
|
mac80211_hwsim module and start wpa_supplicant, hostapd, and various
|
||||||
|
test tools. run-tests.sh is then used to run through all the defined
|
||||||
|
test cases and stop-wifi.sh to stop the programs and unload the kernel
|
||||||
|
module.
|
||||||
|
|
||||||
|
run-all.sh can be used to run the same test cases under different
|
||||||
|
conditions:
|
||||||
|
|
||||||
|
# run normal test cases
|
||||||
|
./run-all.sh
|
||||||
|
|
||||||
|
# run normal test cases under valgrind
|
||||||
|
./run-all.sh valgrind
|
||||||
|
|
||||||
|
# run normal test cases with Linux tracing
|
||||||
|
./run-all.sh trace
|
||||||
|
|
||||||
|
# run P2P test cases with concurrent station interface
|
||||||
|
./run-all.sh concurrent
|
||||||
|
|
||||||
|
# run P2P test cases with concurrent station interface under valgrind
|
||||||
|
./run-all.sh concurrent-valgrind
|
||||||
|
|
||||||
|
run-all.sh directs debug logs into the logs subdirectory. Log file names
|
||||||
|
include the current UNIX timestamp and a postfix to identify the
|
||||||
|
specific log:
|
||||||
|
- log0 = wpa_supplicant debug log for the first radio
|
||||||
|
- log1 = wpa_supplicant debug log for the second radio
|
||||||
|
- log2 = wpa_supplicant debug log for the third radio
|
||||||
|
- hostapd = hostapd debug log
|
||||||
|
- hwsim0 = wlantest debug log
|
||||||
|
- hwsim0.dump = pcap containing all frames exchanged during the tests
|
||||||
|
- tcpdump = tcpdump output
|
||||||
|
- run = debug prints from the test scripts
|
||||||
|
- trace.dat = Linux tracing record (if enabled)
|
||||||
|
|
||||||
|
|
||||||
|
For manual testing, ./start.sh can be used to initialize interfaces and
|
||||||
|
programs and run-tests.py to execute one or more test
|
||||||
|
cases. run-tests.py output verbosity can be controlled with -d (more
|
||||||
|
verbose debug output) and -q (less verbose output) on the command
|
||||||
|
line. "-f <test_*.py>" can be used to specify that all test cases from a
|
||||||
|
single file are to be run. Test name as the last command line argument
|
||||||
|
can be specified that a single test case is to be run (e.g.,
|
||||||
|
"./run-tests.py test_ap_pmf_required").
|
||||||
|
|
||||||
|
|
||||||
|
Adding/modifying test cases
|
||||||
|
---------------------------
|
||||||
|
|
||||||
|
All the test cases are defined in the test_*.py files. These are python
|
||||||
|
scripts that can use the local helper classes to interact with the test
|
||||||
|
components. While various python constructs can be used in the scripts,
|
||||||
|
only a minimal level of python knowledge should really be needed to
|
||||||
|
modify and add new test cases. The easiest starting point for this is
|
||||||
|
likely to take a look at some of the example scripts. When working on a
|
||||||
|
new test, run-tests.py with -d and the test case name on the command
|
||||||
|
line is a convenient way of verifying functionality.
|
||||||
|
|
||||||
|
run-tests.py will automatically import all test cases from the test_*.py
|
||||||
|
files in this directory. All functions starting with the "test_" prefix
|
||||||
|
in these files are assumed to be test cases.
|
||||||
68
tests/hwsim/example-hostapd.config
Normal file
68
tests/hwsim/example-hostapd.config
Normal file
|
|
@ -0,0 +1,68 @@
|
||||||
|
#CC=ccache gcc
|
||||||
|
|
||||||
|
CONFIG_DRIVER_TEST=y
|
||||||
|
CONFIG_DRIVER_NONE=y
|
||||||
|
CONFIG_DRIVER_NL80211=y
|
||||||
|
CONFIG_RSN_PREAUTH=y
|
||||||
|
|
||||||
|
#CONFIG_TLS=internal
|
||||||
|
#CONFIG_INTERNAL_LIBTOMMATH=y
|
||||||
|
#CONFIG_INTERNAL_LIBTOMMATH_FAST=y
|
||||||
|
CONFIG_TLS=openssl
|
||||||
|
|
||||||
|
CONFIG_EAP=y
|
||||||
|
CONFIG_EAP_MD5=y
|
||||||
|
CONFIG_EAP_TLS=y
|
||||||
|
CONFIG_EAP_MSCHAPV2=y
|
||||||
|
CONFIG_EAP_PEAP=y
|
||||||
|
CONFIG_EAP_GTC=y
|
||||||
|
CONFIG_EAP_TTLS=y
|
||||||
|
CONFIG_EAP_SIM=y
|
||||||
|
CONFIG_EAP_AKA=y
|
||||||
|
CONFIG_EAP_AKA_PRIME=y
|
||||||
|
CONFIG_EAP_GPSK=y
|
||||||
|
CONFIG_EAP_GPSK_SHA256=y
|
||||||
|
CONFIG_EAP_SAKE=y
|
||||||
|
CONFIG_EAP_PAX=y
|
||||||
|
CONFIG_EAP_PSK=y
|
||||||
|
CONFIG_EAP_VENDOR_TEST=y
|
||||||
|
CONFIG_EAP_FAST=y
|
||||||
|
CONFIG_EAP_IKEV2=y
|
||||||
|
CONFIG_EAP_TNC=y
|
||||||
|
CONFIG_EAP_UNAUTH_TLS=y
|
||||||
|
ifeq ($(CONFIG_TLS), openssl)
|
||||||
|
CONFIG_EAP_PWD=y
|
||||||
|
endif
|
||||||
|
CONFIG_PKCS12=y
|
||||||
|
CONFIG_RADIUS_SERVER=y
|
||||||
|
CONFIG_IPV6=y
|
||||||
|
CONFIG_TLSV11=y
|
||||||
|
CONFIG_TLSV12=y
|
||||||
|
|
||||||
|
CONFIG_FULL_DYNAMIC_VLAN=y
|
||||||
|
CONFIG_LIBNL32=y
|
||||||
|
CONFIG_PEERKEY=y
|
||||||
|
CONFIG_IEEE80211W=y
|
||||||
|
CONFIG_IEEE80211R=y
|
||||||
|
CONFIG_IEEE80211N=y
|
||||||
|
CONFIG_IEEE80211AC=y
|
||||||
|
|
||||||
|
CONFIG_WPS=y
|
||||||
|
CONFIG_WPS2=y
|
||||||
|
CONFIG_WPS_UPNP=y
|
||||||
|
CONFIG_WPS_NFC=y
|
||||||
|
#CONFIG_WPS_STRICT=y
|
||||||
|
CONFIG_WPA_TRACE=y
|
||||||
|
CONFIG_WPA_TRACE_BFD=y
|
||||||
|
|
||||||
|
CONFIG_P2P_MANAGER=y
|
||||||
|
CONFIG_DEBUG_FILE=y
|
||||||
|
CONFIG_WPA_CLI_EDIT=y
|
||||||
|
CONFIG_ACS=y
|
||||||
|
CONFIG_NO_RANDOM_POOL=y
|
||||||
|
CONFIG_WNM=y
|
||||||
|
CONFIG_INTERWORKING=y
|
||||||
|
CONFIG_HS20=y
|
||||||
|
#CONFIG_SQLITE=y
|
||||||
|
CONFIG_SAE=y
|
||||||
|
CFLAGS += -DALL_DH_GROUPS
|
||||||
111
tests/hwsim/example-wpa_supplicant.config
Normal file
111
tests/hwsim/example-wpa_supplicant.config
Normal file
|
|
@ -0,0 +1,111 @@
|
||||||
|
#CC=ccache gcc
|
||||||
|
|
||||||
|
CONFIG_TLS=openssl
|
||||||
|
#CONFIG_TLS=internal
|
||||||
|
#CONFIG_INTERNAL_LIBTOMMATH=y
|
||||||
|
#CONFIG_INTERNAL_LIBTOMMATH_FAST=y
|
||||||
|
|
||||||
|
CONFIG_IEEE8021X_EAPOL=y
|
||||||
|
|
||||||
|
CONFIG_EAP_MD5=y
|
||||||
|
CONFIG_MSCHAPV2=y
|
||||||
|
CONFIG_EAP_TLS=y
|
||||||
|
CONFIG_EAP_PEAP=y
|
||||||
|
CONFIG_EAP_TTLS=y
|
||||||
|
CONFIG_EAP_GTC=y
|
||||||
|
CONFIG_EAP_OTP=y
|
||||||
|
CONFIG_EAP_PSK=y
|
||||||
|
CONFIG_EAP_PAX=y
|
||||||
|
CONFIG_EAP_LEAP=y
|
||||||
|
CONFIG_EAP_SIM=y
|
||||||
|
CONFIG_EAP_AKA=y
|
||||||
|
CONFIG_EAP_AKA_PRIME=y
|
||||||
|
CONFIG_EAP_VENDOR_TEST=y
|
||||||
|
CONFIG_EAP_TLV=y
|
||||||
|
CONFIG_EAP_SAKE=y
|
||||||
|
CONFIG_EAP_GPSK=y
|
||||||
|
CONFIG_EAP_GPSK_SHA256=y
|
||||||
|
CONFIG_EAP_EKE=y
|
||||||
|
CONFIG_EAP_TNC=y
|
||||||
|
CONFIG_EAP_FAST=y
|
||||||
|
CONFIG_EAP_IKEV2=y
|
||||||
|
|
||||||
|
ifeq ($(CONFIG_TLS), openssl)
|
||||||
|
CONFIG_EAP_PWD=y
|
||||||
|
endif
|
||||||
|
|
||||||
|
CONFIG_USIM_SIMULATOR=y
|
||||||
|
CONFIG_SIM_SIMULATOR=y
|
||||||
|
|
||||||
|
#CONFIG_PCSC=y
|
||||||
|
CONFIG_IPV6=y
|
||||||
|
CONFIG_DRIVER_NONE=y
|
||||||
|
CONFIG_PKCS12=y
|
||||||
|
CONFIG_CTRL_IFACE=unix
|
||||||
|
|
||||||
|
CONFIG_WPA_CLI_EDIT=y
|
||||||
|
|
||||||
|
CONFIG_OCSP=y
|
||||||
|
|
||||||
|
#CONFIG_ELOOP_POLL=y
|
||||||
|
|
||||||
|
#CONFIG_CTRL_IFACE_DBUS=y
|
||||||
|
#CONFIG_CTRL_IFACE_DBUS_NEW=y
|
||||||
|
#CONFIG_CTRL_IFACE_DBUS_INTRO=y
|
||||||
|
|
||||||
|
CONFIG_PEERKEY=y
|
||||||
|
CONFIG_IEEE80211W=y
|
||||||
|
CONFIG_IEEE80211R=y
|
||||||
|
CONFIG_IEEE80211N=y
|
||||||
|
|
||||||
|
CONFIG_DEBUG_FILE=y
|
||||||
|
|
||||||
|
CONFIG_WPS=y
|
||||||
|
CONFIG_WPS2=y
|
||||||
|
#CONFIG_WPS_STRICT=y
|
||||||
|
CONFIG_WPS_UPNP=y
|
||||||
|
CONFIG_WPS_NFC=y
|
||||||
|
CONFIG_WPS_ER=y
|
||||||
|
#CONFIG_WPS_REG_DISABLE_OPEN=y
|
||||||
|
|
||||||
|
CONFIG_DRIVER_NL80211=y
|
||||||
|
CFLAGS += -I/usr/include/libnl3
|
||||||
|
CONFIG_LIBNL32=y
|
||||||
|
|
||||||
|
CONFIG_IBSS_RSN=y
|
||||||
|
|
||||||
|
CONFIG_AP=y
|
||||||
|
CONFIG_P2P=y
|
||||||
|
CONFIG_WIFI_DISPLAY=y
|
||||||
|
|
||||||
|
CONFIG_BGSCAN_SIMPLE=y
|
||||||
|
CONFIG_BGSCAN_LEARN=y
|
||||||
|
|
||||||
|
CONFIG_WPA_TRACE=y
|
||||||
|
CONFIG_WPA_TRACE_BFD=y
|
||||||
|
|
||||||
|
CONFIG_TDLS=y
|
||||||
|
CONFIG_TDLS_TESTING=y
|
||||||
|
CONFIG_NO_RANDOM_POOL=y
|
||||||
|
|
||||||
|
CONFIG_TLSV11=y
|
||||||
|
CONFIG_TLSV12=y
|
||||||
|
|
||||||
|
#CONFIG_HT_OVERRIDES=y
|
||||||
|
|
||||||
|
CONFIG_DEBUG_LINUX_TRACING=y
|
||||||
|
|
||||||
|
CONFIG_INTERWORKING=y
|
||||||
|
CONFIG_HS20=y
|
||||||
|
|
||||||
|
CONFIG_AUTOSCAN_EXPONENTIAL=y
|
||||||
|
CONFIG_AUTOSCAN_PERIODIC=y
|
||||||
|
|
||||||
|
CONFIG_EXT_PASSWORD_TEST=y
|
||||||
|
|
||||||
|
CONFIG_EAP_UNAUTH_TLS=y
|
||||||
|
|
||||||
|
CONFIG_SAE=y
|
||||||
|
CFLAGS += -DALL_DH_GROUPS
|
||||||
|
|
||||||
|
CONFIG_WNM=y
|
||||||
Loading…
Reference in a new issue