From 883ab1b9f1426e145352d58d955f6f54a02ae7c5 Mon Sep 17 00:00:00 2001 From: Jouni Malinen Date: Sun, 29 Sep 2013 15:55:43 +0300 Subject: [PATCH] tests: Document automated mac80211_hwsim test setup This provides guidance on how to build the test setup, how to run the tests, and how to add new test cases. Signed-hostap: Jouni Malinen --- tests/hwsim/README | 175 ++++++++++++++++++++++ tests/hwsim/example-hostapd.config | 68 +++++++++ tests/hwsim/example-wpa_supplicant.config | 111 ++++++++++++++ 3 files changed, 354 insertions(+) create mode 100644 tests/hwsim/README create mode 100644 tests/hwsim/example-hostapd.config create mode 100644 tests/hwsim/example-wpa_supplicant.config diff --git a/tests/hwsim/README b/tests/hwsim/README new file mode 100644 index 000000000..d2de7afa8 --- /dev/null +++ b/tests/hwsim/README @@ -0,0 +1,175 @@ +Automated hostapd/wpa_supplicant testing with mac80211_hwsim +------------------------------------------------------------ + +This directory contains testing infrastructure and test cases to run +automated tests of full hostapd and wpa_supplicant functionality. This +testing is done with the help of mac80211_hwsim which is Linux kernel +driver that simulates IEEE 802.11 radios without requiring any +additional hardware. This setup most of the hostapd and wpa_supplicant +functionality (and large parts of the Linux cfg80211 and mac80211 +functionality for that matter) to be tested. + +mac80211_hwsim is loaded with five simulated radios to allow different +device combinations to be tested. wlantest is used analyze raw packets +captured through the hwsim0 monitor interface that capture all frames +sent on all channels. tcpdump is used to store the frames for +analysis. Three wpa_supplicant processed are used to control three +virtual radios and one hostapd process is used to dynamically control +the other two virtual radios. hwsim_test is used to verify that data +connection (both unicast and broadcast) works between two netdevs. + +The python scripts and tools in this directory control test case +execution. They interact wpa_supplicant and hostapd through control +interfaces to perform the operations. In addition, wlantest_cli and +hwsim_test are used to verify that operations have been performed +correctly and that the network connection works in the expected way. + +These test cases are run automatically against the hostap.git commits +for regression testing and to help in keeping the hostap.git master +branch in stable state. Results from these tests are available here: +http://buildbot.w1.fi:8010/waterfall + + +Building binaries for testing +----------------------------- + +You will need to build (or use already built) components to be +tested. These are available in the hostap.git repository and can be +built for example as follows: + +cd ../../wpa_supplicant +cp ../tests/hwsim/example-wpa_supplicant.config .config +make clean +make +cd ../hostapd +cp ../tests/hwsim/example-hostapd.config .config +make clean +make +cd ../wlantest +make clean +make +cd ../mac80211_hwsim/tools +make + +The test scripts can find the binaries in the locations where they were +built. It is also possible to install hwsim_test and wlantest_cli +somewhat on the path to use pre-built tools. + + +wpaspy +------ + +The python scripts use wpaspy.py to interact with the wpa_supplicant +control interface. This needs to be installed in a way that python can +find it. For example, with this command: + +sudo cp ../wpaspy/wpaspy.py /usr/local/lib/python2.7/dist-packages + + + +mac80211_hwsim +-------------- + +mac80211_hwsim kernel module is available from the upstream Linux +kernel. Some Linux distributions enable it by default. If that's not the +case, you can either enable it in the kernel configuration +(CONFIG_MAC80211_HWSIM=m) and rebuild your kernel or use Backports with +CPTCFG_MAC80211_HWSIM=m to replace the wireless LAN components in the +base kernel. + + +sudo +---- + +Some parts of the testing process requires root privileges. The test +scripts are currently using sudo to achieve this. To be able to run the +tests, you'll probably want to enable sudo with a timeout to not expire +password entry very quickly. For example, use this in the sudoers file: + +Defaults env_reset,timestamp_timeout=180 + +Or on a dedicated test system, you could even disable password prompting +with this in sudoers: + +%sudo ALL=NOPASSWD: ALL + + +Other network interfaces +------------------------ + +Some of the test scripts are still using hardcoded interface names, so +the easiest way of making things work is to avoid using other network +devices that may use conflicting interface names. For example, unload +any wireless LAN driver before running the tests and make sure that +wlan0..4 gets assigned as the interface names for the mac80211_hwsim +radios. It may also be possible to rename the interface expectations in +run-tests.py to allow other names to be used. + + +Running tests +------------- + +Simplest way to run a full set of the test cases is by running +run-all.sh in tests/hwsim directory. This will use start.sh to load the +mac80211_hwsim module and start wpa_supplicant, hostapd, and various +test tools. run-tests.sh is then used to run through all the defined +test cases and stop-wifi.sh to stop the programs and unload the kernel +module. + +run-all.sh can be used to run the same test cases under different +conditions: + +# run normal test cases +./run-all.sh + +# run normal test cases under valgrind +./run-all.sh valgrind + +# run normal test cases with Linux tracing +./run-all.sh trace + +# run P2P test cases with concurrent station interface +./run-all.sh concurrent + +# run P2P test cases with concurrent station interface under valgrind +./run-all.sh concurrent-valgrind + +run-all.sh directs debug logs into the logs subdirectory. Log file names +include the current UNIX timestamp and a postfix to identify the +specific log: +- log0 = wpa_supplicant debug log for the first radio +- log1 = wpa_supplicant debug log for the second radio +- log2 = wpa_supplicant debug log for the third radio +- hostapd = hostapd debug log +- hwsim0 = wlantest debug log +- hwsim0.dump = pcap containing all frames exchanged during the tests +- tcpdump = tcpdump output +- run = debug prints from the test scripts +- trace.dat = Linux tracing record (if enabled) + + +For manual testing, ./start.sh can be used to initialize interfaces and +programs and run-tests.py to execute one or more test +cases. run-tests.py output verbosity can be controlled with -d (more +verbose debug output) and -q (less verbose output) on the command +line. "-f " can be used to specify that all test cases from a +single file are to be run. Test name as the last command line argument +can be specified that a single test case is to be run (e.g., +"./run-tests.py test_ap_pmf_required"). + + +Adding/modifying test cases +--------------------------- + +All the test cases are defined in the test_*.py files. These are python +scripts that can use the local helper classes to interact with the test +components. While various python constructs can be used in the scripts, +only a minimal level of python knowledge should really be needed to +modify and add new test cases. The easiest starting point for this is +likely to take a look at some of the example scripts. When working on a +new test, run-tests.py with -d and the test case name on the command +line is a convenient way of verifying functionality. + +run-tests.py will automatically import all test cases from the test_*.py +files in this directory. All functions starting with the "test_" prefix +in these files are assumed to be test cases. diff --git a/tests/hwsim/example-hostapd.config b/tests/hwsim/example-hostapd.config new file mode 100644 index 000000000..e0348da06 --- /dev/null +++ b/tests/hwsim/example-hostapd.config @@ -0,0 +1,68 @@ +#CC=ccache gcc + +CONFIG_DRIVER_TEST=y +CONFIG_DRIVER_NONE=y +CONFIG_DRIVER_NL80211=y +CONFIG_RSN_PREAUTH=y + +#CONFIG_TLS=internal +#CONFIG_INTERNAL_LIBTOMMATH=y +#CONFIG_INTERNAL_LIBTOMMATH_FAST=y +CONFIG_TLS=openssl + +CONFIG_EAP=y +CONFIG_EAP_MD5=y +CONFIG_EAP_TLS=y +CONFIG_EAP_MSCHAPV2=y +CONFIG_EAP_PEAP=y +CONFIG_EAP_GTC=y +CONFIG_EAP_TTLS=y +CONFIG_EAP_SIM=y +CONFIG_EAP_AKA=y +CONFIG_EAP_AKA_PRIME=y +CONFIG_EAP_GPSK=y +CONFIG_EAP_GPSK_SHA256=y +CONFIG_EAP_SAKE=y +CONFIG_EAP_PAX=y +CONFIG_EAP_PSK=y +CONFIG_EAP_VENDOR_TEST=y +CONFIG_EAP_FAST=y +CONFIG_EAP_IKEV2=y +CONFIG_EAP_TNC=y +CONFIG_EAP_UNAUTH_TLS=y +ifeq ($(CONFIG_TLS), openssl) +CONFIG_EAP_PWD=y +endif +CONFIG_PKCS12=y +CONFIG_RADIUS_SERVER=y +CONFIG_IPV6=y +CONFIG_TLSV11=y +CONFIG_TLSV12=y + +CONFIG_FULL_DYNAMIC_VLAN=y +CONFIG_LIBNL32=y +CONFIG_PEERKEY=y +CONFIG_IEEE80211W=y +CONFIG_IEEE80211R=y +CONFIG_IEEE80211N=y +CONFIG_IEEE80211AC=y + +CONFIG_WPS=y +CONFIG_WPS2=y +CONFIG_WPS_UPNP=y +CONFIG_WPS_NFC=y +#CONFIG_WPS_STRICT=y +CONFIG_WPA_TRACE=y +CONFIG_WPA_TRACE_BFD=y + +CONFIG_P2P_MANAGER=y +CONFIG_DEBUG_FILE=y +CONFIG_WPA_CLI_EDIT=y +CONFIG_ACS=y +CONFIG_NO_RANDOM_POOL=y +CONFIG_WNM=y +CONFIG_INTERWORKING=y +CONFIG_HS20=y +#CONFIG_SQLITE=y +CONFIG_SAE=y +CFLAGS += -DALL_DH_GROUPS diff --git a/tests/hwsim/example-wpa_supplicant.config b/tests/hwsim/example-wpa_supplicant.config new file mode 100644 index 000000000..536c56bef --- /dev/null +++ b/tests/hwsim/example-wpa_supplicant.config @@ -0,0 +1,111 @@ +#CC=ccache gcc + +CONFIG_TLS=openssl +#CONFIG_TLS=internal +#CONFIG_INTERNAL_LIBTOMMATH=y +#CONFIG_INTERNAL_LIBTOMMATH_FAST=y + +CONFIG_IEEE8021X_EAPOL=y + +CONFIG_EAP_MD5=y +CONFIG_MSCHAPV2=y +CONFIG_EAP_TLS=y +CONFIG_EAP_PEAP=y +CONFIG_EAP_TTLS=y +CONFIG_EAP_GTC=y +CONFIG_EAP_OTP=y +CONFIG_EAP_PSK=y +CONFIG_EAP_PAX=y +CONFIG_EAP_LEAP=y +CONFIG_EAP_SIM=y +CONFIG_EAP_AKA=y +CONFIG_EAP_AKA_PRIME=y +CONFIG_EAP_VENDOR_TEST=y +CONFIG_EAP_TLV=y +CONFIG_EAP_SAKE=y +CONFIG_EAP_GPSK=y +CONFIG_EAP_GPSK_SHA256=y +CONFIG_EAP_EKE=y +CONFIG_EAP_TNC=y +CONFIG_EAP_FAST=y +CONFIG_EAP_IKEV2=y + +ifeq ($(CONFIG_TLS), openssl) +CONFIG_EAP_PWD=y +endif + +CONFIG_USIM_SIMULATOR=y +CONFIG_SIM_SIMULATOR=y + +#CONFIG_PCSC=y +CONFIG_IPV6=y +CONFIG_DRIVER_NONE=y +CONFIG_PKCS12=y +CONFIG_CTRL_IFACE=unix + +CONFIG_WPA_CLI_EDIT=y + +CONFIG_OCSP=y + +#CONFIG_ELOOP_POLL=y + +#CONFIG_CTRL_IFACE_DBUS=y +#CONFIG_CTRL_IFACE_DBUS_NEW=y +#CONFIG_CTRL_IFACE_DBUS_INTRO=y + +CONFIG_PEERKEY=y +CONFIG_IEEE80211W=y +CONFIG_IEEE80211R=y +CONFIG_IEEE80211N=y + +CONFIG_DEBUG_FILE=y + +CONFIG_WPS=y +CONFIG_WPS2=y +#CONFIG_WPS_STRICT=y +CONFIG_WPS_UPNP=y +CONFIG_WPS_NFC=y +CONFIG_WPS_ER=y +#CONFIG_WPS_REG_DISABLE_OPEN=y + +CONFIG_DRIVER_NL80211=y +CFLAGS += -I/usr/include/libnl3 +CONFIG_LIBNL32=y + +CONFIG_IBSS_RSN=y + +CONFIG_AP=y +CONFIG_P2P=y +CONFIG_WIFI_DISPLAY=y + +CONFIG_BGSCAN_SIMPLE=y +CONFIG_BGSCAN_LEARN=y + +CONFIG_WPA_TRACE=y +CONFIG_WPA_TRACE_BFD=y + +CONFIG_TDLS=y +CONFIG_TDLS_TESTING=y +CONFIG_NO_RANDOM_POOL=y + +CONFIG_TLSV11=y +CONFIG_TLSV12=y + +#CONFIG_HT_OVERRIDES=y + +CONFIG_DEBUG_LINUX_TRACING=y + +CONFIG_INTERWORKING=y +CONFIG_HS20=y + +CONFIG_AUTOSCAN_EXPONENTIAL=y +CONFIG_AUTOSCAN_PERIODIC=y + +CONFIG_EXT_PASSWORD_TEST=y + +CONFIG_EAP_UNAUTH_TLS=y + +CONFIG_SAE=y +CFLAGS += -DALL_DH_GROUPS + +CONFIG_WNM=y