Commit graph

250 commits

Author SHA1 Message Date
Johannes Berg
a572f0b960 hwsim tests: Print failed tests nicer
There's no reason to format the failed tests as a python
list, just print a (space-separated) list of test names.

Signed-hostap: Johannes Berg <johannes.berg@intel.com>
2013-11-17 16:46:41 +02:00
Jouni Malinen
0c9057970b tests: Do not override existing .config from build.sh
This allows the build.sh script to be used to rebuild binaries based on
existing configuration in addition to the initial case of preparing
suitable build configuration.

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-11-09 15:40:06 +02:00
Jouni Malinen
b4b89ae3aa tests: Remove duplicated mkdir for LOGDIR
LOGDIR directory is created automatically if LOGDIR variable was not
set. If the variable is set, that directory is expected to exist.

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-11-09 15:34:29 +02:00
Michal Kazior
d65a593b1e tests: Use proper admin group in start.sh
wpa_supplicant configuration files had a hardcoded GROUP=admin. The
start.sh script failed on a system without admin group (some systems
have it named adm).

Generate configuration files with appropriate GROUP in the log dir
and use them.

Signed-hostap: Michal Kazior <michal.kazior@tieto.com>
2013-11-09 15:31:35 +02:00
Jouni Malinen
483691bd73 tests: Verify GAS query while associated
Signed-hostap: Jouni Malinen <jouni@qca.qualcomm.com>
2013-11-08 00:41:58 +02:00
Jouni Malinen
836a374566 tests: Verify GAS fragmentation
Signed-hostap: Jouni Malinen <jouni@qca.qualcomm.com>
2013-11-08 00:22:33 +02:00
Jouni Malinen
469f5f3c93 tests: Use only new scan results for Hotspot 2.0 test cases
It looks like old scan results (different SSID for the same BSSID) could
cause test failures.

Signed-hostap: Jouni Malinen <jouni@qca.qualcomm.com>
2013-11-08 00:22:33 +02:00
Johannes Berg
4ecf11c559 hwsim tests: Make channel configuration for VM tests easier
Add a CHANNELS configuration to the script running the VM
that can be added to the vm-config file to allow running
the tests with hwsim devices supporting more than a single
channel.

Eventually, with the (hopefully) upcoming dynamic work in
mac80211_hwsim, this might go away entirely, but for now
this allows testing more code paths.

Signed-hostap: Johannes Berg <johannes.berg@intel.com>
2013-11-07 14:01:06 +02:00
Jouni Malinen
b4264f8f67 tests: Verify disallow_aps with Interworking
Signed-hostap: Jouni Malinen <jouni@qca.qualcomm.com>
2013-11-07 00:12:30 +02:00
Jouni Malinen
6a0b400273 tests: Verify no-duplicate-networks with Interworking connection
Signed-hostap: Jouni Malinen <jouni@qca.qualcomm.com>
2013-11-07 00:12:30 +02:00
Jouni Malinen
dd6f538ea5 tests: Verify BSS additional/removal during HT co-ex scan
This verifies that hostapd can add and remove a secondary BSS
during an ongoing HT co-ex scan.

Signed-hostap: Jouni Malinen <jouni@qca.qualcomm.com>
2013-11-06 15:52:40 +02:00
Jouni Malinen
742117ad88 tests: BSS removal during ongoing HT or ACS scan
These verify that hostapd is able to remove a BSS while the radio is
still in the progress of the initial channel parameter determination.

Signed-hostap: Jouni Malinen <jouni@qca.qualcomm.com>
2013-11-06 01:11:25 +02:00
Jouni Malinen
a364852be8 tests: Adjust BSS add/remove test cases for primary BSS constraint
Since removal of the primary BSS is now going to remove all the BSSs
for a radio, these two test cases need changes to not trigger false
failures.

Signed-hostap: Jouni Malinen <jouni@qca.qualcomm.com>
2013-11-06 01:08:54 +02:00
Jouni Malinen
66f0bdb465 tests: Handle hostapd dying more gracefully
Catch exceptions from operations that try to remove hostapd interface
and rename the log file. If these operations fail due to socket
connection issues, hostapd has likely died or gotten stuck somewhere.
Report the test case as a failure and stop test run cleanly.

Signed-hostap: Jouni Malinen <jouni@qca.qualcomm.com>
2013-11-05 13:21:58 +02:00
Johannes Berg
5061a301e5 hwsim tests: Check kernel messages for warnings/bugs
When a test passes but the kernel printed warnings, consider
the test to have failed.

Signed-hostap: Johannes Berg <johannes.berg@intel.com>
2013-11-05 10:17:35 +02:00
Jouni Malinen
d355372ce4 tests: HS 2.0 excluded SSID
Signed-hostap: Jouni Malinen <jouni@qca.qualcomm.com>
2013-11-04 14:48:33 +02:00
Jouni Malinen
d1ba402fe5 tests: HS 2.0 required roaming consortium
Signed-hostap: Jouni Malinen <jouni@qca.qualcomm.com>
2013-11-04 14:48:28 +02:00
Jouni Malinen
2232edf8e2 tests: Use dictionary as add_cred_values() parameter
This makes it more convenient to use this function for cases that modify
credential parameters.

Signed-hostap: Jouni Malinen <jouni@qca.qualcomm.com>
2013-11-04 13:12:34 +02:00
Jouni Malinen
5c5656c37a tests: Do not require hlr_auc_gw for HS 2.0 username/password
Signed-hostap: Jouni Malinen <jouni@qca.qualcomm.com>
2013-11-04 13:12:28 +02:00
Jouni Malinen
8454e1fd33 tests: Add preliminary version of DFS test cases
These are still disabled by default since they require mac80211_hwsim
changes that are not yet in the upstream kernel and because the test
cases do not fully work yet. Anyway, they are quite convenient for
development use, so good to have them available in the main repository.

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-11-03 21:57:39 +02:00
Jouni Malinen
1b3926ef73 tests: Add a test case for HT40 co-ex scan
Signed-hostap: Jouni Malinen <j@w1.fi>
2013-11-03 21:30:31 +02:00
Jouni Malinen
b47750beba tests: Add test cases for automatic channel selection
This extends the Hostapd class to support monitor interface events and
STATUS command similarly to the WpaSupplicant class so that internal
hostapd state can be verified in more detail.

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-11-03 21:30:31 +02:00
Jouni Malinen
77990cd7a9 tests: Add test case for multi-BSS configuration file
Verify that a single configuration file can be used to initiate multiple
BSSes in hostapd and that these BSSes can then be dynamically removed.

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-11-03 20:20:50 +02:00
Jouni Malinen
e114c49cfc tests: Add an EAP-TLS test case
This fixes the user.key file (incorrect key was copied previously) and
adds a test case for EAP-TLS with WPA2-Enterprise.

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-11-03 19:51:06 +02:00
Jouni Malinen
22b99086ce tests: Add more EAP test cases
This increases EAP method coverage for WPA2-Enterprise to include
EAP-pwd, EAP-GPSK, EAP-SAKE, EAP-EKE, EAP-IKEv2, EAP-PAX, and EAP-PSK.

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-11-03 19:51:06 +02:00
Jouni Malinen
b5476d940e tests: Add run-tests.py --no-reset
This can be used to request the previously used default behavior where
the devices are not stopped at the end of a test case if a single test
case is run.

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-11-03 11:31:48 +02:00
Jouni Malinen
5230bf64e8 tests: hostapd behavior on dynamic addition of invalid config
Verify that hostapd handles ENABLE command on invalid configuration
correctly and allows the configuration to be fixed or the interface to
be removed.

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-11-03 11:31:48 +02:00
Jouni Malinen
1640a2e424 tests: Add test cases for SAE
Signed-hostap: Jouni Malinen <j@w1.fi>
2013-11-03 11:31:48 +02:00
Jouni Malinen
f5f59e4f57 tests: Clean up output formatting with str.format()
Signed-hostap: Jouni Malinen <j@w1.fi>
2013-11-02 17:07:13 +02:00
Jouni Malinen
bafd8bfa57 tests: Remove trailing whitespace
Signed-hostap: Jouni Malinen <j@w1.fi>
2013-11-02 15:40:36 +02:00
Jouni Malinen
f01edbcfac tests: Rename stop-wifi.sh to stop.sh
This makes script naming more consistent with start.sh.

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-11-02 15:39:25 +02:00
Jouni Malinen
78060b25a1 tests: Simplify test_ap_bss_add_remove implementation
Use lists and loops to avoid duplicated operations.

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-11-02 13:53:23 +02:00
Jouni Malinen
57661377df tests: Use a helper function to rename wpa_supplicant/hostapd logs
Signed-hostap: Jouni Malinen <j@w1.fi>
2013-11-02 13:04:06 +02:00
Jouni Malinen
8f47f31dd8 tests: Always write run-tests.py debug logs a file
Remove the -l command like option from run-tests.py and always enable
writing of debug level logs to files. The stdout debug verbosity is
controlled independently of the debug log files.

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-11-02 12:20:59 +02:00
Jouni Malinen
75428961d9 tests: Split hostapd debug log into per test case files
Signed-hostap: Jouni Malinen <j@w1.fi>
2013-11-02 11:53:38 +02:00
Jouni Malinen
d956022c11 tests: Ignore old scan results in test_grpform_pbc
This is needed to avoid invalid PBC session overlap detection when
the previous test case used active PBC mode and the old BSS entry
in cfg80211 may still be valid when starting the next test case.

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-11-02 11:27:16 +02:00
Jouni Malinen
c65f23ab64 tests: Use single channel scans to speed up test cases
A full scan in these specific test cases does not add anything to the
coverage, so use a single channel scan for the station connection to
remove undesired extra time needed for a full scan.

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-11-02 11:22:16 +02:00
Jouni Malinen
de684a2b83 tests: Add run-tests.py --shuffle-test
This optional argument can be used to randomize the order in which the
test cases are run. This can provide more coverage on testing
interactions of common use cases in various different sequences. Such
issues have already been found even with the fixed order of test cases,
but being able to reorder the tests makes this more efficient.

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-11-02 10:30:37 +02:00
Johannes Berg
fdbc092389 hwsim tests: Add build script
This is easier than to copy/paste from the README. We may
want to extend it later to change the .config for some
common differences between systems (e.g., libnl/libbfd).

Signed-off-by: Johannes Berg <johannes.berg@intel.com>
2013-11-02 10:17:46 +02:00
Johannes Berg
a5d7da3fd3 hwsim tests: Prefill database in VM tests
In some cases, e.g., with the VM tests if the VM crashes, it
can be useful to know which tests should have run but didn't
(or didn't finish). In order to catch these more easily, add
an option to prefill the database with all tests at the very
beginning of the testing (in a new NOTRUN state) and use the
option in the VM tests.

Signed-hostap: Johannes Berg <johannes.berg@intel.com>
2013-11-02 10:17:44 +02:00
Johannes Berg
63f83fac0d hwsim tests: Create results database in VM tests
Create a results.db in the output directory when running
the tests in a VM. To make that easier, create the tables
in the python script if they don't exist.

Signed-hostap: Johannes Berg <johannes.berg@intel.com>
2013-11-02 10:16:28 +02:00
Johannes Berg
4a1b8901ab hwsim tests: Refactor test results reporting
Refactor the test reporting to treat the different results
(success/skip/failure) identically. This makes the timing
seem a bit longer, but cleans up the code which will allow
for adding more checks (e.g., on the captured data files)
later.

Signed-hostap: Johannes Berg <johannes.berg@intel.com>
2013-11-02 10:03:14 +02:00
Johannes Berg
1cd3eae362 hwsim tests: Allow setting KERNEL and KVMARGS
Rather than just having KERNELDIR, allow setting KERNEL directly.
Also remove the -s option that prevents running multiple machines
at the same time, but add a KVMARGS= variable that can be used to
restore that if needed.

Signed-hostap: Johannes Berg <johannes.berg@intel.com>
2013-11-02 10:01:09 +02:00
Jouni Malinen
a633397756 tests: Test dynamic BSS addition/removal
test_ap_bss_add_remove verifies hostapd behavior when BSSes are
added/removed in multi-BSS configuration.

Signed-hostap: Jouni Malinen <jouni@qca.qualcomm.com>
2013-10-31 23:04:50 +02:00
Jouni Malinen
a1ddc79adf tests: Remove separate error/failed text file
The new sqlite database for results deprecated this.

Signed-hostap: Jouni Malinen <jouni@qca.qualcomm.com>
2013-10-31 14:39:09 +02:00
Jouni Malinen
904b6e7e75 tests: Remove separate results text file
The new sqlite database for results deprecated this.

Signed-hostap: Jouni Malinen <jouni@qca.qualcomm.com>
2013-10-31 14:36:26 +02:00
Jouni Malinen
b72c300267 tests: Use a symlink for default LOGDIR between scripts
This allows run-tests.py to use the same logs/<date> default logdir as
start.sh which is quite convenient for manual test runs.

Signed-hostap: Jouni Malinen <jouni@qca.qualcomm.com>
2013-10-31 14:27:57 +02:00
Jouni Malinen
353e5067ff tests: Split wpa_supplicant debug logs into per test case files
Signed-hostap: Jouni Malinen <jouni@qca.qualcomm.com>
2013-10-31 13:19:38 +02:00
Jouni Malinen
c9aa430896 tests: Split run-tests.py logger info into per test case files
The run-tests.py -l argument does not take an argument value anymore.
Instead, debug output is directed to a separate file <test>.log for each
test case.

Signed-hostap: Jouni Malinen <jouni@qca.qualcomm.com>
2013-10-31 12:51:08 +02:00
Jouni Malinen
e955226208 tests: Remove the 'test_' prefix from test module names
This is unnecessary extra complexity for user, so use the 'test_' prefix
only internally within the python scripts and file names.

Signed-hostap: Jouni Malinen <jouni@qca.qualcomm.com>
2013-10-31 11:49:58 +02:00