Commit graph

120 commits

Author SHA1 Message Date
Jouni Malinen
fd0465b85e tests: Print a summary of SKIP reasons
This makes it easier to determine reasons for test cases being skipped.

Signed-off-by: Jouni Malinen <j@w1.fi>
2019-12-27 16:03:50 +02:00
Jouni Malinen
88b6c6e244 tests: Allow test selection with prefix wildcards
Test names can now use wildcard in the end (e.g., ap_wpa2_psk*) to match
all test cases with the specified prefix.

Signed-off-by: Jouni Malinen <j@w1.fi>
2019-07-09 16:10:44 +03:00
Jouni Malinen
f82a840878 tests: Fix ERP test cases to close separate AS (ifname=as-erp)
The earlier change from using apdev[1] to using a no-interface
ifname=as-erp in the ERP related test cases ended up leaving the
separate authentication server interface running after the end of the
test case. This left UDP port 18128 open and any consecutive test case
needing this port failed to start another authentication server.

This can be reproduced with following test case sequence:
fils_auth_gtk_rekey authsrv_unknown_user

Fixes: e374def207 ("tests: Start ERP authentication server without AP")
Signed-off-by: Jouni Malinen <j@w1.fi>
2019-05-25 19:07:11 +03:00
Jouni Malinen
e467d7ab4a tests: Show ifname in country code not cleared messages
Signed-off-by: Jouni Malinen <j@w1.fi>
2019-04-13 12:06:09 +03:00
Johannes Berg
61876c7eed tests: Fix trace-cmd recording with python3
The change to python3 broke trace-cmd recording, two
strings need to be bytes instead. Fix that.

Also add a flush() that seems to be needed now.

Signed-off-by: Johannes Berg <johannes.berg@intel.com>
2019-04-12 20:43:33 +03:00
Jouni Malinen
e01a492caa tests: Helper function for DISCONNECT + ABORT_SCAN + wait
Use a helper function to perform this common sequence to disconnect and
stop any possibly started reconnection attempt.

Signed-off-by: Jouni Malinen <j@w1.fi>
2019-03-17 17:58:33 +02:00
Jouni Malinen
b21540e682 tests: Close wpa_supplicant control interface more robustly
Some of the test cases left behind attached control interface monitor
sockets that could result in hitting the wpa_supplicant socket TX queue
limit. Try to be a bit more careful about detaching and closing the
sockets to avoid this.

Signed-off-by: Jouni Malinen <j@w1.fi>
2019-03-17 16:08:30 +02:00
Jouni Malinen
fab49f6145 tests: Python coding style cleanup (pylint3 bad-whitespace)
Signed-off-by: Jouni Malinen <jouni@codeaurora.org>
2019-03-16 18:52:09 +02:00
Jouni Malinen
a8b8da1132 tests: Python coding style cleanup (pylint3 unneeded-not)
Use more readable "foo not in bar" construction for the couple of places
that did "not foo in bar".

Signed-off-by: Jouni Malinen <jouni@codeaurora.org>
2019-03-15 11:34:32 +02:00
Jouni Malinen
2f22ed4fab tests: Fix run-tests.py module file (-l) handling
Prior cleanup broke the indentation here and made the addition of test
cases unreachable.

Fixes: 0663ae22ff ("tests: Do not use tabs for indentation")
Signed-off-by: Jouni Malinen <jouni@codeaurora.org>
2019-03-15 11:18:20 +02:00
Jouni Malinen
5a3a131993 tests: Mark log files to use utf-8 encoding
This seems to be needed when using python3 in VM for the ssid_utf8 test
case debug prints from the control interface requests. This breaks
python2 support for the same logging entries, but there does not seem to
be any easy way of addressing this in a manner that works for both
python versions, so move ahead with the python3-only support from now
on.

Signed-off-by: Jouni Malinen <jouni@codeaurora.org>
2019-02-18 18:19:06 +02:00
Masashi Honma
432496cdd8 tests: Switch default python version to 3
Signed-off-by: Masashi Honma <masashi.honma@gmail.com>
2019-02-17 17:26:01 +02:00
Johannes Berg
b513ed429d tests: Remove useless test list from run-tests.py help
There's no point in printing out a 3k+ long list, just remove it.

Signed-off-by: Johannes Berg <johannes.berg@intel.com>
2019-02-05 23:51:36 +02:00
Jouni Malinen
6395b9eb3e tests: Read binary file in add_log_file() for python3
python3 requires sqlite3.Binary() input to be bytes instead of str, so
open the files for binary mode reading.

Signed-off-by: Jouni Malinen <j@w1.fi>
2019-02-04 12:26:33 +02:00
Masashi Honma
8c851c2e36 tests: Use python3 compatible function attribute
This patch is made by using 2to3 command.

$ find . -name *.py | xargs 2to3 -f funcattrs -w -n

Signed-off-by: Masashi Honma <masashi.honma@gmail.com>
2019-01-26 13:00:03 +02:00
Masashi Honma
35d8c2545e tests: Use python3 compatible dict operation
This patch is made by using 2to3 command.

$ find . -name *.py | xargs 2to3 -f dict -w -n

Signed-off-by: Masashi Honma <masashi.honma@gmail.com>
2019-01-26 12:53:05 +02:00
Masashi Honma
89896c0008 tests: Use python3 compatible print statement
This patch is made by using 2to3 command.

$ find . -name *.py | xargs 2to3 -f print -w -n

Signed-off-by: Masashi Honma <masashi.honma@gmail.com>
2019-01-26 12:37:25 +02:00
Masashi Honma
bab493b904 tests: Use python3 compatible "except" statement
This patch is made by using 2to3 command.

$ find . -name *.py | xargs 2to3 -f except -w -n

Signed-off-by: Masashi Honma <masashi.honma@gmail.com>
2019-01-26 12:11:35 +02:00
Jouni Malinen
c55799e54c tests: Check per-wiphy specific country code between test cases
This allows more accurate logging of failures related to the cfg80211
country=98 (intersection) case. This version is trying to give some more
time to allow the country code to clear, but that does not seem to be
sufficient with the current cfg80211 implementation for country=98 (but
might be for other cases).

The additional check for country=98 at the beginning of each test case
is an attempt to force cfg80211 to restore world roaming state with a
new association and disconnection at the station side detected after the
AP side has already stopped. This is needed after the Linux kernel
commit 113f3aaa81bd ("cfg80211: Prevent regulatory restore during STA
disconnect in concurrent interfaces").

Signed-off-by: Jouni Malinen <j@w1.fi>
2019-01-01 11:46:50 +02:00
Vasyl Vavrychuk
2494bcef24 tests: Add option --dbus for logging D-Bus messages
Signed-off-by: Vasyl Vavrychuk <vvavrychuk@gmail.com>
2018-02-17 18:21:47 +02:00
Jouni Malinen
8171d70406 tests: Write TEST-START and TEST-STOP into kernel logs with timestamps
This makes it easier to synchronize log entries in the kernel log
(seconds from boot) and wpa_supplicant/hostapd (UNIX timestamp).

Signed-off-by: Jouni Malinen <j@w1.fi>
2017-05-27 11:08:16 +03:00
Masashi Honma
d760db19b1 tests: Save the log of wmediumd
Signed-off-by: Masashi Honma <masashi.honma@gmail.com>
2017-03-01 00:35:40 +02:00
Jouni Malinen
1edc31904a tests: Beacon request - active scan mode and many BSSs
Signed-off-by: Jouni Malinen <jouni@qca.qualcomm.com>
2017-01-03 19:53:03 +02:00
Jouni Malinen
bb5d761c2e tests: Do not attach wpa_supplicant monitor interface unnecessarily
There is no need to attach the monitor interface was events when issuing
only a single INTERFACES command.

Signed-off-by: Jouni Malinen <jouni@qca.qualcomm.com>
2016-12-14 13:56:15 +02:00
Jouni Malinen
78a37200f8 tests: Reset wlan5 device before RELOG command
This moves the wpa_supplicant debug entries from the end of a test case
using a dynamically added wlan5 interface to the correct test case,
i.e., the test case that added the interface instead of whatever test
case happens to follow this.

Signed-off-by: Jouni Malinen <jouni@qca.qualcomm.com>
2016-09-23 16:03:59 +03:00
Jonathan Afek
8efc83d4e7 tests: Add support for wlantest for remote hwsim tests
Use a monitor interface given in the command line that is not also a
station or an AP as a monitor running wlantest on the channel used by
the test. This makes all the tests that use wlantest available for
execution on real hardware on remote hosts.

Signed-off-by: Jonathan Afek <jonathanx.afek@intel.com>
2016-05-28 16:34:09 +03:00
Jouni Malinen
0663ae22ff tests: Do not use tabs for indentation
Be more consistent with indentation (always uses spaces in Python
files).

Signed-off-by: Jouni Malinen <j@w1.fi>
2016-04-25 00:19:40 +03:00
Janusz Dziedzic
2147b3a7f8 tests: Print traceback if test fails
This is useful in case we hit a problem in test code.

Signed-off-by: Janusz Dziedzic <janusz.dziedzic@tieto.com>
2016-04-24 20:16:30 +03:00
Johannes Berg
1a4f18d8f4 tests: run-tests: Print more details about NameError
If encountering a NameError, print the entire traceback so that
it's actually debuggable.

Signed-off-by: Johannes Berg <johannes.berg@intel.com>
2015-11-28 00:21:20 +02:00
Johannes Berg
f2a438f51d tests: Print test name to kernel log buffer
Print the test name to the kernel log buffer so debugging
cross-test issues becomes easier.

Signed-off-by: Johannes Berg <johannes.berg@intel.com>
2015-11-24 16:28:24 +02:00
Jouni Malinen
a302ee342c tests: Use sqlite3.Binary() with the log files
This is needed to avoid issues in some cases where 8-bit bytestrings may
be present in the otherwise text debug log.

Signed-off-by: Jouni Malinen <j@w1.fi>
2015-09-07 16:53:23 +03:00
Jouni Malinen
1f9427c9cd tests: Explicitly close hostapd global control socket on RELOG
Without this, the run-tests.py socket could have been left in attached
to receive all hostapd global events during a test case. This could hit
the limit of pending messages on the socket since there is nothing
clearing this socket during the execution of a test case. Fix this by
explicitly closing the socket after having completed the RELOG command.

Signed-off-by: Jouni Malinen <j@w1.fi>
2015-07-26 18:12:09 +03:00
Anton Nayshtut
41a256ecd9 tests: Add FST module tests
Signed-off-by: Jouni Malinen <jouni@qca.qualcomm.com>
2015-07-16 18:26:16 +03:00
Johannes Berg
d2002f8334 tests: Allow running with arbitrary working directory
It's somewhat annoying that you can only run parallel-vm.py as
./parallel-vm.py, not from elsewhere by giving the full path,
so fix that by resolving the paths correctly in the scripts where
needed.

Signed-off-by: Johannes Berg <johannes.berg@intel.com>
2015-03-07 12:23:07 +02:00
Jouni Malinen
cfb5576d93 tests: Rename dmesg files to a unique name
This allows multiple dmesg files to be saved if a test case is executed
multiple times similarly to the other logfiles.

Signed-off-by: Jouni Malinen <j@w1.fi>
2015-02-21 16:07:53 +02:00
Jouni Malinen
c4668009d0 tests: Remove unnecessary use of sudo from test cases
run-tests.py is running as root, so sudo does not need to be used
anymore from within each test case.

Signed-off-by: Jouni Malinen <jouni@qca.qualcomm.com>
2015-02-07 15:37:13 +02:00
Eliad Peller
a918e48af1 tests: Add printk tracer to trace-cmd
Signed-off-by: Eliad Peller <eliadx.peller@intel.com>
2015-02-05 13:48:40 +02:00
Jouni Malinen
a66d2248a0 tests: Close wlan5 control interface monitor more explicitly
There were couple of common cases where the control interface for the
dynamic wpa_supplicant instance could have been left in attached state
until Python ends up cleaning up the instance. This could result in
issues if many monitor interface events were queued for that attached
socket. Make this less likely to cause issues by explicitly detaching
and closing control interfaces before moving to the next test case.

Signed-off-by: Jouni Malinen <j@w1.fi>
2015-01-18 17:13:55 +02:00
Jouni Malinen
81e787b750 tests: Convert test skipping to use exception
Instead of returning "skip" from the test function, raise the new
HwsimSkip exception to indicate a test case was skipped.

Signed-off-by: Jouni Malinen <j@w1.fi>
2015-01-08 22:43:47 +02:00
Jouni Malinen
51c5aeb42a tests: Convert "hwsim-SKIP" exception to use a custom class
This makes the design a bit cleaner for catching the exceptions.

Signed-off-by: Jouni Malinen <j@w1.fi>
2015-01-07 16:31:22 +02:00
Jouni Malinen
2ec82e67c1 tests: wpa_supplicant D-Bus interface
This adds new files with test cases to verify both the old and new
wpa_supplicant D-Bus interface.

Signed-off-by: Jouni Malinen <j@w1.fi>
2015-01-02 22:50:27 +02:00
Jouni Malinen
1942b68172 tests: Disable terminal echo for stdin control (-i)
This makes vm-run.sh output cleaner to parse for parallel-vm.py.

Signed-off-by: Jouni Malinen <j@w1.fi>
2014-12-25 16:37:12 +02:00
Jouni Malinen
6ac593ec47 tests: Optimize run-tests.py test case dict iteration
Use a single iteration through the module dictionary rather than
iterating over the keys and separately fetching values to get the
function.

Signed-off-by: Jouni Malinen <j@w1.fi>
2014-12-22 20:08:11 +02:00
Jouni Malinen
9363f5e065 tests: Optimize run-tests.py --prefill-tests startup time
It took significant part of the startup latency to prefill the database
with test cases due to the SQL COMMIT operation between each added row.
Move COMMIT to outside the loop to speed startup significantly.

Signed-off-by: Jouni Malinen <jouni@qca.qualcomm.com>
2014-11-27 20:08:15 +02:00
Jouni Malinen
499c5e9dc6 tests: Follow test sequence from run-tests.py command line
It can be useful to specify an exact order of test cases and also to
allow the same test case to be run multiple times when the list of tests
is provided on the command line.

Signed-off-by: Jouni Malinen <jouni@qca.qualcomm.com>
2014-11-27 19:48:41 +02:00
Jouni Malinen
f6d1c5d9f8 tests: Fix regression in START line total count
The previous changes to enable stdin control broke the previous case of
showing the total number of test cases in the START lines. Fix that by
using a separate variable for the total number of test casess instead of
using length of the list of remaining test cases.

Signed-off-by: Jouni Malinen <j@w1.fi>
2014-11-19 22:02:08 +02:00
Jouni Malinen
0d7456e9fd tests: Add run-tests.py -i to control execution from stdin
The new -i command line argument can be used to control test case
execution from stdin to run-tests.py and vm-run.sh. This can be used,
e.g., to run multiple repeated test sequences in a virtual machine
without havign to restart the VM between each iteration.

Signed-off-by: Jouni Malinen <j@w1.fi>
2014-11-19 15:20:53 +02:00
Jouni Malinen
cb8cb33e55 tests: Allow run-tests.py to be run from the vm directory
Some of the paths in run-tests.py are hardcoded and could not handle
tests/hwsim/vm as the working directory. Modify the design enough to
allow ../run-tests.py -L to be used from the vm directory.

Signed-off-by: Jouni Malinen <j@w1.fi>
2014-11-19 15:20:53 +02:00
Jouni Malinen
5e3c0b02a7 tests: Make run-tests.py process test case selection arguments
This allows a list of matching test cases to be produced without having
to run the test cases. Previously, -L output included all defined test
cases regardless of what else was included on the command line.

Signed-off-by: Jouni Malinen <j@w1.fi>
2014-11-19 15:20:52 +02:00
Ilan Peer
3bfa5f10a9 tests: Add support for reading modules list from a file
The new run-tests.py argument "-l <file>" can now be used to specify the
test modules using a text file.

Signed-off-by: Ilan Peer <ilan.peer@intel.com>
2014-10-26 23:27:26 +02:00