Commit graph

258 commits

Author SHA1 Message Date
Johannes Berg
1cd3eae362 hwsim tests: Allow setting KERNEL and KVMARGS
Rather than just having KERNELDIR, allow setting KERNEL directly.
Also remove the -s option that prevents running multiple machines
at the same time, but add a KVMARGS= variable that can be used to
restore that if needed.

Signed-hostap: Johannes Berg <johannes.berg@intel.com>
2013-11-02 10:01:09 +02:00
Jouni Malinen
a633397756 tests: Test dynamic BSS addition/removal
test_ap_bss_add_remove verifies hostapd behavior when BSSes are
added/removed in multi-BSS configuration.

Signed-hostap: Jouni Malinen <jouni@qca.qualcomm.com>
2013-10-31 23:04:50 +02:00
Jouni Malinen
a1ddc79adf tests: Remove separate error/failed text file
The new sqlite database for results deprecated this.

Signed-hostap: Jouni Malinen <jouni@qca.qualcomm.com>
2013-10-31 14:39:09 +02:00
Jouni Malinen
904b6e7e75 tests: Remove separate results text file
The new sqlite database for results deprecated this.

Signed-hostap: Jouni Malinen <jouni@qca.qualcomm.com>
2013-10-31 14:36:26 +02:00
Jouni Malinen
b72c300267 tests: Use a symlink for default LOGDIR between scripts
This allows run-tests.py to use the same logs/<date> default logdir as
start.sh which is quite convenient for manual test runs.

Signed-hostap: Jouni Malinen <jouni@qca.qualcomm.com>
2013-10-31 14:27:57 +02:00
Jouni Malinen
353e5067ff tests: Split wpa_supplicant debug logs into per test case files
Signed-hostap: Jouni Malinen <jouni@qca.qualcomm.com>
2013-10-31 13:19:38 +02:00
Jouni Malinen
c9aa430896 tests: Split run-tests.py logger info into per test case files
The run-tests.py -l argument does not take an argument value anymore.
Instead, debug output is directed to a separate file <test>.log for each
test case.

Signed-hostap: Jouni Malinen <jouni@qca.qualcomm.com>
2013-10-31 12:51:08 +02:00
Jouni Malinen
e955226208 tests: Remove the 'test_' prefix from test module names
This is unnecessary extra complexity for user, so use the 'test_' prefix
only internally within the python scripts and file names.

Signed-hostap: Jouni Malinen <jouni@qca.qualcomm.com>
2013-10-31 11:49:58 +02:00
Jouni Malinen
4cd8343f4d tests: Remove the 'test_' prefix from test names
This is unnecessary extra complexity for user and reports, so use the
'test_' prefix only internally within the python scripts.

Signed-hostap: Jouni Malinen <jouni@qca.qualcomm.com>
2013-10-31 11:43:45 +02:00
Jouni Malinen
d05159ccc5 tests: Replace tcpdump with wlantest file write
There is no need to run a separate tcpdump process to capture frames
from hwsim0 since wlantest is already doing that can write the results
to a file.

Signed-hostap: Jouni Malinen <jouni@qca.qualcomm.com>
2013-10-31 11:35:02 +02:00
Jouni Malinen
79b16a5c0d tests: Remove last-debug creation
This file was used for adding debug info into the buildbot logs. The new
sqlite database-based design will replace that, so there is no need to
create last-debug and getting rid of it will make it easier to split the
run log into per-test case files.

Signed-hostap: Jouni Malinen <jouni@qca.qualcomm.com>
2013-10-31 11:26:52 +02:00
Jouni Malinen
6a5b4b1a66 tests: Fix sqlite run column to be an integer
Commit 781b65cfbb ended up accidentally
changing this from an integer to a string. Fix this by not converting
the variable into a string.

Signed-hostap: Jouni Malinen <jouni@qca.qualcomm.com>
2013-10-31 11:08:16 +02:00
Johannes Berg
970d3b096f hwsim tests: Add scripts to run in a VM
Instead of running on the host, it can be useful to run in a
VM, particularly to test kernel rather than userspace changes,
so add a few scripts that allow doing so easily.

The basic idea is that the VM kernel is the same architecture
as the host kernel, so the host's root filesystem can be used
(in read-only mode) to run everything. Only a log filesystem
is mounted read-write and will get all the test output.

The kernel console output is collected to a special 'console'
file in the logs directory and kernel crashes are detected.

Signed-hostap: Johannes Berg <johannes.berg@intel.com>
2013-10-31 11:08:16 +02:00
Johannes Berg
e8e238c436 hwsim tests: Silence chown warnings
If running in a VM with the logs going to a host filesystem
chown isn't actually possible, silence any warnings from it.

Signed-hostap: Johannes Berg <johannes.berg@intel.com>
2013-10-31 11:08:16 +02:00
Johannes Berg
044dd1c267 hwsim tests: Silence modprobe warnings
If module loading isn't enabled (e.g., in VM tests) then
don't try to load/unload the modules.

Signed-hostap: Johannes Berg <johannes.berg@intel.com>
2013-10-31 11:08:16 +02:00
Johannes Berg
8f946511c9 hwsim tests: Reset at the end of a test
Resetting at the beginning causes the reset logging/tracing
data to leak from the previous test into the next, and the
last one being missed at all - reset at the end of each run
instead. Also reset before all tests just in case running a
test actually crashed the python script.

Signed-hostap: Johannes Berg <johannes.berg@intel.com>
2013-10-31 11:08:16 +02:00
Johannes Berg
0d5a54048b hwsim tests: Allow collecting dmesg
In addition to tracing, allow collecting dmesg. There's no
provision for actually looking at it and finding problems
in it yet though.

Signed-hostap: Johannes Berg <johannes.berg@intel.com>
2013-10-31 11:08:16 +02:00
Johannes Berg
0141fa5244 hwsim tests: Pass --logdir to run-tests.py
Instead of passing the log directory for each option
(-l, -r, -e, and -T) pass it once and make the other
options just take the filename (optionally, even).

This will also make it easier to extend later.

Signed-hostap: Johannes Berg <johannes.berg@intel.com>
2013-10-31 11:07:52 +02:00
Johannes Berg
958bf1ba39 hwsim tests: Move logging into timestamped subdirectory
Instead of logging many files called "<timestamp>-*", log into
a new "<timestamp>/*" in the logs/ subdirectory and only put
the last-debug file into logs/. If a LOGDIR is specified in the
environment, instead just put everything into that directory
(so the caller should make sure to give it a timestamp or so)
and skip the creation of last-debug entirely.

Also clean up a bit and pass the LOGDIR from run-all.sh to
start.sh rather than having start.sh create the timestamp and
run-all.sh detect the latest one when having run start.sh.

Signed-hostap: Johannes Berg <johannes.berg@intel.com>
2013-10-31 11:07:26 +02:00
Johannes Berg
435e37df7c hwsim tests: Allow run-tests.py to start tracing
In order to get tracing per test, allow run-tests.py to start
and stop tracing per test case. This is implemented using a
python 'with' context so it starts/stops automatically at the
right spots.

Instead of starting global tracing, also use it from run-all.sh
and put the trace files into the log dir.

Note that this only works right if you use a separate log dir
for all test runs as the trace files aren't timestamped.

Signed-hostap: Johannes Berg <johannes.berg@intel.com>
2013-10-31 10:47:44 +02:00
Johannes Berg
0648c3b8f5 hostapd: Add -T Linux tracing option
Just like wpa_supplicant, give hostapd the -T option to
send all debug messages into the Linux tracing buffer.
Enable this option for hwsim test builds by default.

Signed-hostap: Johannes Berg <johannes.berg@intel.com>
2013-10-31 10:47:44 +02:00
Johannes Berg
f62ec3696d hwsim tests: Pass run-all arguments on
Allow run-all to pass on extra arguments from the command
line that aren't built by the script itself.

Signed-hostap: Johannes Berg <johannes.berg@intel.com>
2013-10-31 10:29:05 +02:00
Johannes Berg
aee9446d0c hwsim tests: Pass commit as an argument
Rather than saving the current commit to a file, pass it
as an argument to run-tests.py.

Signed-hostap: Johannes Berg <johannes.berg@intel.com>
2013-10-31 10:29:00 +02:00
Jouni Malinen
3b464a3c21 tests: Silence chmod errors
These will fail if used on a readonly file system and are only needed
for valgrind runs.

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-10-30 19:58:23 +02:00
Johannes Berg
f72434b43e tests: Refactor run-all.sh
Reuse the code rather than duplicating the implementation
of starting the tests. To make that easier, allow passing
multiple modules with -f to run-tests.py.

Signed-hostap: Johannes Berg <johannes.berg@intel.com>
2013-10-30 19:58:23 +02:00
Johannes Berg
95194219df tests: Reduce duplication in startup code
Instead of hard-coding four different cases, use variables
(and printf) to reduce the duplication.

Signed-hostap: Johannes Berg <johannes.berg@intel.com>
2013-10-30 19:58:23 +02:00
Johannes Berg
0b1faa8bf1 tests: Use argparse module with hwsim
Instead of re-implementing a command-line parser, use the
argparse module.

The only real change (I hope) is that the test module must
now be given to the -f option without the .py suffix.

Also, --help now works, and if a test module/test name is
given that doesn't exist, the valid list is printed.

Signed-hostap: Johannes Berg <johannes.berg@intel.com>
2013-10-30 19:58:23 +02:00
Johannes Berg
2f3a3edd5d tests: Import sqlite3 only if used
Might help some systems that don't have/need it.

Signed-hostap: Johannes Berg <johannes.berg@intel.com>
2013-10-30 19:20:07 +02:00
Johannes Berg
781b65cfbb tests: Use proper SQL construction in hwsim test reporting
Don't construct SQL strings on the fly but instead use the
argument placeholders.

Signed-hostap: Johannes Berg <johannes.berg@intel.com>
2013-10-30 19:19:07 +02:00
Johannes Berg
5fecc0f525 tests: Allow specifying LOGDIR for hwsim scripts
Instead of hardcoding logs/, allow putting LOGDIR into
the environment. This helps run tests in a VM where the
rootfs might be mounted read-only.

Signed-hostap: Johannes Berg <johannes.berg@intel.com>
2013-10-30 15:55:58 +02:00
Jouni Malinen
789b9f1d52 tests: Enable verbose logger_stdout debug in hostapd
This makes hostapd debug logs more helpful for debugging.

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-10-30 00:38:57 +02:00
Jouni Malinen
709f18d501 tests: Add a test for dynamic hostapd config change
test_ap_change_ssid verifies that the SET and RELOAD control interface
commands can be used to change the SSID of a hostapd-controlled WPA2-PSK
AP, i.e., for a case where the PSK needs to be derived again after the
SSID change.

Signed-hostap: Jouni Malinen <jouni@qca.qualcomm.com>
2013-10-29 17:17:04 +02:00
Jouni Malinen
8fba2e5d42 tests: Add Hotspot 2.0 test cases for connecting with username
The test_ap_hs20_username* test cases verify that a username/password
credential can be used for Hotspot 2.0 connection and that the network
type is reported correctly.

Signed-hostap: Jouni Malinen <jouni@qca.qualcomm.com>
2013-10-29 14:38:31 +02:00
Jouni Malinen
59f8a3c69d tests: Add external_sim test cases with roaming network
test_ap_hs20_ext_sim_roaming is otherwise identical with
test_ap_hs20_ext_sim, but the network is operated by a roaming partner.

Signed-hostap: Jouni Malinen <jouni@qca.qualcomm.com>
2013-10-29 14:23:31 +02:00
Jouni Malinen
bbe867672e tests: Add helper functions for Hotspot 2.0 test cases
These make it easier to add new test cases for Interworking network
selection.

Signed-hostap: Jouni Malinen <jouni@qca.qualcomm.com>
2013-10-29 14:20:29 +02:00
Jouni Malinen
f4defd91cd tests: Verify that STATUS sp_type is valid for Interworking
This verifies that the network type (home/roaming/unknown) is available
and set properly in the test_ap_hs20_ext_sim test case.

Signed-hostap: Jouni Malinen <jouni@qca.qualcomm.com>
2013-10-29 14:12:23 +02:00
Johannes Berg
b3e2ce3bae tests: Do not require wpaspy installation
Since the scripts must be run from the source directory to
find the tests to run, they can use a relative path to the
wpaspy module instead of requiring it to be installed.

Signed-hostap: Johannes Berg <johannes.berg@intel.com>
2013-10-27 18:28:28 +02:00
Jouni Malinen
f3be6eea9b tests: Allow test case descriptions to be written into database
"run-tests.py -S <db file> -L" can now be used to update a database
table with the current set of test cases and their descriptions.

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-10-27 12:41:10 +02:00
Jouni Malinen
b74b7e87bb tests: Add support for sqlite results database
This is more convenient to use directly than going through the
text-based results file.

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-10-27 10:04:47 +02:00
Jouni Malinen
00606de898 tests: Unload cfg80211 on stop-wifi.sh
cfg80211 may keep some state (e.g., regulatory domain), so make sure
this gets cleared between each full testing cycle.

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-10-27 00:08:51 +03:00
Jouni Malinen
e201e5ab61 tests: Add P2P channel selection test cases for group re-invocation
Signed-hostap: Jouni Malinen <j@w1.fi>
2013-10-26 23:50:11 +03:00
Jouni Malinen
799716a8f1 tests: Add P2P channel selection test cases with world roaming
This verifies that passive scan 5 GHz channels are not selected with and
without p2p_add_cli_chan=1.

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-10-26 23:49:54 +03:00
Jouni Malinen
46f2cfcefc tests: Add P2P channel selection test cases for group formation
Signed-hostap: Jouni Malinen <j@w1.fi>
2013-10-26 23:49:10 +03:00
Jouni Malinen
e7ecab4a3b Use ARRAY_SIZE() macro
Replace the common sizeof(a)/sizeof(a[0]) constructions with a more
readable version.

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-10-26 17:49:05 +03:00
Jouni Malinen
b9ab2f2e5c tests: Add pre-authorized invitation-to-active-group
test_p2p_go_invite_auth is similar to test_p2p_go_invite with the main
difference being in the peer device pre-authorizing the invitation
instead of processing invitation at upper layers after having received
it.

Signed-hostap: Jouni Malinen <jouni@qca.qualcomm.com>
2013-10-21 18:24:20 +03:00
Jouni Malinen
ebc61dc2db tests: Add test cases for GAS operations
Signed-hostap: Jouni Malinen <jouni@qca.qualcomm.com>
2013-10-21 18:24:20 +03:00
Jouni Malinen
72c052d525 tests: Add domain_suffix_match validation
This verifies that domain_suffix_match works correctly both for the
matching and mismatching cases.

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-10-20 21:38:02 +03:00
Jouni Malinen
c7afc0789c tests: Add negative TLS test case to verify trust root validation
Signed-hostap: Jouni Malinen <j@w1.fi>
2013-10-20 21:38:02 +03:00
Jouni Malinen
d93a240731 tests: Stop test run on NOTE command failure
There is no point trying to go through a test case if the NOTE command
to write TEST-START entry does not succeed. This avoids some excessive
waits on buildbot trying to forcefully kill the programs on its timeout
if wpa_supplicant gets stuck waiting for something (like the current
issue with libnl events and commands having a chance of hitting a
blocking wait on netlink messages).

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-10-20 21:38:02 +03:00
Jouni Malinen
033e48ee2d tests: Force GO to have an old entry in test_p2p_go_invite
This adds more coverage to testing by forcing the GO to be found with an
older entry in the BSS table and with that entry having a different
operating channel. Such a case has found issues with PD retries and
scanning if incorrect frequency and SSID is selected for the group.
Instead of relying on the old BSS entry with different operating channel
to happen based on a specific test case sequence, force this to happen
for this test case every time.

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-10-20 21:38:01 +03:00
Jouni Malinen
efd43d85bd tests: Test case for Hotspot 2.0 with external SIM processing
This verifies network selection using 3GPP Network Information and
connection using EAP-SIM with GSM authentication handled by an external
program.

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-10-20 13:41:23 +03:00
Jouni Malinen
0076a18f39 tests: Remove cred before changing domain
This is in preparation for supporting multiple domain values after which
setting the domain value does not replace the old value, but adds a new
one.

Signed-hostap: Jouni Malinen <jouni@qca.qualcomm.com>
2013-10-18 14:13:45 +03:00
Jouni Malinen
dfe6ce4eeb tests: Make sure hlr_auc_gw exits and clean up sockets
If a test run is terminated forcefully, hlr_auc_gw could have been left
running and/or some of the control interface files could still remain.
This could result in hlr_auc_gw not starting again for the next test
run.

Signed-hostap: Jouni Malinen <jouni@qca.qualcomm.com>
2013-10-17 21:05:06 +03:00
Jouni Malinen
0d4c5494bc tests: Skip EAP-SIM/AKA/AKA' if hlr_auc_gw not available
Do not fail the EAP-SIM/AKA/AKA' test cases if hlr_auc_gw is not
available since hlr_auc_gw is not considered required part for the test
setup.

Signed-hostap: Jouni Malinen <jouni@qca.qualcomm.com>
2013-10-17 21:04:55 +03:00
Jouni Malinen
f9174c3b4e tests: Flush stdout after each status line print
This makes it more convenient to follow live test execution status.

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-09-29 20:42:37 +03:00
Jouni Malinen
9626962d2f tests: Add test cases for WPA2-Enterprise
Signed-hostap: Jouni Malinen <j@w1.fi>
2013-09-29 20:35:26 +03:00
Jouni Malinen
479cbb3892 tests: Start RADIUS authentication server
This can be used to run WPA2-Enterprise test cases.

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-09-29 19:14:16 +03:00
Jouni Malinen
997ebcfa5c tests: Auto detect group name
Some older systems used admin group, but adm group seems to be more
common nowadays, so detect this automatically rather than assume admin
group is used.

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-09-29 16:41:17 +03:00
Jouni Malinen
66767f0f4a tests: Merge start-p2p-concurrent.sh functionality into start.sh
Signed-hostap: Jouni Malinen <j@w1.fi>
2013-09-29 16:37:25 +03:00
Jouni Malinen
0d84c400f9 tests: Add mechanism for test cases to indicate a skipped run
This can be used by test cases that depend on some external component
that may not always be available to indicate clearly that a test case
was skipped rather than passed or failed.

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-09-29 16:21:42 +03:00
Jouni Malinen
9b00f24a76 tests: run-tests.py -L to print out list of test cases
This can be used to get a summary of all the available test cases.

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-09-29 16:11:48 +03:00
Jouni Malinen
48d8a43a28 tests: Adjust debug verbosity levels to get rid of extra info
Signed-hostap: Jouni Malinen <j@w1.fi>
2013-09-29 16:07:01 +03:00
Jouni Malinen
883ab1b9f1 tests: Document automated mac80211_hwsim test setup
This provides guidance on how to build the test setup, how to run the
tests, and how to add new test cases.

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-09-29 15:55:43 +03:00
Jouni Malinen
5f3eddac8e tests: Add test cases for P2P service discovery
Signed-hostap: Jouni Malinen <j@w1.fi>
2013-09-29 13:15:26 +03:00
Jouni Malinen
7ebf841fc5 tests: Test GO functionality with legacy STAs
Signed-hostap: Jouni Malinen <j@w1.fi>
2013-09-29 12:42:05 +03:00
Jouni Malinen
d85bb75d47 tests: Make run-tests.py command argument parser more flexible
This allow -e, -r, and -f arguments to be given in any order.

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-09-28 18:35:17 +03:00
Jouni Malinen
3b7475d365 tests: Write test case result summary into a file
This can be used to build statistics from test execution without having
to go through number of log files.

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-09-28 18:31:00 +03:00
Jouni Malinen
1fc114a16a tests: Add test duration and time to PASS/FAIL reports
Signed-hostap: Jouni Malinen <j@w1.fi>
2013-09-28 18:20:32 +03:00
Jouni Malinen
8d0825d655 tests: Exit early if startup fails
There is no point trying to go through the tests if hostapd and
wpa_supplicant processes cannot be started properly.

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-09-28 18:19:38 +03:00
Jouni Malinen
6ca3a98bc2 tests: Wait for driver scan state to clear between tests
cfg80211/mac80211 seems to getting stuck with scans every now and then.
Check for this special state and delay return from reset() until the
driver has stopped the scan operation. This reduces likelihood of
failing multiple test cases in a row because of a single error.

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-09-28 18:18:33 +03:00
Jouni Malinen
ef2bd5a344 tests: Add a test case for P2P invitation
test_p2p_go_invite verifies that GO can invite a peer in a case where
the operating channel and peer's listen channel are different.

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-09-28 12:09:40 +03:00
Jouni Malinen
d3b082371d tests: Speed up stop-wifi.sh
Split the long waits to iterations of shorter waits that can be stopped
based on processes having died.

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-09-28 11:06:46 +03:00
Jouni Malinen
53b63a0a3f tests: Add support for using Linux kernel tracing functionality
run-all.sh and start.sh scripts can now take 'trace' command line
argument to request Linux tracing information from mac80211, cfg80211,
and wpa_supplicant to be recorded.

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-09-28 11:06:46 +03:00
Jouni Malinen
9d507452dd tests: Clear global monitor interface on dump_monitor()
This avoids some issues with the global control interface socket
getting stuck with send buffer limit reached if test cases did not
use wait_global_event().

Signed-hostap: Jouni Malinen <jouni@qca.qualcomm.com>
2013-09-25 16:45:45 +03:00
Jouni Malinen
ea0e92eef5 tests: Check wpa_supplicant connection after reset
Some test runs have resulted in wpa_supplicant being unresponsive or
very slow on the control interface. Check for that type of issues as
part of the reset sequence that is done between each test case.

Signed-hostap: Jouni Malinen <jouni@qca.qualcomm.com>
2013-09-03 12:13:25 +03:00
Jouni Malinen
18b12b56c1 tests: Add a test case for invitation after client removal
test_persistent_group_invite_removed_client verifies that a client
that is removed from a group can rejoin based on new invitation.

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-09-01 21:53:03 +03:00
Jouni Malinen
2c914e2464 tests: Use a helper function for waiting session termination
Signed-hostap: Jouni Malinen <j@w1.fi>
2013-09-01 21:35:10 +03:00
Jouni Malinen
451afb4fd3 tests: Add test cases for persistent P2P groups
Signed-hostap: Jouni Malinen <j@w1.fi>
2013-09-01 21:35:10 +03:00
Jouni Malinen
672f3a72e4 tests: Extend test_autogo_2cli to test client removal
Signed-hostap: Jouni Malinen <j@w1.fi>
2013-09-01 21:35:10 +03:00
Jouni Malinen
b162675fd9 tests: Add test cases for per-STA PSK in P2P group
Signed-hostap: Jouni Malinen <j@w1.fi>
2013-09-01 21:35:10 +03:00
Jouni Malinen
dd850f4a11 tests: Make test_autogo validate group removal event on client
Use test_autogo to verify that P2P client is able to notice explicit
group session termination message from the GO. This would have caught
the regression in processing P2P deauthentication notifications.

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-08-31 17:59:16 +03:00
Jouni Malinen
846be88975 tests: Use longer timeout in test_ap_wps_er_add_enrollee
This is another one of the test cases that can time out frequently
under valgrind during WPS exchange. Increase the timeout to make
false error reports less likely to occur.

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-08-31 11:54:23 +03:00
Jouni Malinen
7e3f110b08 tests: Increase WPS timeout to 30 seconds
Number of crypto operations seem to take very long time in the valgrind
tests (about five seconds for public key generation for M1 and M2 on a
virtual server) and this is enough to push the test runs to hit the
timeout frequently even when there is no real error. Make this less
frequent by increasing WPS test case timeout from 15 to 30 seconds to
avoid issues based on the test scripts.

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-08-31 10:45:03 +03:00
Jouni Malinen
b1ce1ec09b tests: Increase P2P group formation timeout to 20 seconds
Number of crypto operations seem to take very long time in the valgrind
tests (about five seconds for passphrase to PSK mapping and for public
key generation for M1 and M2 on a virtual server) and this is enough to
push the test runs to hit the timeout frequently even when there is no
real error. Make this less frequent by increasing group formation
timeout from 15 to 20 seconds to avoid issues based on the test scripts
(15 + config time seconds timeout in the protocol may still kick in,
though).

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-08-31 10:33:44 +03:00
Jouni Malinen
18ece3133e tests: Wait longer for program deinit
Wait a bit longer for wpa_supplicant/hostapd to exit and clear control
interface files separately if either any of the cleanup steps fail to do
so.

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-08-27 16:02:15 +03:00
Jouni Malinen
15f7b02c95 tests: Wait for hostapd/wpa_supplicant to reply at start (concurrent)
Instead of waiting for a fixed duration, poll hostapd/wpa_supplicant
through the control interface until they are ready to reply in order to
avoid starting test case execution too early. This is like commit
b76579e0c0 but for the concurrent P2P test
cases that were forgotten from that commit.

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-08-27 16:01:04 +03:00
Jouni Malinen
dcfb4d652a tests: Detect BSSID mismatch in test_ibss_rsn
It looks like mac80211 does not always manage to join the same IBSS and
this can result in test_ibss_rsn failures. Detect this case and try to
get the IBSSes to merge by running a scan prior to continuing with the
test case.

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-08-27 03:11:02 +03:00
Jouni Malinen
3d5b356ac0 tests: Wait for all 4-way handshake pairs in test_ibss_rsn
The handshake between STA2 and STA1 was not tracked previously. For
completeness, include that in the list of handshakes that are explicitly
waited for to complete prior to starting the connectivity tests. Since
all key setup should be done at this point, remove the extra wait before
the last connectivity test between STA1 and STA2.

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-08-27 02:02:28 +03:00
Jouni Malinen
853b49a030 tests: Increase test_ap_wps_init connection timeout
It looks like there may not be enough time to complete this test case in
15 seconds under valgrind on a virtual host every time, so increase the
timeout to 30 seconds to get more consistent test results.

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-08-26 16:54:05 +03:00
Jouni Malinen
b76579e0c0 tests: Wait for hostapd/wpa_supplicant to reply at start
Instead of waiting for a fixed duration, poll hostapd/wpa_supplicant
through the control interface until they are ready to reply in order to
avoid starting test case execution too early.

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-08-26 16:47:15 +03:00
Jouni Malinen
9e88556a1c tests: Kill hostapd/wpa_supplicant more forcefully if needed
Try to make sure new testing can be started by forcing
hostapd/wpa_supplicant to be killed with SIGKILL if needed. In addition,
wait a bit longer when killing the processes to avoid issues with the
next test run starting before the old one has been fully terminated.

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-08-26 16:38:13 +03:00
Jouni Malinen
024913a156 tests: Stop valgrind.bin regardless of which option is started
The previous design of having to provide valgrind parameter to
stop-wifi.sh did not work since the new test setup may no match the
previous one. Instead of trying to figure out what needs to be stopped
based on previous run, stop all wpa_supplicant/hostapd processes running
under valgrind unconditionally to avoid cases where new set of processes
cannot be started due to existing processed.

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-08-26 11:28:32 +03:00
Jouni Malinen
e5a3983819 tests: Use more enforced order for the RSN IBSS steps
test_ibss_rsn has been failing pretty frequently due to various issues
in getting all connections up. Try to address this initially by avoiding
concurrent operations and explicitly waiting for 4-way handshake
completion before testing data connection. Once this test case is
workign more robustly, a more difficult case with concurrent operations
can be added as a separate test case.

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-08-25 23:10:37 +03:00
Jouni Malinen
433107f0a2 tests: Silence wlantest clear_tdls_counters output from stdout
This removes unnecessary "OK" lines from test logs.

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-08-25 21:48:51 +03:00
Jouni Malinen
be1a40f26f tests: Redirect tcpdump output to a file
Signed-hostap: Jouni Malinen <j@w1.fi>
2013-08-25 21:43:21 +03:00
Jouni Malinen
c548fb27c8 tests: Save debug log to a file and clean up stdout status
This makes it easier to figure out what failed and allows builbot to
integrate multiple logs and state information about the test cases.

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-08-25 20:15:54 +03:00
Jouni Malinen
f84692764a tests: Sleep even longer before starting valgrind tests
It looks like some virtual host setups are too slow to get
wpa_supplicant initialized under valgrind and this was triggering test
failures. Work around this by using even a longer wait before test case
execution.

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-08-24 23:47:40 +03:00
Jouni Malinen
d011246f79 tests: Make wpa_supplicant and hostapd executable through valgrind
It looks like valgrind could not under buildbot with the default file
mode. Fix this by making wpa_supplicant and hostapd binaries readable
and executable by everyone.

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-08-24 21:46:44 +03:00
Jouni Malinen
789959c493 tests: Continue even if device reset fails
This may provide better testing coverage by allowing temporary issues
during the test cycle to be ignored.

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-08-24 21:16:04 +03:00
Jouni Malinen
0c43a982c7 tests: Add option to run test cases under valgrind/concurrent P2P
./run-all.sh can now take an optional argument to select whether to run
all test cases as before (default), to run these under valgrind
("valgrind"), to run P2P test cases with concurrent station interface
("concurrent"), or the concurrent P2P test cases under valgrind
("concurrent-valgrind").

valgrind cases report errors if a test case fails or valgrind reports an
error.

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-08-24 20:33:21 +03:00