Commit graph

151 commits

Author SHA1 Message Date
Jouni Malinen
997ebcfa5c tests: Auto detect group name
Some older systems used admin group, but adm group seems to be more
common nowadays, so detect this automatically rather than assume admin
group is used.

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-09-29 16:41:17 +03:00
Jouni Malinen
66767f0f4a tests: Merge start-p2p-concurrent.sh functionality into start.sh
Signed-hostap: Jouni Malinen <j@w1.fi>
2013-09-29 16:37:25 +03:00
Jouni Malinen
0d84c400f9 tests: Add mechanism for test cases to indicate a skipped run
This can be used by test cases that depend on some external component
that may not always be available to indicate clearly that a test case
was skipped rather than passed or failed.

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-09-29 16:21:42 +03:00
Jouni Malinen
9b00f24a76 tests: run-tests.py -L to print out list of test cases
This can be used to get a summary of all the available test cases.

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-09-29 16:11:48 +03:00
Jouni Malinen
48d8a43a28 tests: Adjust debug verbosity levels to get rid of extra info
Signed-hostap: Jouni Malinen <j@w1.fi>
2013-09-29 16:07:01 +03:00
Jouni Malinen
883ab1b9f1 tests: Document automated mac80211_hwsim test setup
This provides guidance on how to build the test setup, how to run the
tests, and how to add new test cases.

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-09-29 15:55:43 +03:00
Jouni Malinen
5f3eddac8e tests: Add test cases for P2P service discovery
Signed-hostap: Jouni Malinen <j@w1.fi>
2013-09-29 13:15:26 +03:00
Jouni Malinen
7ebf841fc5 tests: Test GO functionality with legacy STAs
Signed-hostap: Jouni Malinen <j@w1.fi>
2013-09-29 12:42:05 +03:00
Jouni Malinen
d85bb75d47 tests: Make run-tests.py command argument parser more flexible
This allow -e, -r, and -f arguments to be given in any order.

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-09-28 18:35:17 +03:00
Jouni Malinen
3b7475d365 tests: Write test case result summary into a file
This can be used to build statistics from test execution without having
to go through number of log files.

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-09-28 18:31:00 +03:00
Jouni Malinen
1fc114a16a tests: Add test duration and time to PASS/FAIL reports
Signed-hostap: Jouni Malinen <j@w1.fi>
2013-09-28 18:20:32 +03:00
Jouni Malinen
8d0825d655 tests: Exit early if startup fails
There is no point trying to go through the tests if hostapd and
wpa_supplicant processes cannot be started properly.

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-09-28 18:19:38 +03:00
Jouni Malinen
6ca3a98bc2 tests: Wait for driver scan state to clear between tests
cfg80211/mac80211 seems to getting stuck with scans every now and then.
Check for this special state and delay return from reset() until the
driver has stopped the scan operation. This reduces likelihood of
failing multiple test cases in a row because of a single error.

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-09-28 18:18:33 +03:00
Jouni Malinen
ef2bd5a344 tests: Add a test case for P2P invitation
test_p2p_go_invite verifies that GO can invite a peer in a case where
the operating channel and peer's listen channel are different.

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-09-28 12:09:40 +03:00
Jouni Malinen
d3b082371d tests: Speed up stop-wifi.sh
Split the long waits to iterations of shorter waits that can be stopped
based on processes having died.

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-09-28 11:06:46 +03:00
Jouni Malinen
53b63a0a3f tests: Add support for using Linux kernel tracing functionality
run-all.sh and start.sh scripts can now take 'trace' command line
argument to request Linux tracing information from mac80211, cfg80211,
and wpa_supplicant to be recorded.

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-09-28 11:06:46 +03:00
Jouni Malinen
9d507452dd tests: Clear global monitor interface on dump_monitor()
This avoids some issues with the global control interface socket
getting stuck with send buffer limit reached if test cases did not
use wait_global_event().

Signed-hostap: Jouni Malinen <jouni@qca.qualcomm.com>
2013-09-25 16:45:45 +03:00
Jouni Malinen
ea0e92eef5 tests: Check wpa_supplicant connection after reset
Some test runs have resulted in wpa_supplicant being unresponsive or
very slow on the control interface. Check for that type of issues as
part of the reset sequence that is done between each test case.

Signed-hostap: Jouni Malinen <jouni@qca.qualcomm.com>
2013-09-03 12:13:25 +03:00
Jouni Malinen
18b12b56c1 tests: Add a test case for invitation after client removal
test_persistent_group_invite_removed_client verifies that a client
that is removed from a group can rejoin based on new invitation.

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-09-01 21:53:03 +03:00
Jouni Malinen
2c914e2464 tests: Use a helper function for waiting session termination
Signed-hostap: Jouni Malinen <j@w1.fi>
2013-09-01 21:35:10 +03:00
Jouni Malinen
451afb4fd3 tests: Add test cases for persistent P2P groups
Signed-hostap: Jouni Malinen <j@w1.fi>
2013-09-01 21:35:10 +03:00
Jouni Malinen
672f3a72e4 tests: Extend test_autogo_2cli to test client removal
Signed-hostap: Jouni Malinen <j@w1.fi>
2013-09-01 21:35:10 +03:00
Jouni Malinen
b162675fd9 tests: Add test cases for per-STA PSK in P2P group
Signed-hostap: Jouni Malinen <j@w1.fi>
2013-09-01 21:35:10 +03:00
Jouni Malinen
dd850f4a11 tests: Make test_autogo validate group removal event on client
Use test_autogo to verify that P2P client is able to notice explicit
group session termination message from the GO. This would have caught
the regression in processing P2P deauthentication notifications.

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-08-31 17:59:16 +03:00
Jouni Malinen
846be88975 tests: Use longer timeout in test_ap_wps_er_add_enrollee
This is another one of the test cases that can time out frequently
under valgrind during WPS exchange. Increase the timeout to make
false error reports less likely to occur.

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-08-31 11:54:23 +03:00
Jouni Malinen
7e3f110b08 tests: Increase WPS timeout to 30 seconds
Number of crypto operations seem to take very long time in the valgrind
tests (about five seconds for public key generation for M1 and M2 on a
virtual server) and this is enough to push the test runs to hit the
timeout frequently even when there is no real error. Make this less
frequent by increasing WPS test case timeout from 15 to 30 seconds to
avoid issues based on the test scripts.

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-08-31 10:45:03 +03:00
Jouni Malinen
b1ce1ec09b tests: Increase P2P group formation timeout to 20 seconds
Number of crypto operations seem to take very long time in the valgrind
tests (about five seconds for passphrase to PSK mapping and for public
key generation for M1 and M2 on a virtual server) and this is enough to
push the test runs to hit the timeout frequently even when there is no
real error. Make this less frequent by increasing group formation
timeout from 15 to 20 seconds to avoid issues based on the test scripts
(15 + config time seconds timeout in the protocol may still kick in,
though).

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-08-31 10:33:44 +03:00
Jouni Malinen
18ece3133e tests: Wait longer for program deinit
Wait a bit longer for wpa_supplicant/hostapd to exit and clear control
interface files separately if either any of the cleanup steps fail to do
so.

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-08-27 16:02:15 +03:00
Jouni Malinen
15f7b02c95 tests: Wait for hostapd/wpa_supplicant to reply at start (concurrent)
Instead of waiting for a fixed duration, poll hostapd/wpa_supplicant
through the control interface until they are ready to reply in order to
avoid starting test case execution too early. This is like commit
b76579e0c0 but for the concurrent P2P test
cases that were forgotten from that commit.

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-08-27 16:01:04 +03:00
Jouni Malinen
dcfb4d652a tests: Detect BSSID mismatch in test_ibss_rsn
It looks like mac80211 does not always manage to join the same IBSS and
this can result in test_ibss_rsn failures. Detect this case and try to
get the IBSSes to merge by running a scan prior to continuing with the
test case.

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-08-27 03:11:02 +03:00
Jouni Malinen
3d5b356ac0 tests: Wait for all 4-way handshake pairs in test_ibss_rsn
The handshake between STA2 and STA1 was not tracked previously. For
completeness, include that in the list of handshakes that are explicitly
waited for to complete prior to starting the connectivity tests. Since
all key setup should be done at this point, remove the extra wait before
the last connectivity test between STA1 and STA2.

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-08-27 02:02:28 +03:00
Jouni Malinen
853b49a030 tests: Increase test_ap_wps_init connection timeout
It looks like there may not be enough time to complete this test case in
15 seconds under valgrind on a virtual host every time, so increase the
timeout to 30 seconds to get more consistent test results.

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-08-26 16:54:05 +03:00
Jouni Malinen
b76579e0c0 tests: Wait for hostapd/wpa_supplicant to reply at start
Instead of waiting for a fixed duration, poll hostapd/wpa_supplicant
through the control interface until they are ready to reply in order to
avoid starting test case execution too early.

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-08-26 16:47:15 +03:00
Jouni Malinen
9e88556a1c tests: Kill hostapd/wpa_supplicant more forcefully if needed
Try to make sure new testing can be started by forcing
hostapd/wpa_supplicant to be killed with SIGKILL if needed. In addition,
wait a bit longer when killing the processes to avoid issues with the
next test run starting before the old one has been fully terminated.

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-08-26 16:38:13 +03:00
Jouni Malinen
024913a156 tests: Stop valgrind.bin regardless of which option is started
The previous design of having to provide valgrind parameter to
stop-wifi.sh did not work since the new test setup may no match the
previous one. Instead of trying to figure out what needs to be stopped
based on previous run, stop all wpa_supplicant/hostapd processes running
under valgrind unconditionally to avoid cases where new set of processes
cannot be started due to existing processed.

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-08-26 11:28:32 +03:00
Jouni Malinen
e5a3983819 tests: Use more enforced order for the RSN IBSS steps
test_ibss_rsn has been failing pretty frequently due to various issues
in getting all connections up. Try to address this initially by avoiding
concurrent operations and explicitly waiting for 4-way handshake
completion before testing data connection. Once this test case is
workign more robustly, a more difficult case with concurrent operations
can be added as a separate test case.

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-08-25 23:10:37 +03:00
Jouni Malinen
433107f0a2 tests: Silence wlantest clear_tdls_counters output from stdout
This removes unnecessary "OK" lines from test logs.

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-08-25 21:48:51 +03:00
Jouni Malinen
be1a40f26f tests: Redirect tcpdump output to a file
Signed-hostap: Jouni Malinen <j@w1.fi>
2013-08-25 21:43:21 +03:00
Jouni Malinen
c548fb27c8 tests: Save debug log to a file and clean up stdout status
This makes it easier to figure out what failed and allows builbot to
integrate multiple logs and state information about the test cases.

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-08-25 20:15:54 +03:00
Jouni Malinen
f84692764a tests: Sleep even longer before starting valgrind tests
It looks like some virtual host setups are too slow to get
wpa_supplicant initialized under valgrind and this was triggering test
failures. Work around this by using even a longer wait before test case
execution.

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-08-24 23:47:40 +03:00
Jouni Malinen
d011246f79 tests: Make wpa_supplicant and hostapd executable through valgrind
It looks like valgrind could not under buildbot with the default file
mode. Fix this by making wpa_supplicant and hostapd binaries readable
and executable by everyone.

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-08-24 21:46:44 +03:00
Jouni Malinen
789959c493 tests: Continue even if device reset fails
This may provide better testing coverage by allowing temporary issues
during the test cycle to be ignored.

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-08-24 21:16:04 +03:00
Jouni Malinen
0c43a982c7 tests: Add option to run test cases under valgrind/concurrent P2P
./run-all.sh can now take an optional argument to select whether to run
all test cases as before (default), to run these under valgrind
("valgrind"), to run P2P test cases with concurrent station interface
("concurrent"), or the concurrent P2P test cases under valgrind
("concurrent-valgrind").

valgrind cases report errors if a test case fails or valgrind reports an
error.

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-08-24 20:33:21 +03:00
Jouni Malinen
3ed2814a1e tests: Use logger.info() instead of print to get one stream
print and logger.info() were directing output to different locations
(stdout and stderr, respectively) which resulted in buildbot showing
reordered entries. Use logger consistently to avoid that.

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-08-24 19:48:04 +03:00
Jouni Malinen
4896ddaea1 tests: Check for exceptions during TEST-START/STOP
It looks like the NOTE commands can time out in some cases. Avoid
stopping the test run in such a case to get more coverage if this is a
temporary issue.

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-08-24 19:41:08 +03:00
Jouni Malinen
76d320ef3f tests: Update concurrent P2P setup to match start.sh changes
The design for launching AP mode interfaces was changed, but
start-p2p-concurrent.sh was not updated with matching changes.

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-08-24 19:01:32 +03:00
Jouni Malinen
9d86f37f78 tests: Validate AP connection in each TDLS step
Make sure the connection between each STA and the AP is in working state
between each TDLS setup/teardown operation.

Signed-hostap: Jouni Malinen <jouni@qca.qualcomm.com>
2013-07-31 13:38:34 +03:00
Jouni Malinen
ec9589fdac tests: Pass apdev instead of bssid to TDLS functions
This makes it easier to add new validation steps into TDLS setup and
teardown processes.

Signed-hostap: Jouni Malinen <jouni@qca.qualcomm.com>
2013-07-31 13:27:41 +03:00
Jouni Malinen
fdae3c40f6 tests: Add a test case for IBSS RSN
Signed-hostap: Jouni Malinen <j@w1.fi>
2013-07-21 16:01:54 +03:00
Jouni Malinen
0fa28afe35 tests: Use wpa_supplicant global control interface for P2P
This allows more consistent interface to be used regardless of which
P2P driver design is used (especially for P2P management operations
over netdev vs. dedicated P2P_DEVICE).

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-07-01 00:36:33 +03:00