Commit graph

789 commits

Author SHA1 Message Date
Jouni Malinen
b76579e0c0 tests: Wait for hostapd/wpa_supplicant to reply at start
Instead of waiting for a fixed duration, poll hostapd/wpa_supplicant
through the control interface until they are ready to reply in order to
avoid starting test case execution too early.

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-08-26 16:47:15 +03:00
Jouni Malinen
9e88556a1c tests: Kill hostapd/wpa_supplicant more forcefully if needed
Try to make sure new testing can be started by forcing
hostapd/wpa_supplicant to be killed with SIGKILL if needed. In addition,
wait a bit longer when killing the processes to avoid issues with the
next test run starting before the old one has been fully terminated.

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-08-26 16:38:13 +03:00
Jouni Malinen
024913a156 tests: Stop valgrind.bin regardless of which option is started
The previous design of having to provide valgrind parameter to
stop-wifi.sh did not work since the new test setup may no match the
previous one. Instead of trying to figure out what needs to be stopped
based on previous run, stop all wpa_supplicant/hostapd processes running
under valgrind unconditionally to avoid cases where new set of processes
cannot be started due to existing processed.

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-08-26 11:28:32 +03:00
Jouni Malinen
e5a3983819 tests: Use more enforced order for the RSN IBSS steps
test_ibss_rsn has been failing pretty frequently due to various issues
in getting all connections up. Try to address this initially by avoiding
concurrent operations and explicitly waiting for 4-way handshake
completion before testing data connection. Once this test case is
workign more robustly, a more difficult case with concurrent operations
can be added as a separate test case.

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-08-25 23:10:37 +03:00
Jouni Malinen
433107f0a2 tests: Silence wlantest clear_tdls_counters output from stdout
This removes unnecessary "OK" lines from test logs.

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-08-25 21:48:51 +03:00
Jouni Malinen
be1a40f26f tests: Redirect tcpdump output to a file
Signed-hostap: Jouni Malinen <j@w1.fi>
2013-08-25 21:43:21 +03:00
Jouni Malinen
c548fb27c8 tests: Save debug log to a file and clean up stdout status
This makes it easier to figure out what failed and allows builbot to
integrate multiple logs and state information about the test cases.

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-08-25 20:15:54 +03:00
Jouni Malinen
f84692764a tests: Sleep even longer before starting valgrind tests
It looks like some virtual host setups are too slow to get
wpa_supplicant initialized under valgrind and this was triggering test
failures. Work around this by using even a longer wait before test case
execution.

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-08-24 23:47:40 +03:00
Jouni Malinen
d011246f79 tests: Make wpa_supplicant and hostapd executable through valgrind
It looks like valgrind could not under buildbot with the default file
mode. Fix this by making wpa_supplicant and hostapd binaries readable
and executable by everyone.

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-08-24 21:46:44 +03:00
Jouni Malinen
789959c493 tests: Continue even if device reset fails
This may provide better testing coverage by allowing temporary issues
during the test cycle to be ignored.

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-08-24 21:16:04 +03:00
Jouni Malinen
0c43a982c7 tests: Add option to run test cases under valgrind/concurrent P2P
./run-all.sh can now take an optional argument to select whether to run
all test cases as before (default), to run these under valgrind
("valgrind"), to run P2P test cases with concurrent station interface
("concurrent"), or the concurrent P2P test cases under valgrind
("concurrent-valgrind").

valgrind cases report errors if a test case fails or valgrind reports an
error.

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-08-24 20:33:21 +03:00
Jouni Malinen
3ed2814a1e tests: Use logger.info() instead of print to get one stream
print and logger.info() were directing output to different locations
(stdout and stderr, respectively) which resulted in buildbot showing
reordered entries. Use logger consistently to avoid that.

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-08-24 19:48:04 +03:00
Jouni Malinen
4896ddaea1 tests: Check for exceptions during TEST-START/STOP
It looks like the NOTE commands can time out in some cases. Avoid
stopping the test run in such a case to get more coverage if this is a
temporary issue.

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-08-24 19:41:08 +03:00
Jouni Malinen
76d320ef3f tests: Update concurrent P2P setup to match start.sh changes
The design for launching AP mode interfaces was changed, but
start-p2p-concurrent.sh was not updated with matching changes.

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-08-24 19:01:32 +03:00
Jouni Malinen
9d86f37f78 tests: Validate AP connection in each TDLS step
Make sure the connection between each STA and the AP is in working state
between each TDLS setup/teardown operation.

Signed-hostap: Jouni Malinen <jouni@qca.qualcomm.com>
2013-07-31 13:38:34 +03:00
Jouni Malinen
ec9589fdac tests: Pass apdev instead of bssid to TDLS functions
This makes it easier to add new validation steps into TDLS setup and
teardown processes.

Signed-hostap: Jouni Malinen <jouni@qca.qualcomm.com>
2013-07-31 13:27:41 +03:00
Jouni Malinen
fdae3c40f6 tests: Add a test case for IBSS RSN
Signed-hostap: Jouni Malinen <j@w1.fi>
2013-07-21 16:01:54 +03:00
Jouni Malinen
0fa28afe35 tests: Use wpa_supplicant global control interface for P2P
This allows more consistent interface to be used regardless of which
P2P driver design is used (especially for P2P management operations
over netdev vs. dedicated P2P_DEVICE).

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-07-01 00:36:33 +03:00
Jouni Malinen
441d6d23b2 tests: Initialize global wpa_supplicant control interface
Signed-hostap: Jouni Malinen <j@w1.fi>
2013-06-30 20:21:07 +03:00
Jouni Malinen
cd7f1b9a1d tests: Add test cases for Fast BSS Transition
Signed-hostap: Jouni Malinen <j@w1.fi>
2013-05-24 16:05:40 +03:00
Jouni Malinen
280cd8a9a2 tests: Allow hwsim_test and wlantest_cli to be used from PATH
This makes it easier to support some test environments where the
repository is shared between hosts.

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-05-11 11:34:49 +03:00
Jouni Malinen
8c87f65f32 tests: Use /var/run/hostapd-global to avoid socket file in work dir
This makes some cases more convenient with read-only file systems.

Signed-hostap: Jouni Malinen <jouni@qca.qualcomm.com>
2013-05-10 17:09:55 +03:00
Jouni Malinen
6edaee9c82 tests: Test WPS ER UPnP functionality to add Enrollees
Signed-hostap: Jouni Malinen <j@w1.fi>
2013-04-28 22:00:58 +03:00
Jouni Malinen
590160d502 tests: Add a test case for AP auto configuration on config token build
Signed-hostap: Jouni Malinen <jouni@qca.qualcomm.com>
2013-04-27 22:15:06 +03:00
Jouni Malinen
8e80dd5b0c tests: Use ignore_old_scan_res with some more WPS tests
The scan results from old test cases showing different SSID on the AP
can result in false failures, so avoid those by ignoring old scan
results from cfg80211.

Signed-hostap: Jouni Malinen <jouni@qca.qualcomm.com>
2013-04-27 22:12:45 +03:00
Jouni Malinen
f59a10d405 tests: Generate a tarball of test results if a test fails
This makes it easier to collect failure data from test servers for
further analysis.

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-04-26 13:48:48 +03:00
Jouni Malinen
79c3cbaf12 tests: Capture hwsim0 packet dump during test run
Signed-hostap: Jouni Malinen <j@w1.fi>
2013-04-26 12:36:56 +03:00
Jouni Malinen
ea295abcd5 tests: Add test cases for NFC+WPS
Signed-hostap: Jouni Malinen <j@w1.fi>
2013-04-01 19:34:53 +03:00
Jouni Malinen
c7860beb52 tests: Use Wlantest class in test_autogo_tdls
Signed-hostap: Jouni Malinen <j@w1.fi>
2013-04-01 10:37:44 +03:00
Jouni Malinen
93a06242ce tests: Add a test case for Hotspot 2.0 network selection
Signed-hostap: Jouni Malinen <j@w1.fi>
2013-04-01 01:01:24 +03:00
Jouni Malinen
6b67117a34 tests: Stop wpa_supplicant before hostapd
Stopping the AP first was not ideal for the test cases since it could
result in wpa_supplicant trying to connect back and start a scan at the
end of a test case and cause problems for the following test case that
tried to scan in the beginning while the previously started scan was
still in progress.

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-04-01 00:59:50 +03:00
Jouni Malinen
62d58f7ae3 tests: Use FLUSH command to clear wpa_supplicant state
Signed-hostap: Jouni Malinen <j@w1.fi>
2013-03-31 20:32:33 +03:00
Jouni Malinen
6dacb8e98b tests: Ignore old scan results in couple more WPS test cases
These have shown failures due to obsolete scan results from previous
test cases being present.

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-03-31 20:01:57 +03:00
Jouni Malinen
1a8e4761da tests: Use a more compact summary of test results
Do not print the potentially long list of passed test cases. In case of
failure(s), make sure the failed test list is the last item in the
report.

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-03-31 18:10:17 +03:00
Jouni Malinen
803edd1c09 tests: Fix BSS table flushing and old result ignoring for WPS tests
The WPS tests are more prone to fail if scan results from the previous
test cases are allowed to remain in the wpa_supplicant BSS table during
the consecutive test since the test setup uses the same BSSID for the
test APs that change their configuration. Avoid these mostly bogus
issues by enforcing wpa_supplicant to drop and ignore old scan results
during the WPS test cases.

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-03-31 18:05:42 +03:00
Jouni Malinen
347c77e6cc tests: Remove obsolete test script
The generic test cases and run-tests.py have removed need of this
separate test script.

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-03-31 18:05:42 +03:00
Jouni Malinen
7585f349c7 tests: Add command line argument -f for specifying a test file
This can be used to run all test cases from a single test file.

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-03-31 18:05:42 +03:00
Jouni Malinen
63a020ead8 tests: Use a single set of scripts for running both P2P and AP tests
This removes the unnecessary separation of P2P (no hostapd) and AP
tests. The same scripts can be used to prepare for these tests and to
execute the tests.

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-03-31 18:05:31 +03:00
Jouni Malinen
0b20f514de tests: Make hostapd test setup more similar to the P2P tests
Signed-hostap: Jouni Malinen <j@w1.fi>
2013-03-31 18:05:31 +03:00
Jouni Malinen
ae3ad3284a tests: Pass AP devices to test cases
This removes hardcoded assumptions about AP interfaces from test_*.py.

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-03-31 18:05:31 +03:00
Jouni Malinen
302b7a1b09 tests: Add test cases for WPS operations
Signed-hostap: Jouni Malinen <j@w1.fi>
2013-03-31 12:46:05 +03:00
Jouni Malinen
fde43cddb9 tests: Fix test_discovery to force full device discovery
P2P_PROV_DISC fails if the peer is not fully discovered and the previous
version of the test cases did not force this (i.e., it could have
started PD just based on having seen a Probe Request frame from the
peer).

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-03-30 20:31:22 +02:00
Jouni Malinen
5924d4c105 P2P: Add a test case for discovering a P2P client in a group
Signed-hostap: Jouni Malinen <j@w1.fi>
2013-03-30 20:17:11 +02:00
Jouni Malinen
b12b7c7a1f tests: Make test_discovery report PD failures with more detail
Signed-hostap: Jouni Malinen <j@w1.fi>
2013-03-30 16:10:04 +02:00
Jouni Malinen
5b06bdf795 tests: Add SA Query test with PMF
Signed-hostap: Jouni Malinen <j@w1.fi>
2013-03-30 15:31:50 +02:00
Jouni Malinen
835a546b20 tests: Add test cases for PMF
Signed-hostap: Jouni Malinen <j@w1.fi>
2013-03-30 13:47:22 +02:00
Jouni Malinen
5126138c9e tests: Add roaming test cases
Signed-hostap: Jouni Malinen <j@w1.fi>
2013-03-30 11:28:39 +02:00
Jouni Malinen
fcbccf1aae tests: Set umask to avoid unexpected issues with hostapd sockets
buildbot seems to be running with more limited umask by default and that
prevented the hostapd per-interface socket from being accessed.

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-03-30 11:11:55 +02:00
Jouni Malinen
8bc0799b93 tests: Use the ap_ifname variable instead of hardcoding ifname
Signed-hostap: Jouni Malinen <j@w1.fi>
2013-03-30 10:31:18 +02:00
Jouni Malinen
894541ea9c tests: Get rid of the unnecessary add_tests() functions
Signed-hostap: Jouni Malinen <j@w1.fi>
2013-03-29 21:03:55 +02:00
Jouni Malinen
b61e418c35 tests: Add helper functions for connection to an AP
Signed-hostap: Jouni Malinen <j@w1.fi>
2013-03-29 20:41:57 +02:00
Jouni Malinen
e259d186cf tests: Add helper functions for starting hostapd AP
Signed-hostap: Jouni Malinen <j@w1.fi>
2013-03-29 20:33:25 +02:00
Jouni Malinen
0165c4be10 tests: Add TDLS test cases for WEP and open network
Signed-hostap: Jouni Malinen <j@w1.fi>
2013-03-29 19:15:43 +02:00
Jouni Malinen
e492837b67 tests: Add TDLS test cases with WPA-PSK and WPA+WPA2-PSK
Signed-hostap: Jouni Malinen <j@w1.fi>
2013-03-29 18:51:48 +02:00
Jouni Malinen
b8cd4c542f tests: Configure hostapd dynamically during the tests
This makes it more convenient to change hostapd parameters between the
test cases.

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-03-29 18:37:03 +02:00
Jouni Malinen
9f46d57f2b tests: Rename AP testing scripts to be more generic
Signed-hostap: Jouni Malinen <j@w1.fi>
2013-03-29 17:19:47 +02:00
Jouni Malinen
a9bdfd49a6 tests: Add more test cases for TDLS functionality
Signed-hostap: Jouni Malinen <j@w1.fi>
2013-03-27 16:40:49 +02:00
Jouni Malinen
dc4e222cb7 tests: Use more thorough validation of TDLS frame paths
Signed-hostap: Jouni Malinen <j@w1.fi>
2013-03-27 16:05:52 +02:00
Jouni Malinen
059555878e tests: Add forgotten new files to the repository
Signed-hostap: Jouni Malinen <j@w1.fi>
2013-03-27 14:40:49 +02:00
Jouni Malinen
81266da728 tests: Add initial AP tests with hostapd
Signed-hostap: Jouni Malinen <j@w1.fi>
2013-03-27 14:29:01 +02:00
Jouni Malinen
d014a15046 tests: Fix TDLS counter validation
Signed-hostap: Jouni Malinen <j@w1.fi>
2013-03-27 13:32:36 +02:00
Jouni Malinen
8a244cd8a0 tests: Add concurrent version of P2P test setup
A virtual station interface in scanning state can change the P2P
operations quite a bit, so running all the P2P tests also in such
setup can increase testing coverage quite a bit.

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-03-19 10:58:28 +02:00
Jouni Malinen
4441b9d571 tests: Use social channel scan to speed up tests
The discovery and group formation test cases do not need to run the full
scan as the first step, so get rid of it to make the tests complete more
quickly.

Signed-hostap: Jouni Malinen <jouni@qca.qualcomm.com>
2013-03-18 17:36:12 +02:00
Jouni Malinen
47c34473f9 tests: Give document string for each test case as a description
Signed-hostap: Jouni Malinen <j@w1.fi>
2013-03-17 11:19:24 +02:00
Jouni Malinen
1a4d80b8d5 tests: Group formation with PBC
Signed-hostap: Jouni Malinen <j@w1.fi>
2013-03-17 11:10:59 +02:00
Jouni Malinen
5743006d36 tests: Add P2P device discovery and provision discovery test case
Signed-hostap: Jouni Malinen <j@w1.fi>
2013-03-17 10:58:48 +02:00
Jouni Malinen
d963f03772 tests: Use social-only scan by default to speed up tests
Signed-hostap: Jouni Malinen <j@w1.fi>
2013-03-16 11:47:01 +02:00
Jouni Malinen
c4fff52c89 tests: Add GO Negotiation without pre-authorization
This is the most common sequence for initiating GO Negotiation, so
include a test case that uses threads to follow both peers at the
same time to allow GO Negotiation to be re-initiated.

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-03-16 11:36:59 +02:00
Jouni Malinen
4d7d61b689 tests: Fix wait duration
Signed-hostap: Jouni Malinen <j@w1.fi>
2013-03-16 11:21:22 +02:00
Jouni Malinen
89ffe06d68 tests: Do not hide error return when using run-all.sh
Signed-hostap: Jouni Malinen <j@w1.fi>
2013-03-09 23:29:22 +02:00
Jouni Malinen
2a4291ffdc tests: Register test cases automatically from test_p2p_*.py
This removes need for editing run-p2p-tests.py whenever new test files
are added.

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-03-09 23:19:59 +02:00
Jouni Malinen
823d572f99 tests: Add a script for running all the automated tests
Signed-hostap: Jouni Malinen <j@w1.fi>
2013-03-09 21:40:31 +02:00
Jouni Malinen
07a2e61b43 tests: Use wlantest to validate TDLS behavior
Signed-hostap: Jouni Malinen <j@w1.fi>
2013-03-09 19:01:56 +02:00
Jouni Malinen
e5774e1888 tests: Reset devices based on each test
Make sure the devices are in known state before starting each test case.

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-03-09 19:00:57 +02:00
Jouni Malinen
7cb08cdbd6 tests: Add a test case for using TDLS in a P2P group
Signed-hostap: Jouni Malinen <j@w1.fi>
2013-03-09 18:24:32 +02:00
Jouni Malinen
78ebbf0916 tests: Add autogo test with two clients
Signed-hostap: Jouni Malinen <j@w1.fi>
2013-03-09 17:51:22 +02:00
Jouni Malinen
8b2b687eb8 tests: Start wlantest on hwsim0
This is in preparation for using wlantest to verify correct behavior in
various test cases.

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-03-09 17:39:11 +02:00
Jouni Malinen
f3f8ee882f tests: Add support for separate P2P group interfaces
Signed-hostap: Jouni Malinen <j@w1.fi>
2013-03-09 17:34:08 +02:00
Jouni Malinen
9e7308be40 tests: Reset device state at the end of full test run
Signed-hostap: Jouni Malinen <j@w1.fi>
2013-03-09 16:34:20 +02:00
Jouni Malinen
4ea8d3b5cf tests: Add autonomous GO testing
Signed-hostap: Jouni Malinen <j@w1.fi>
2013-03-09 16:30:25 +02:00
Jouni Malinen
dd34860dcd tests: Allow selected tests to be specified
Command line parameter to run-p2p-tests.py can now be used to select
which test case is run instead of always running all test cases.

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-03-09 16:07:01 +02:00
Jouni Malinen
caf518a743 tests: Add GO Negotiation provisioning method validation
Signed-hostap: Jouni Malinen <j@w1.fi>
2013-03-09 16:07:01 +02:00
Jouni Malinen
f7b1a75036 tests: Validate GO Negotiation status code
Signed-hostap: Jouni Malinen <j@w1.fi>
2013-03-09 16:07:01 +02:00
Jouni Malinen
809079d3ea tests: GO Negotiation with both devices using GO intent 15
Signed-hostap: Jouni Malinen <j@w1.fi>
2013-03-09 16:07:01 +02:00
Jouni Malinen
c68f9a61c6 tests: Report group formation results as a dict
Signed-hostap: Jouni Malinen <j@w1.fi>
2013-03-09 16:07:01 +02:00
Jouni Malinen
a311c61dd6 tests: Add test execution mechanism
run-p2p-tests.py can now be used to run all P2P test cases. The
actual test cases are defined in test_p2p_*.py files.

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-03-02 12:55:07 +02:00
Jouni Malinen
731e60f5c5 tests: Add start/stop scripts and configuration files
Signed-hostap: Jouni Malinen <j@w1.fi>
2013-03-02 11:39:54 +02:00
Jouni Malinen
3eb29b7b47 tests: Check connectivity in the P2P group
Use hwsim_test to verify that the formed group has working data
connectivity.

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-03-02 11:38:56 +02:00
Jouni Malinen
1ae73b03be Add a starting point for mac80211_hwsim-based testing
This will hopefully grow over time to become a much more complete
testing mechanism that uses mac80211_hwsim to verify various
wpa_supplicant and hostapd functions automatically.

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-03-02 01:05:03 +02:00