Commit graph

5971 commits

Author SHA1 Message Date
Jouni Malinen
de684a2b83 tests: Add run-tests.py --shuffle-test
This optional argument can be used to randomize the order in which the
test cases are run. This can provide more coverage on testing
interactions of common use cases in various different sequences. Such
issues have already been found even with the fixed order of test cases,
but being able to reorder the tests makes this more efficient.

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-11-02 10:30:37 +02:00
Johannes Berg
fdbc092389 hwsim tests: Add build script
This is easier than to copy/paste from the README. We may
want to extend it later to change the .config for some
common differences between systems (e.g., libnl/libbfd).

Signed-off-by: Johannes Berg <johannes.berg@intel.com>
2013-11-02 10:17:46 +02:00
Johannes Berg
a5d7da3fd3 hwsim tests: Prefill database in VM tests
In some cases, e.g., with the VM tests if the VM crashes, it
can be useful to know which tests should have run but didn't
(or didn't finish). In order to catch these more easily, add
an option to prefill the database with all tests at the very
beginning of the testing (in a new NOTRUN state) and use the
option in the VM tests.

Signed-hostap: Johannes Berg <johannes.berg@intel.com>
2013-11-02 10:17:44 +02:00
Johannes Berg
63f83fac0d hwsim tests: Create results database in VM tests
Create a results.db in the output directory when running
the tests in a VM. To make that easier, create the tables
in the python script if they don't exist.

Signed-hostap: Johannes Berg <johannes.berg@intel.com>
2013-11-02 10:16:28 +02:00
Johannes Berg
4a1b8901ab hwsim tests: Refactor test results reporting
Refactor the test reporting to treat the different results
(success/skip/failure) identically. This makes the timing
seem a bit longer, but cleans up the code which will allow
for adding more checks (e.g., on the captured data files)
later.

Signed-hostap: Johannes Berg <johannes.berg@intel.com>
2013-11-02 10:03:14 +02:00
Johannes Berg
1cd3eae362 hwsim tests: Allow setting KERNEL and KVMARGS
Rather than just having KERNELDIR, allow setting KERNEL directly.
Also remove the -s option that prevents running multiple machines
at the same time, but add a KVMARGS= variable that can be used to
restore that if needed.

Signed-hostap: Johannes Berg <johannes.berg@intel.com>
2013-11-02 10:01:09 +02:00
Jouni Malinen
c092d83e26 P2P: Clear pending group formation data on group removal
It was possible for the wpa_s->show_group_started and wpa_s->go_params
to be left set when a P2P group was removed before group formation had
completed. In case a separate P2P group interface was not used, this
could rsult in all future scans using the hardcoded DIRECT-* SSID and as
such, not find the network they were trying to find. Fix this by
clearing these P2P parameters on group removal.

Signed-hostap: Jouni Malinen <jouni@qca.qualcomm.com>
2013-11-01 10:31:55 +02:00
Jouni Malinen
9100b6607b P2P: Debug print reason for specific SSID for scan
It can be useful to see whether the specific P2P SSID was used for scan
based on p2p_in_provisioning or show_group_started when debugging issues
where this case shows up unexpectedly.

Signed-hostap: Jouni Malinen <jouni@qca.qualcomm.com>
2013-11-01 10:30:50 +02:00
Jouni Malinen
a633397756 tests: Test dynamic BSS addition/removal
test_ap_bss_add_remove verifies hostapd behavior when BSSes are
added/removed in multi-BSS configuration.

Signed-hostap: Jouni Malinen <jouni@qca.qualcomm.com>
2013-10-31 23:04:50 +02:00
Jouni Malinen
2aec4f3c0c Allow add-BSS operation to re-use existing netdev
When removing and re-adding the first wlan# netdev to hostapd
dynamically, the netdev is already present and should not be removed and
re-added to maintain its state as not-added-by-hostapd so that it does
not get removed automatically.

Signed-hostap: Jouni Malinen <jouni@qca.qualcomm.com>
2013-10-31 22:34:25 +02:00
Jouni Malinen
5592065850 hostapd: Allow a single BSS to be removed from an interface
The global control interface command "REMOVE <ifname>" can now be used
to remove a single virtual interface (BSS) without affecting other
virtual interfaces on the same radio.

Signed-hostap: Jouni Malinen <jouni@qca.qualcomm.com>
2013-10-31 22:30:15 +02:00
Kyeyoon Park
2e2fff37e9 hostapd: Allow a single BSS to be added to an interface
The global control interface command "ADD
bss_config=<phyname>:<config file>" can now be used to add a single
virtual interface (BSS) to an interface.

Signed-hostap: Jouni Malinen <jouni@qca.qualcomm.com>
2013-10-31 22:30:15 +02:00
Kyeyoon Park
a1fb5692af hostapd: Make hostapd_interface_init_bss() available externally
Signed-hostap: Jouni Malinen <jouni@qca.qualcomm.com>
2013-10-31 22:00:31 +02:00
Kyeyoon Park
66936c6af8 hostapd: Make hostapd_init() available externally
Signed-hostap: Jouni Malinen <jouni@qca.qualcomm.com>
2013-10-31 22:00:31 +02:00
Kyeyoon Park
390e489c0d hostapd: Allow the first BSS in a multi-BSS setup to be removed
This moves the vif added check from core hostapd to the driver wrapper
(only driver_nl80211.c uses this) and reorders operations a bit to allow
the first BSS (vif) to be removed from a multi-BSS setup.

Signed-hostap: Jouni Malinen <jouni@qca.qualcomm.com>
2013-10-31 22:00:31 +02:00
Kyeyoon Park
834ee56f78 nl80211: Make wpa_driver_nl80211_data::first_bss pointer
This is needed to allow the first BSS to be removed and replaced by the
next one in the list.

Signed-hostap: Jouni Malinen <jouni@qca.qualcomm.com>
2013-10-31 22:00:31 +02:00
Jouni Malinen
748c0ac0e7 nl80211: Fix monitor interface reference counting
The first user of monitor interface was not counted and that could
result in the monitor interface getting removed if the initial interface
was removed from a multi-BSS setup.

Signed-hostap: Jouni Malinen <jouni@qca.qualcomm.com>
2013-10-31 22:00:31 +02:00
Jouni Malinen
08e55ebbb0 nl80211: Add a debug print for DEL_BEACON
This is quite helpful in debugging beaconing issues.

Signed-hostap: Jouni Malinen <jouni@qca.qualcomm.com>
2013-10-31 22:00:12 +02:00
Kyeyoon Park
33b0b330ce hostapd: Fix error path in hostapd_add_iface()
Incorrect count variable was used in freeing up the BSS data.

Signed-hostap: Jouni Malinen <jouni@qca.qualcomm.com>
2013-10-31 15:59:12 +02:00
Helmut Schaa
770ecdf27c ACS: Do not get stuck while failing to do a subsequent scan
Return control flow to hostapd by calling hostapd_acs_completed()
if requesting a scan from the underlying device fails.

Signed-hostapd: Helmut Schaa <helmut.schaa@googlemail.com>
2013-10-31 15:12:21 +02:00
Michal Kazior
813d4bac5a DFS: Add support for multi-BSS
If radar was detected single BSS is notified about it. This caused only
that single BSS to be stopped and restarted. However, due to nl80211
interface combinations the BSS was not started on a new channel and
other BSSes remained operating on the old channel.

The downside is that hostapd_disable_iface() causes deauth frames to be
sent. This is undesired but on the other hand it doesn't make sense to
create workarounds that imitate CSA's 'block tx'. For proper Tx
quiescing CSA should be properly implemented.

Signed-off-by: Michal Kazior <michal.kazior@tieto.com>
2013-10-31 15:12:17 +02:00
Michal Kazior
954e71d270 DFS: Reset cac_started properly
If CAC was stopped but hostapd was not terminated cac_started would
remain set to 1.

Signed-off-by: Michal Kazior <michal.kazior@tieto.com>
2013-10-31 14:49:38 +02:00
Michal Kazior
6a398ddc9a DFS: Sanitize channel availability checks
Fixes corner case of holes in channel list and simplifies availability
checks.

Signed-hostap: Michal Kazior <michal.kazior@tieto.com>
2013-10-31 14:49:06 +02:00
Michal Kazior
32595da608 DFS: Fix HT40/VHT calculation
Decouple HT/VHT offset/center-freq calculations from channel lookup.
This will be necessary for further improvements on the DFS codebase.

Signed-hostap: Michal Kazior <michal.kazior@tieto.com>
2013-10-31 14:49:03 +02:00
Jouni Malinen
a1ddc79adf tests: Remove separate error/failed text file
The new sqlite database for results deprecated this.

Signed-hostap: Jouni Malinen <jouni@qca.qualcomm.com>
2013-10-31 14:39:09 +02:00
Jouni Malinen
904b6e7e75 tests: Remove separate results text file
The new sqlite database for results deprecated this.

Signed-hostap: Jouni Malinen <jouni@qca.qualcomm.com>
2013-10-31 14:36:26 +02:00
Jouni Malinen
b72c300267 tests: Use a symlink for default LOGDIR between scripts
This allows run-tests.py to use the same logs/<date> default logdir as
start.sh which is quite convenient for manual test runs.

Signed-hostap: Jouni Malinen <jouni@qca.qualcomm.com>
2013-10-31 14:27:57 +02:00
Jouni Malinen
353e5067ff tests: Split wpa_supplicant debug logs into per test case files
Signed-hostap: Jouni Malinen <jouni@qca.qualcomm.com>
2013-10-31 13:19:38 +02:00
Jouni Malinen
c9aa430896 tests: Split run-tests.py logger info into per test case files
The run-tests.py -l argument does not take an argument value anymore.
Instead, debug output is directed to a separate file <test>.log for each
test case.

Signed-hostap: Jouni Malinen <jouni@qca.qualcomm.com>
2013-10-31 12:51:08 +02:00
Jouni Malinen
e955226208 tests: Remove the 'test_' prefix from test module names
This is unnecessary extra complexity for user, so use the 'test_' prefix
only internally within the python scripts and file names.

Signed-hostap: Jouni Malinen <jouni@qca.qualcomm.com>
2013-10-31 11:49:58 +02:00
Jouni Malinen
4cd8343f4d tests: Remove the 'test_' prefix from test names
This is unnecessary extra complexity for user and reports, so use the
'test_' prefix only internally within the python scripts.

Signed-hostap: Jouni Malinen <jouni@qca.qualcomm.com>
2013-10-31 11:43:45 +02:00
Jouni Malinen
d05159ccc5 tests: Replace tcpdump with wlantest file write
There is no need to run a separate tcpdump process to capture frames
from hwsim0 since wlantest is already doing that can write the results
to a file.

Signed-hostap: Jouni Malinen <jouni@qca.qualcomm.com>
2013-10-31 11:35:02 +02:00
Jouni Malinen
79b16a5c0d tests: Remove last-debug creation
This file was used for adding debug info into the buildbot logs. The new
sqlite database-based design will replace that, so there is no need to
create last-debug and getting rid of it will make it easier to split the
run log into per-test case files.

Signed-hostap: Jouni Malinen <jouni@qca.qualcomm.com>
2013-10-31 11:26:52 +02:00
Jouni Malinen
6a5b4b1a66 tests: Fix sqlite run column to be an integer
Commit 781b65cfbb ended up accidentally
changing this from an integer to a string. Fix this by not converting
the variable into a string.

Signed-hostap: Jouni Malinen <jouni@qca.qualcomm.com>
2013-10-31 11:08:16 +02:00
Johannes Berg
970d3b096f hwsim tests: Add scripts to run in a VM
Instead of running on the host, it can be useful to run in a
VM, particularly to test kernel rather than userspace changes,
so add a few scripts that allow doing so easily.

The basic idea is that the VM kernel is the same architecture
as the host kernel, so the host's root filesystem can be used
(in read-only mode) to run everything. Only a log filesystem
is mounted read-write and will get all the test output.

The kernel console output is collected to a special 'console'
file in the logs directory and kernel crashes are detected.

Signed-hostap: Johannes Berg <johannes.berg@intel.com>
2013-10-31 11:08:16 +02:00
Johannes Berg
e8e238c436 hwsim tests: Silence chown warnings
If running in a VM with the logs going to a host filesystem
chown isn't actually possible, silence any warnings from it.

Signed-hostap: Johannes Berg <johannes.berg@intel.com>
2013-10-31 11:08:16 +02:00
Johannes Berg
044dd1c267 hwsim tests: Silence modprobe warnings
If module loading isn't enabled (e.g., in VM tests) then
don't try to load/unload the modules.

Signed-hostap: Johannes Berg <johannes.berg@intel.com>
2013-10-31 11:08:16 +02:00
Johannes Berg
8f946511c9 hwsim tests: Reset at the end of a test
Resetting at the beginning causes the reset logging/tracing
data to leak from the previous test into the next, and the
last one being missed at all - reset at the end of each run
instead. Also reset before all tests just in case running a
test actually crashed the python script.

Signed-hostap: Johannes Berg <johannes.berg@intel.com>
2013-10-31 11:08:16 +02:00
Johannes Berg
0d5a54048b hwsim tests: Allow collecting dmesg
In addition to tracing, allow collecting dmesg. There's no
provision for actually looking at it and finding problems
in it yet though.

Signed-hostap: Johannes Berg <johannes.berg@intel.com>
2013-10-31 11:08:16 +02:00
Johannes Berg
0141fa5244 hwsim tests: Pass --logdir to run-tests.py
Instead of passing the log directory for each option
(-l, -r, -e, and -T) pass it once and make the other
options just take the filename (optionally, even).

This will also make it easier to extend later.

Signed-hostap: Johannes Berg <johannes.berg@intel.com>
2013-10-31 11:07:52 +02:00
Johannes Berg
958bf1ba39 hwsim tests: Move logging into timestamped subdirectory
Instead of logging many files called "<timestamp>-*", log into
a new "<timestamp>/*" in the logs/ subdirectory and only put
the last-debug file into logs/. If a LOGDIR is specified in the
environment, instead just put everything into that directory
(so the caller should make sure to give it a timestamp or so)
and skip the creation of last-debug entirely.

Also clean up a bit and pass the LOGDIR from run-all.sh to
start.sh rather than having start.sh create the timestamp and
run-all.sh detect the latest one when having run start.sh.

Signed-hostap: Johannes Berg <johannes.berg@intel.com>
2013-10-31 11:07:26 +02:00
Johannes Berg
435e37df7c hwsim tests: Allow run-tests.py to start tracing
In order to get tracing per test, allow run-tests.py to start
and stop tracing per test case. This is implemented using a
python 'with' context so it starts/stops automatically at the
right spots.

Instead of starting global tracing, also use it from run-all.sh
and put the trace files into the log dir.

Note that this only works right if you use a separate log dir
for all test runs as the trace files aren't timestamped.

Signed-hostap: Johannes Berg <johannes.berg@intel.com>
2013-10-31 10:47:44 +02:00
Johannes Berg
0648c3b8f5 hostapd: Add -T Linux tracing option
Just like wpa_supplicant, give hostapd the -T option to
send all debug messages into the Linux tracing buffer.
Enable this option for hwsim test builds by default.

Signed-hostap: Johannes Berg <johannes.berg@intel.com>
2013-10-31 10:47:44 +02:00
Johannes Berg
f62ec3696d hwsim tests: Pass run-all arguments on
Allow run-all to pass on extra arguments from the command
line that aren't built by the script itself.

Signed-hostap: Johannes Berg <johannes.berg@intel.com>
2013-10-31 10:29:05 +02:00
Johannes Berg
aee9446d0c hwsim tests: Pass commit as an argument
Rather than saving the current commit to a file, pass it
as an argument to run-tests.py.

Signed-hostap: Johannes Berg <johannes.berg@intel.com>
2013-10-31 10:29:00 +02:00
Jouni Malinen
3b464a3c21 tests: Silence chmod errors
These will fail if used on a readonly file system and are only needed
for valgrind runs.

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-10-30 19:58:23 +02:00
Johannes Berg
f72434b43e tests: Refactor run-all.sh
Reuse the code rather than duplicating the implementation
of starting the tests. To make that easier, allow passing
multiple modules with -f to run-tests.py.

Signed-hostap: Johannes Berg <johannes.berg@intel.com>
2013-10-30 19:58:23 +02:00
Johannes Berg
95194219df tests: Reduce duplication in startup code
Instead of hard-coding four different cases, use variables
(and printf) to reduce the duplication.

Signed-hostap: Johannes Berg <johannes.berg@intel.com>
2013-10-30 19:58:23 +02:00
Johannes Berg
0b1faa8bf1 tests: Use argparse module with hwsim
Instead of re-implementing a command-line parser, use the
argparse module.

The only real change (I hope) is that the test module must
now be given to the -f option without the .py suffix.

Also, --help now works, and if a test module/test name is
given that doesn't exist, the valid list is printed.

Signed-hostap: Johannes Berg <johannes.berg@intel.com>
2013-10-30 19:58:23 +02:00
Johannes Berg
2f3a3edd5d tests: Import sqlite3 only if used
Might help some systems that don't have/need it.

Signed-hostap: Johannes Berg <johannes.berg@intel.com>
2013-10-30 19:20:07 +02:00