Create a results.db in the output directory when running
the tests in a VM. To make that easier, create the tables
in the python script if they don't exist.
Signed-hostap: Johannes Berg <johannes.berg@intel.com>
Refactor the test reporting to treat the different results
(success/skip/failure) identically. This makes the timing
seem a bit longer, but cleans up the code which will allow
for adding more checks (e.g., on the captured data files)
later.
Signed-hostap: Johannes Berg <johannes.berg@intel.com>
test_ap_bss_add_remove verifies hostapd behavior when BSSes are
added/removed in multi-BSS configuration.
Signed-hostap: Jouni Malinen <jouni@qca.qualcomm.com>
This allows run-tests.py to use the same logs/<date> default logdir as
start.sh which is quite convenient for manual test runs.
Signed-hostap: Jouni Malinen <jouni@qca.qualcomm.com>
The run-tests.py -l argument does not take an argument value anymore.
Instead, debug output is directed to a separate file <test>.log for each
test case.
Signed-hostap: Jouni Malinen <jouni@qca.qualcomm.com>
This is unnecessary extra complexity for user, so use the 'test_' prefix
only internally within the python scripts and file names.
Signed-hostap: Jouni Malinen <jouni@qca.qualcomm.com>
This is unnecessary extra complexity for user and reports, so use the
'test_' prefix only internally within the python scripts.
Signed-hostap: Jouni Malinen <jouni@qca.qualcomm.com>
Commit 781b65cfbb ended up accidentally
changing this from an integer to a string. Fix this by not converting
the variable into a string.
Signed-hostap: Jouni Malinen <jouni@qca.qualcomm.com>
Resetting at the beginning causes the reset logging/tracing
data to leak from the previous test into the next, and the
last one being missed at all - reset at the end of each run
instead. Also reset before all tests just in case running a
test actually crashed the python script.
Signed-hostap: Johannes Berg <johannes.berg@intel.com>
In addition to tracing, allow collecting dmesg. There's no
provision for actually looking at it and finding problems
in it yet though.
Signed-hostap: Johannes Berg <johannes.berg@intel.com>
Instead of passing the log directory for each option
(-l, -r, -e, and -T) pass it once and make the other
options just take the filename (optionally, even).
This will also make it easier to extend later.
Signed-hostap: Johannes Berg <johannes.berg@intel.com>
In order to get tracing per test, allow run-tests.py to start
and stop tracing per test case. This is implemented using a
python 'with' context so it starts/stops automatically at the
right spots.
Instead of starting global tracing, also use it from run-all.sh
and put the trace files into the log dir.
Note that this only works right if you use a separate log dir
for all test runs as the trace files aren't timestamped.
Signed-hostap: Johannes Berg <johannes.berg@intel.com>
Reuse the code rather than duplicating the implementation
of starting the tests. To make that easier, allow passing
multiple modules with -f to run-tests.py.
Signed-hostap: Johannes Berg <johannes.berg@intel.com>
Instead of re-implementing a command-line parser, use the
argparse module.
The only real change (I hope) is that the test module must
now be given to the -f option without the .py suffix.
Also, --help now works, and if a test module/test name is
given that doesn't exist, the valid list is printed.
Signed-hostap: Johannes Berg <johannes.berg@intel.com>
Since the scripts must be run from the source directory to
find the tests to run, they can use a relative path to the
wpaspy module instead of requiring it to be installed.
Signed-hostap: Johannes Berg <johannes.berg@intel.com>
"run-tests.py -S <db file> -L" can now be used to update a database
table with the current set of test cases and their descriptions.
Signed-hostap: Jouni Malinen <j@w1.fi>
There is no point trying to go through a test case if the NOTE command
to write TEST-START entry does not succeed. This avoids some excessive
waits on buildbot trying to forcefully kill the programs on its timeout
if wpa_supplicant gets stuck waiting for something (like the current
issue with libnl events and commands having a chance of hitting a
blocking wait on netlink messages).
Signed-hostap: Jouni Malinen <j@w1.fi>
This can be used by test cases that depend on some external component
that may not always be available to indicate clearly that a test case
was skipped rather than passed or failed.
Signed-hostap: Jouni Malinen <j@w1.fi>
This makes it easier to figure out what failed and allows builbot to
integrate multiple logs and state information about the test cases.
Signed-hostap: Jouni Malinen <j@w1.fi>
print and logger.info() were directing output to different locations
(stdout and stderr, respectively) which resulted in buildbot showing
reordered entries. Use logger consistently to avoid that.
Signed-hostap: Jouni Malinen <j@w1.fi>
It looks like the NOTE commands can time out in some cases. Avoid
stopping the test run in such a case to get more coverage if this is a
temporary issue.
Signed-hostap: Jouni Malinen <j@w1.fi>
This allows more consistent interface to be used regardless of which
P2P driver design is used (especially for P2P management operations
over netdev vs. dedicated P2P_DEVICE).
Signed-hostap: Jouni Malinen <j@w1.fi>
Stopping the AP first was not ideal for the test cases since it could
result in wpa_supplicant trying to connect back and start a scan at the
end of a test case and cause problems for the following test case that
tried to scan in the beginning while the previously started scan was
still in progress.
Signed-hostap: Jouni Malinen <j@w1.fi>
Do not print the potentially long list of passed test cases. In case of
failure(s), make sure the failed test list is the last item in the
report.
Signed-hostap: Jouni Malinen <j@w1.fi>
This removes the unnecessary separation of P2P (no hostapd) and AP
tests. The same scripts can be used to prepare for these tests and to
execute the tests.
Signed-hostap: Jouni Malinen <j@w1.fi>
