Commit graph

68 commits

Author SHA1 Message Date
Jouni Malinen
71c41d45af tests: Provide more details of parallel testing with curses UI
This extends parallel-vm.py to show more details about testing progress
from each VM.

Signed-off-by: Jouni Malinen <j@w1.fi>
2014-10-19 11:19:49 +03:00
Jouni Malinen
a87e123a96 tests: Add Python-version of parallel-vm.sh
This is a more advanced version of the simple parallel-vm.sh script.
Status of each VM is printed out during the test and results are
provided in more convenient format in the end.

Signed-off-by: Jouni Malinen <j@w1.fi>
2014-10-12 21:49:36 +03:00
Jouni Malinen
76f999b01d tests: Show number of remaining parallel VMs
Make the parallel-vm.sh output a bit more helpful by showing a count of
running VMs.

Signed-off-by: Jouni Malinen <j@w1.fi>
2014-05-15 18:38:11 +03:00
Jouni Malinen
09e38c2fce tests: Add scripts to allow parallel execution of tests in VMs
"parallel-vm.sh <number of VMs> [arguments..]" can now be used to run
multiple VMs in parallel to speed up full test cycle significantly. In
addition, the "--split srv/total" argument used in this design would
also make it possible to split this to multiple servers to speed up
testing.

Signed-off-by: Jouni Malinen <j@w1.fi>
2014-03-24 23:37:42 +02:00
Jouni Malinen
1cfb58d400 tests: Build hlr_auc_gw separately for code coverage analysis
This improves accuracy of the code coverage reports with hostapd-as-AS
and hlr_auc_gw getting analyzed separately.

Signed-off-by: Jouni Malinen <j@w1.fi>
2014-02-15 17:08:38 +02:00
Johannes Berg
ac5e8631e6 hwsim tests: vm: add optional time-warp
To test the code under the influence of time jumps, add the option
(--timewarp) to the VM tests to reset the clock all the time, which
makes the wall clock time jump speed up 20x, causing gettimeofday()
to be unreliable for timeout calculations.

Signed-hostap: Johannes Berg <johannes.berg@intel.com>
2014-01-17 12:00:20 +02:00
Johannes Berg
fd77e594a4 hwsim tests: vm: read a config file from $HOME
The vm-config in the subdirectory is less useful as it
will get removed by "git clean" and similar, so read a
config file from ~/.wpas-vm-config in addition.

Signed-off-by: Johannes Berg <johannes.berg@intel.com>
2014-01-14 17:08:04 +02:00
Jouni Malinen
9489637b42 tests: Verify dynamic wpa_supplicant interface addition/removal
Signed-hostap: Jouni Malinen <j@w1.fi>
2013-12-31 15:45:18 +02:00
Jouni Malinen
4472aafbe0 tests: Generate a combined code coverage report
This combines coverage from all three separate reports into a single
report.

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-12-28 16:32:54 +02:00
Jouni Malinen
3f33b3ad8c tests: Collect code coverage separately from each component in vm
Use a more robust design for collecting the gcov logs from the case
where test cases are run within a virtual machine. This generates a
writable-from-vm build tree for each component separately so that the
lcov and gcov can easily find the matching source code and data files.
In addition, prepare the reports automatically at the end of the
vm-run.sh --codecov execution.

Signed-hostap: Jouni Malinen <j@w1.fi>
2013-12-27 18:11:07 +02:00
Jouni Malinen
f4bfa2d27f tests: Allow gcov to be used when running test cases within vm
Signed-hostap: Jouni Malinen <j@w1.fi>
2013-12-26 13:37:06 +02:00
Johannes Berg
625188e5bf hwsim tests: Handle regdomain requests in vm scripts
In order to handle regulatory domain requests, crda needs to be
installed on the host, but we also need to install a uevent helper in
the VM so that it gets executed (since we don't run udev).

Signed-hostap: Johannes Berg <johannes.berg@intel.com>
2013-12-24 06:49:57 +02:00
Johannes Berg
667a158d08 tests: VM test script: Copy gcov data if present
If there's code coverage analysis data, copy it out of the VM
to be able to analyse it later. Also add a description to the
README file about how to use it.

Signed-hostap: Johannes Berg <johannes.berg@intel.com>
2013-12-07 17:45:12 -08:00
Johannes Berg
4ecf11c559 hwsim tests: Make channel configuration for VM tests easier
Add a CHANNELS configuration to the script running the VM
that can be added to the vm-config file to allow running
the tests with hwsim devices supporting more than a single
channel.

Eventually, with the (hopefully) upcoming dynamic work in
mac80211_hwsim, this might go away entirely, but for now
this allows testing more code paths.

Signed-hostap: Johannes Berg <johannes.berg@intel.com>
2013-11-07 14:01:06 +02:00
Johannes Berg
a5d7da3fd3 hwsim tests: Prefill database in VM tests
In some cases, e.g., with the VM tests if the VM crashes, it
can be useful to know which tests should have run but didn't
(or didn't finish). In order to catch these more easily, add
an option to prefill the database with all tests at the very
beginning of the testing (in a new NOTRUN state) and use the
option in the VM tests.

Signed-hostap: Johannes Berg <johannes.berg@intel.com>
2013-11-02 10:17:44 +02:00
Johannes Berg
63f83fac0d hwsim tests: Create results database in VM tests
Create a results.db in the output directory when running
the tests in a VM. To make that easier, create the tables
in the python script if they don't exist.

Signed-hostap: Johannes Berg <johannes.berg@intel.com>
2013-11-02 10:16:28 +02:00
Johannes Berg
1cd3eae362 hwsim tests: Allow setting KERNEL and KVMARGS
Rather than just having KERNELDIR, allow setting KERNEL directly.
Also remove the -s option that prevents running multiple machines
at the same time, but add a KVMARGS= variable that can be used to
restore that if needed.

Signed-hostap: Johannes Berg <johannes.berg@intel.com>
2013-11-02 10:01:09 +02:00
Johannes Berg
970d3b096f hwsim tests: Add scripts to run in a VM
Instead of running on the host, it can be useful to run in a
VM, particularly to test kernel rather than userspace changes,
so add a few scripts that allow doing so easily.

The basic idea is that the VM kernel is the same architecture
as the host kernel, so the host's root filesystem can be used
(in read-only mode) to run everything. Only a log filesystem
is mounted read-write and will get all the test output.

The kernel console output is collected to a special 'console'
file in the logs directory and kernel crashes are detected.

Signed-hostap: Johannes Berg <johannes.berg@intel.com>
2013-10-31 11:08:16 +02:00