Cipher suite (CCMP, TKIP, GCMP, ..) and key management testing ============================================================== wpa_supplicant and hostapd include number of extensions that allow special test builds to be used for testing functionality related to correct implementation of IEEE 802.11. These extensions allow behavior to be modified and invalid operations to be performed to verify behavior of other devices in unexpected situations. While most of the testing extensions are focused on the fully automated testing framework with mac80211_hwsim (see tests/hwsim subdirectory), many of these can be used for over-the-air testing of the protocol as well. Since some of the testing extensions can result in exposing key information or allowing non-compliant behavior, these changes are disabled in default wpa_supplicant and hostapd builds for production purposes. Testing functionality can be enabled by adding CONFIG_TESTING_OPTIONS=y into build configuration (hostapd/.config and wpa_supplicant/.config). Testing setup ------------- These tests can be run as black-box testing without having to modify the tested device at all or without knowing details of its functionality. The test commands in wpa_supplicant/hostapd control interfaces are used to perform unexpected operations and normal data traffic is used to verify reaction of the tested device to such operations. In theory, the test functionality is available with most drivers supported by wpa_supplicant/hostapd, but the most reliable results are likely available through ath9k-based devices. If you are using something else, it is strongly recommended that you'll run the first tests with sniffer captures and verify that the test tools are behaving correctly. wpa_supplicant is used to control a test device in station mode to test an AP and hostapd is similarly used to control a test device in AP mode to test a station. Various data traffic generators could be used to test the behavior, but this document focuses on using ping to test unicast traffic and arping to test broadcast traffic. To keep things simple and to reduce interference from unrelated traffic, the steps here assume static IPv4 addresses are used and IPv6 is disabled. The tests here use WPA2-Personal for simplicity. WPA2-Enterprise and other cipher suites can also be tested for more complete coverage. Example hostapd.conf for the test tool in AP mode: driver=nl80211 hw_mode=g channel=1 ieee80211n=1 interface=wlan0 ctrl_interface=/var/run/hostapd ctrl_interface_group=adm ssid=test-psk wpa=2 wpa_key_mgmt=WPA-PSK wpa_pairwise=CCMP wpa_passphrase=12345678 Example wpa_supplicant.conf for the test tool in station mode: ctrl_interface=DIR=/var/run/wpa_supplicant GROUP=adm network={ ssid="test-psk" key_mgmt=WPA-PSK psk="12345678" } The examples in this document assume following IPv4 address configuration: Test tool (either AP or station mode): 192.168.1.1/24 Device under test: 192.168.1.2/24 Data traffic tests ------------------ ping is used to test whether unicast frames go through on the data link. It should be noted that ping may need to use broadcast ARP at the beginning if the other device is not yet in the ARP table, so working broadcast and unicast connectivity may be needed to get this started. Example: $ ping -n -c 5 192.168.1.2 PING 192.168.1.2 (192.168.1.2) 56(84) bytes of data. 64 bytes from 192.168.1.2: icmp_seq=1 ttl=64 time=43.7 ms 64 bytes from 192.168.1.2: icmp_seq=2 ttl=64 time=67.9 ms 64 bytes from 192.168.1.2: icmp_seq=3 ttl=64 time=900 ms 64 bytes from 192.168.1.2: icmp_seq=4 ttl=64 time=5.81 ms 64 bytes from 192.168.1.2: icmp_seq=5 ttl=64 time=135 ms --- 192.168.1.2 ping statistics --- 5 packets transmitted, 5 received, 0% packet loss, time 4004ms rtt min/avg/max/mdev = 5.811/230.605/900.223/337.451 ms This shows working unicast data connectivity. $ ping -n -c 5 192.168.1.2 PING 192.168.1.2 (192.168.1.2) 56(84) bytes of data. --- 192.168.1.2 ping statistics --- 5 packets transmitted, 0 received, 100% packet loss, time 4033ms This shows not working unicast data connectivity. arping is used to test broadcast connectivity. Example: $ arping -b -I wlan0 192.168.1.2 -c 5 ARPING 192.168.1.2 from 192.168.1.1 wlan0 Unicast reply from 192.168.1.2 [] 119.695ms Unicast reply from 192.168.1.2 [] 144.496ms Unicast reply from 192.168.1.2 [] 166.788ms Unicast reply from 192.168.1.2 [] 2.283ms Unicast reply from 192.168.1.2 [] 2.234ms Sent 5 probes (5 broadcast(s)) Received 5 response(s) This shows working broadcast data connectivity. $ arping -b -I wlan0 192.168.1.2 -c 5 ARPING 192.168.1.2 from 192.168.1.1 wlan0 Sent 5 probes (5 broadcast(s)) Received 0 response(s) This shows not working broadcast data connectivity. If testing results do not look consistent, the testing state can be cleared by disconnection and reconnecting the station (the test tool or the DUT) to the network. Sniffer and wlantest -------------------- It is useful to get a wireless sniffer capture from the operating channel of the AP to be able to confirm DUT behavior if any of the data tests indicate reason to believe something is not working as expected. wlantest (from the wlantest directory of hostap.git) can be used to decrypt and analyze a sniffer capture. For example: wlantest -r wlan0.pcap -n decrypted.pcap -p 12345678 The debug prints and comments in the generated file indicate where unexpected behavior has been detected, e.g., when the test tool ends up clearing its packet number to test replay protection. That can help in checking whether the DUT actually replies to a frame that it was supposed to drop due replay. Testing replay protection on a station device --------------------------------------------- Start hostapd and use hostapd_cli on the test device to control testing operations. Connect the DUT to the network. <3>AP-STA-CONNECTED This indicates that the connection was completed successfully. Verify that broadcast and unicast traffic works correctly (if not, something is wrong in the test setup and that needs to be resolved before being able to run any tests). Verify that unicast traffic works and issue the following command in hostapd_cli: > raw RESET_PN OK Verify that unicast traffic does not work anymore. If it does, the DUT does not implement replay protection correctly for unicast frames. Note that unicast traffic can recover once the packet number from the test device increases beyond the value used prior to that RESET_PN command. Verify that broadcast traffic works and issue the following command in hostapd_cli: > raw RESET_PN ff:ff:ff:ff:ff:ff OK Verify that broadcast traffic does not work anymore. If it does, the DUT does not implement replay protection correctly for broadcast frames. Note that broadcast traffic can recover once the packet number from the test device increases beyond the value used prior to that RESET_PN command. Testing replay protection on an AP device ----------------------------------------- Start the AP (DUT) and start wpa_supplicant on the test device to connect to the network. Use wpa_cli to control the test device. <3>SME: Trying to authenticate with (SSID='test-psk' freq=5240 MHz) <3>CTRL-EVENT-REGDOM-CHANGE init=CORE type=WORLD <3>Trying to associate with (SSID='test-psk' freq=5240 MHz) <3>Associated with <3>WPA: Key negotiation completed with [PTK=CCMP GTK=CCMP] <3>CTRL-EVENT-CONNECTED - Connection to completed [id=0 id_str=] Verify that unicast traffic works and issue the following command in wpa_cli: > raw RESET_PN OK Verify that unicast traffic does not work anymore. If it does, the DUT does not implement replay protection correctly. Note that unicast traffic can recover once the packet number from the test device increases beyond the value used prior to that RESET_PN command. IEEE 802.11 protocol uses unicast frames in station-to-AP direction, so there is no need to test AP replay protection behavior separately with the broadcast IPv4 traffic (which would be converted to unicast frames on the link layer). Testing GTK reinstallation protection on a station device (group handshake) --------------------------------------------------------------------------- Use the procedure describe above for testing replay protection, but with the following hostapd_cli commands: Test broadcast connectivity; should work > raw RESEND_GROUP_M1 OK > raw RESET_PN ff:ff:ff:ff:ff:ff OK Test broadcast connectivity; should not work; if it does, the device does not implement protection for delayed retransmission of Group Key Message 1/2. Testing GTK reinstallation protection on a station device (4-way handshake) --------------------------------------------------------------------------- Use the procedure described above for testing replay protection for broadcast traffic, but with the following hostapd_cli commands: Test broadcast connectivity; should work > raw RESEND_M3 OK > raw RESET_PN ff:ff:ff:ff:ff:ff OK Test broadcast connectivity; should not work; if it does, the device does not implement protection for delayed retransmission of 4-way handshake EAPOL-Key Message 3/4. Variant 1: Include extra Message 1/4 Otherwise same as above, but replace RESEND_M3 command with: > raw RESEND_M1 OK > raw RESEND_M3 OK Variant 2: Include two extra Message 1/4 Otherwise same as above, but replace RESEND_M3 command with: > raw RESEND_M1 change-anonce OK > raw RESEND_M1 OK > raw RESEND_M3 OK Testing TK reinstallation protection on a station device (4-way handshake) -------------------------------------------------------------------------- Use the procedure described above for testing replay protection for unicast traffic, but with the following hostapd_cli commands: Test unicast connectivity; should work > raw RESEND_M3 OK > raw RESET_PN OK Test unicast connectivity; should not work; if it does, the device does not implement protection for delayed retransmission of 4-way handshake EAPOL-Key Message 3/4. Variant 1: Include extra Message 1/4 Otherwise same as above, but replace RESEND_M3 command with: > raw RESEND_M1 OK > raw RESEND_M3 OK Variant 2: Include two extra Message 1/4 Otherwise same as above, but replace RESEND_M3 command with: > raw RESEND_M1 change-anonce OK > raw RESEND_M1 OK > raw RESEND_M3 OK Testing ANonce generation on an AP device ----------------------------------------- Start the AP (DUT) and start wpa_supplicant on the test device to connect to the network. Use wpa_cli to control the test device. <3>SME: Trying to authenticate with (SSID='test-psk' freq=5240 MHz) <3>CTRL-EVENT-REGDOM-CHANGE init=CORE type=WORLD <3>Trying to associate with (SSID='test-psk' freq=5240 MHz) <3>Associated with <3>WPA: Key negotiation completed with [PTK=CCMP GTK=CCMP] <3>CTRL-EVENT-CONNECTED - Connection to completed [id=0 id_str=] Show the ANonce from the first 4-way handshake, request PTK rekeying, and show the ANonce from the second 4-way handshake: > GET anonce df8c61d1f1f7aca9f1739dd888199547f4af2b8b07f8bf15b45ea271da0072b2 > raw KEY_REQUEST 0 1 OK > GET anonce d8ddcb716f28abfdf1352a05d51e7a70f58802122e99d13c730c3c0f09594aac If the ANonce values are same, the AP did not update the ANonce for rekeying (it should have as shown in the example above). Testing FT Reassociation Request frame retransmission on an AP device --------------------------------------------------------------------- This test case requires a sniffer to be used and manually analyzed. Enable FT on the DUT AP (likely two AP devices needed), connect test tool to the AP using FT protocol (e.g., connect to another AP first and then use the "ROAM " command), and do the following steps: - verify unicast traffic from the AP to test station (either ping from the AP or from a device behind the AP); this needs to work - wpa_cli "raw RESEND_ASSOC" - verify unicast traffic from the AP to test station (either ping from the AP or from a device behind the AP); this is likely to fail, but the real analysis is done based on the sniffer capture In the sniffer capture, find the last Reassociation Request frame from the test station (this is more or less identical to the previous one and the only one that should not have Authentication frame exchange before it). Look at the last used PN in a unicast Data frame from the AP to the test station before the last Reassociation Request frame and the PN in the following unicast Data frame after the last Reassociation Request frame. If the PN goes down (e.g., is reset to 1), this would be a sign of a likely security vulnerability. The AP's TK configuration should be verified (i.e., whether it is configuring the same TK again and then allowing it to be used with reused PN values).