Starting March 27, 2025, we recommend using android-latest-release instead of aosp-main to build and contribute to AOSP. For more information, see Changes to AOSP.

Run CTS Verifier multidevice tests

This page contains instructions for using Better Together CTS Verifier (CTS-V) tests for Android 16 QPR2 or higher.

Set up host-side multidevice tests

This section explains how to set up multidevice tests.

Verify that your desktop machine meets the operating system requirements for CTS.
Follow steps 2 and 5 of Install desktop software to install and verify that adb, AAPT2, and Python are correctly installed on your desktop.
- Your Python version should be 3.11 or higher. To determine your Python version, run python3 --version. If the version is lower than 3.11, install the latest official Python release. For further information, see the Downloads section of python.org.
- Certain tests require the host to have the Python venv module. On Debian and Ubuntu systems, this module might not be installed by default. To determine if your Python version has the venv module, run python3 -m venv venv. If this command fails, an error message is shown. Follow the prompt to install the python3.x-venv package.
Prepare two matching devices under test (DUTs), each with CTS-V set up.
- For information on setting up a DUT, see Set up the DUT.
- For instructions on setting up CTS-V, see Setup.
Go to the setup section for your test type:
- For NFC tests, go to Set up NFC tests.
- For Wi-Fi AP connection tests, go to Set up Wi-Fi AP connection tests.
- To test the CDM module, go to Set up standard two-device tests and then go to Set up CDM tests.
If your test isn't in this list, proceed to Set up standard two-device tests

Set up NFC tests

NFC tests use one DUT and one PN532 NFC chip.

To set up NFC tests:

Purchase a PN532 NFC chip. We recommend the All-In-One PN532.
On the DUT, navigate to the Settings app.
Enable NFC.
Position the NFC chip:
- For phones, position the DUT's NFC reader as shown in Figure 1:
  
  Figure 1. NFC chip positioning.
- For other device types, position the chip next to the device's NFC antenna.
Attach the PN532 NFC chip to your testing workstation using a USB cable.

Optional: Set up Wi-Fi AP connection tests

The Wi-Fi access point (AP) connection tests (CtsWifiConnectionTests) test connectivity between a DUT and AP. We strongly recommend that you run these tests, but they aren't required in CTS-V Android 16 16 QPR2.

These tests require a DUT and an OpenWrt Banana Pi R3 AP.

To set up Wi-Fi AP connection tests:

Purchase the Banana Pi R3 AP and accessories listed in this table:

Item	Quantity
BPi-R3 board, similar to the Banana Pi BPI-R3 Router Board with MediaTek MT7986 chip design support Wi-Fi 6 ,2G DDR RAM ,8G eMMC flash onboard	1
BPi-R3 aluminum case, similar to the BPI-R3 Iron case	1
BPi-R3 aluminum heatsink (cooling fan), similar to the BPI-R3 aluminum heatsink with fan	1
2 and 5 GHz antenna with cable, similar to the 5DB antenna in the BPI shop	8
Power adapter, similar to the 12V/2A DC power supply	1

To make your purchase, see the Easy to buy section of the Banana Pi BPI-R3 page.

Set up the access point. For information on setting up the access point, see Set up the Banana Pi BPI-R3 AP.
Optional: If you don't have a shield box, we recommend the JTP-SR101 shield box. Purchase this box using the following information:

Dong Guan Zheng Sheng Electronics Technology Co., LTD
Bohui Industrial Park, Panlong Road, Liaobu Town, Dongguan City, Guangdong Province, China
Contact: Forest Pan
Email: forest.pan@jtpmak.cn
Phone (China): +86 18676993556

Note: A shield box allows for external signal isolation and for a DUT and AP to be 10 cm apart
Connect the DUT and AP to the host and place in an RF shield box. The DUT and AP should be at least 10 cm apart. Figure 2 shows this configuration:

Figure 2. DUT and AP in shield box.
Use SSH to verify that the AP is accessible from the host.

Set up standard two-device tests

For the default two-device setup:

Place two matching Android DUTs approximately 20 cm apart.
To set up a clean environment, place both devices in a shield box.

Note: If you have issues with these tests and require help, verify tests have been run with both devices in a shield box.
Optional: Set up an OTA sniffer for Wi-Fi debugging.

Set up CDM tests

The test_permissions_sync() test case has different behavior depending on the build type of the devices on which the test is executed. It is critical that both debuggable (userdebug or eng) and non-debuggable (user) builds are tested by OEMs and that tests pass for both.

Exemption

The CDD clause for the permissions sync API implementation only requires it to be able to successfully transfer data between devices over a secure channel. Since the secure channel implementation isn't a CDD compliance requirement, this test can be skipped on non-debuggable (user) builds, but only if you want to opt out of supporting the CDM permissions sync feature.

The tests must pass on debuggable builds without exception.

Prerequisites for testing on non-debuggable builds

If you aren't exempt by the previous exemption clauses, verify that you meet the following prerequisites.

The secure channel uses AVF (AttestationVerificationFramework) to verify the trustworthiness of hardware. Attestations generated by both parties contain several pieces of information about themselves to ensure that there hasn't been any unauthorized alteration in their system. AVF checks for the following states during the verification process:

Device has access to internet
Device uses verified boot and the build must be signed with a release-key, not a dev-key
Device is bootloader locked. For detailed instructions, see locking the bootloader
OS, key boot, and key vendor patch levels are within 12 months. Don't use a build older than a year
Device attestation is backed by one of the vendor-approved root certificates. Specify your trusted root certificates in the vendor_required_attestation_certificates.xml resource overlay.

Run host-side multidevice tests (AOSP 16 or later)

CTS Verifier 16 introduces the support for host-side multidevice tests. These tests can be executed using automated scripts on the host, instead of the manual test operation on the device. After each test is completed, results are automatically uploaded to the DUT and displayed in the CTS Verifier app.

This section explains how to run the host-side multidevice tests.

Run multidevice tests

To run a multidevice test:

On your test workstation, launch the cts-v-host console from the directory where the CTS-V zip package was unzipped:
```
./android-cts-verifier/android-cts-v-host/tools/cts-v-host-tradefed
```
From within the CTS Verifier app on the DUT, click Host-side Tests. Figure 3 shows the host-side tests in the CTS Verifier app:

Figure 3. Host-side multidevice tests in CTS Verifier app.

A list of test host-side multidevice test modules is displayed.
Identify the name of the test module that you want to run. For example, the CompanionDeviceManager module is listed as CtsCompanionDeviceManagerMultiDeviceTestCases.

Note: If you tap on a host-side module in the CTS Verifier app, you won't see a list of individual test cases. Instead the UI automatically populates with the test case results after the module has been executed on the host.
In the cts-v-host console, run the following command:
```
run cts-v-host -m test_module_name
```
For example:
```
run cts-v-host -m CtsCompanionDeviceManagerMultiDeviceTestCases
```
Note: If you run cts-v-host without using the -m test_module_name, the command attempts to run all test modules, ignoring deviations in device and hardware requirements. Instead, execute the run cts-v-host-default-devices command to run the tests with the standard setup as a single subplan.
Note: You can indicate the specific DUTs to test using their serial numbers: run cts-v-host -m test_module_name -s\ device1_serial -s device2_serial ...

After the xTS console finishes running the tests, the results appear in the CTS Verifier app. Tests marked green have passed. Tests marked red have failed. Figure 4 shows example results for the CtsCompanionDeviceManager tests:

Figure 4. Host-side multidevice test results in CTS Verifier app.

Note: Host-side results are automatically added to the overall CTS-V test report.

Optional: Run Wi-Fi AP connection tests

Follow these steps to run Wi-Fi AP connection tests:

Edit the testbed configuration file (WifiConnectionTestbed.yaml). This file is in the directory where CTS-Verifier is unzipped:
```
./android-cts-verifier/android-cts-v-host/testcases/CtsWifiConnectionTests/x86_64/connection/WifiConnectionTestbed.yaml
```
Change the value of the hostname field to the IP address of the AP, based on your local SSH settings. To identify the IP address, see Find the IP address of the AP.

The following example shows the location of the hostname field in the WifiConnectionTestbed.yaml file:
```
TestBeds:
- Name: WifiConnectionTestbed
  Controllers:
    # Specify settings for the AP.
    OpenWrtDevice:
    - hostname: AP-IP
      skip_init_reboot: True
```
In the cts-v-host console, run the following command:
```
run everything -m CtsWifiConnectionTests
```
Note: For further information on the cts-v-host console, see Run multidevice tests.

Troubleshoot multidevice tests

This section provides troubleshooting help for possible issues.

Fix No response for GetFirmwareVersion during NFC tests

If you receive the message verify_firmware_version RuntimeError: No response for GetFirmwareVersion while running the multidevice tests, the tests can't access the PN532 NFC board.

To fix this issue, identify the serial path used by the PN532 NFC board on your host, such as dev/ttyUSB1, then manually specify it using the --module-arg argument in the console:

run cts-v-host -m CtsNfcHceMultiDeviceTestCases --module-arg CtsNfcHceMultiDeviceTestCases:pn532_serial_path:/dev/ttyUSB1

Fix Transaction failed error message during NFC tests

If you receive the message Transaction failed, check device logs for more information. for all NFC test cases, it is likely because the DUT's NFC chip can't detect the PN532.

If you have multiple devices connected to the host, and some of them don't have a PN532 placed on top, the wrong DUT may have been selected. For more information, see Set up NFC tests.

To fix this problem, do one of the following:

Set the correct DUT's serial in your host-side test command using the -s flag.
Disconnect all non-DUT devices from the host.

CDM test case `test_permissions_sync` is ignored

If the test is being run on non-debuggable devices, then see if you are exempted. Otherwise, verify that both devices meet the prerequisites.