CTS device interaction helper module

For Android 11 or higher, the Compatibility Test Suite (CTS) device interaction helper modules let you customize how certain CTS tests interact with the user interface (UI) on a specific device. This means that actions, such as replacing a UI element that isn't covered by the Android Compatibility Definition Document (CDD) or API docs, can be done while still passing CTS.

OEMs who want to customize the Android UI during product development and need to pass CTS might be able to implement helper modules. If you use the default Android implementation, then no additional work is needed.

Implementing helper modules

Requirements to customize the UI

Check the CDD or Mainline modules for any UI requirements. If the desired UI is covered by the CDD or Mainline modules, then that UI can't be customized.

If the CTS tests that interact with the desired UI don't use the helper framework, then that UI can't be customized. Work with the test owner to convert the test module before the UI can be changed.

Otherwise, you can customize the UI.

Implementation workflow

  1. Customize the UI as needed for your specific product.
  2. Designate the existing AOSP helper modules as subclasses for the CTS test modules that need to interact with the UI. Replace the necessary interactions appropriately for the customized UI. Replacements vary depending on the type of changes.
    • The OEM subclasses are in an OEM package, such as com.[oem].cts.helpers.
    • Each OEM subclass is named with a common prefix that distinguishes it from the AOSP implementation, which has the prefix Default.
  3. Build the helpers into an APK following these test runner conventions.
    • Android.bp should declare android_test_helper_app with the same name as the contained package.
    • AndroidManifest.xml for the APK must declare a metadata property named interaction-helpers-prefix with the value of the class prefix chosen in the previous bullet point.
    • The app should depend on cts-helpers-core, cts-helpers-interfaces, and com.android.cts.helpers.aosp. If the OEM helpers fully implement all relevant interfaces, then com.android.cts.helpers.aosp is optional.
  4. Set the ro.vendor.cts_interaction_helper_packages property in the device image to include the name of the APK. If you need to separate your helper implementations across multiple APKs, this property can contain a colon-separated list of packages.
  5. Ensure that the APK is available in the testcases directory when running Tradefed for CTS. If needed, confirm that the expected helper implementation class is chosen by examining logcat messages.
  6. Optional, but highly recommended: Submit your helper implementation to AOSP or make it available for third-party testing.

Example helper implementation

For example, CtsPrintTestCases expects a helper with the interface defined in ICtsPrintHelper. The AOSP implementation is called com.android.cts.helpers.aosp.DefaultCtsPrintHelper.

If you customize the print UI, you can create com.oem.cts.helpers.OemCtsPrintHelper that subclasses DefaultCtsPrintHelper. android_test_helper_app in Android.bp is named com.oem.cts.helpers, which produces com.oem.cts.helpers.apk, and declares interaction-helpers-prefix as Oem in AndroidManifest.xml.

The device property ro.vendor.cts_interaction_helper_packages is set to com.oem.cts.helpers.

Reference implementations

Reference implementations include interfaces under cts/libs/helpers and the default AOSP helpers under cts/helpers. The top-level interface is documented in cts/libs/helpers/core/src/com/android/cts/helpers/ICtsDeviceInteractionHelper.java.

To connect the CTS test to its helpers, test owners can use the @Rule definition documented in cts/libs/helpers/core/src/com/android/cts/helpers/DeviceInteractionHelperRule.java.

Each CTS module that uses the framework and its expected helper behavior is documented in an interface defined under cts/libs/helpers/core/src/com/android/cts/helpers.

Running CTS tests

Testing without helpers

Other than one property, the option to test without helpers doesn’t exist at runtime on the device, but optionally modifies how CTS tests interact with the device. If you need to run CTS without the helper implementations, you have two options:

  • Remove the ro.vendor.cts_interaction_helper_packages property from the device. This prevents the helpers from being used on that build entirely.
  • Remove the helper APK from the testcases directory before running CTS. This prevents the helpers from being used by any runs until the APK is restored to testcases.

You can alter the default settings with Tradefed arguments and the ro.vendor.cts_interaction_helper_packages property control, which is where the helper APK is loaded from.

See the following for the expected values or ranges for each of the available settings.

  • ro.vendor.cts_interaction_helper_packages is a colon-separated string containing package names. It can take any value that is a valid package choice for the OEM’s helper implementation.
  • cts-tradefed accepts a device-interaction-helper:property-name argument that temporarily changes the expected property for one test run, such as --module-arg 'CtsPrintTestCases:{device-interaction-helper}property-name:debug.cts.hlp'. The value of the property name can be any property that you set on the device. The value of the property follows the same restrictions as the ro.vendor.cts_interaction_helper_packages property described above.

Testing with customizations

By default, reference implementations pass CTS on stock Android. Check that partner implementations pass CTS with UI customizations. Run whichever CTS modules cover the UI or features you customized.

Certain CTS modules or helpers might not support some customizations yet.

  • A CTS module that interacts with the UI you want to customize might not use the helper framework. CTS modules are expected to convert to the helper framework based on demand and the test owner priorities. File requests for conversion early in the process to ensure that the conversion gets on the schedule, similar to requesting CTS changes to support your planned features.
  • Functions provided by an existing helper might not fully address the customizations you want to make. Helper functions should abstract away UI dependencies. If a helper function indirectly has a UI dependency, this can be treated similarly to bugs in CTS.