Self-instrumenting tests example

When an instrumentation test is started, its target package is restarted with instrumentation code injected and initiated for execution. One exception is that the target package here can't be the Android application framework itself, such as the package android, because doing so leads to the paradoxical situation where Android framework would need to be restarted, which is what supports the system functions, including the instrumentation itself.

This means that an instrumentation test cannot inject itself into Android framework, a.k.a. the system server, for execution. In order to test the Android framework, the test code can invoke only public API surfaces, or those exposed using Android Interface Definition Language AIDL available in the platform source tree. For this category of tests, it's not meaningful to target any particular package. Therefore, it's customary for such instrumentations to be declared to target its own test application package, as defined in its own <manifest> tag of AndroidManifest.xml.

Depending on the requirements, test application packages in this category may also:

• Bundle activities needed for testing.
• Share the user ID with the system.
• Be signed with the platform key.
• Be compiled against the framework source rather than the public SDK.

This category of instrumentation tests is sometimes referred to as self-instrumentation. Here are some examples of self-instrumentation tests in the platform source:

• frameworks/base/core/tests/
• frameworks/base/services/tests/servicestests

The example covered here is writing a new instrumentation test with target package set at its own test application package. This guide uses the following test to serve as an example:

• Hello World Instrumentation Test

It's recommended to browse through the code first to get a rough impression before proceeding.

Decide on a source location

Typically your team will already have an established pattern of places to check in code, and places to add tests. Most teams own a single git repository, or share one with other teams but have a dedicated sub directory that contains component source code.

Assuming the root location for your component source is at <component source root>, most components have src and tests folders under it, and some additional files such as Android.mk (or broken up into additional .mk files), the manifest file AndroidManifest.xml, and the test configuration file 'AndroidTest.xml'.

Since you are adding a brand new test, you'll probably need to create the tests directory next to your component src, and populate it with content.

In some cases, your team might have further directory structures under tests due to the need to package different suites of tests into individual apks. And in this case, you'll need to create a new sub directory under tests.

Regardless of the structure, you'll end up populating the tests directory or the newly created sub directory with files similar to what's in instrumentation directory in the sample gerrit change. The details of each file are explained later in this document.

Manifest file

As with an app project, each instrumentation test module requires a manifest file called AndroidManifest.xml. To automatically include this file using theBUILD_PACKAGE core makefile, provide this file next to the Android.mk file for your test module.

If you aren't familiar with the AndroidManifest.xml file, refer to the App Manifest Overview

Following is a sample AndroidManifest.xml file:

<manifest xmlns:android="http://schemas.android.com/apk/res/android"
  android:sharedUserId="android.uid.system"
  package="android.test.example.helloworld" >

    <application>
       <uses-library android:name="android.test.runner"/>
    </application>

    <instrumentation android:name="androidx.test.runner.AndroidJUnitRunner"
                     android:targetPackage="android.test.example.helloworld"
                     android:label="Hello World Test"/>

</manifest>

Some select remarks on the manifest file:

<manifest xmlns:android="http://schemas.android.com/apk/res/android"
    package="android.test.example.helloworld" >

The package attribute is the application package name: this is the unique identifier that the Android application framework uses to identify an application (or in this context: your test application). Each user in the system can only install one application with that package name.

Furthermore, this package attribute is the same as what ComponentName#getPackageName() returns, and also the same you would use to interact with various pm sub commands use adb shell.

Note that although the package name is typically in the same style as a Java package name, it actually has very few things to do with it. In other words, your application (or test) package may contain classes with any package names, though on the other hand, you could opt for simplicity and have your top level Java package name in your application or test identical to the application package name.

android:sharedUserId="android.uid.system"

This declares that at installation time, this APK file should be granted the same user ID, i.e. runtime identity, as the core platform. Note that this is dependent on the apk being signed with same certificate as the core platform (see LOCAL_CERTIFICATE in a previous section), yet they are different concepts:

• some permissions or APIs are signature protected, which requires same signing certificate
• some permissions or APIs requires the system user identity of the caller, which requires the calling package to share user ID with system, if it's a separate package from core platform itself

<uses-library android:name="android.test.runner" />

This is required for all Instrumentation tests since the related classes are packaged in a separate framework JAR library file, therefore requires additional classpath entries when the test package is invoked by application framework.

android:targetPackage="android.test.example.helloworld"

You might have noticed that the targetPackage here is declared the same as the package attribute declared in the manifest tag of this file. As mentioned in testing basics, this category of instrumentation test are typically intended for testing framework APIs, so it's not very meaningful for them to have a specific targeted application package, other then itself.

Simple configuration file

Each new test module must have a configuration file to direct the build system with module metadata, compile-time dependencies and packaging instructions. In most cases, the Soong-based, Blueprint file option is sufficient. For details, see Simple Test Configuration.

Complex configuration file

For these more complex cases, you also need to write a test configuration file for Android's test harness, Trade Federation.

The test configuration can specify special device setup options and default arguments to supply the test class. See the example at /platform_testing/tests/example/instrumentation/AndroidTest.xml.

A snapshot is included here for convenience:

<configuration description="Runs sample instrumentation test.">
  <target_preparer class="com.android.tradefed.targetprep.TestFilePushSetup"/>
  <target_preparer class="com.android.tradefed.targetprep.TestAppInstallSetup">
    <option name="test-file-name" value="HelloWorldTests.apk"/>
  </target_preparer>
  <target_preparer class="com.android.tradefed.targetprep.PushFilePreparer"/>
  <target_preparer class="com.android.tradefed.targetprep.RunCommandTargetPreparer"/>
  <option name="test-suite-tag" value="apct"/>
  <option name="test-tag" value="SampleInstrumentationTest"/>

  <test class="com.android.tradefed.testtype.AndroidJUnitTest">
    <option name="package" value="android.test.example.helloworld"/>
    <option name="runner" value="android.support.test.runner.AndroidJUnitRunner"/>
  </test>
</configuration>

Some select remarks on the test configuration file:

<target_preparer class="com.android.tradefed.targetprep.TestAppInstallSetup">
  <option name="test-file-name" value="HelloWorldTests.apk"/>
</target_preparer>

This tells Trade Federation to install the HelloWorldTests.apk onto the target device using a specified target_preparer. There are many target preparers available to developers in Trade Federation and these can be used to ensure the device is setup properly prior to test execution.

<test class="com.android.tradefed.testtype.AndroidJUnitTest">
  <option name="package" value="android.test.example.helloworld"/>
  <option name="runner" value="android.support.test.runner.AndroidJUnitRunner"/>
</test>

This specifies the Trade Federation test class to use to execute the test and passes in the package on the device to be executed and the test runner framework which is JUnit in this case.

For more information, see Test Module Configs.

JUnit4 features

Using android-support-test library as test runner enables adoption of new JUnit4 style test classes, and the sample gerrit change contains some very basic use of its features. See the example at /platform_testing/tests/example/instrumentation/src/android/test/example/helloworld/HelloWorldTest.java.

While testing patterns are usually specific to component teams, there are some generally useful usage patterns.

@RunWith(JUnit4.class)
public class HelloWorldTest {

A significant difference in JUnit4 is that tests are no longer required to inherit from a common base test class; instead, you write tests in plain Java classes and use annotation to indicate certain test setup and constraints. In this example, we are instructing that this class should be run as a JUnit4 test.

    @BeforeClass
    public static void beforeClass() {
    ...
    @AfterClass
    public static void afterClass() {
    ...
    @Before
    public void before() {
    ...
    @After
    public void after() {
    ...
    @Test
    @SmallTest
    public void testHelloWorld() {
    ...

The @Before and @After annotations are used on methods by JUnit4 to perform pre test setup and post test teardown. Similarly, the @BeforeClass and @AfterClass annotations are used on methods by JUnit4 to perform setup before executing all tests in a test class, and teardown afterwards. Note that the class-scope setup and teardown methods must be static. As for the test methods, unlike in earlier version of JUnit, they no longer need to start the method name with test, instead, each of them must be annotated with @Test. As usual, test methods must be public, declare no return value, take no parameters, and may throw exceptions.

Instrumentation class access

Although not covered in the basic hello world example, it's fairly common for an Android test to require access Instrumentation instance: this is the core API interface that provides access to application contexts, activity lifecycle related test APIs and more.

Because the JUnit4 tests no longer require a common base class, it's no longer necessary to obtain Instrumentation instance via InstrumentationTestCase#getInstrumentation(), instead, the new test runner manages it via InstrumentationRegistry where contextual and environmental setup created by instrumentation framework is stored.

To access the instance of Instrumentation class, simply call static method getInstrumentation() on InstrumentationRegistry class:

Instrumentation instrumentation = InstrumentationRegistry.getInstrumentation()

Build and test locally

For the most common use cases, employ Atest.

For more complex cases requiring heavier customization, follow the instrumentation instructions.