ABI stability

Application Binary Interface (ABI) stability is a prerequisite of framework-only updates because vendor modules may depend on the Vendor Native Development Kit (VNDK) shared libraries that reside in the system partition. Within an Android release, newly-built VNDK shared libraries must be ABI-compatible to previously released VNDK shared libraries so vendor modules can work with those libraries without recompilation and without runtime errors. Between Android releases, VNDK libraries can be changed and there are no ABI guarantees.

To help ensure ABI compatibility, Android 9 includes a header ABI checker, as described in the following sections.

About VNDK and ABI compliance

The VNDK is a restrictive set of libraries that vendor modules may link to and which enable framework-only updates. ABI compliance refers to the ability of a newer version of a shared library to work as expected with a module that is dynamically linked to it (i.e. works as an older version of the library would).

About exported symbols

An exported symbol (also known as a global symbol) refers to a symbol that satisfies all of the following:

• Exported by the public headers of a shared library.
• Appears in the .dynsym table of the .so file corresponding to the shared library.
• Has WEAK or GLOBAL binding.
• Visibility is DEFAULT or PROTECTED.
• Section index is not UNDEFINED.
• Type is either FUNC or OBJECT.

The public headers of a shared library are defined as the headers available to other libraries/binaries through the export_include_dirs, export_header_lib_headers, export_static_lib_headers, export_shared_lib_headers, and export_generated_headers attributes in Android.bp definitions of the module corresponding to the shared library.

About reachable types

A reachable type is any C/C++ built-in or user-defined type that is reachable directly or indirectly through an exported symbol AND exported through public headers. For example, libfoo.so has function Foo, which is an exported symbol found in the .dynsym table. The libfoo.so library includes the following:

typedef struct foo_private foo_private_t;

typedef struct foo {
  int m1;
  int *m2;
  foo_private_t *mPfoo;
} foo_t;

typedef struct bar {
  foo_t mfoo;
} bar_t;

bool Foo(int id, bar_t *bar_ptr);

typedef struct foo_private {
  int m1;
  float mbar;
} foo_private_t;

cc_library {
  name : libfoo,
  vendor_available: true,
  vndk {
    enabled : true,
  }
  srcs : ["src/*.cpp"],
  export_include_dirs : [
    "exported"
  ],
}

Looking at Foo, direct/indirect reachable types include:

A similar explanation can be given for types reachable through base class specifiers and template parameters as well.

Ensure ABI compliance

ABI compliance must be ensured for the libraries marked vendor_available: true and vndk.enabled: true in the corresponding Android.bp files. For example:

cc_library {
    name: "libvndk_example",
    vendor_available: true,
    vndk: {
        enabled: true,
    }
}

For data types reachable directly or indirectly by an exported function, the following changes to a library are classified as ABI-breaking:

* Both public and private member functions must not be changed or removed because public inline functions can refer to private member functions. Symbol references to private member functions can be kept in caller binaries. Changing or removing private member functions from shared libraries can result in backward-incompatible binaries.

** The offsets to public or private data members must not be changed because inline functions can refer to these data members in their function body. Changing data member offsets can result in backward-incompatible binaries.

*** While these don't change the memory layout of the type, there are semantic differences that could lead to libraries not functioning as expected.

Use ABI compliance tools

When a VNDK library is built, the library's ABI is compared with the corresponding ABI reference for the version of the VNDK being built. Reference ABI dumps are located in:

${ANDROID_BUILD_TOP}/prebuilts/abi-dumps/vndk/<PLATFORM_VNDK_VERSION>/<BINDER_BITNESS>/<ARCH>/source-based

For example, on building libfoo for x86 at API level 27, libfoo's inferred ABI is compared with its reference at:

${ANDROID_BUILD_TOP}/prebuilts/abi-dumps/vndk/27/64/x86/source-based/libfoo.so.lsdump

ABI breakage error

On ABI breakages, the build log displays warnings with the warning type and a path to the abi-diff report. For example, if libbinder's ABI has an incompatible change, the build system throws an error with a message similar to the following:

*****************************************************
error: VNDK library: libbinder.so's ABI has INCOMPATIBLE CHANGES
Please check compatibility report at:
out/soong/.intermediates/frameworks/native/libs/binder/libbinder/android_arm64_armv8-a_cortex-a73_vendor_shared/libbinder.so.abidiff
******************************************************
---- Please update abi references by running
platform/development/vndk/tools/header-checker/utils/create_reference_dumps.py -l libbinder ----

Build VNDK library ABI checks

When a VNDK library is built:

1. header-abi-dumper processes the source files compiled to build the VNDK library (the library's own source files as well as source files inherited through static transitive dependencies), to produce .sdump files that correspond to each source.
   
2. header-abi-linker then processes the .sdump files (using either a version script provided to it or the .so file corresponding to the shared library) to produce a .lsdump file that logs all of the ABI information corresponding to the shared library.
   
3. header-abi-diff compares the .lsdump file with a reference .lsdump file to produce a diff report that outlines the differences in the ABIs of the two libraries.
   

header-abi-dumper

The header-abi-dumper tool parses a C/C++ source file and dumps the ABI inferred from that source file into an intermediate file. The build system runs header-abi-dumper on all compiled source files while also building a library that includes the source files from transitive dependencies.

Currently .sdump files are in JSON format, which isn't guaranteed to be stable across future releases. As such, .sdump file formatting should be considered a build system implementation detail.

For example, libfoo.so has the following source file foo.cpp:

#include <stdio.h>
#include <foo_exported.h>

bool Foo(int id, bar_t *bar_ptr) {
    if (id > 0 && bar_ptr->mfoo.m1 > 0) {
        return true;
    }
    return false;
}

You can use header-abi-dumper to generate an intermediate .sdump file that represents the ABI presented by the source file using:

$ header-abi-dumper foo.cpp -I exported -o foo.sdump -- -I exported -x c++

This command tells header-abi-dumper to parse foo.cpp with the compiler flags following --, and emit the ABI information that is exported by the public headers in the exported directory. The following is foo.sdump generated by header-abi-dumper:

{
 "array_types" : [],
 "builtin_types" :
 [
  {
   "alignment" : 4,
   "is_integral" : true,
   "linker_set_key" : "_ZTIi",
   "name" : "int",
   "referenced_type" : "_ZTIi",
   "self_type" : "_ZTIi",
   "size" : 4
  }
 ],
 "elf_functions" : [],
 "elf_objects" : [],
 "enum_types" : [],
 "function_types" : [],
 "functions" :
 [
  {
   "function_name" : "FooBad",
   "linker_set_key" : "_Z6FooBadiP3foo",
   "parameters" :
   [
    {
     "referenced_type" : "_ZTIi"
    },
    {
     "referenced_type" : "_ZTIP3foo"
    }
   ],
   "return_type" : "_ZTI3bar",
   "source_file" : "exported/foo_exported.h"
  }
 ],
 "global_vars" : [],
 "lvalue_reference_types" : [],
 "pointer_types" :
 [
  {
   "alignment" : 8,
   "linker_set_key" : "_ZTIP11foo_private",
   "name" : "foo_private *",
   "referenced_type" : "_ZTI11foo_private",
   "self_type" : "_ZTIP11foo_private",
   "size" : 8,
   "source_file" : "exported/foo_exported.h"
  },
  {
   "alignment" : 8,
   "linker_set_key" : "_ZTIP3foo",
   "name" : "foo *",
   "referenced_type" : "_ZTI3foo",
   "self_type" : "_ZTIP3foo",
   "size" : 8,
   "source_file" : "exported/foo_exported.h"
  },
  {
   "alignment" : 8,
   "linker_set_key" : "_ZTIPi",
   "name" : "int *",
   "referenced_type" : "_ZTIi",
   "self_type" : "_ZTIPi",
   "size" : 8,
   "source_file" : "exported/foo_exported.h"
  }
 ],
 "qualified_types" : [],
 "record_types" :
 [
  {
   "alignment" : 8,
   "fields" :
   [
    {
     "field_name" : "mfoo",
     "referenced_type" : "_ZTI3foo"
    }
   ],
   "linker_set_key" : "_ZTI3bar",
   "name" : "bar",
   "referenced_type" : "_ZTI3bar",
   "self_type" : "_ZTI3bar",
   "size" : 24,
   "source_file" : "exported/foo_exported.h"
  },
  {
   "alignment" : 8,
   "fields" :
   [
    {
     "field_name" : "m1",
     "referenced_type" : "_ZTIi"
    },
    {
     "field_name" : "m2",
     "field_offset" : 64,
     "referenced_type" : "_ZTIPi"
    },
    {
     "field_name" : "mPfoo",
     "field_offset" : 128,
     "referenced_type" : "_ZTIP11foo_private"
    }
   ],
   "linker_set_key" : "_ZTI3foo",
   "name" : "foo",
   "referenced_type" : "_ZTI3foo",
   "self_type" : "_ZTI3foo",
   "size" : 24,
   "source_file" : "exported/foo_exported.h"
  }
 ],
 "rvalue_reference_types" : []
}

foo.sdump contains ABI information exported by the source file foo.cpp and the public headers, for example,

• record_types. Refer to structs, unions, or classes defined in the public headers. Each record type has information about its fields, its size, access specifier, the header file it's defined in, and other attributes.
• pointer_types. Refer to pointer types directly/indirectly referenced by the exported records/functions in the public headers, along with the type the pointer points to (through the referenced_type field in type_info). Similar information is logged in the .sdump file for qualified types, built-in C/C++ types, array types, and lvalue and rvalue reference types. Such information allows recursive diffing.
• functions. Represent functions exported by public headers. They also have information about the function's mangled name, the return type, the types of the parameters, the access specifier, and other attributes.

header-abi-linker

The header-abi-linker tool takes the intermediate files produced by header-abi-dumper as input then links those files:

The tool merges the type graphs in all the intermediate files given to it, taking into account one-definition (user-defined types in different translation units with the same fully qualified name, might be semantically different) differences across translation units. The tool then parses either a version script or the .dynsym table of the shared library (.so file) to make a list of the exported symbols.

For example, libfoo consists of foo.cpp and bar.cpp. header-abi-linker could be invoked to create the complete linked ABI dump of libfoo as follows:

header-abi-linker -I exported foo.sdump bar.sdump \
                  -o libfoo.so.lsdump \
                  -so libfoo.so \
                  -arch arm64 -api current

Example command output in libfoo.so.lsdump:

{
 "array_types" : [],
 "builtin_types" :
 [
  {
   "alignment" : 1,
   "is_integral" : true,
   "is_unsigned" : true,
   "linker_set_key" : "_ZTIb",
   "name" : "bool",
   "referenced_type" : "_ZTIb",
   "self_type" : "_ZTIb",
   "size" : 1
  },
  {
   "alignment" : 4,
   "is_integral" : true,
   "linker_set_key" : "_ZTIi",
   "name" : "int",
   "referenced_type" : "_ZTIi",
   "self_type" : "_ZTIi",
   "size" : 4
  }
 ],
 "elf_functions" :
 [
  {
   "name" : "_Z3FooiP3bar"
  },
  {
   "name" : "_Z6FooBadiP3foo"
  }
 ],
 "elf_objects" : [],
 "enum_types" : [],
 "function_types" : [],
 "functions" :
 [
  {
   "function_name" : "Foo",
   "linker_set_key" : "_Z3FooiP3bar",
   "parameters" :
   [
    {
     "referenced_type" : "_ZTIi"
    },
    {
     "referenced_type" : "_ZTIP3bar"
    }
   ],
   "return_type" : "_ZTIb",
   "source_file" : "exported/foo_exported.h"
  },
  {
   "function_name" : "FooBad",
   "linker_set_key" : "_Z6FooBadiP3foo",
   "parameters" :
   [
    {
     "referenced_type" : "_ZTIi"
    },
    {
     "referenced_type" : "_ZTIP3foo"
    }
   ],
   "return_type" : "_ZTI3bar",
   "source_file" : "exported/foo_exported.h"
  }
 ],
 "global_vars" : [],
 "lvalue_reference_types" : [],
 "pointer_types" :
 [
  {
   "alignment" : 8,
   "linker_set_key" : "_ZTIP11foo_private",
   "name" : "foo_private *",
   "referenced_type" : "_ZTI11foo_private",
   "self_type" : "_ZTIP11foo_private",
   "size" : 8,
   "source_file" : "exported/foo_exported.h"
  },
  {
   "alignment" : 8,
   "linker_set_key" : "_ZTIP3bar",
   "name" : "bar *",
   "referenced_type" : "_ZTI3bar",
   "self_type" : "_ZTIP3bar",
   "size" : 8,
   "source_file" : "exported/foo_exported.h"
  },
  {
   "alignment" : 8,
   "linker_set_key" : "_ZTIP3foo",
   "name" : "foo *",
   "referenced_type" : "_ZTI3foo",
   "self_type" : "_ZTIP3foo",
   "size" : 8,
   "source_file" : "exported/foo_exported.h"
  },
  {
   "alignment" : 8,
   "linker_set_key" : "_ZTIPi",
   "name" : "int *",
   "referenced_type" : "_ZTIi",
   "self_type" : "_ZTIPi",
   "size" : 8,
   "source_file" : "exported/foo_exported.h"
  }
 ],
 "qualified_types" : [],
 "record_types" :
 [
  {
   "alignment" : 8,
   "fields" :
   [
    {
     "field_name" : "mfoo",
     "referenced_type" : "_ZTI3foo"
    }
   ],
   "linker_set_key" : "_ZTI3bar",
   "name" : "bar",
   "referenced_type" : "_ZTI3bar",
   "self_type" : "_ZTI3bar",
   "size" : 24,
   "source_file" : "exported/foo_exported.h"
  },
  {
   "alignment" : 8,
   "fields" :
   [
    {
     "field_name" : "m1",
     "referenced_type" : "_ZTIi"
    },
    {
     "field_name" : "m2",
     "field_offset" : 64,
     "referenced_type" : "_ZTIPi"
    },
    {
     "field_name" : "mPfoo",
     "field_offset" : 128,
     "referenced_type" : "_ZTIP11foo_private"
    }
   ],
   "linker_set_key" : "_ZTI3foo",
   "name" : "foo",
   "referenced_type" : "_ZTI3foo",
   "self_type" : "_ZTI3foo",
   "size" : 24,
   "source_file" : "exported/foo_exported.h"
  }
 ],
 "rvalue_reference_types" : []
}

The header-abi-linker tool:

• Links the .sdump files provided to it (foo.sdump and bar.sdump), filtering out the ABI information not present in the headers residing in the directory: exported.
• Parses libfoo.so, and collects information about the symbols exported by the library through its .dynsym table.
• Adds _Z3FooiP3bar and _Z6FooBadiP3foo.

libfoo.so.lsdump is the final generated ABI dump of libfoo.so.

header-abi-diff

The header-abi-diff tool compares two .lsdump files representing the ABI of two libraries and produces a diff report stating the differences between the two ABIs.

The ABI diff file is in protobuf text format. The format is subject to change in future releases.

For example, you have two versions of libfoo: libfoo_old.so and libfoo_new.so. In libfoo_new.so, in bar_t, you change the type of mfoo from foo_t to foo_t *. Since bar_t is a reachable type, this should be flagged as an ABI breaking change by header-abi-diff.

To run header-abi-diff:

header-abi-diff -old libfoo_old.so.lsdump \
                -new libfoo_new.so.lsdump \
                -arch arm64 \
                -o libfoo.so.abidiff \
                -lib libfoo

Example command output in libfoo.so.abidiff:

lib_name: "libfoo"
arch: "arm64"
record_type_diffs {
  name: "bar"
  type_stack: "Foo-> bar *->bar "
  type_info_diff {
    old_type_info {
      size: 24
      alignment: 8
    }
    new_type_info {
      size: 8
      alignment: 8
    }
  }
  fields_diff {
    old_field {
      referenced_type: "foo"
      field_offset: 0
      field_name: "mfoo"
      access: public_access
    }
    new_field {
      referenced_type: "foo *"
      field_offset: 0
      field_name: "mfoo"
      access: public_access
    }
  }
}

The libfoo.so.abidiff contains a report of all ABI breaking changes in libfoo. The record_type_diffs message indicates a record has changed and lists the incompatible changes, which include:

• The size of the record changing from 24 bytes to 8 bytes.
• The field type of mfoo changing from foo to foo * (all typedefs are stripped off).

The type_stack field indicates how header-abi-diff reached the type that changed (bar). This field may be interpreted as Foo is an exported function that takes in bar * as parameter, that points to bar, which was exported and changed.

Enforce ABI and API

To enforce the ABI and API of VNDK shared libraries, ABI references must be checked into ${ANDROID_BUILD_TOP}/prebuilts/abi-dumps/vndk/. To create these references, run the following command:

${ANDROID_BUILD_TOP}/development/vndk/tools/header-checker/utils/create_reference_dumps.py

After creating the references, any change made to the source code that results in an incompatible ABI/API change in a VNDK library now results in a build error.

To update ABI references for specific libraries, run the following command:

${ANDROID_BUILD_TOP}/development/vndk/tools/header-checker/utils/create_reference_dumps.py -l <lib1> -l <lib2>

For example, to update libbinder ABI references, run:

${ANDROID_BUILD_TOP}/development/vndk/tools/header-checker/utils/create_reference_dumps.py -l libbinder