Android-CustomSDK

Building Custom SDK with all Hidden APIs to work with Android System Applications with Android Studio.

This can be done either by customizing the sdk itself or importing the library into Android Studio. But the convenient way is that to build a custom sdk provided as the first step.

I. Building Custom SDK

1. Download the aosp repo

2. $source build/envsetup.sh

3. $lunch [your-build-variant]

4. [your-build-variant] could be sdk_phone_x86_64, sdk_phone_x86, sdk-eng if you're building for phone/generic build and it could be aosp_car_x86_64, aosp_car_x86 if you're building for generic car. Always prefer to use x86_64 bit variant.

5. $make sdk

6. After $make sdk, the sdk wil be generated at /root_aosp/out/host/linux-x86/sdk/ on Linux and /root_aosp/out/host/darwin-x86/sdk/ on Mac OS

7. If you want build SDK for Windows OS, then run $make win_sdk instead of $make sdk. The generated sdk will be at /root_aosp/out/host/windows-x86/sdk/

8. Go to /aosp_root/out/target/common/obj/JAVA_LIBRARIES/framework_intermediates/ and copy classes-header.jar file

9. Copy your SDK's android.jar file under /Sdk/platforms/android-[xx]/ and copy android.jar file

10. Create a new folder and extract android.jar contents.

11. Extract classes-header.jar on the same new folder where we extracted android.jar file. Replace all the files if the file name already exists.

12. Make a new android.jar file by "/new_folder$jar cfM ../android.jar ." inside the new folder.

13. The above command will generate a new android.jar file outside the new folder.

14. Copy the generated android.jar file and replace it in your SDK directory /Sdk/platforms/android-[xx]/

II. Importing the library into Android Studio

1. Copy "classes.jar" from /aosp_root/out/target/common/obj/JAVA_LIBRARIES/framework_intermediates/

2. Paste the library under /root_project_dir/app/system_libs/ as "framework.jar"

3. Add it as a dependency in your app module [build.gradle] file - implementation files('system_libs/framework.jar')

4. Configure your project's [build.gradle] file with the below lines of code for java compilation.

5. Configure your module's [build.gradle] file with the below code that brings the framework.jar file as a top library and moves SDK to the last. So that it points to framework.jar APIs instead of SDK.

III. AOSP Prebuilt SDK

1. Prebuilt SDK in AOSP can be found at /aosp_root/prebuilts/sdk/current/system/android.jar. But access to Hidden APIs and Framework AIDL services were limited.

IV. Building Custom Emulator

1. Download the aosp repo

2. $source build/envsetup.sh

3. $lunch [your-build-variant]

4. [your-build-variant] could be sdk_phone_x86_64, sdk_phone_x86 if you're building for phone/generic build and it could be aosp_car_x86_64, aosp_car_x86 if you're building for generic car. Always prefer to use x86_64 bit variant.

5. $make -j16

6. $emulator

7. Your emulator will be launched automatically from /aosp_root/prebuilts/android-emulator/linux-x86_64/ if you are building on Linux and /aosp_root/prebuilts/android-emulator/darwin-x86_64/ if you are building on Mac OS based on the system images generated on /aosp_root/out/target/product/[your-build-variant]

V. Packaging and sharing Custom Emulator

1. Download the aosp repo

2. $source build/envsetup.sh

3. $lunch [your-build-variant]

4. [your-build-variant] could be sdk_phone_x86_64, sdk_phone_x86 if you're building for phone/generic build and it could be aosp_car_x86_64, aosp_car_x86 if you're building for generic car. Always prefer to use x86_64 bit variant.

5. $make -j16 sdk sdk_repo

6. This will generate two files under /aosp-root/out/host/linux-x86/sdk/[your-build-variant]/ or in /aosp-root/out/host/darwin-x86/sdk/[your-build-variant]/ if you are building on Mac OS

   sdk-repo-linux-system-images-eng.[username].zip

   repo-sys-img.xml

7. sdk-repo-linux-system-images-eng.[username].zip is the actual emulator system image that you normally see in /Sdk/system-images/ in an unzipped format.

8. If you want to share this emulator among many developers, then host the both files repo-sys-img.xml and sdk-repo-linux-system-images-eng.[username].zip in an online environment.

9. Make sure that you change the URL in repo-sys-img.xml of sdk-repo-linux-system-images-eng.[username].zip hosted file.

10. After hosting the file, Go to Android Studio > Tools > SDK Manager > SDK Update Sites > Tap Add Button and provide name and URL of repo-sys-img.xml hosted file and then, tap ok.

11. This adds your custom emulator system image to the System Images page of the SDK (/Sdk/system-images/).

12. Then go to Android Studio > Tools > AVD Manager > and create a new virtual device with your custom emulator system image.

13. If you want to use the custom emulator locally, then just unzip the file sdk-repo-linux-system-images-eng.[username].zip in /Sdk/system-images/ directory and should be in the below folder structure.

14. The folder structure for the system images in /Sdk/system-images/ should be,

    android-[xx]/[TagId]/x86_64/ - this is for x86_64 version

    android-[xx]/[tagId]/x86/ - this is for x86 version

    [xx] is the android API level (ex. 29)

    [TagId] is the emulator variant. It could be android-automotive if you are building for Car and could be default if you are building for others.

15. And then, edit source.properties and package.xml:

    Abi - should be x86_64 or x86

    TagId - is the emulator variant. It could be android-automotive if you are building for Car and could be default if you are building for others.

    TagDisplay or display - Give the name as you like. ex) "Android Automotive System Image" for Car Variant.

    display-name - could be "Intel x86 Atom_64 System Image" for "x86_64" or "Intel x86 Atom System Image" for "x86"

16. If you can't find package.xml and was not auto-generating then just copy it from other emulator system images from your SDK.

17. Then go to Android Studio > Tools > AVD Manager > and create a new virtual device with your custom emulator system image.

VI. Error Handling when building SDK / Emulator

1. The Product Variants sdk_phone_x86_64, aosp_x86_64 is in /aosp_root/build/target/product/ directory.

2. The Product Variant aosp_car_x86_64 is in /aosp_root/device/generic/car/ directory.

3. Building sdk_phone_x86_64 for phone variant will not produce any error and will produce SDK and Emulator System Image without any problem.

4. But for building aosp_car_x86_64 for Car will produce some development tools missing errors.

5. The cause of the error for the Car variant aosp_car_x86_64 is that it does not include host tools and libs unlike sdk_phone_x86_64 has host and tools defined.

   Copy the host tools and libs from sdk_phone_x86_64 to aosp_car_x86_64

   # Define the host tools and libs that are parts of the SDK
   -include sdk/build/product_sdk.mk
   -include development/build/product_sdk.mk

6. Step 5 was not even tried before and it is still a suggestion only.

7. The errors that I experienced are:

   deployagent.jar was not found in /aosp_root/out/target/product/[your-build-variant]/system/framework/ directory.

   deployagent was not found in /aosp_root/out/target/product/[your-build-variant]/system/bin/ directory.

   These errors are produced by /aosp_root/development/build/sdk.atree.

8. To solve this problem, first build for aosp_car_x86_64-userdebug, then build aosp_car_x86 variant and then build aosp_car_x86_64.

9. After all, copy deployagent.jar from /aosp_root/out/target/product/aosp_car_x86/system/framework/ and deployagent from /aosp_root/out/target/product/aosp_car_x86/system/bin/ to /aosp_root/out/target/product/aosp_car_x86_64/system/framework/ and /aosp_root/out/target/product/aosp_car_x86_64/system/bin/

10. Then try $make sdk sdk_repo again and sdk and emulator system images will be produced.

11. The whole idea is to solve the error is, copy all the missing packaging development tools from other product variants from aosp_root/out/target/product/[other-build-variant]/ to your aosp_root/out/target/product/[your-build-variant]/

VII. Preparing Writable Emulator

1. After building custom emulator from above steps, create a new Android Virtual Device (AVD) from AVD Manager, Android Studio.

2. Try remounting the emulator by:

    $ adb root
   
    $ adb disable-verity
   
    $ adb reboot
   
    $ adb root
   
    $ adb remount
   

3. If the remount was unsuccessful, then try launching emulator as a writable system.

   i) List the available AVDs:

    Sdk/emulator$ ./emulator -list-avds
   
    [avd-name-1]
    [avd-name-2]
   

   ii) Launch the writable emulator system:

    Sdk/emulator$ ./emulator -writable-system -avd [any-avd-name]
   

   iii) Try remounting the emulator:

    $ adb root
   
    $ adb remount
   

4. With the remounted (or) writable emulator system, you can directly push your system applications into system.

    $ adb push [app-name].apk /system/product/priv-app/[app-name]/
   
    $ adb reboot
   

VIII. Signing KeyStore

1. Signing the keystore (.jks) with the android platform certificates requires the certificate (platform.pem) and the key (platform.x509.pk8) that can be found in the below path of AOSP

    aosp_root/build/target/product/security/
   

2. Generate a new or use the existing keystore file from Android Studio.

3. Download the shell script keytool-importkeypair from https://github.com/getfatday/keytool-importkeypair

4. Run the script. It will import the key/certificate pair with the keystore file. Then you can directly use keystore file to sign the apk with platform certificate.

    $ keytool-importkeypair -k platform.keystore -p platform -pk8 platform.pk8 -cert platform.x509.pem -alias platform
   

IX. Allowing Privileged App Permission

1. Permissions for all the system privileged apps can be found in:

    aosp_root/frameworks/base/etc/permissions
   

2. Push your permission file [package_name].xml to the below directory:

    $ adb push [package_name].xml /system/product/etc/permissions/
   

3. On Android 10 and above, the device won't boot without these permissions.

References

1. Building Custom SDK by kwagjj > https://kwagjj.wordpress.com/2017/10/27/building-custom-android-sdk-from-aosp-and-adding-it-to-android-studio/

2. Standalone SDK > https://grapheneos.org/build#standalone-sdk

3. Importing Framework Library into Android Studio by kwagjj > https://kwagjj.wordpress.com/2017/08/10/using-framework-jar-in-android-studio/

4. Building JAR File > https://www.codejava.net/java-core/tools/using-jar-command-examples

5. Restrictions on Non-SDK interfaces > https://developer.android.com/distribute/best-practices/develop/restrictions-non-sdk-interfaces

6. Embedded Android by Karim Yaghmour > https://www.oreilly.com/library/view/embedded-android/9781449327958/ch04.html

7. Building and Sharing AVD > https://source.android.com/setup/create/avd

8. Building Android Car Emulator > https://www.embien.com/blog/building-android-car-emulator/

9. Creating Acloud Instance > https://source.android.com/setup/start

10. Cuttlefish Virtual Android Devices > https://source.android.com/setup/create/cuttlefish

11. Run apps on the Android Emulator > https://developer.android.com/studio/run/emulator

12. Signing build for release > https://source.android.com/devices/tech/ota/sign_builds

13. Signing your app > https://developer.android.com/studio/publish/app-signing

14. Allowing Priv-app Permissions > https://source.android.com/devices/tech/config/perms-whitelist

15. Carbon - Creates images for your source code > https://carbon.now.sh/

License

Copyright 2021 LiteKite Startup

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS, 
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.