NDK Development:
====
This document describes how one can modify the NDK and generate
new experimental release packages for it.
I. Getting the sources:
---
The sources live under the "ndk" and "development/ndk" directories in
the Android source tree:
- "ndk" contains the main build scripts and documentation
- "development/ndk" contains platform-specific headers and samples
If you have downloaded the full Android source tree through the "repo"
tool, you can start directly there. Otherwise, you can just get these
two repositories with the following:
mkdir workdir
cd workdir
git clone https://android.googlesource.com/platform/ndk.git ndk
git clone https://android.googlesource.com/platform/development.git development
export NDK=`pwd`/ndk
II. Building the platforms tree:
---
You need to do that once if you want to use the content of $NDK to build
samples, tests or anything else:
$NDK/build/tools/gen-platforms.sh
What the script does is populate the $NDK/platforms and $NDK/samples
directories from the content of development/ndk.
What is under development/ndk is segregated by API level. This makes it
easier to add a new platform to the tree, but is not well-suited to building
stuff. The gen-platforms.sh script will gather all files appropriately
and place the result inside $NDK/platforms and $NDK/samples.
Note: These directories are listed by $NDK/.gitignore, so they won't appear
on your git status. You can remove them if you want by running:
$NDK/build/tools/dev-cleanup.sh
which also removes all intermediate files and directories from $NDK.
III. Prebuilt binaries:
---
The NDK requires several prebuilt binary executables to work properly, these
include the following:
- toolchain binaries for the cross-compiler and associated tools
- gdbserver binaries required for native debugging
These are not provided in the NDK's git repositories. However, there are
several ways to get them:
### 1. From a previous NDK release package:
By far the easiest thing to do is to copy the binaries from a previous
NDK installation. You can do that with a command like the following one:
cp -r $PREVIOUS_NDK/toolchains/* $NDK/toolchains/
NOTE: The binaries are listed in $NDK/.gitignore and will not appear
in your git status.
### 2. Download and rebuild directly from the internet:
IMPORTANT: This is *very* long.
The NDK comes with several scripts that can be used to rebuild the
binaries from scratch, after downloading their sources from
android.googlesource.com.
There are several ways to do that, the most naive one, which will
always work but will be *very* long (expect a few hours on a typical
dual-core machine) is to do the following:
$NDK/build/tools/rebuild-all-prebuilt.sh
This will perform all the steps required to rebuild the binaries,
which include:
- downloading the sources from android.googlesource.com
- patching them with appropriate changes, if needed
- rebuilding everything from scratch
- copying the generated binaries to the proper location under $NDK
You will need about 30G of free space in your /tmp directory to be
able to do that, and *plenty* of free time.
IMPORTANT: If you plan to generate NDK release packages, even
experimental ones, we strongly suggest you to use the individual
steps described in 3/ below.
IMPORTANT:
Since NDK r5, Windows binaries can be built on Linux by using the
--mingw option, which requires that you have the "mingw32" package
installed on your system. For example:
$NDK/build/tools/rebuild-all-prebuilt.sh --mingw
We do not officially support building these binaries directly on
Windows (either through Cygwin or MSys) anymore, due to the vast
number of problems these environments create when trying to do so.
### 3. Download, rebuild, package, install in separate steps:
If you plan to generate your own NDK release packages, it is better
to rebuild your binaries using separate steps, as in:
- Download the sources from the Internet, patch them, then
package the result in a simple tarball.
- For every target system (linux-x86, darwin-x86 and windows),
rebuild the binaries from the same source tarball.
- Package and collect all prebuilt binaries into a single
directory that will be used when packaging NDK releases.
Here are more details on how to do that:
#### 3.a/ Download + patching + packaging sources:
Use the following command to download, patch and package the
sources:
$NDK/build/tools/download-toolchain-sources.sh --package
This will create a large tarball containing all sources ready to be
used by the following step. The generated file path will be dumped at
the script when it completes its operation and should be something
like:
/tmp/android-ndk-toolchain-<date>.tar.bz2
Note that if you don't use the --package option, you will need to
provide the name of a directory where the patched sources will be
copied instead, as in:
$NDK/build/tools/download-toolchain-sources.sh <target-src-dir>
#### 3.b/ Build the binaries:
Use the following command to rebuild the binaries from the source
tarball that was created in the previous section with the --package
option:
$NDK/build/tools/rebuild-all-prebuilt.sh --toolchain-pkg=<file>
Where <file> points to the package generated by the
download-toolchain-sources.sh script.
In the case where you downloaded the sources to a directory instead,
use the --toolchain-src-dir option instead, as with:
$NDK/build/tools/rebuild-all-prebuilt.sh --toolchain-src-dir=<path>
This will rebuild all the prebuilt binaries for your host platforms
and place them in a directory named:
/tmp/ndk-prebuilt/prebuilt-<date>/
These binary packages include the following:
- host-specific toolchain binaries. e.g.
`arm-linux-androideabi-4.6-linux-x86.tar.bz2`.
- toolchain specific device binaries, e.g.
`arm-gdbserver.tar.bz2`.
IMPORTANT:
To generate Windows binaries on Windows, install the "mingw32"
package on your system, then use the --mingw option, as in:
$NDK/build/tools/rebuild-all-prebuilt.sh --mingw --toolchain-pkg=<file>
Note that device-specific binaries (e.g. gdbserver) cannot be
rebuilt with this option.
#### 3.c/ Copy the binaries to your NDK tree:
Simply go to your NDK tree, and unpack the binary tarballs in place,
for example:
cd $NDK
tar xjf <path>/*.tar.bz2
Where <path> is a directory containing all the tarballs (e.g. it
could be simply /tmp/ndk-prebuilt/prebuilt-<date>)
This will put the corresponding files at the correct location.
#### 3.c/
It is a good idea to save the generated toolchain binaries into
an archive. To do that, use the --package option, as in:
$NDK/build/tools/rebuild-all-prebuilt.sh --package
This will generate a package file containing all the prebuilts, that
can be unpacked directly into your $NDK directory. The package name is
printed at the end, e.g."android-ndk-prebuild-<date>-<system>.tar.bz2".
Where <date> is the current date, and <system> is your system name.
Then, to unpack:
cd $NDK
tar xjf /tmp/android-ndk-prebuilt-<date>-<system>.tar.bz2
The generated package can easily be shared with other people.
IV. Generate new package releases:
---
You can generate new experimental NDK release packages once you're satisfied
with your changes, in order to share them with other people. There are two
ways to do that:
### 1. Using the 'make-release.sh' script:
The simplest, and also the slowest way, to generate a new NDK release
is to invoke this script, with:
$NDK/build/tools/make-release.sh
NOTE: THIS WILL BE VERY VERY LONG. The script will do all the steps
described in section III *from* scratch, and this can take several
hours on a dual-core machine.
You should only use it in case of desperation, or if you don't want
to deal with all the details exposed in section III or below.
### 2. Using a previous NDK release package:
This is the second simplest way to generate a new package, and it will
be extremely quick because it will pick the prebuilt binaries directly
from the previous package.
Do the following:
cd $NDK
build/tools/package-release.sh --prebuilt-ndk=<file>
Where <file> points to a previous NDK package (i.e. archive file).
NOTE: This method can only be used to generate a single release package
for the current host system.
### 3. Using prebuilt tarballs:
If you have generated prebuilt binary tarballs with the steps described
in section III.3 above, you can use these to generate release packages
as well.
Assuming that you have collected prebuilt tarballs for all three supported
host systems (i.e. linux-x86, darwin-x86 and windows) under a directory,
do the following:
cd $NDK
build/tools/package-release.sh --prebuilt-dir=<path>
The generated NDK package release will have a name that looks like:
/tmp/ndk-release/android-ndk-<release>-<system>.zip
Where <release> is by default the current date in ISO format
(e.g. 20100915), and <system> corresponds to the host system where the
NDK release is supposed to run.
The script 'package-release.sh' provides a few additional options:
--release=<name> Change the name of the release
--systems=<list> Change the list of host systems to package for
--platforms=<list> List of API levels to package in the NDK
--out-dir=<path> Specify a different output directory for the
final packages (instead of /tmp/ndk-release)
Use --help to list them all.
====
This document describes how one can modify the NDK and generate
new experimental release packages for it.
I. Getting the sources:
---
The sources live under the "ndk" and "development/ndk" directories in
the Android source tree:
- "ndk" contains the main build scripts and documentation
- "development/ndk" contains platform-specific headers and samples
If you have downloaded the full Android source tree through the "repo"
tool, you can start directly there. Otherwise, you can just get these
two repositories with the following:
mkdir workdir
cd workdir
git clone https://android.googlesource.com/platform/ndk.git ndk
git clone https://android.googlesource.com/platform/development.git development
export NDK=`pwd`/ndk
II. Building the platforms tree:
---
You need to do that once if you want to use the content of $NDK to build
samples, tests or anything else:
$NDK/build/tools/gen-platforms.sh
What the script does is populate the $NDK/platforms and $NDK/samples
directories from the content of development/ndk.
What is under development/ndk is segregated by API level. This makes it
easier to add a new platform to the tree, but is not well-suited to building
stuff. The gen-platforms.sh script will gather all files appropriately
and place the result inside $NDK/platforms and $NDK/samples.
Note: These directories are listed by $NDK/.gitignore, so they won't appear
on your git status. You can remove them if you want by running:
$NDK/build/tools/dev-cleanup.sh
which also removes all intermediate files and directories from $NDK.
III. Prebuilt binaries:
---
The NDK requires several prebuilt binary executables to work properly, these
include the following:
- toolchain binaries for the cross-compiler and associated tools
- gdbserver binaries required for native debugging
These are not provided in the NDK's git repositories. However, there are
several ways to get them:
### 1. From a previous NDK release package:
By far the easiest thing to do is to copy the binaries from a previous
NDK installation. You can do that with a command like the following one:
cp -r $PREVIOUS_NDK/toolchains/* $NDK/toolchains/
NOTE: The binaries are listed in $NDK/.gitignore and will not appear
in your git status.
### 2. Download and rebuild directly from the internet:
IMPORTANT: This is *very* long.
The NDK comes with several scripts that can be used to rebuild the
binaries from scratch, after downloading their sources from
android.googlesource.com.
There are several ways to do that, the most naive one, which will
always work but will be *very* long (expect a few hours on a typical
dual-core machine) is to do the following:
$NDK/build/tools/rebuild-all-prebuilt.sh
This will perform all the steps required to rebuild the binaries,
which include:
- downloading the sources from android.googlesource.com
- patching them with appropriate changes, if needed
- rebuilding everything from scratch
- copying the generated binaries to the proper location under $NDK
You will need about 30G of free space in your /tmp directory to be
able to do that, and *plenty* of free time.
IMPORTANT: If you plan to generate NDK release packages, even
experimental ones, we strongly suggest you to use the individual
steps described in 3/ below.
IMPORTANT:
Since NDK r5, Windows binaries can be built on Linux by using the
--mingw option, which requires that you have the "mingw32" package
installed on your system. For example:
$NDK/build/tools/rebuild-all-prebuilt.sh --mingw
We do not officially support building these binaries directly on
Windows (either through Cygwin or MSys) anymore, due to the vast
number of problems these environments create when trying to do so.
### 3. Download, rebuild, package, install in separate steps:
If you plan to generate your own NDK release packages, it is better
to rebuild your binaries using separate steps, as in:
- Download the sources from the Internet, patch them, then
package the result in a simple tarball.
- For every target system (linux-x86, darwin-x86 and windows),
rebuild the binaries from the same source tarball.
- Package and collect all prebuilt binaries into a single
directory that will be used when packaging NDK releases.
Here are more details on how to do that:
#### 3.a/ Download + patching + packaging sources:
Use the following command to download, patch and package the
sources:
$NDK/build/tools/download-toolchain-sources.sh --package
This will create a large tarball containing all sources ready to be
used by the following step. The generated file path will be dumped at
the script when it completes its operation and should be something
like:
/tmp/android-ndk-toolchain-<date>.tar.bz2
Note that if you don't use the --package option, you will need to
provide the name of a directory where the patched sources will be
copied instead, as in:
$NDK/build/tools/download-toolchain-sources.sh <target-src-dir>
#### 3.b/ Build the binaries:
Use the following command to rebuild the binaries from the source
tarball that was created in the previous section with the --package
option:
$NDK/build/tools/rebuild-all-prebuilt.sh --toolchain-pkg=<file>
Where <file> points to the package generated by the
download-toolchain-sources.sh script.
In the case where you downloaded the sources to a directory instead,
use the --toolchain-src-dir option instead, as with:
$NDK/build/tools/rebuild-all-prebuilt.sh --toolchain-src-dir=<path>
This will rebuild all the prebuilt binaries for your host platforms
and place them in a directory named:
/tmp/ndk-prebuilt/prebuilt-<date>/
These binary packages include the following:
- host-specific toolchain binaries. e.g.
`arm-linux-androideabi-4.6-linux-x86.tar.bz2`.
- toolchain specific device binaries, e.g.
`arm-gdbserver.tar.bz2`.
IMPORTANT:
To generate Windows binaries on Windows, install the "mingw32"
package on your system, then use the --mingw option, as in:
$NDK/build/tools/rebuild-all-prebuilt.sh --mingw --toolchain-pkg=<file>
Note that device-specific binaries (e.g. gdbserver) cannot be
rebuilt with this option.
#### 3.c/ Copy the binaries to your NDK tree:
Simply go to your NDK tree, and unpack the binary tarballs in place,
for example:
cd $NDK
tar xjf <path>/*.tar.bz2
Where <path> is a directory containing all the tarballs (e.g. it
could be simply /tmp/ndk-prebuilt/prebuilt-<date>)
This will put the corresponding files at the correct location.
#### 3.c/
It is a good idea to save the generated toolchain binaries into
an archive. To do that, use the --package option, as in:
$NDK/build/tools/rebuild-all-prebuilt.sh --package
This will generate a package file containing all the prebuilts, that
can be unpacked directly into your $NDK directory. The package name is
printed at the end, e.g."android-ndk-prebuild-<date>-<system>.tar.bz2".
Where <date> is the current date, and <system> is your system name.
Then, to unpack:
cd $NDK
tar xjf /tmp/android-ndk-prebuilt-<date>-<system>.tar.bz2
The generated package can easily be shared with other people.
IV. Generate new package releases:
---
You can generate new experimental NDK release packages once you're satisfied
with your changes, in order to share them with other people. There are two
ways to do that:
### 1. Using the 'make-release.sh' script:
The simplest, and also the slowest way, to generate a new NDK release
is to invoke this script, with:
$NDK/build/tools/make-release.sh
NOTE: THIS WILL BE VERY VERY LONG. The script will do all the steps
described in section III *from* scratch, and this can take several
hours on a dual-core machine.
You should only use it in case of desperation, or if you don't want
to deal with all the details exposed in section III or below.
### 2. Using a previous NDK release package:
This is the second simplest way to generate a new package, and it will
be extremely quick because it will pick the prebuilt binaries directly
from the previous package.
Do the following:
cd $NDK
build/tools/package-release.sh --prebuilt-ndk=<file>
Where <file> points to a previous NDK package (i.e. archive file).
NOTE: This method can only be used to generate a single release package
for the current host system.
### 3. Using prebuilt tarballs:
If you have generated prebuilt binary tarballs with the steps described
in section III.3 above, you can use these to generate release packages
as well.
Assuming that you have collected prebuilt tarballs for all three supported
host systems (i.e. linux-x86, darwin-x86 and windows) under a directory,
do the following:
cd $NDK
build/tools/package-release.sh --prebuilt-dir=<path>
The generated NDK package release will have a name that looks like:
/tmp/ndk-release/android-ndk-<release>-<system>.zip
Where <release> is by default the current date in ISO format
(e.g. 20100915), and <system> corresponds to the host system where the
NDK release is supposed to run.
The script 'package-release.sh' provides a few additional options:
--release=<name> Change the name of the release
--systems=<list> Change the list of host systems to package for
--platforms=<list> List of API levels to package in the NDK
--out-dir=<path> Specify a different output directory for the
final packages (instead of /tmp/ndk-release)
Use --help to list them all.
