Skip to content

Commit

Permalink
Initial Commit
Browse files Browse the repository at this point in the history
  • Loading branch information
darajnish committed Sep 19, 2021
1 parent 0745d11 commit 8d6fc4f
Show file tree
Hide file tree
Showing 7 changed files with 342 additions and 0 deletions.
97 changes: 97 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,97 @@
# youtube-dl android builds

Scripts to build binary packages to run youtube-dl (with FFmpeg) on Android, with Python and just the required dependencies. These scripts use the [termux build system](https://github.com/termux/termux-packages) to build the entire required packages and then the unnecessary packages and the files are trimmed off to get just the required minimal build. The list of required packages to run `youtube-dl` are in the file `required-packages.txt`, which is used by the script to trim off unnecessary packages for our build.

Note: These packages work only for Android 5 (Lollipop) and above.

## Running

#### Requirements

- [cURL](https://curl.haxx.se/)
- [Git](https://git-scm.com/)
- [Docker](https://www.docker.com/)
- A GNU/Linux System

#### Steps

Once, the above requirements are met, make sure you've an internet connection and the docker service is running in your system. Remove any previous `termux/package-builder` container, if you have already in your docker (if you were using the termux build system previously).

Other than all the above, you're good to go. Just start the build by running the script `autobuild.sh`, the script will do the rest and create a build for each of the [Android ABIs](https://developer.android.com/ndk/guides/abis) and place the respective packages in the same directory.

```bash
./autobuild.sh
```

However, if you don't want to use `autobuild.sh` or you just want to build just for a single architecture, then it can be achieved using the following steps.

First, clone the termux build system repository
```bash
git clone "https://github.com/termux/termux-packages"
```
Download the latest youtube-dl script
```bash
curl -L "https://yt-dl.org/downloads/latest/youtube-dl" -o "youtube-dl"
```

Copy the required scripts into `termux-packages`
```bash
cp -v build-youtubedl.sh required-packages.txt youtube-dl.sh youtube-dl termux-packages
```

Make sure the docker service is running and switch to termux docker container
```bash
cd termux-packages && ./scripts/run-docker.sh
```

Create a build directory and change into it
```bash
mkdir -v build
```

Run `build-youtubedl.sh` to start building the entire required packages, change the value of ARCH variable to the android architecture (`x86_64`, `i686`, `arm`, `aarch64`) for which you want to build the packages. For, [`armeabi-v7a`](https://developer.android.com/ndk/guides/abis#v7a) ARCH=arm, and for [`arm64-v8a`](https://developer.android.com/ndk/guides/abis#arm64-v8a) ARCH=aarch64, and for [`x86`](https://developer.android.com/ndk/guides/abis#x86) ARCH=i686 and for [`x86_64`](https://developer.android.com/ndk/guides/abis#86-64) ARCH=x86_64
```bash
ARCH=arm ../build-youtubedl.sh build
```
Similarly, extract the required packages
```bash
ARCH=arm ../build-youtubedl.sh extract
```
And the same way, build the packages
```bash
ARCH=arm ../build-youtubedl.sh package
```
If you need to clean the current `build`directory and the build environment
```bash
ARCH=arm ../build-youtubedl.sh clean
```

Exit from the termux build system
```bash
exit
```
Finally, you have a compressed tarball package for running youtube-dl on android, you may copy it out to the main directory.
```bash
cp -v youtubedl-arm.tar.xz ..
```


Extracting the tarball and running youtube-dl on an android system is easy and can be done with the help of busybox tar applet.
Assuming, we've copied busybox and the package into our android system at `/data/local/tmp` we can run youtube-dl as follows,
```bash
cd /data/local/tmp
chmod 755 busybox
mkdir test
cd test
../busybox tar -xf ../youtubedl-arm.tar.xz
./usr/bin/youtube-dl "https://www.youtube.com/watch?v=aqz-KE-bpKQ"
```

## Credits

- [youtubedl-android guide](https://github.com/yausername/youtubedl-android/blob/master/BUILD_FFMPEG.md) for giving an overall insight of how it's done
- [Termux Wiki](https://wiki.termux.com/wiki/Building_packages) for providing a detailed guide on how to build packages using the termux build system

## License

The scripts are under under [GPLv3](LICENSE)
29 changes: 29 additions & 0 deletions autobuild.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
#!/bin/bash

DEPS=(curl git docker)

echo "=> Checking for required programs.."
for prog in ${DEPS[@]} ; do
if ! ${prog} --version &>/dev/null ; then
echo "${prog} is not installed! Install ${prog} first!"
exit 2
fi
done

if [[ ! -f "youtube-dl" ]]; then
echo "=> Downloading the latest youtube-dl.."
curl -L "https://yt-dl.org/downloads/latest/youtube-dl" -o "youtube-dl"
fi

echo "=> Preparing the build environment.."
git clone "https://github.com/termux/termux-packages"
cp -v youtubedl-builder.sh build-youtubedl.sh required-packages.txt youtube-dl.sh youtube-dl termux-packages

echo "=> Building starts.."
cd termux-packages
./scripts/run-docker.sh ./youtubedl-builder.sh

echo "=> Copying the built packages.."
cp -v youtubedl-x86_64.tar.xz youtubedl-i686.tar.xz youtubedl-arm.tar.xz youtubedl-aarch64.tar.xz ..

echo "=> Build Finished!"
135 changes: 135 additions & 0 deletions build-youtubedl.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,135 @@
#!/bin/bash

if [[ -z "$ARCH" ]]; then
export ARCH=arm #x86_64, i686, arm, aarch64
fi

PKGFILE="../required-packages.txt"
YOUTUBEDL="../youtube-dl"
YOUTUBEDLSH="../youtube-dl.sh"
PKGDIR="../ffmpeg-packges-${ARCH}"
PREFIX="/data/data/sng.darajnish.ytube_dl/files/"


rm_obvious () {
rm -rvf data/data/com.termux/files/usr/include/
rm -rvf data/data/com.termux/files/usr/lib/pkgconfig/
rm -rvf data/data/com.termux/files/usr/lib/cmake/
rm -rvf data/data/com.termux/files/usr/lib/glib-2.0
rm -rvf data/data/com.termux/files/usr/bin/
rm -rvf data/data/com.termux/files/usr/share
rm -rvf data/data/com.termux/files/usr/etc/fonts/conf.d/
}

build() {
export TERMUX_ARCH="${ARCH}"
export TERMUX_PREFIX="${PREFIX}/usr"
export TERMUX_ANDROID_HOME="${PREFIX}/home"

_CDIR="$(pwd)"
cd ..
./clean.sh
./build-package.sh librav1e
./build-package.sh ffmpeg
./build-package.sh python
cd "${_CDIR}"
mv -v ../output "${PKGDIR}"
}

extract_clean() {
if [[ ! -f "$PKGFILE" ]]; then
echo "Error: pkgfile.txt not found in the current directory!"
exit 2
fi

if [[ ! -d "$PKGDIR" ]]; then
echo "Error: The packages directory ${PKGDIR} doesn't exist!"
exit 2
fi

if [[ $(ls -l | head -n1) != 'total 0' ]]; then
echo "Error: The directory must be empty!"
exit 2
fi

while read line; do
if [[ $line == 'ffmpeg'* ]]; then
dpkg-deb -xv "${PKGDIR}/${line}_${ARCH}.deb" .
rm -rvf data/data/com.termux/files/usr/include/
rm -rvf data/data/com.termux/files/usr/share/doc/
rm -rvf data/data/com.termux/files/usr/share/man/
rm -rvf data/data/com.termux/files/usr/lib/pkgconfig/
rm -rvf data/data/com.termux/files/usr/share/ffmpeg/examples/
elif [[ $line == 'python'* ]]; then
dpkg-deb -xv "${PKGDIR}/${line}_${ARCH}.deb" .
rm -rvf data/data/com.termux/files/usr/lib/pkgconfig/
rm -rvf data/data/com.termux/files/usr/share/
rm -rvf data/data/com.termux/files/usr/include/
rm -rvf data/data/com.termux/files/usr/bin/2to3*
rm -rvf data/data/com.termux/files/usr/bin/pydoc*
rm -rvf data/data/com.termux/files/usr/bin/python*-config
rm -rvf data/data/com.termux/files/usr/lib/python3.9/unittest
rm -rvf data/data/com.termux/files/usr/lib/python3.9/site-packages/xcbgen/
rm -rvf data/data/com.termux/files/usr/lib/python3.9/ensurepip/
rm -rvf data/data/com.termux/files/usr/lib/python3.9/lib2to3/
rm -rvf data/data/com.termux/files/usr/lib/python3.9/distutils/
rm -rvf data/data/com.termux/files/usr/lib/python3.9/pydoc_data/
rm -rvf data/data/com.termux/files/usr/lib/python3.9/pydoc.py
elif [[ $line == 'ca-certificates'* ]]; then
dpkg-deb -xv "${PKGDIR}/${line}_all.deb" .
rm_obvious
else
dpkg-deb -xv "${PKGDIR}/${line}_${ARCH}.deb" .
rm_obvious
fi
done < "${PKGFILE}"

cp -v /home/builder/lib/android-ndk/toolchains/llvm/prebuilt/linux-x86_64/lib64/clang/*/lib/linux/${ARCH}/libomp.so data/data/com.termux/files/usr/lib/

mkdir -v "data/data/com.termux/files/usr/lib/youtube-dl"
cp -v "${YOUTUBEDL}" "data/data/com.termux/files/usr/lib/youtube-dl"

cp -v "${YOUTUBEDLSH}" "data/data/com.termux/files/usr/bin/youtube-dl"
chmod +x "data/data/com.termux/files/usr/bin/youtube-dl"
}

package() {
#export XZ_OPT=-e9
cd "data/data/com.termux/files"
XZ_OPT=-e9 tar -acvf "../../../../../youtubedl-${ARCH}.tar.xz" *
cd "../../../../"
}

clean() {
rm -rvf *
../clean.sh
}

show_help() {
cat << "EOF"
Usage: build-youtubedl.sh [build | extract | package]
build Execute the build task
extract Execute extraction and cleaning
package Execute the packaging task
clean Cleans off the current directory
EOF
}


if [[ "$1" == "build" ]]; then
build
elif [[ "$1" == "extract" ]]; then
extract_clean
elif [[ "$1" == "package" ]]; then
package
elif [[ "$1" == "clean" ]]; then
clean
elif [[ "$1" == "--help" ]]; then
show_help
else
show_help
fi

45 changes: 45 additions & 0 deletions required-packages.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
ca-certificates_20210603
bzip2_1.0.8-6
libffi_3.4.2
libandroid-support_28-2
openssl_1.1.1l
libcrypt_0.2-3
libgnutls_3.6.16
libunistring_0.9.10-4
libidn2_2.3.2
libnettle_3.7.3
readline_8.1.1
zlib_1.2.11-5
libaom_3.1.2-1
libass_0.15.2
fribidi_1.0.10
glib_2.66.1-1
libiconv_1.16-3
pcre_8.45
libpng_1.6.37-3
libbz2_1.0.8-6
libc++_21d-1
libgraphite_1.3.14
freetype_2.11.0
libuuid_1.0.3-4
libxml2_2.9.12
fontconfig_2.13.1-6
libdav1d_0.9.2
librav1e_0.4.1-2
libsoxr_0.1.3-2
libx264_1\:0.161.3049
libx265_3.3
xvidcore_1.3.7
libvorbis_1.3.7
libmp3lame_3.100-3
libopus_1.3.1-4
libvpx_1.10.0
libgmp_6.2.1
libandroid-glob_0.6-2
liblzma_5.2.5-1
libvidstab_1.1.0
libwebp_1.2.1
harfbuzz_2.9.1
libogg_1.3.5
python_3.9.7
ffmpeg_4.4-2
Binary file added youtube-dl
Binary file not shown.
22 changes: 22 additions & 0 deletions youtube-dl.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
#!/bin/sh

if [[ -z "$YOUTUBEDL_ROOT" ]]; then
export YOUTUBEDL_ROOT="$(realpath $(dirname ${0})/..)"
fi

_CACHEDIR="${YOUTUBEDL_ROOT}/cache"
_LIBDIR="${YOUTUBEDL_ROOT}/lib"
_BINDIR="${YOUTUBEDL_ROOT}/bin"
_SSLFILE="${YOUTUBEDL_ROOT}/etc/tls/cert.pem"
_YOUTUBEDL="${YOUTUBEDL_ROOT}/lib/youtube-dl/youtube-dl"

export PATH="$PATH:${_BINDIR}"
export LD_LIBRARY_PATH="${_LIBDIR}"
export PYTHONHOME="${YOUTUBEDL_ROOT}"
export SSL_CERT_FILE="${_SSLFILE}"

if [[ ! -d "${_CACHEDIR}" ]]; then
mkdir "${_CACHEDIR}"
fi

python "${_YOUTUBEDL}" --cache-dir "${_CACHEDIR}" ${@}
14 changes: 14 additions & 0 deletions youtubedl-builder.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
#!/bin/bash

mkdir -pv build
cd build

for arch in x86_64 i686 arm aarch64 ; do
echo "=> Building for ${arch}"

export ARCH="${arch}"
../build-youtubedl.sh build
../build-youtubedl.sh extract
../build-youtubedl.sh package
../build-youtubedl.sh clean
done

0 comments on commit 8d6fc4f

Please sign in to comment.