Logo Computer scientist,
engineer, and educator
• Software  • The KBOX3 project

KBOX logo

KBOX3 development diary

For comments and general information on the previous version of KBOX, KBOX2, please see here

February 2017

February 8

My first update in over a year! I'm not getting anywhere near as much correspondence about KBOX3 as I used to. Whether that's because there is no further interest, or because everything that is likely to work now works as well as can be expected, I'm not sure. Anyway, the project is still alive, and I do respond to e-mails about it.

It's been brought to my attention that the link to Nano on the downloads page is to the older, KBOX2 version. Although the version of Nano is the same, KBOX2 binaries were not built to be position-independent, which many modern Android releases insist on. Fixed now.

I have a build of "Popeye", the chess problem solver. At present it's only a binary, not a package, but I understand it does work. Please contact me if you're interested.

July 2015

July 31

Despite my previous pessimism, KBOX3 does work on (at least some) 64-bit Android devices. I guess that I've become so used to Google repeatedly changing Android in ways that break KBOX, I was too quick to jump to the assumption that they had done so again. However, it turns out this time to be a "d'oh!" moment on my part.

In some shell scripts I had specified the interpreter like this:

when the usual formula is:
This was a hangover from the very early days of KBOX, when the reference to the Android shell in /system/bin was necessary. It has not been necessary for several years, but I hadn't quite gotten out of the habit of using it. Refering to the Android shell wasn't a problem on 32-bit devices, but on a 64-bit Android the stock shell is a 64-bit binary, and everything in KBOX is 32-bit. So when we try to load the libfakechroot.so library into the process, it fails because the bit-widths don't match.

This problem only affects packages that use shell scripts, and I've been through the whole distribution and (I hope) fixed all such scripts. I have uploaded corrected versions of the broken packages to my downloads page. However, it's possible that I have missed some scripts, so do please let me know if you get errors about not being able to load a 32-bit library into a 64-bit process, and I'll try to fix them.

As a side note, KBOX3 only works on 64-bit Android devices because the platform provides some measure of 32-bit backward compatibility. Although it would be possible, in principle, to rebuild the whole of KBOX3 to be 64-bit-clean, I can see no benefit at all in doing so

July 29

KBOX3 does not currently work on any 64-bit Android device. Yep, Google has found another way to stop KBOX working, although I doubt it was intentional. On a 64-bit device, even if the KBOX shell starts up, attempting to run any program will result an error message of this form:
"/data/data/jackpal.androidterm/kbox3/lib/libfakechroot.so" is 32-bit instead of 64-bit
And, because libfakechroot.so is not loaded, the whole operating basis of KBOX is undermined. This library is used to fake the Linux-like filesystem on top of the Android filesystem that all KBOX executables depend on.

At present, this means that KBOX3 won't work on the HTC One M8S and M8, the Nexus 9, and the Samsung Galaxy S6, and a few other devices. However, it seems likely that more vendors will move to a 64-bit version of Android 5 in future, and this could well spell the end for KBOX. On a side note, I suspect that the move to 64-bit is to take advantage of the newer ARMv8 architecture, not because any Android app is likely to need a 64-bit address space.

The odd thing is that libfakechroot.so is indeed a 32-bit executable, as is everything else in KBOX3. I can't for the life of me understand why Android thinks this library ought to be 64-bit, when it is being preloaded into a 32-bit application. Of course, there are a great many things about Android that I don't understand. In principle, I could build a 64-bit version of libfakechroot.so, but I can't think of any reason for it to work with 32-bit code.

The truly bizarre thing, however, is that the KBOX shell starts up at all, given this problem. It seems that it does start, and it finds files where they should be, which means that libfakechroot.so is doing what it should. It's only when the shell tries to launch further executables that Android starts complaining about the library bit-width.

I fear that the only solution is to rebuild the whole of KBOX as 64-bit. Since I've only recently recovered from the pain of rebuilding it as position-independent to suit Android 5, that isn't a job I intend to undertake for a long time, if ever.

July 1

At long last, gcc is now available for KBOX3. The package can be obtained from the Downloads page. This is an early release and there could well still be problems. For more information, see here.

June 2015

June 30

A lot of folks who have tried KBOX3 have found that the shell won't even start up, because of a reported missing dlopen symbol. Support for the dlopen() function (for dynamically loading libraries) has been a serious nuisance throughout the whole of the KBOX project. Support for this function seems to vary from device to device in an unpredictable way. Unfortunately, I've never been able to reproduce the problem.

I've recently had a contribution of a modified version of libfakechroot.so, which is known to make KBOX3 work on at least one Android 5 device. I'm recommending that anybody who sees errors about missing dlopen download and install fixed version 1 of libfakechroot.so. Just copy this over the existing libfakechroot.so version in the lib directory.

I've called this 'fixed version 1' because I can't be sure that other modified versions won't be needed for other systems, unfortunately.

On a more positive note, I've had a new build of GCC from Cyd Haselton which should restore C compiler support to KBOX3, much missed until now. I was unable to build GCC on my desktop workstation in a way that worked on Android 5 (because of the position-independent binary requirements, among other things). The trick — which I confess I would not have thought of myself — was to build GCC on the device under Android 4.4, which did not have these restrictions. I'm still working on getting this new build packaged up for distribution, but it's looking promising so far.

March 2015

March 4

I've released a first attempt at some utilities to support building Java applications directly on the Android device. For more information, see here.

March 3

I've built gnuchess for KBOX3, which was straightforward. I was hoping to build the ncurses interface 'cboard' but, alas, the source is full of wide-character functions that have no implementation in Android Bionic. These problems might be surmountable, if there's enough demand for this to make it worth the effort.

March 2

Python 3 is now working on KBOX3. The problem, as so often that case with Android 5, was figuring out how to specify all the necessary compiler/linker switches in the rather complicated build system. Python modules are built as shared objects — it should be possible, in principle to link them statically to the main executable, but that didn't work for me at all — and the linker needs these binaries' dependencies (e.g., -lm) specified explicitly, even though they are specified on the main executable. Since the Python build system does not do this without a good deal of fiddling, I assume that's not something that is necessary with desktop Linux.

March 1

I managed to build the latest (at this time) version of Perl — 5.20.2. I made use of Alex Suylok's cross-compiler patch for Perl, because building Perl is a real bear otherwise. I can't begin to understand why building Perl needs to be quite the nightmare that it is, but there are many things I don't understand, so nobody should be surprised by that. For KBOX2 I unpacked the Perl binaries from Google's Android Scripting tools, but these don't work on Android 5, so I had to build from scratch. I've only run simple tests, and I have no idea how this build of Perl will perform with anything that involves dynamic loading (but I can guess it won't be an unmitigated success.) Still, for routine stuff it seems fine.

February 2015

February 28

I've managed to port Git (code repository management utility) to KBOX3. The problem, as it so often is, was figuring out to get the appropriate compiler switches and library references into Git's complicated configuration system. I've only tested with basic pull and push operations, which seem to work fine, including with https:. However, I have not included any CA certificates in the package, so operations that work over SSL need to have GIT_SSL_NO_VERIFY=1 defined in the environment. Since a growing number of utilities need CA certificates, perhaps a better approach would be provide them in a separate package, and build these utilities so they look in the appropriate directory?

February 27

Work on support for Android 5 has started at last. Because the differences are so substantial, I've decided to start a whole new project — KBOX3 — and retire KBOX2. Supporting one Linux distribution single-handed in my spare time is difficult enough; two would be a nightmare. I don't know to what extent KBOX3 will work with Android versions prior to 5, but I'm not aiming to support them, nor old hardware. KBOX2 will remain available, in its current form, for the forseeable future.

Porting Linux applications to run under Android 5 is no easier than porting them to Android 3-4 was. KBOX3 takes the same basic approach, of providing a virtual filesystem that looks like a normal root filesystem. However, there are now even more restrictions on access to system files and devices, along with the absurd and demoralizing requirement to build everything in position-independent code. At the time of writing, even Google's own builds of scripting languages like Perl and Lua remain disfunctional on Android 5, and there's no indication that this will change any time soon.

The situation has been worsened by the fact that later versions of the Google Android NDK — which are required for Android 5 — have even more bits missing than previous versions. In some cases, where the problems are justing missing header files, I've been able to fix the problems just by copying in the missing bits from a previous release. However, a number of standard C library calls that were just useless stubs in the old versions of Android Bionic are now missing altogether. In a sense that's a good thing — it's surely better that something fail to build, than that it builds and then fails in use. But despite the missing code, most of the packages I built for KBOX2 did seem to work, and considerable changes are sometimes required to replace the functionality that has been removed, even with useless stubs.

A particularly irritating finding with Android 5 is that there are some functions defined in the NDK that don't exist in the actual on-device Bionic implementation. Some packages that rely on auto-configuration to probe the compiler to find what it offers can fail, because the generated code contains referencs to missing functions.

All in all, a horrible job made more horrible. Still, the packages that I started KBOX to use — Busybox, Dropbear, and Vim — are all working, as are about 80% of the other stuff from KBOX2. The big, complex packages like Perl and gcc are currently not available and, as they were contributed by other people — all of whom are volunteers as I am — I don't know when that situation will change.

Copyright © 1994-2015 Kevin Boone. Updated Feb 08 2017