Ich hatte einen komischen Traum. Ich bin im Grunewald spazieren gegangen. Plötzlich stand ich vor einem verlassenen Gelände des U.S. Militärs, ein Lauschposten, direkt an der Grenze, die durch den angrenzenden Bach ging. Das Gebäude stand und sah aus wie ein überdimensionierter, weisser Bienenkorb. Neben dem Eingang in den Bienenkorb, der eine Eichentür war, hing eine Gedenktafel auf der Stand: «Dem Gedenken an die Opfer des übermässigen Genusses der Betelnuss.» Darunter waren die Namen und Geburts- und Sterbedaten einer Familie in blauer Schrift auf weisser Emaille. Die Namen waren persisch - oder was ich mir einbilde persisch zu sein. Ein Toter war ein 13 jähriger Junge, er starb am 14.5.1978. Seine Eltern starben am 17.5.1978. An die Geburtsdaten kann ich mich mehr erinnern, nur noch an den des Sohn und dass der Vater 78 Jahre alt geworden ist.

Als ich wach geworden bin, musste ich mich erst mal schlau machen, was die Betelnuss genau ist. Ich verbinde nichts mit den beiden Tagen im Mai 1978. Ich war noch nie im Grunewald.

In the last few weeks I had developed an Android based counter for Lego® Great Ball Contraption systems. The counter should measure contactless the throughput of passing balls.

The hypothesis was: a colored or white Lego ball moves in front of a black background. The Android system camera captures every X milliseconds an image. The first image is captured with no passing ball and the image data is stored in memory for comparison. Every following image is checked for color changed against the first image. If the amount of pixels with a changed color (against the background from the first image) reached a percentage threshold, the ball count will be increased by one. Because of the camera sensor noise the color change value must also reach a threshold to be measured. In the implementation these two parameters can be configured. This improves the detection rate depending on the light environment of the GBC setup.

Measure the color change also reduces false positive ball counts because of environment light changes.

Nevertheless, Lego® also offers not only colored balls but also black and white only balls (footballs/soccer balls). If the user has also black and white balls he can also switch on a brightness detection based on the luminance. This follows the same rules like the color change detection but has a different threshold configuration parameter. Usage of the brightness detection might increase the false positive rate.

After a ball detection

After the systems has detected a passing ball in front of the camera, the system stops analyzing the given images for an amount of time. Otherwise it might happen that the same ball is counted twice.

The delay is depending on the layout of the GBC system and for this the delay time is also configurable.

Two good parameters are:

Capture delay

Delay between two image captures of the camera.
Value: 30ms

Capture delay after detection

The delay of time between the detection of a ball in an image and the next time checking again for a passing ball.
Value: 150ms

This allows a theoretical throughput of 5 Lego balls per second. Note: GBC is defined with 1 Lego ball per second. But as you know we ignore the standard in our constructions. Because of this the Android GBC Ball Counter implementation contains also a balls per second calculation.

Limitation

If two balls are passing very close succession there are false negative counts. This means, the two balls are counted as one ball. This is the greatest limitation of this non-contact method. The problem can be limited only by structural measures on GBC system.

Implementation

Image format

I use the YUV 4:2:0 8 bit image format delivered by the Camera2 Android API. This API was introduced with Android 5.0 (Lollipop).

The YUV image format has some advantages over the JPEG format: it doesn't pipe the image data through the RGB calculation and compression routines which takes a lot of time and other resources and doesn't produce more important information.

Android 5.0 Bug

Due to a bug in Android 5.0 the YUV subsystem doesn't deliver correct UV (chrominance) information. only the first 600+ bytes contains color information. The rest of the bytes are always zero. Only the Y (luminance) values are correct. This means: for Android 5.0 the implementation only supports the brightness ball passing detection algorithm.

More information

The implementation of the GBC is open source under the terms of the GNU General Public License 3 and the source code is available on Github. There you can dive deeper into the ball detection strategy.

How it works in practice