XBM images and animations on a Nokia LCD

(Instant link to Github repository)

Everything basically started with planning a new PCB revision for my 4chord MIDI project. The version of the Nokia LCD I've been using so far had a few shortcomings that wouldn't fit my ideas for the new revision. Mainly, the flat flex cable connection came with too many design limitations, including adding a back light (see 4chord MIDI's roadmap).

So I was on the lookout for alternative versions of this display, probably trying every choice there is out there along the way, and eventually settled on the breakout board style one. You'll find them from Sparkfun, Adafruit and, well, China.

The Nokia 5110 LCD breakout board

I very much liked how the LCD is just clipped into the PCB, and connected straight to the copper via some adapter touching the PCB and the inside of the LCD. Even better, some LEDs soldered next to it took care of the back light part (admittedly, the diffused back light available for the Olimex 3310 LCD had a nicer, final touch, but what can you do).

So far so good, but how to actually use it in my own PCB? I couldn't find any design or Gerber files of the breakout board in the wilds of the internet, but also didn't care enough to contact anyone about that. Instead, I decided the most satisfying solution for myself will be to just create a KiCad component myself, and so I did (basic "reverse engineering" the breakout board was straight-forward with a caliper and multimeter, but to get understanding in more detail, I found useful additional information about the display and its different types from the serdisplib project).

Based on that KiCad component, I created a simple breakout board and sent it off to OSH Park. Since it usually takes around two weeks until the PCBs end up in Finland, I thought I could refresh my memory about controlling the LCD in the meantime.

Displaying images the old way

Having a look at the LCD handling code I wrote some time ago, and the accompanying shell scripts that should make life easier ..it quickly became obvious, there's a lot of room for improvement. The main reason is probably the LCD's TLS8204 driver's memory arrangement, which is very convenient for displaying 8px high text, but not so much for displaying everything else.

Generally, to display images on the LCD, the XBM image format seems perfect. It's basically C code representing all the black and white pixels in a char array. But those pixel arrangements are nowhere near to be used as-is with the Nokia LCDs. It would require the original image to be rotated 90 degrees counter-clockwise, and flipped vertically afterwards. The resulting XBM image would then need to be re-arranged row and column wise to finally match the memory layout.

I mostly use GIMP to create images for the LCD, so I'd just rotate and flip them as part of the creation process within GIMP. Using ImageMagick and run convert -transpose on the image will work, too. The memory re-arrangement was done with a shell script wrapper that would parse the transposed XBM image into a C code template and compile a binary, which eventually outputs a char array that can be dumped directly into the LCD driver's memory.

Finding an easier alternative way

The fact that an XBM image is just C code and reading it in is a simple #include, made me insist on keeping the actual processing in C. Maybe a future version will use Python. This does seem like a prime job to (finally) get more acquainted with NumPy.

But yeah, since it will require to include the XBM source directly into the C source, code, the actual processing will still require some parsing, code generation and compilation to output the raw image data. The previous concept to wrap it all into a shell script and generate a single .c and .h file from it, did actually work quite nicely. I mainly wanted to get rid of the separated rotate and flip transformation and do that directly on the fly inside the processing C code. Time to flip around the bits on paper.

As you can see, tears and sweat made the papers all wrinkly.. (using them as coasters might have had some effect on that, too). Having it written down and visualized helped making it clear what had to be done, and the initial proof of concept implementation verified that I was on the right track.

..and then some.

Getting to know each bit in the process, and understanding how they travel from the initial pixel drawn in GIMP to the LCD driver's memory location, sparked some more ideas. Sure, displaying images on the LCD - either some full screen images or partial images later on as part of a GUI (once again, the 4chord MIDI project appears for reference) is fun and useful, but how about some animation?

Sure, each frame could be just transformed into a full screen image and then replaced every some milliseconds, but that would mean extra storage of 504 bytes for each frame (84x48 pixel resolution being stored in 84 single byte wide rows and six byte high columns). Storing instead only the changes between each frame can significantly decrease the required program memory. Well, to a certain extend, at some point it might be the complete opposite.

So instead of simply taking one single XBM image, rotate and flip it, and re-arrange it for the LCD memory, the same needs to be done for two individual images, followed by a byte by byte comparison. The generated code is then simply a combination of "memory address" and the address' new value. Since there are 504 bytes used with the LCD, a struct with a 16bit address and a 8bit value is needed for each single pixel change happening within a frame transition

struct frame_diff {
    uint16_t addr;
    uint8_t data;
};

struct frame_transition {
     uint16_t diffcnt;
     struct frame_diff diffs[];
};

This means, any pixel change between two frames will require 3 instead of one 1 byte. Hence, if less than one third of the pixels are different from one frame to the other, this will be a more efficient storage solution. But if more change, this is unfortunately a bad solution.

Some additional logic could help to sort this out, and actually just use the image data as-is if the threshold is crossed. Also, splitting the screen into an upper and lower part and store them separately would mean, the addr part inside the frame_diff struct could be stored in a one byte uint8_t variable, which in best case will further decrease the required program memory. Neither of this is however implemented at the moment of this writing. But it's a consideration for future use, if the current implementation will cause some real-world problems.

Wrapping it together

Regardless of these known potential flaws, I was happy enough with the general concept to move on and automatize the code generation. If really necessary, I could always just do some manual adjustments.

Since the main parts of the LCD data generation is the same for single images and animations, I decided the code generation should be handled by one single script that takes a list of .xbm files as parameter and offers two options:

1. generate individual char array data for each given XBM file
2. generate an animation by using the first XBM file as key frame, and generate frame diff data for each other frame - looping back to the first one.

Also, the C part should be use the same code for each option, separating the specific parts with some preprocessor defines. Unlike the previous implementation, where everything was one template file that got parsed to include the XBM code itself, this implementation will only have to generate a header file.

So to achieve the two options - individual set of graphics, and animation built from a set of graphics, three different header file templates will have the specific preprocessor defines set up, and parsing the original XBM file(s) into it will generate code for either of:

• char array nokia_gfx_filename for an individual image
• char array code the animation key frame (practically the same as before, except the variable name is always nokia_gfx_keyframe instead of constructed from the image's file name)
• frame diff struct code between two given files

Each code simply outputs the image data variable declaration on stderr, and the image data itself on stout. The shell script wrapping it all together is then redirecting them to a header and source file respectively.

For everything else from here on, it's probably best to just have a look at the code and its example. But yeah, let's have a little animation here:

This is the graphically very elaborate example that I've dumped along with the code on GitHub. If used as single images animation, the total code size is 5054 bytes. With the frame diff style animation, the exact same animation could be decreased to 2210 bytes without any additional tweaking.

Note, the ghosting you can see in the animation is mostly caused by my sad attempts to video-to-gif conversion, in reality the effect is a lot more subtle.

Some final words on creating XBM images

XBM images are binary bitmaps, so a pixel is either black and therefore on (bit value 1 in C code) or white and therefore off (0 in C code). This is ideal for monochromatic LCDs like the Nokia 5110/3310, and is good enough for text and some simple user interface. But actual images are rarely just black and white. Even black and white pictures are usually grayscale images.

You can basically convert any random image file (JPG, PNG, whatever) to XBM with ImageMagick or GIMP (and probably a lot other tools), just make sure it's scaled to fit the display's 84x48 resolution. However, to get decent results, you'll probably want to fiddle with the colors.

First off, desaturate it to get a grayscale image (Colors->Desaturate... in GIMP), play with the color levels, contrasts etc., and ultimately set the threshold (Colors->Thresholds... in GIMP) to select what will be black and what will be white. Without setting the threshold levels, GIMP will use the defaults when writing the XBM file.

Also, when adding text to the image, I'd recommend to disable anti-aliasing (there's a checkbox for that in the Font tool settings in GIMP).