Apple II video to HDMI using a Raspberry Pi Zero / Pico

Example output from an Apple II video to HDMI adapter that I have been toying with. Pictured is one of the many colour models. The dual colour mode represents how the Apple II's video is inherently monochrome and yet the use of artifact colour results in a decent colour depth for such a small storage footprint.

In September 2021, Rob, a fellow Apple II enthusiast posted some images of an RGBtoHDMI, a Raspberry Pi based HDMI video solution, being setup on his Apple II. It sparked my interest because back in 2015 I looked into using the Raspberry Pi as a HDMI video display solution. This was around the time I was working on the "A2 Video Streamer" project. I couldn't see how the Raspberry Pi could capture signals consistently and fast enough to process the A2 video stream so I implemented a solution using the BeagleBone Black instead. Even though my devices worked, neither of them were user friendly enough to warrant further development. I don't see these projects as failures but as learning opportunities and stepping stones to bigger and better solutions. The RGBtoHDMI has given me new ideas to investigate.

The RGBtoHDMI is an amazing project. (https://github.com/hoglet67/RGBtoHDMI/wiki) It began as a HDMI interface for the BBC Micro but because of its flexibility it has been modified to support many computer systems of the same era, including the Apple II and the Apple IIGS. It's made up of two main parts. One being the CPLD which handles the level shifting, digital or analog signal sampling and the bit shifting while the Raspberry Pi Zero handles the pixel clock and frame image generation. It generates the right frequencies that allow it to synchronise to a computer's video clock. However, this comes at a price. It pretty much maxes out the RPi Zero's CPU. Even though it can output Apple II video to HDMI, using this device for the Apple II, in my opinion, is not overly efficient (I'm talking about the TTL signals here and not the composite one). The Apple II provides its own pixel clock so the majority of RGBtoHDMI's awesomeness, in this situation, is not needed. Instead, I would prefer to use the RPi Zero's CPU resources to perform better video emulation. The source code for RGBtoHDMI is huge and complex. I wanted to take a more simplistic approach.

        


Raspberry Pi Zero being used for video output and a few of the many failed attempts at getting a stable picture.

I wanted to revisit the Raspberry Pi to see if it was capable of being a simple modern day Apple II video solution. Especially now with the tiny size of some Pi models. I started off by using the RPi Zero to sample Apple IIc video signals directly (after going through level shifting of course). To get the maximum amount of power available out of the RPi Zero, bare metal programming was used. Not true bare metal. The manufacturer's firmware is still being used to setup the communications and the video but then everything after that is bare metal. Even then, using the general GPIO interface, I wasn't able to obtain a stable picture. I tried running the program in C, in assembler (manually optimised), using DMA and even running the code on the Video Processing Unit (VPU) but with no luck. The Apple II pixel clock being just above 14MHz means that sampling needs to be performed at around the 30 mega samples per second mark. I was getting in the high 20s but the readings were not consistent. We have a situation where the Apple II is outputting the video stream at a consistent rate but the RPi Zero only wants to read the data when it feels like it. The RPi Zero is not a microcontroller. It's a computer and the GPIO is isolated from the CPU. This means that it is difficult to control high speed parallel data coming in. In most situations the RPi Zero will be the master and dictating when and how the data is being transferred. We don't have that luxury here so a new plan was needed.

Digging deeper into the RPi Zero, there is an interface which shows potential as a high speed parallel means of reading in data. That interface is the RPi's Secondary Memory Interface (SMI). There isn't a lot of documentation on the interface or example projects, especially when it comes to bare metal. However I did find enough to get me started and plenty to keep me experimenting. The worst part of this interface is that the bus is driven when it's not active. This is to protect memory chips from having floating pins, which is not good for them. However, we are not interfacing with memory chip but an Apple II video source instead. The consistent Apple II video stream doesn't fit in well with this interface. I needed to find something to isolate the two. At first I was concerned about the SMI shorting out the Apple II motherboard but I found a great solution. The 74HCT4066 chip worked quite well as an isolator. This is an analog switch which has a small amount of "on resistance" that allows this chip to be used as a level shifter. Also the chip's control lines were used to isolate the Apple II video signal when the SMI interface drove the data bus. I did manage to pull in data at speeds of 15 mega samples per second but for some reason it would not work at the higher speeds. At that point, my wired connections between the IIc and the devices were a mess and not as short and organised as they are shown in the photo above. I wasn't surprised I had issues. Frustrated with not being able to get anywhere with the project I took a break. Time was running out for the year and I wasn't going to lug my development machine with me on holidays. Any more work in this area was going to have to wait until the following year.

Raspberry Pi Pico and a HDMI breakout board being used for video output. Monochrome output.

What I did have with me was my laptop and the Raspberry Pi Pico which I wanted to try out. I was able to quickly and easily setup a development environment for the RPi Pico which allowed me to play around with video examples over the holidays. When I returned back home, it only took me a weekend to get the RPi Pico working as a video output device for the Apple IIc. The RPi Pico is a totally different beast compared to the RPi Zero. It's a microcontroller and is able to control the GPIO pins at the same rate as its system clock, which is rated to 133MHz. This A2 video solution is based on top of the PicoDVI project (https://github.com/Wren6991/picodvi). It uses a HDMI socket but the signals are actually DVI signals. These DVI signals are being bit banged out of the RPi Pico in a semi-compliant format. It's amazing that this actually works since DVI and HDMI are normally reserved for devices that run ten times the speed of the RPi Pico or have dedicated hardware. This solution has several drawbacks. The RPi Pico needs to be overclocked to 252MHz for a 640x480 60Hz picture and even more if you want higher resolutions. Also, due to not being fully DVI compliant means that working on a given monitor is just pot luck. I found that a monochrome picture worked well but I could not get any colour examples working properly on the holiday apartment's TV or my home DELL monitor (even after trying the recommended work arounds). I'm sure I could have found a monitor to make this work but I just didn't see this as a solution we could move forward with. That's not to say the RPi Pico is not useful. Using the RPi Pico as an Apple II to VGA adapter would be a better alternative. There are several projects around which demonstrate the RPi Pico's ability to generate VGA signals. The RP Pico's VGA output would need to be RGB565 (16bit colour) since there are not enough pins on the Pico to directly make an RGB888 (24bit colour) solution. Maybe you could but with more hardware. However, using a common Apple II colour palette (https://docs.google.com/spreadsheets/d/1rKR6A_bVniSCtIP_rrv8QLWJdj4h6jEU1jJj0AebWwg/edit#gid=1819819314) fits in perfectly into the RGB565 colours.

Taking it to the next level will require RGB888 (24bit colour / 16 million colours) and a lot of processing power for colour emulation. That's why I moved back to working with the RPi Zero. Again I tried my luck with the SMI interface and yet again I fell flat on my face. Getting SMI working with DMA and bringing in the data at a paced rate would be the ultimate goal. This would leave the RPi Zero totally free to do just one job ie colour emulation. To get a working solution in the meantime meant that I had to compromise. I figured I could get something operational now and then later work backwards to achieve the ultimate result.

Logic analyser output showing one line of an Apple II video signal. Things to note.

1. The clock (14M) is too fine to make out using this scale. Only after zooming in would you see a nice square wave.

2. The start of line can be determined by using the SYNC or WNDW signals.

3. The GR signal goes from high to low indicating that the previous line was a graphics line but the current line is part of the text section. The Apple II screen is in MIXED mode ie graphics and text.

Before leading into the working solution I just wanted to express how simple the Apple II video stream really is. There are three signals which give you most of what is needed for a video display. The most important of the signals is the serial data out (SEROUT) which carries the monochrome 560 pixels per line of the Apple II screen. All the Apple II video display modes are a subset of the 560 pixels. Then there is the clock (14M). Its frequency is just over 14 MHz. It does not matter that the frequency of the US version is slightly different to the PAL version because the only thing needed for processing is the clock's edge. The third signal is used to calculate the horizontal and vertical line synchronisation. The sync (SYNC) signal can be used for this if you want to start at the very left of the screen, including blank padding, or the window (WNDW) signal which starts just before the 560 Apple II pixels. The vertical sync can be determined by reading the specific sync pattern or just by measuring the time between sync pulses. Other less important signals can be used to make the display cleaner. For example the GR signal can be used to kill the colour for lines that are displaying text (this removes the colour and makes text look clear and sharp). The LDPS signal can be used to re-align different video modes. The number of line pixels is always 560 but some video modes start closer than others to the left hand border.


Raspberry Pi Pico and Pi Zero being used for video output. 


Example of different monochrome and colour modes. The colour mode shown here is what the AppleWin emulator calls "Composite Idealized". The last mode fades the picture in and out but obviously the processing power needed is not enough using the current setup.

A working video solution was generated by using a RPi Zero and a RPi Pico. The RPi Pico acts as a serial to parallel converter and a buffer which stores data until the RPi Zero is ready to read that data. This means the RPi Zero's GPIO does not have to operate at a high speed. I would have preferred to have used a hardware serial to parallel FIFO but I just couldn't find anything that was cheap and available in a DIP package. In its current configuration the RPi Pico is not even working up a sweat. Only one PIO module is operating which means that the other PIO module and the two CPU cores are not being used. An 8 bit bus was enough to get this device to work. I was considering going to 16 bits but then I would loose the serial port debugging pins. Also, the archaic RPi Zero GPIO pin numbering makes routing PCB tracks more difficult. So, less bus lines in this case may be better. The GPIO is being called directly and for now, it is not using the SMI interface. Eight pixel bits are being read and processed on the fly before being sent to the frame buffer. Since the bitstream always represents a monochrome picture, emulation is needed to convert this into colour information. I'm experimenting with several different colour models and pushing the limits to see how much processing power can be used before the video picture turns to shit. At the moment the data from the RPi Pico is being polled which is a very inefficient way of doing communications. Even before going to the ultimate DMA solution, I could rewrite the code to be interrupt or state driven to get more out of the system. There are several other simple patches that could be done to increase performance like adding a bigger buffer in the RPi Pico or adding buffering to the RPi Zero (this would allow for processing during the blanking times). However then you start to compromise between the video delay and the emulation quality. For now I'm just happy that it works.

The adapter can be developed into a package for the IIc using the IIc's unique video port, the IIe using the 60pin auxiliary port or for the older Apple II models using the fiddly method (that's the technical word for it) by using alligator clips and connecting directly to chip pin legs (like in a2heaven's "Apple II VGA Scaler" card). A lot of the code is currently hard coded since the aim was to concentrate on getting a working system. It's currently operating in the centre of a 640 x 480 frame running at 60Hz and using ARGB8888. The A2's horizontal resolution is doubled and the second line is illuminated or turned off depending on the scan line display setting. I've added two user buttons. One button toggles the different display modes and the second button toggles between the colour palettes of which I only have two setup. I love how user friendly this solution is. Plug it in and it's up within two seconds. Most of that time seems to be due to the monitor syncing up. I did consider doing an FPGA implementation instead. I purchased a Scarab FPGA with HDMI outputs a few years back but I never got around to doing anything with it. It wasn't exactly cheap and they don't look to be available anymore. There are cheaper FPGA alternatives around today like the "Tang Nano 4K" which looks promising (I have one on order). So, is this going to be the best Apple II video solution? Definitely not. I still prefer to use a CRT when I can. When a CRT is not available then I love the VidHD, especially as a IIgs solution. All I wish is that we can make a device with great colour emulation for the HDMI platform that is cheap, easy to make and is readily available. Making a VGA version may not be a bad idea either. As a DIY project I couldn't think of anything simpler. The bonus is that if you don't like any of my colour models then you'll have the option to re-program the device and display whatever suits you.

Video of the device going through its current display modes. They come in pairs. Every second one is the same as the one before except it is set to have 50% scan lines. https://docs.google.com/open?id=18Pt6o6ZSSOQmiXkR6-xle7y614uFe4Pn