Continuing on the first post, we are going to tackle the first part of this project, the displays.

The display

As mentioned before, I settled for the VS035ZSM panel by BOE. While there are panels out there that fit better to what I want to achieve, this will do for now. Its specs are:

• resolution: 1440 * 1600
• size: 3.5'
• input: Dual MIPI DSI
• framerate: 90 fps
• color depth: 8-bit
• driver ic: NT57860

Getting the datasheets

Unfortunately, both, display datasheet and MIPI protocol datasheets require an NDA. The MIPI datasheets can be found easily with a few Google searches. I also tried contacting BOE, but got no response. The good news are that at the time of writing this article, the display datasheet can be found by googling VS035ZSM datasheet on the images tab. Still, BOE has omitted the initialization code in its datasheet to make our lives even harder. Fortunately, Project Northstar uses the same display, and since it is open source, the initialization code can be found. It would be also great to have the datasheet of the display driver ic (NT57860), but it could not be found anywhere, and Novatek ignored my emails.

The bridge

The goal is to have an FPGA receive video, process it and send it to the display. I could not find a device that can handle MIPI at the speeds I want, with a low price and great availability. This is where an RGB to MIPI bridge ASIC comes to play. It can convert a 24-bit(8-bits per color) video stream to MIPI DSI, giving me a much bigger selection of FPGAs to work with. Some briges are:

• Toshiba TC358768 or TC358778
• Solomon Systech SSD2828
• Lontium LT8918

The SSD2828 ASIC was chosen due to its availability and the documentation and sample code that can be found on the web.

A sanity check

Before connecting the SSD2828 to the display, it would be a great idea to evaluate the functionality of the ASIC first. Since the VS035ZSM is quite complex and requires speeds way higher than what I can measure with any oscilloscope, I decided to test a simpler and smaller display first. The AUO H154QN01 display used on the ipod nano 6th gen. Most importantly, others have worked with this display and documented the process.

Starting on the prototype

The prototype board was designed in Kicad. Instead of designing and paying for 4 separate boards & solder stencils just for a one time prototype, I placed all boards in a single pcb. When the boards arrived, I split them using a sharpened box cutter, and lots of patience. JLCPCB permits this method, as long as they don’t have to perform any cutting that allows for the PCBs to split easily. With that out of the way, the design process began.

The MIPI signals have a differential impedance of 100Ω. There is no need for any termination resistors, since these are included in the devices. No pin package delays are provided for the displays, or the ssd2828. The ASIC offers the ability to tune the skew of each lane to compensate for PCB errors. The lanes all have the same number of VIA transitions and are length matched.

Receiving the prototype

Each PCB can be separated by scoring the line on the silkscreen. Keep out areas have been placed around the cutting areas for all layers in order to not damage the internal layers when separating.

The AUO display running a test pattern generated by the microcontroller.

You should be able to spot a kinda major error on the design. The FPC connectors are facing the wrong way! Thankfully all of the connectors are pointed to the opposite direction, so the error cancels it self out.

Probably for cost cutting reasons, some MIPI display drivers support only command mode and omit the video mode entirely. This AUO display is one of those. MIPI provides DCS commands for writing pixels in command mode, but the ssd2828 cannot utilize these commands to convert RGB video. For this demo, the microcontroller generates the pixel data and sends it through the SPI bus to the ssd2828 to be transmitted as DCS commands.

When first powering up the BOE display (after quadruple checking the voltage rails and the connector) AND (making voltage dividers between the 3v3 microcontroller and the 1v8 tolerant display for the reset and PWM signals) the display showed the left part of the picture above. I was relieved that it was not on fire and also confused by this behavior. The blue,red,green,white,black pattern was visible. The display had expected a compressed video stream, and this is how it was interpreting my test pattern. The MIPI D-PHY V1.01 can support up to 1Gbps per lane for a total of 4Gbps total throughput. For bigger resolutions and framerates, either a second MIPI port is used, upping the bandwidth to 8Gbps, or VESA-DSC(Display Stream Compression) is used. Many VR displays support both methods simultaneously. After some trial and error I was able to remove the code that initialized the compression.

The next day I tried getting the clocks from 350Mbps to 1Gbps per lane. Then the horizontal stripes on the right picture above appeared. Could they be the result of a bad PCB or something else? After taking a look at the Project Northstar display settings, they mentioned that for higher speeds, the horizontal porches had to be increased. It worked!

The BOE display running @60 fps with the ssd2828 test pattern.

Partial mode

For my application, only a portion of the screen is needed. At first I was considering of driving the entire display and adding borders horizontally and vertically to simulate a smaller screen. After some digging I found that the MIPI specification offers a partial mode, where only a portion of the framebuffer is updated. This way, processing power and bandwidth can be reduced.

Unfortunately, this mode is not implemented correctly on the NT57860 display driver. While this mode works for cropping the framebuffer vertically(letterboxing) it does not work correctly for cropping horizontally(pillarboxing). In the above image an FPGA produces a pattern with black and white vertical stripes. Due to the bad implementation of the horizontal crop, the image gets shifted. Still this issue can be solved by disabling horizontal cropping, and making the FPGA add black bars on the left and right part of the video, to give the illusion of this feature working properly.

Even with this setback, the bandwidth gained by a smaller image should allow for higher framerates to be achieved. The screen can handle up to 90 fps.

Programming

I don’t have much to say here, the datasheet for the ssd2828 was a bit of a mess, and writing drivers for it was a pain in the ass. Here is the project repo. On the Readme I also linked some projects and application notes that were good references for this project.

Conclusion

The next post of this series will be about designing a custom FPGA board for video processing. Bye.