Communication Protocols
This area covers communication protocols used by image sensors, display protocols, and longer range protocols. Bridge chips, which convert between protocols are also covered.
Image Sensor Communication Protocols. View
Before reviewing image sensors, we need to understand the protocols which they use to communicate. Longer distance protocols will be discussed later.
I2C is a serial (1 data bit wide) communications protocol.
Serial Camera Control Bus (SCCB) runs on top of I2C and is used for controlling cameras. It is the OmniVision-specific term for standard I2C.
MIPI-CSI Camera Command Set (CCS) is a simple scheme for configuring registers over an I2C bus, supporting various register address sizes.
Serial Peripheral Interface Protocol (SPI) is a parallel interface. It can be 1,2,4 or 8 bits wide. FPGAs use SPI for external Flash Memory and often external RAM access. The Himax cameras can be controlled using either SPI or I2C. In some cameras, images can be transmitted over SPI, albeit slowly. The SPI protocol is not ideal for video as it lacks dedicated lines for essential timing signals like Vertical Sync (VSync) and Horizontal Sync (VSync).
Wireless Protocols. View
Communication using radio waves.
Wifi Radio protocols on microcontrollers are limited to 40 Mhz, meaning <= 20Mbits/sec for TCP. Compression is needed for tolerable video. Often H,264 compression is used.
3G/4G/5G For longer distance, one needs to use the cellular phone network. AD9363 is one promising chip.
USB. View
USB Video Device Class (UVC) transmits video and camera control over USB.
USB is used for connecting multiple devices to a host computer. USB UVC transmits video over USB. Before protocol overhead: USB 2 runs at speeds up to 480 Mbits/s. USB 3 runs at speeds up to 5Gbits/s (Gen1x1), 10Gbit/s (Gen2x1, Gen1x2), 20Gbit/s (Gen2x2). USB-4 runs st speeds of 20/40/80 Gbits/s. USB uses 8/10 bit encoding, so only four fifths of the bandwidth is available for data. USB has both a host interface and a device interface. FPGAs are well supported with the device interface, both with hard-core USB device cores, and by the well-known less-than $10 FTDI chips. The $23 EZ-USB™ FX5 enables a low cost FPGA to function as a USB host.
Other Wired Protocols. View
The world is moving to high speed serial protocols.
They have fewer wires than parallel interfaces making them cheaper to produce. They use differential signalling which reduces noise, and reduces the voltage swings which reduces power consumption.
Display Protocols. View
Often the last stage in a video pipeline is to display it. These display protocols are listed from simplest to most complex and slowest to fastest signaling frequency.
Display Parallel Interface (DPI) and RGB are two names for the same thing: parallel output to a display. This is not so much a standard as a simple way of doing things. One pixel is transferred per clock cycle, up to 24 bits wide. Also clock, control signals, HSync, VSync, and often Blank, are transmitted at the same time. Hackaday article [21].
Flat Panel Display Link (FPD-Link), confusingly also called LVDS, was released by National Semiconductor in 1996 for sending images from laptops to flat panel displays. It sends each RGB color on a separate differential wire pair at a higher frequency.
Transition-minimized Differential Signaling (TMDS) is the physical layer used by DVI and HDMI. TMDS uses current-mode logic with open-collector drivers. GateMate LVDS can drive TMDS for low DVI resolution output, but the TMDS and LVDS protocols are not really compatible. For higher frequencies a bridge is needed.
DVI runs on top of the TMDS physical layer. DVI is a free standard for transmitting video. DVI allocates one pair to each color, and one pair to the clock. DVI has horizontal and vertical sync signals, and an empty frame around the image (the porch). DVI encodes 8 bits of color in 10 bits. DVI can run at speeds from 24.5Mhz (640*480 @60Hz) (easy to debug) up to 161+ Mhz (1600 × 1200 @ 60 Hz). Dual Channel DVI supports twice as much data, and can run at higher frequencies.
There are many DVI output solutions for FPGAs, but a shortage of DVI input solutions for FPGAs. SiPeed has an undocumented LVDS driven DVI in/out board [13] for their Tang Primer 25K. It includes a DVI output demo, but no DVI input demo. There is a closed source Xilinx DVI input solution, and a low bandwidth DVI input solution for the ECP5 ULX3M boards.
HDMI is a proprietary superset of DVI and includes audio inserted into that empty frame (front and back porch). Neither DVI nor HDMI are compressed signals. Hackaday article [19].
DisplayPort uses Low Voltage Differential Signaling, It has at least 4 unidirectional data channels, and a half duplex bidirectional auxiliary channel for command and control. DisplayPort is packet based and more complex than DVI. There are 4 displayPort repositories on github. The Parreto/DisplayPort repository appears mature, and is free for internal use.
Embedded DisplayPort (eDP) is a display panel interface standard for portable and embedded devices. It can have fewer wires than DisplayPort, making it an even more complex protocol. Hackaday article [21].
MIPI Display Serial Interface (MIPI-DSI) drives displays and is simpler than MIPI-CSI. DSI requires fewer wires, does not include camera control and can be uni-directional. FPGAs which do not support MIPI-CSI may be able to support MIPI DSI, at least the parallel version. While the official MIPI-DSI spec is also not freely available, the protocol is well described here [15 ] and here [6]. Hackaday article [20].
Please Comment
You can comment on this page by replying to the following Mastodon toot. Eventually the entire thread will be displayed, not just the toot.
Built with the Forest WIki.
User contributed content is licensed under a
Creative Commons CC By 4.0 License.