Recommendations
General advice on which technology to use when doing image processing. It depends strongly on your bandwidth requirements.
If an ASIC or application exists which does what you need, use it.
For applications with low bandwidth, and small computational loads, a mass produced microcontroller is the cheapest solution. Libmpix running on the Zephyr operating system is a good choice. Once they get vector processing integrated it will be a better choice.
The only Open Source FPGAs with MIPI are the Lattice NX17/40 chips.
If Lattice NX17/40 is not an option, but you want to use Open Source FPGAs, and you need less than 1.66 Mpixels, use a Digital Video Port camera.
If a DVP camera does not meet your needs, consider using a microcontroller as a bridge with MIPI-CSI in and a parallel DPI or RGB port out. Examples include Raspberry Pi Zero Bridge, and maybe ESP-p4, CH32H417 and Stm-32 N6 bridges. . You will need CDC circuits, available as Open Source Verilog, or provided in hard core by GateMate. You will be limited to 1.66 MPixels.
For applications over 1.66 MegaPixels, a closed source FPGA with MIPI is needed.
To easily connect an Open Source FPGA to USB, use EZ-USB FX3 with single-ended wires. FX5 also supports LVDS which can run faster.
For a lower cost USB display, use DVI output to a DVI→ USB Dongle.
For even lower unit cost USB connections, use a microcontroller as a bridge, but the engineering costs will be vastly higher. H.264 or M-JPEG will improve the video performance.
For bandwidths up to 5 Gbits/s and cable lengths up to 15 meters use the [MIPI A-Phy standard(/mipi-a-phy). While MIPI C-Phy is proprietary, the A-Phy standard is more open. A rapidly increasing number of products should be available.
For bandwidths up to 5Gbits/s, cable lengths up to 15 meters, and simpler engineering, use one of the protocols not designed by a committee. FPD-link or GMSL. GMSL even allows the control signals to pass via a different side channel.
For displays, the Gowin 1K and GateMate support 640 * 480 DVI @60 FPS. The Lattice ECP5 supports 1920x1080 DVI @ 30FPS. Gowin 20K supports (3840 × 2160s) DVI. FGPAs can drive TFT LCD screens. Otherwise one of the above mentioned microcontrollers can be used to drive DVI.
Military applications should also consider the supply chains used in producing their video processing pipeines.
Supply Chains. View
Since Covid and the Ukrainian war supply chain disruptions, people are concerned about the source of their video components.
The Polish military are particularly concerned about not using Chinese components. They complain that even the capacitors and resistors are made in China. While Lattice is an American company, and Canon, maker of the Raspberry Pi image sensor, is a Japanese company, all of those parts are made in mainland China or in Taiwan. I even used two Chinese AI’s, DeepSeek and Kimi K2 to proofread (but not edit) this article. So it is hard to not be dependent on tech from China.
China is perfectly happy selling parts to both the Russian and the Western militaries. The more those two groups squander their resources fighting over Ukraine, the better it is for the Chinese.
Western countries which want to avoid Chinese suppliers for military applications, may want to use the British Raspberry Pi Zero, the German GateMate, the Taiwanese HiMax cameras, the European STM32N6, the Korean Samsung cameras and the Japanese Sony and Canon cameras. However, one must always verify the actual manufacturing and assembly location of these components.
Please Comment
You can comment on this page by replying to the following Mastodon toot. Eventually the entire thread will be displayed, not just the toot.
Built with the Forest WIki.
User contributed content is licensed under a
Creative Commons CC By 4.0 License.