Applications

Real-time dewarping

of fisheye images is a computationally and memory intensive process well suited to an FPGA, particularly those with large memories.  First Person View (FPV) drones use fisheye lenses because this reduces costs, only need to be focused once, and provide a wide field of view. But AI and computer vision algorithms cannot use them; the distortion is too great. For real-time applications such as dog fights, an AI controller would need dewarped images.

Drone Applications

Open Source FPGAs are great for building innovative applications which do not already exist and for reducing the cost of existing applications.

Multispectral Cameras for drones are closed source, expensive and include 5 cameras. Red, Green, Blue , infrared, and a fisheye First Person View (FPV) for navigation. An Open Source multispectral camera is needed.

For fire fighting, one just needs three cameras: a fisheye RGB FPV camera for navigation and a rectilinear RGB camera paired with a rectilinear infrared camera for mapping.

For agriculture, 5 cameras are also used. But photosynthesis spectrum peaks at two frequencies: infrared (650nn) and blue (450nM). So only two cameras are needed for mapping photosynthesis. An open source agriculture-specific camera is needed. In this time of steadily worsening climate change causing rising food prices, we should be doing whatever we can to help global agriculture.

Drone recognition would help in protecting infrastructure. In WWII ships had silhouette sheets to recognize enemy ships. We now need 3-D silhouette sheets for drone recognition.

Drone camera applications would need to integrate with existing Open Source drone control software such as Open HD.

OpenHD is Open Source software for controlling a Drone.  It accepts MIP-CSI or USB video, and broadcasts it or records it.  For broadcasting, it uses a modified version of WiFi; it does not check for packet confirmation.  This allows errors to be transmitted, but significantly reduces latency.  Low cost FPGAs  do not support CSI, so FPGA pipelines on drones need to use one of the bridge solutions described later.

Protecting infrastructure against drones could use an inexpensive grid of upward facing fisheye camera. The grid could both detect low flying drones, and triangulate their exact 3-D positions. More expensive high resolution cameras with physical zoom could be used to detect drones at higher altitudes.

RunCam WifiLink V2 ($69) is Open Source and transmits DVI video over Wifi.  It uses H.264 or H.265 encoding to compress and then transmit 720p video at 120 Frames Per Second (FPS) over Wifi. It is used for drones. It uses Infineon chips. It brought down prices in that market.

Other Applications

Video pipelines are not just for the drone market. There is a class of vision impaired people who would benefit from good edge detection algorithms. Thick black lines on their glasses would help them navigate through doors, and follow the sidewalk. The Canny and other edge detector algorithms detect fine lines, but are not so good at recognizing the major features needed to support this functionality. FPGAs can use more computationally intensive algorithms to accomplish this task.

Stereoscopic vision currently uses computationally expensive disparity maps. Edge detection followed by stereoscopic analysis may reduce computational complexity, and run on less expensive FPGAs.