Drone Applications

Real-time dewarping

Real-time dewarping of fisheye images is a computationally and memory intensive process well suited to an FPGA, particularly those with large memories.  First Person View (FPV) drones use fisheye lenses because this reduces costs, only need to be focused once, and provide a wide field of view. But AI and computer vision algorithms cannot use them; the distortion is too great. For real-time applications such as dog fights, an AI controller would need dewarped images.

Multispectral Cameras for drones are closed source, expensive and include 5 cameras. Red, Green, Blue , infrared, and a fisheye First Person View (FPV) for navigation. An Open Source multispectral camera is needed.

For fire fighting, one just needs three cameras: a fisheye RGB FPV camera for navigation and a rectilinear RGB camera paired with a rectilinear infrared camera for mapping.

For agriculture, 5 cameras are also used. But photosynthesis spectrum peaks at two frequencies: infrared (650nn) and blue (450nM). So only two cameras are needed for mapping photosynthesis. An open source agriculture-specific camera is needed. In this time of steadily worsening climate change causing rising food prices, we should be doing whatever we can to help global agriculture.

Drone camera applications would need to integrate with existing Open Source drone control software such as Open HD.

OpenHD is Open Source software for controlling a Drone.  It accepts MIP-CSI or USB video, and broadcasts it or records it.  For broadcasting, it uses a modified version of WiFi; it does not check for packet confirmation.  This allows errors to be transmitted, but significantly reduces latency.  Low cost FPGAs  do not support CSI, so FPGA pipelines on drones need to use one of the bridge solutions described later.

RunCam WifiLink V2 ($69) is Open Source and transmits DVI video over Wifi.  It uses H.264 or H.265 encoding to compress and then transmit 720p video at 120 Frames Per Second (FPS) over Wifi. It is used for drones. It uses Infineon chips. It brought down prices in that market.