Applications

Drone Applications.    View      

First Person View (FPV) drones use fisheye lenses to display the view to the pilot. Multispectral cameras are used for Agriculture, firefighting and rescue operations.

Real-time dewarping

Real-time dewarping of fisheye images is a computationally and memory intensive process well suited to an FPGA, particularly those with large memories.  First Person View (FPV) drones use fisheye lenses because this reduces costs, only need to be focused once, and provide a wide field of view. But AI and computer vision algorithms cannot use them; the distortion is too great. For real-time applications such as dog fights, an AI controller would need dewarped images.

Drone camera applications would need to integrate with existing Open Source drone control software such as Open HD.

OpenHD is Open Source software for controlling a Drone.  It accepts MIP-CSI or USB video, and broadcasts it or records it.  For broadcasting, it uses a modified version of WiFi; it does not check for packet confirmation.  This allows errors to be transmitted, but significantly reduces latency.  Low cost FPGAs  do not support CSI, so FPGA pipelines on drones need to use one of the bridge solutions described later.

RunCam WifiLink V2 ($69) is Open Source and transmits DVI video over Wifi.  It uses H.264 or H.265 encoding to compress and then transmit 720p video at 120 Frames Per Second (FPS) over Wifi. It is used for drones. It uses Infineon chips. It brought down prices in that market.

Anti-Drone Applications.    View      

People can hear drones before they can see them. But once they are heard, zoom cameras could find them.

Protecting infrastructure against drones could use an inexpensive grid of upward facing fisheye camera. The grid could both detect low flying drones, and triangulate their exact 3-D positions. More expensive high resolution cameras with physical zoom could be used to detect drones at higher altitudes.

Drone recognition would help in protecting infrastructure. In WWII ships had silhouette sheets to recognize enemy ships. We now need 3-D silhouette sheets for drone recognition.

NeTV2.    View      

An open video development board in a PCI express form factor that supports overlaying content on encrypted video signals. It can alw

The NeTV2 is an FPGA-based development board optimized for open digital video applications, and just like any other development board

Classic mode allows the NeTV2 to add encrypted pixels to an encrypted video stream, without ever decrypting the video stream. It does this by observing the initial cryptographic handshake between the the video source and video sink. So, for example, using NeTV Classic Mode, you could add an opaque text overlay to a live video stream, but you could not add a translucent text overlay, as that would require decrypting the original video stream to compute the alpha blending.

Libre mode works only with unencrypted video feeds, but has full access to the entire video stream. This lets you arbitrarily manipulate pixels in real time, either onboard or by plugging into a host using the PCI-express 2.0 x4 interface for offboarding compute to a companion GPU or cloud service.

Tiny Videos.    View      

TimVideos.us is a group of exciting projects which together create a system for doing both recording and live event streaming for conferences, meetings, user groups and other presentations.

https://code.timvideos.us/

Other Applicaitons.    View      

Video pipelines are not just for the drone market.

There is a class of vision impaired people who would benefit from good edge detection algorithms. Thick black lines on their glasses would help them navigate through doors, and follow the sidewalk. The Canny and other edge detector algorithms detect fine lines, but are not so good at recognizing the major features needed to support this functionality. FPGAs can use more computationally intensive algorithms to accomplish this task.

Stereoscopic vision currently uses computationally expensive disparity maps. Edge detection followed by stereoscopic analysis may reduce computational complexity, and run on less expensive FPGAs.

NuEyes NuLoups.    View      

A beautiful example of what can be done with FPGA based video processing pipelines. I know the people who wrote the Verilog. Sadly not open source.

Engineered for dental and medical use and designed in conjunction with health care experts, NuLoupes are built for real world use cases. NuLoupes brings surgical loupes into the 21st century with live 3D stereoscopic imaging for depth perception, continuous magnification from 1x to 13x, all in a light weight ergonomic design for comfort and all day use. Launching in late 2025.