Camera Hardware

A consumer thinks of a digital camera as something with a lens, a case, a USB port and some buttons. An electrical engineer thinks of a camera as something with a sensor, possibly an image processing pipeline, and some wires supporting a communication protocol. Cameras for engineers do not have a case, they may also need a separate lens. Consumers usually use one camera at a time. Engineers may integrate multiple cameras together into a larger system.

Two cameras working together can provide stereoscopic images. Cell phones integrate multiple camera chips with lenses with different focal lengths into a single end-user device. An FPGA can be used to integrate cameras responding to different frequencies creating multispectral or hyper-spectral cameras. Newsrooms often switch between groups of cameras. Poland could benefit from grids of inexpensive upward-facing cameras for infrastructure protection against drones.

Camera sensors can be RGB, infrared, hypers-pectral, polarized or logarithmic. Cameras can have either rectilinear lenses or less expensive fisheye lenses. Fisheye cameras have a fixed focal length. Regular lenses need focusing and therefore are more expensive. The Waveshare OV5640 with Fisheye lens costs $16. The WaveShare OV5640 with a regular lens, autofocus and onboard flash costs $26. Regular lenses can have physical zoom. Vibration isolation hardware is available.

Like film cameras, digital cameras with a global shutter record the entire image at a single point in time. Like monitors, digital cameras with a rolling shutter capture the image line by line, making them less expensive. This causes problems when the image is moving quickly, or is illuminated by flickering fluorescent lighting.

There are certified Open Source Camera modules [8]. Here are some other options.

WebCams are a consumer device, but with the bridge solutions discussed below, they can be used with FPGAs.  They are inexpensive and very easy to use, just plug them into a USB port.

Raspberry Pi Cameras have the biggest brand name among image sensors. The key advantages of the Raspberry Pi camera ecosystem are the price, the richness of the ecosystem, and how easy they are to use.

Adafruit OV5640 Camera Boards ($10-$22.50) use an Omnivision sensor, a Raspberry Pi connector and are open source hardware. The closed source WaveShare OV5640 transfers up to 16 bits over DVP at 24Mhz.

OmniVision was an American company, bought by the Chinese in 2016. They produce a large number of camera sensors with user controllable image pipelines. The chips deliver video over MIPI, DVP and USB. They also offer Image Signal Processors (ISPs) without sensors using MIPI in. Generally their products have a tremendous range of functionality, but at the cost of a very complex api. The specific datasheets list the operations supported.

HiMax is a fabless Taiwanese company selling both displays and image sensors. They have a very interesting and innovative catalogue of products. Their website is beautifully designed. Their documentation is a pleasure to read. They have the highest bandwidth DVP image sensor which I have seen. 1280 x 960 30FPS 16 bits @ 85MHz. The camera control supports both I2C and SPI. They also have AI camera chips, and an interesting 3D facial recognition product for computer security. When you walk away from your computer, you are automatically logged off.

Sony Block Cameras have both physical zoom and autofocus.  Sadly autofocus cannot be used on drones because of the vibrations.  Sony Block cameras have a 4 lane LVDS interface.

AMS-OSRAM, sells a number of cameras with LVDS output. They are investing €567 million, 40% from the European Union, in expanding their image sensor fab in Austria.

RealSense is an Intel spinoff which sells cameras in cases with integrated depth perception or facial recognition.

Arducam is very active with both low end Raspberry Pi and high end Nvidia Jetson cameras. They have an interesting stack for connecting lots  of different cameras to USB 2 and 3.

Waveshare produces a number of inexpensive camera boards using the Omnivision sensors.

Other camera companies also exist but are not covered in this space limited article.

.  Camera selection depends on the application requirements. FPV drones need light weight fisheye lenses with a large field of view. Mapping drones need rectilinear lenses.  Suicide drones need cheap cameras. Surveillance drones are safe from ground fire at higher altitudes and so should use the more expensive cameras with high resolution (64 Mpixels) and physical zoom. For ground based camera nets looking upwards for drones, unless they are closely spaced, fisheye lenses are needed. For guard towers, monitoring a fragment of a perimeter, regular lenses would be better. Cameras fighting fires need infrared sensors.

Here is a searchable database of image sensors [4]. It includes 16 vendors with 874 sensors, but does not include the following image sensor vendors: AMS OSRAM, Gpixel, Nuvoton, Renesas, Samsung, STMicroelectronics, Teledyne and Tepedyne Flir.

When choosing a camera it is best to use one with a large installed base and good technical support. In general, camera data sheets are incomplete, they do not provide all of the details, and so the cameras can be difficult to configure. Libcamera manages the complexity of many cameras making them easier to configure. A large installed base ensures that all of the Libcamera bugs for that device have already been found.