A quick dossier for understanding complete characterization, tuning and benchmarking of camera-based systems
At PathPartner, we deal with image quality tuning a lot, since we leverage our expertise in advanced imaging technologies to provide IQ tuning services needed in the camera R&D chain, from camera modules to the final imaging product. We also have a fully equipped in-house imaging lab that includes all necessary tools needed for optimizing camera image quality, calibration, testing and benchmarking to deliver practical solutions for digital camera-based products. In addition to this, we work with customers from various domains and provide image quality tuning solutions and services.
Image quality tuning is a continual process. We understand image quality and its subtleties well enough to accomplish fabulous results. Whether you want a tuning from scratch or need to refine an existing one, we can help. Why not check our imaging technologies expertise on numerous platforms?
To convert the imaging information coming from the sensor into a final good-looking picture and video using complex image signal processing pipeline, requires many iterations which is termed as Camera Tuning or ISP Tuning.We also need to understand the need for Image quality assessment (IQA), which is divided into three main applications:
- Based on the image quality evaluation results, the camera parameters or algorithms can be tuned to achieve the maximum possible image or video quality.
- The IQA results can be used to benchmark cameras or algorithms with the other in the market. The image quality metrics will give customers an easy comparison of the products.
- IQA results can optimize the camera by sending the image quality metrics to particular modules. The encoders are optimized based on the type of compression technique, quantization ratio, motion compensation, scaling and SNR values.
Image Quality (IQ) Tuning Suite
Our success in all kinds of camera markets—particularly in high-performance global security/surveillance cameras, automotive imaging, medical imaging, video-centric solutions, etc., where image quality and its precision in decision making in real-time through algorithms is essential—has been achieved through a proprietary suite of image processing algorithms enhancing images under varied conditions and in real-time. Our expertise in quadrant and platforms allows us to provide imaging technology.
Figure 1: Basic ISP Pipeline
Image signal processing (ISP) is a method to transform the sensors raw data to the final compressed image. Tuning involves optimizing the ISP parameters that help to calibrate and modify the filters and LUTs (look-up table) for existing parameters like exposure control, white balance correction, auto - focus, noise reduction, sharpness improvement, black level adjustment, gamma and color correction and tone mapping. The image processing pipeline plays a key role in overall image quality. The increased complexity with image processing is required by a corresponding increase in image resolution. However, due to many parameters, tuning an image pipeline for a particular camera system is a challenging task because different lenses, camera modules, and sensors have different characteristics.
Figure 2: Difference between unprocessed and processed image
Why ISP Pipeline?
Several kinds of artifacts degrade the image quality captured by the camera. The image pipeline generates a digital color image from raw data produced by a pipeline camera sensor in real-time. To address these artifacts, a digital camera embeds complex digital signal processing (DSP) in the image processor referred to as “Image Pipeline”. Other artifacts are introduced by the physical nature or working principle of the image sensor, e.g., by the temporal and spatial integrations of the image signal and by the color filters spectral distributions. Hence, the image pipeline stands out as a vital member of the camera system that is crucial for generating high-quality digital images.
Why Image Quality Tuning?
Image quality has no standard metric and different applications expect quality in different ways since the same camera can be used in a various way. A camera needs to take quality pictures in various lighting scenarios, such as indoors, in bright sunlight and relative darkness. Image processing needs to be readily flexible and configurable to maximize quality for every scene. A digital camera system comprises a lens, camera module and sensor. All components play a part in image quality and interact with each other. Therefore, the image pipeline must be tuned for particular camera system components based on their characteristics. The challenging part of this task arises due to the following three critical parameters:
- Physical, optical or electrical characteristics being different for lens and image sensors
- Non-availability of standard image quality metrics
- Subjectivity due to individual preferences for image quality
Figure 3: Typical Image Quality (IQ) Tuning Flow
The Two Phases of Tuning: Objective & Subjective TuningAs soon as your camera system gets enabled, several algorithms start working on analyzing camera optics hardware and the software following things like
- Distance between the camera lens and the objects in the scene to determine the focus values.
- Colour temperature of the scene and light intensity to establish accurate white balance settings.
- It also measures the amount of light present in the scene to appropriately calculate the exposure to maintain a right amount of brightness.
- Capturing images under controlled lighting environment of standard test charts and IQ parameters are measured using Imatest© software.
- Auto white balance (AWB) & color accuracy & saturation, auto exposure (AE) error, noise, uniformity, defective pixels, sharpness, dynamic range, distortion, AE/AWB convergence time, etc.
- Customer preference-based KPI
- Evaluate vision algorithm accuracy due to ISP parameter configuration changes
Subjective tuning captures all varieties of pictures within the wild, identifying systematic image quality issues and altering the ISP setting to handle them. To address an image problem, we have to analyze different pictures to determine the root cause. Let’s say one of the captured pictures looked blur; our thought process goes something like this: Is that blurriness a result of focus failure or camera stabilization? Or is it an issue we need to address through tuning? Was it observed in this one picture, or was it a recurring issue? And if so, is there a specific sort of scene where it’s an issue? Inquiring these questions can help us narrow down which components in the ISP we should try to fix. Subjective IQ evaluation includes:
- Analyzing images/videos of indoor & outdoor real-life scenes in different lighting conditions.
- In Images: Colourfulness, memory color (green grass, sky, sea, fruits, etc.), skin tone, noise level & details (aliasing, moiré patterns, sharpening artifacts, etc.), auto white balance (AWB) decision, brightness (AE) and tonal range, flare, distortion & vignetting (optics), motion artifacts (rolling shutter artifacts), etc.
- In Videos: Video blockiness, motion artifacts, AE convergence & AWB convergence/oscillations, details/texture, noise level etc.
State-of-the-art Lab to Carry Out IQ Tuning
Our in-house image quality lab has state-of-the-art facilities with advanced image quality tuning, benchmarking, camera calibration, best in class hardware and software tools, test charts for various feature analysis to assist you with your camera needs to assure image quality. Our procedures offer objective and subjective measurements of how well an imaging system performs across a various internal standard image quality metrics as per the use-case or customer KPIs and delivers comprehensive, detailed product benchmark and comparison reports.
Camera-based Automotive Imaging Solutions
Starting with commonplace rear-view cameras, cars rely on cameras for a wide range of features, including obstacle detection, driver monitoring, bird’s-eye views and even side mirror replacement. Autonomous vehicles rely intensely on camera images to complement their lidar and radar sensors. All this implies that having an effective and well-implemented camera tuning strategy for your car advancement endeavors is more vital than ever.
PathPartner’s Advanced Driver Assistance Systems (ADAS) – extensive experience in imaging and vision systems coupled with ready-to-use solution accelerators – is well-poised to be your trusted partner in your advanced driver assistance solution development. Check out our experience in an autonomous system that supports multiple cameras for both human and computer vision technologies ensuring the solution works in real-world conditions.
Putting It All Together
The image pipeline needs to be flexible enough to accommodate dynamically changing user preferences. To get the best image quality, an image pipeline must be tuned based on the sensor characteristics, the lens and the camera module. Moreover, tuning should also consider typical lighting conditions where the camera system will be used. Quality depends on the scene being shot, the particular lighting conditions and the users’ specific preferences. The image-processing pipeline's flexibility also results in significant time-to-market and cost reduction savings since the same pipeline can be leveraged across multiple applications by accommodating differences in individual sensor and lens combinations, output displays, and varying lighting conditions.
With a good understanding of overall image quality tuning, now you can make sure you have:
If you’ve got any more questions about image quality, or you’d like to discuss with us please feel free to reach us at firstname.lastname@example.org
- Case study - IQ tuning & benchmarking for AI - Powered Surveillance Camera
- Case Study - IQ tuning & benchmarking for camera manufacturer
- Role of Machine Vision technology in Packaging Industry
- 10 Amazing AI-Based Video Analytics Use Cases in Retail
- How to Integrate Industrial Robots and Machine Vision in Manufacturing: Beyond Industry 4.0
- Case study - Development of a Wireless Smart Home Monitoring Camera
- Connected cameras
- Imaging algorithms
- Plug and play camera modules