Image pre-processing is an integral part of the computer vision pipeline. Pre-processing entails cleaning up the image and making sure that it is ready to be fed into the image recognition pipeline. Several techniques are used in pre-processing, such as denoising, color enhancement, high dynamic range, artifact removal, image stabilization, and so on.
For whatever reasons, computer vision technologies, such as object detection or deep learning algorithms, often make headlines, whereas image pre-processing has largely been ignored by the press. However, pre-processing is an equally important part of the overall process and deserves special attention, depending on the use case. After all, detecting an object in an image that is dark does not make sense and the job of pre-processing is precisely to “clean up” the image so that the object of interest can be easily detected.
Pre-Processing Algorithms Optimizing Images
Consider facial recognition at the airport. There are many potential problems that an image detection algorithm could be susceptible to in the absence of proper pre-processing. For example, the quality of optics may limit the quality of the picture. The number of faces in a picture may be too high, limiting the number of pixels available per face for processing. Moving objects may throw shadows or obscure the view. If outdoors, weather conditions such as cloudiness, fog, or smoke may impact the incoming image. If indoors, ambient light conditions may vary depending on the time of day.
All of these scenarios can be enhanced by an image pre-processing pipeline and the image can be brought to a standard input format so the face recognition algorithm always receives a standard, pre-defined set of values. The pre-processing algorithms can tweak the parameters for each of the conditions described in the preceding paragraph and optimize the image in such a way that a convolutional neural network (CNN) engine can provide the best results.
Challenges Faced by Companies Entering the Pre-Processing Market
Many companies are focusing on pre-processing, but their efforts have primarily focused on intellectual property (IP) and software. Image pre-processing, by definition, is a vast collection of algorithms and is highly application dependent. The optimized software implementation is also central processing unit (CPU) dependent and vendors almost have to choose a side.
While many companies have jumped into this industry, companies like Almalence, Morpho, Irida Labs, Itseez (acquired by Intel’s IOTG Business Group), and ZMicro all have some sort of pre-processing solutions optimized for different chipsets. All of them have been in business for several years and are shipping their products. Algolux is taking pre-processing a step further. The company has developed an engine based on machine learning techniques that optimizes the overall pre-processing pipeline to achieve optimal results.
The pre-processing IP industry has traditionally relied on providing value-added services or licensing and/or a royalty-type business model. While the royalty approach has its upside, many original equipment manufacturers (OEMs) are reluctant to add costs to their bill of materials (BOM) and thus they shy away from the royalty model. This leaves vendors with the choice of services and licensing that makes it harder to scale business. Even if the OEM agrees to royalties, the per-unit payment is often too low to have a sizeable impact. The IP must also be customized for the particular OEM application and may require upfront customization from the vendor prior to closing the sale. This adds to the overall cost of development, making it harder to monetize.
Recent Acquisitions Signal a Positive Future
While the dollar figures and revenue associated with pre-processing companies remains much smaller in comparison with the other types of computer vision product companies, ARM’s acquisition of Apical for $350 million is an implicit endorsement of the importance of pre-processing. Apical developed synthesizable pre-processing IP, among other things. Itseez’s acquisition by Intel is yet another success story in the pre-processing market, although the price of the deal has not been disclosed publicly.
The computer vision application industry is in the early stages of its ramp-up and many facets will require enhancement. In time, we should see more winners emerge in the image pre-processing space.