Camera Image Correction Technology

In today's business world, the use of the built-in camera functions of mobile devices continues to expand.
For example, photos of customer business cards are taken and saved using smart phones. Photos of what is written on marker boards are likewise taken and attached to the books of minutes. However, depending on the conditions when taking photos, there are cases in which the images can not be made out easily due to image distortion or the degradation of brightness and saturation.Note1

Fuji Xerox developed "Camera Image Correction Technology" in order to improve the visibility of document images of written information taken by cameras on such mobile devices as smart phones. This technology consists of geometric distortion correction (to correct image distortion) and color reproduction correction (to correct the brightness and saturation of images).

Fig. 1 below shows "geometric distortion correction", which corrects the distortion of images in photos of business cards or written marker boards and only cuts out the necessary regions of the image.

  1. First, regions where the brightness and color change sharply are detected, and then the image contour is extracted.
  2. Next, a straight contour line is extracted according to the image's contour characteristics (e.g., intensity, direction). The characteristics of the region contained within that line (e.g., shape, ratio of long side to short side, area, brightness distribution) are analyzed, and then the desired subject region is extracted.
  3. Finally, distortions in the extracted subject region are corrected, and only the image in the subject region is extracted.

Camera input image(1) Image contour is extracted.(2) Subject region is detected.(3) Subject shape is corrected and the image is cut out.

Fig. 1: Geometric Distortion Correction

Next, the brightness and saturation of the image are corrected. Fig. 2 below shows "color reproduction correction" that reproduces images that are easy to make out.

  1. The noise level of the cutout image is reduced and image sharpness improved.
  2. Next, degradation that affects image brightness is corrected. Here, the appropriate background region is identified, and the changed state of degraded brightness is estimated by statistical analysis. Thus, correction is enabled even when there is a non-uniform degradation of brightness.
  3. The last step is whitening of the image by color conversion. The region in the original document (acknowledged as the white background) is first identified. Then the degree of brightness and saturation within the image is estimated and corrected.

The processes above can therefore reproduce color approximating that of the original document.

Cutout Image(1) Sharpening and smoothing(2) The degradation of image surface brightness is reduced.(3) Brightness and saturation are corrected.

Fig. 2: Color Reproduction Correction

The example below shows how photos taken by smart phones can be corrected by combining geometric distortion correction with color reproduction correction.

Before correctionAfter correction

Before correctionAfter correction

Examples of Image Processing
(Business Card and Marker Board)

The degradation of brightness and saturation refers to what occurs due to certain conditions when taking photos, such as decreased brightness or additional color not shown in the actual subject of the photo.