Smart WelcomEyes Advance

Fuji Xerox has achieved a "zero waiting time experience" for users of our multifunction devices when proceeding from sleep mode recovery to job execution through our unique sensing technology, "Smart WelcomEyes® Technology." We have now further enhanced this technology and developed "Smart WelcomEyes Advance," which offers smarter, more convenient features.

Two cameras have been installed in the multifunction device for Smart WelcomEyes Advance Technology. We replaced the reflection sensor, which was previously used to detect human movement, with a human detection camera, making it possible to detect users within a wider area. We also installed a face recognition camera that can identify users for easy authentication.
The following is an explanation of the positions of the pyroelectric sensor and cameras, and the user detection range.
The pyroelectric sensor, which has a wide detection area while consuming only a small amount of energy, is installed in an aperture at the bottom of the pillar cover on the front of the multifunction device, as it was in the previous Smart WelcomEyes Technology. The aperture in which the pyroelectric sensor is installed has been enlarged, expanding its detection range from 800mm to approximately 1200mm in order to correspond with the larger detection range of the new human detection camera. This makes it possible for the pyroelectric sensor to detect potential users approaching the multifunction device at a greater range to quickly recover the human detection camera from sleep mode.
The human detection camera is installed in the same area in which the reflection sensor was previously located for Smart WelcomEyes Technology-on the pillar cover surface. The camera protrudes slightly from the pillar cover, giving it a detection area with an arc of approximately 180 degrees and a radius of 1200mm from the center of the device's control panel. Within this area, the camera determines whether or not the detected person is a user (someone who intends to use the device) from the direction the person's body is facing. Also, the area with an arc of approximately 180 degrees and a radius of 350mm from the front of the multifunction device is designated as the user detection range, so that even if a person within the 1200mm-radius detection area is not determined to be a user of the device, they are automatically judged to be a user after entering the 350mm range.
The face recognition camera is installed in the upper left of the device's control panel. The images captured with this camera are displayed on the face recognition screen on the control panel. When a user stands in front of the control panel and looks into the screen so that their face fits into the dotted outline, it is detected and verified against preregistered face information. If the detected face information matches a user's preregistered face information, the user can easily log into the device (Fig. 3).

Fig. 1: Positions of the pyroelectric sensor and two cameras

Fig. 2: Area where people are detected

Fig. 3: Position of the user and the face recognition screen

Only the pyroelectric sensor, which consumes very little power, remains active when the machine is not being used, in order to maintain an overall low rate of energy consumption. When the pyroelectric sensor detects the movement of a person near the device, the human detection camera is activated. The human detection camera detects the direction the approaching person's body is facing based on the direction of the person's toes. If the camera determines that a person approaching the device is a user, the device exits sleep mode. When a person approaching the device is determined to be just a passerby, the device is programed to remain in sleep mode.

In order to process the images captured by the human detection camera and face recognition camera, Toshiba Corporation's image recognition processor was implemented. This processor, which is also known for being an effective technology in car mounted cameras, is able to achieve high performance in image recognition and processing while consuming minimal electricity. It does not even require a cooling fan. Since it has high processing performance as well as environmental compatibility while also being compact enough to fit inside the pillar cover, it meets the goals of Fuji Xerox's RealGreen® concept, which has led to its implementation in the Smart WelcomEyes Advance Technology.

Image processing technology for user detection

The image recognition processor performs two types of image processing in real time on the video image captured by the camera's fish-eye lens: distortion correction and viewpoint conversion. After the image processing, this technology detects the direction the person's toes are pointing from the image of the feet created in the viewpoint conversion process, and indicates the person's toes detected within the human detection area in red dots (Fig. 4-1). In the images of a passerby captured by the camera, the person's toes are still detected, but since the trail of red dots is not leading toward the multifunction device, the person is not judged to be a user of the device (Fig. 4-2).

Fig. 4-1: Images captured by the human detection camera (person detected as a user)

Fig. 4-2: Images captured by the human detection camera (person detected as a passerby)

Technology to process the images for face recognition

The image recognition processor detects 14 facial parts as feature points. Then, the facial size and angle are corrected three-dimensionally to extract the feature amount of the face (Fig. 5).The feature amount of the face is affected by lighting, so it becomes difficult to match the face of a person with preregistered face information if the lighting conditions are different. Also, if light shines directly on the camera, it can cause blown-out highlights due to lens flare, making the detection of feature points difficult. Normally, an expensive anti-flare lens would need to be used, but instead, by providing an on-screen outline and guiding the user to align their face with it, we are able to reduce the impact of ambient disturbance factors, thus maintaining the low cost of this technology.

Fig. 5: Results of face and facial parts detection

Smart WelcomEyes Advance is Fuji Xerox's original technology that minimizes the number of steps necessary for the user to start using a multifunction device, providing an office environment in which customers can work efficiently without having to interrupt their thoughts to use the device. We plan to further evolve this technology and offer greater value to customers.

  • Note Smart WelcomEyes and RealGreen are registered trademarks of Fuji Xerox.