Two years in the making, VCX-WebCam 2023 has been designed from the ground up to evaluate both integrated and peripheral consumer-grade video conferencing cameras for video image quality.
In today’s hybrid world, people want to look and project their best selves. The last two years have shown that few integrated webcams are good enough, and peripheral webcams are not much better. Video image quality is about more than just megapixels. Camera sensors, optics, software, hardware, and image processing algorithms all have a key part to play.
At a time when webcams have become more important than ever, the need for a benchmark that provides a reliable, objective and comprehensive measure of webcam image quality has only become greater. Today marks a turning point with the release of VCX-WebCam 2023.
Starting with a rigorous evaluation methodology, VCX-WebCam 2023 evaluates camera performance in controlled conditions that replicate the kind of stressful environments devices are used in for video conferencing. Tests are built on up-to-date evaluation methods that work in many of the same ways modern image processing algorithms do. In addition to basic image quality concerns, tests are focused on common pain points reported by users and industry experts throughout the development process.
With a special emphasis on face-present and face-moving scenes, auto-exposure, white balance, noise and spatial frequency performance, the VCX score provides a comprehensive measurement of camera imaging performance. A high VCX score means a high level of image quality.
VCX-WebCam 2023 was built with transparency in mind by the webcam standards department of the VCX forum. Chaired by Anthony from Intel and with important contributions from Dell, Image Engineering, Lenovo, HP, AMD, PAL labs, Microsoft and many others across the PC ecosystem, this group leveraged the knowledge and experience of leading OEMs, test labs, ISP providers, hardware and software vendors from across the PC ecosystem to build a specification that is both robust and transparent.
Results are objective to avoid bias from human testers while still reflecting the common subjective experience of the end user. VCX-WebCam 2023 paves the way to providing realistic, dependable results indicative of how real people rate webcam image quality and performance.
The specification behind the VCX-WebCam 2023 benchmark consists of 24 KPIs across 6 areas of focus.
Focus areas were selected based on pain points collected from member tuning and customer teams throughout the PC ecosystem. Special attention was paid to areas that have been difficult to test objectively, until now. The specification is a mix of recognized industry test methods from organizations like IEEE and ISO as well as existing VCX methods for mobile phone camera tests. New to the industry are innovative test methods created by VCX for things like exposure, white balance convergence, and presence-driven behaviors. Rather than reinventing the wheel, we felt it was important to leverage the great work these organizations are doing and focus on filling the gaps to build the comprehensive, objective benchmark the ecosystem has been lacking.
Figure 1. Convergence test measuring exposure time, target, and stability
Of particular interest is the special attention paid to skin tone response throughout the VCX-WebCam 2023 test suite. VCX partnered with a movie prop-house to develop affordable, realistic mannequin heads modeled on real people in a range of skin tones. Nearly every test is repeated with these heads to challenge camera system behavior. A VCX-sponsored research study demonstrated a notable difference in how many camera systems respond to different skin colors. We hope that by embedding skin tone into the very core of the test philosophy, VCX-WebCam can help deliver a better experience for everyone.
Figure 2. VCX user study results by skin tone
The 6 key areas of focus for VCX-WebCam 2023 are:
VCX-forum is launching the VCX-WebCam 2023 benchmark specification today and is in the process of testing devices. The first publicly available device scores will be published on the vcx-forum.org website in early 2023.
- Visual Noise – a way of measuring image noise-similary to the way the human eye does.
- Contrast, Dynamic Range and Exposure – These three areas are often linked. How bright your image is, the range of what content is bright” and “dark” and how details in dark areas are reproduced are all key areas that separate better from lesser webcam designs.
- Spatial Frequency Response – a way of measuring not just how “sharp” an image is, but how well image detail is preserved and carefully enhanced without adding unpleasant artifacts.
- Color Reproduction – How well a camera reproduces colors in a pleasing fashion, not just in a test chart but in different faces across many different test scenarios.
- Image Uniformity – a measure of how consistent brightness and color accuracy are across the entire image.
- Framerate – How fast your camera responds to the real scene often depends solely on how often frames are captured. Thresholds for framerate in low and mid-light conditions focus on the places problems happen the most.
- For more information on the VCX-Webcam 2023 standard and to download the specification go to the download page
- For more information on VCX-forum and forum members, go to the membership page
- For press or other inquiries send inquiries to Benjamin Pak at info@vcx-forum.org
Source:
Intel, Industry Partners Set New PC Webcam Standard
Cross-PC ecosystem group raises PC image quality standards for videoconferencing with new VCX-Webcam metric.
www.intel.com
