DxOMark has faced some criticism due to the way they score smartphone cameras. Previously, the protocol they used evaluated the camera’s JPG output rather than on RAW image data. This, along with the video tests they do, cover over 50 different realistic indoor and outdoor scenes. Quality is always rated based on the device’s default (auto) mode and they only intervene to activate the bokeh, flash, and zoom features. This process has enabled the company to be held in high regard for quite a while, but they are now changing the way they test and score smartphone cameras.
Among DxOMark’s new tests include a new zoom test involving indoor, outdoor, and lab environments with the focus being on resolution and image detail, noise, color reproduction, and image artifacts. With more smartphones using a portrait/bokeh feature, there’s a new test specifically for this that evaluates the blur effect and looks at distortion on portrait subjects and artifacts too. The autofocus and sharpness tests now take into account motion in the scene and the speed of image capture. Specifically for the autofocus test, they evaluate not only accuracy but also speed and texture, and the noise when holding the phone as well as when it is mounted.
So this is the new testing methodology that the team has come up with to acquire more data points to calculate a photo and video sub-score. They have even taken this new testing protocol and re-tested some older devices as well. DxOMark pulled any device that had occupied the top spot as well as some others to do re-testing on. This includes the HTC U11, Google Pixel, Pixel XL, iPhone 7, iPhone 7 Plus, and the Galaxy S6 Edge.
If you’re interested in learning how these devices stack up against the new review system then you can find them here. The new tests also give these devices new scores and it actually brings the Google Pixel in line with the same score that the HTC U11 received (which is a 90).