Plots

Distortion metrics can be found here.

Methodology

Large images were first encoded with Mozjpeg at q85 filesizes. Big is 125% of Large. Medium is 60% of Large. Small is 60% of Medium. Tiny is 60% of Small. Everything else was matched to +/- 5% filesize.

All the pictures have been compressed from RGB PNGs.

Formats

This study compares 6 differents encoders:

• AVIF: github.com/AOMediaCodec/libavif/commit/71d008eee44eec0994dafff01d296111f44fc123
  (using AOM: aomedia.googlesource.com/aom/+/bcfe6fbfed315f83ee8a95465c654ee8078dbff9)
• HEIF: github.com/strukturag/libheif/commit/c0fa663f0ae989e62f0b061263521e22adbca564
  (using libde265-dev 1.0.8 and libx265-dev 3.5)
• JPEG-XL: github.com/libjxl/libjxl/-/commit/2741d7b9b33d9bc709522bf10e782f1939d0ad70
• MozJPEG: github.com/mozilla/mozjpeg/commit/fd569212597dcc249752bd38ea58a4e2072da24f
• WebP: chromium.googlesource.com/webm/libwebp/+/e68765af4256287b55334c362fcf0f177f4bbb7c
• WebP2: chromium.googlesource.com/codecs/libwebp2/+/8c7e7cb3a2c457f246bc708ace0b76c35d42d5ad

The Python scripts used to generate the compressed images are available on the GIT repository. The configuration file containing the codec settings is recipes.json (quality_start and quality_end are inclusive).

Metrics

We use these algorithms in order to compare each format:

• SSimulacra: Structural SIMilarity Unveiling Local And Compression Related Artifacts. This is a perceptual metric designed specifically for scoring image compression related artifacts in a way that correlates well with human opinions.
  github.com/cloudinary/ssimulacra/commit/2a830abd42aee4883683d21d77b1f5bcaa462fae
  
  • Command used: ssimulacra [original-png] [decompressed-png]
• SSimulacra2: SSIMULACRA 2 - Structural SIMilarity Unveiling Local And Compression Related Artifacts. Perceptual metric developed by Jon Sneyers (Cloudinary) in July-August 2022.
  github.com/cloudinary/ssimulacra2/commit/9012bce32c3adecccd4633736cec2759129a929d
  
  • Command used: ssimulacra2 [original-png] [decompressed-png]
• DSSIM: Image similarity comparison simulating human perception, based on multiscale SSIM.
  github.com/kornelski/dssim/commit/dc77fc016b783570f18efe2c92d69c09863b9e0c
  
  • Command used: dssim [original-png] [decompressed-png]
• Butteraugli: project that estimates the psychovisual similarity of two images (tools/butteraugli_main.cc in libjxl).
  github.com/libjxl/libjxl/commit/2741d7b9b33d9bc709522bf10e782f1939d0ad70
  
  • Command used: butteraugli [original-png] [decompressed-png] --pnorm 3 (taking 3-norm output)
• SSIM: Structural Similarity algorithm
  
  • Command used: vmaf --json --model version=vmaf_v0.6.1 --feature float_ssim -r [original-y4m] -d [decompressed-y4m] -o [json-output]
  • Uses pooled_metrics.float_ssim.mean
• MSSSIM: Multi-Scale Structural Similarity algorithm
  
  • Command used: vmaf --json --model version=vmaf_v0.6.1 --feature float_ms_ssim -r [original-y4m] -d [decompressed-y4m] -o [json-output]
  • Uses pooled_metrics.float_ms_ssim.mean
• PSNR-HVS: Peak Signal to Noise Ratio taking into account Contrast Sensitivity Function (CSF) and between-coefficient contrast masking of DCT basis functions
  
  • Command used: vmaf --json --model version=vmaf_v0.6.1 --feature psnr_hvs -r [original-y4m] -d [decompressed-y4m] -o [json-output]
  • Uses pooled_metrics.psnr_hvs.mean
• PSNR-AVG: Peak Signal to Noise Ratio in YCbCr colorspace with the weights 8/10, 1/10 and 1/10, respectively
  
  • Command used: vmaf --json --model version=vmaf_v0.6.1 --feature psnr -r [original-y4m] -d [decompressed-y4m] -o [json-output]
  • Uses pooled_metrics.psnr_y.mean * 0.8 + pooled_metrics.psnr_cb.mean * 0.1 + pooled_metrics.psnr_cr.mean * 0.1
• CIEDE2000: a color-difference formula recommended by the CIE in year 2001
  
  • Command used: vmaf --json --model version=vmaf_v0.6.1 --feature ciede -r [original-y4m] -d [decompressed-y4m] -o [json-output]
  • Uses pooled_metrics.ciede2000.mean
• VMAF: Video Multi-Method Assessment Fusion: http://techblog.netflix.com/2016/06/toward-practical-perceptual-video.html
  github.com/Netflix/vmaf/commit/f2661673a078718ddfc1ddb6042048c1b1284946
  
  • Command used: vmaf --json --model version=vmaf_v0.6.1 -r [original-y4m] -d [decompressed-y4m] -o [json-output]
  • Uses pooled_metrics.vmaf.mean
  • [original-png] and [decompressed-png] were converted to [original-y4m] and [decompressed-y4m] with ffmpeg -y -i [png] -pix_fmt yuv444p -vf scale=in_range=full:out_range=full [y4m] (using ffmpeg (5.0.1-3+build1)).

Notes

Updated: 2022-10-04

This page is based on eclipseo.github.io comparison page. Originally developed by xooyoozoo. A list of sources for the images can be found in this text file. The source images were stripped of any metadata, EXIF, XMP, color profile etc. including the gAMA PNG chunk.

LICENSE