What is Image Transcoding?

Image Transcoding is the process of converting JPG or PNG images to make file sizes smaller, thereby reducing the time it takes to send them across the Internet, or to take advantage of certain capabilities that some formats provide that others don't - for example, a byte reduction with newer image formats.

The Instart platform has a set of optimizations that allow for many different transcoding approaches that take advantage of modern compression approaches, device-specific image formats, and the use of modern image analysis technology - SmartVision - to ensure that images are adapted to provide the best performance and quality possible. The system takes many factors into consideration when transcoding images, such as the source format, the actual visual content of the images, the quality levels, and even the requesting device's capabilities.

There are four main types of optimizations done by the service:

  • Lossless optimization while keeping the source image format the same
  • Lossy transcoding of JPGs
  • Lossless transcoding while using more modern image formats
  • Lossy transcoding while using more modern image formats

Transcoding JPG and PNG images in place allows a reduction in image data size in a number of ways, depending on the type. The data is removed in place and the resulting file retains the same name, but the overall data transfer per image is smaller. This type of transcoding is generally always used, as it will make images smaller, even if only slightly.

Additionally, we support transcoding to more modern image formats such as WebP and JPEG XR. In this case the image retains the same name but its source format changes. This offers even better data size reduction, but such images can at this time only be displayed by Google Chrome in the case of WebP, or Microsoft Internet Explorer 10 and above in the case of JPEG XR.

Lossless optimization while keeping the source image format the same

Optimizing JPG and PNG images in place allows reducing image data size by removing unneeded metadata and applying improved lossless compression methods. The data is removed in place and the resulting file has the same name and remains in the same source format. This type of optimization should generally always be used, as it makes images incrementally smaller.

Lossless JPG and PNG optimization can each be enabled or disabled independently.

Lossy transcoding of JPGs

JPG images can also be configured to be transcoded in a lossy manner, if desired, to further reduce data with a minimal effect on quality. (PNG is a lossless format. You can only adjust the quality of a PNG image by transcoding it to WebP or JPEG XR, as described below.)

To do this, we set a quality threshold and a size threshold. We then reduce the quality level of any images that have an initial image quality greater than the quality threshold and are also larger than size threshold.

For example, if the quality threshold is set to 80%, and the size threshold to 500,000 bytes, an image that has a quality level of 100% and is 1 MB in size will be transcoded to a quality level of 80%, while another image that has a quality level of 100% and is 480,000 bytes in size will be left at the original quality level.

Lossless transcoding to modern image formats

JPG/PNG image transcoding to WebP

Images can be transcoded from JPG and PNG formats into a more modern image format, WebP. WebP images can display the same image quality as JPGs and PNGs using significantly less data when the service detects that Google Chrome (which is currently the only browser capable of displaying the format) is the requesting user agent.

JPG/PNG image transcoding to JPEG XR

Images can be transcoded from JPG and PNG formats into a more modern image format, JPEG XR. JPEG XR images can display the same or better image quality as JPGs and PNGs using significantly less data when the service detects that Microsoft Internet Explorer 10 or above (which is currently the only browser capable of displaying the format) is the requesting user agent.

Lossy transcoding to modern image formats

JPG/PNG image transcoding to WebP

If SmartVision is enabled, it will be applied to WebP images transcoded from JPG. This is a lossy transformation.

If SmartVision is not enabled, transcoding of JPG to WebP is lossless by default. For both PNG and JPG images, the transformation can be configured to be lossy to further reduce data with a minimal impact on quality. If enabled, the quality reduction of images transcoded to WebP is performed in a more aggressive manner.

The quality of the resulting image can be configured by setting a transcoding quality percentage.

We can further specify that the alpha and/or non-alpha channels of PNG images can be lossy - we can specify either, both, or neither.

JPG/PNG image transcoding to JPEG XR

Image transcoding to JPEG XR is lossless by default, but if desired, it can be configured to be lossy to further reduce data with a minimal impact on quality.

The quality of the resulting image can be configured by setting a transcoding quality percentage.

We can further specify that the alpha channel of PNG images can also have its own transcoding quality percentage.

What is SmartVision?

SmartVision is a sophisticated image processing technology that automatically analyzes each image as it passes through the service. SmartVision uses cutting-edge computer vision technology to gain an understanding of each image and its unique attributes, such as the volume and distance of objects in the scene, the quality of the original image, and an understanding of the quality-of-experience curve as the image is progressively rendered for an end user. It analyzes the actual content of an image and intelligently determines how much of it can be initially streamed to the browser so that the user can recognize the image on their device without perceiving a degradation in image quality. This allows web applications to send less image data to the browser, reducing time-to-interaction to provide a full-fidelity experience moments later. SmartVision is used by default to make the best possible choices for each image, unless you want to use a fixed quality threshold for all images, which we also support.

The SmartVision processor first takes each image through a primary analysis pass, which produces many identification signatures for each image. The system divides the image up into a number of versions, each at a different quality level. The SmartVision processor then analyzes each of these quality levels against the original image to create information on how each version degrades across a number of image signatures.

At this point the system uses an advanced affinity propagation algorithm, together with a large manually classified image corpus maintained by the service, to identify the characteristics of each image. Examples of these characteristics are images which have zoomed-in close-ups of people or products, zoomed-out images of products with few solid colors, medium-zoomed product images with heavy patterns, or even zoomed-out architectural elements like a house or a broad city view.

Product close-up with complex pattern

Examples of product close-up photos with complex pattern

Person, medium range, low complexity

Examples of photos of persons, medium range, low complexity

External landscape, long view

Examples of external landscape photos, long view

Based on this information, the system builds a visual quality-of-experience map for each image that can be used to make a decision on the best balance between fidelity and performance.

Note

The maximum size of images that the SmartVision algorithm will be applied to is 4000 x 4000, or 16000 pixels. Images larger than this will be transcoded to a fixed quality level of 90% by default (this value is configurable).

What web performance metrics will be positively impacted by using Image Transcoding?

Our testing has found that Image Transcoding can provide the following benefits to the following web performance metrics:

MetricDescription
DomContentLoadedThis event is fired after static image tags are processed and loaded. Since we reduce the image bytes for static image tags, this positively impacts the DomContentLoaded metric.
Document Complete (onload) time and content reductionThe onload event is generally the best indicator of the first paint of the screen. Since image transcoding reduces the image bytes, it reduces the total amount of data transfer required and hence the time required to reach the onload event.

Where does Image Transcoding have the most impact on performance?

This feature benefits sites that have a substantial number of images on the page, and in situations where the number of image bytes is the bottleneck for fast page load performance. Generally we find that this feature works best when the images are over 4 KB in size and you have at least 10 images on the page.

This feature generally shows best when the images are static references in HTML IMG tags, as this impact will be felt before the onload event.

Where does Image Transcoding have the least impact on performance?

Sites with very few JPG and/or PNG images or a large number of small JPG and/or PNG images (less than 4kb) will generally not benefit much from this feature.

We find that text- and code-heavy sites like a SaaS application will receive little to no benefit from this feature due to the fact they typically have minimal images.

What do I need to do to enable Image Transcoding?

Image Transcoding does not require changes to the backend web servers nor to the code that makes up the web site or application. As your web traffic flows through the Instart platform, the service will automatically transcode and cache JPG and PNG images and begin sending them to requesting browsers.

All that is required is that your configuration be set up to enable Image Transcoding on your site. Like all of our features, it can be selectively enabled for certain hosts, specific directories, or under certain conditions.