Hey guys, and welcome back!
Have you ever thought why don’t we compress bodies before for example sending them to client or server? Compressed data is smaller, so it takes lower bandwidth and transmission speed increases, right? But why it isn’t very common? Stay with me! We’ll find out later in this story, Let’s jump into it…
The answer!
Do you know what kind of resource is needed to do compression? Yes, this process requires CPU and that means additional load on your server. Imagine a server that already handles hundreds of thousands of requests, and the average body size is between 2KB to 10KB. It’s kind of small right? For high-traffic servers compressing such a tiny body is not critical and won’t help you very much.
Similarly, the client side also needs computational resources to do decompression. For devices that have limited processing power, this could be a drawback!
While compressing request or response body is worthless, it doesn’t mean that we don’t need compression. In general, we have two types of content — Text based and already compressed.
- HTML, CSS, JavaScript and Json are examples of text-based, and compression is more beneficial.
- Images, Videos and certain binary formats are already compressed and won’t benefit much from additional compression.
Text-based content often contains a lot of repetition. common words, phrases and structures appear frequently (like a HTML file that is full of duplicate tags) and can be encoded more efficiently.
A little more about compression…
Compression reduces data size, enabling faster network transmission and requiring less disk storage. For this we have two forms of compression — Lossy and Lossless. The goal is the same, but it’s done in different ways.
- Lossless compression algorithms focus on reducing repetitive patterns. This process starts by recognizing repeated patterns, and then based on what algorithm you are using, those repeated patterns are removed. In this technique, compression ratio isn’t very much.
- Lossy compression algorithms permanently remove some unnecessary information. Generally, these algorithms aim to maintain acceptable quality that is less noticeable to humans, though reduced quality may be evident in certain scenarios.
As you may guess we never use lossy compression for our text-based content because any loss for example in our HTML file could lead to significant rendering issues, distorted layouts, or even missing crucial elements like images or embedded scripts.
But for images, audios, or videos we have a different story because by losing information, quality reduces but our content won’t be unusable.
Hopefully you don’t need third-party packages this time!
Zlib, as you may be familiar, is a built-in package in Nodejs that includes functionalities for this purpose, and you can do compression with different algorithms. You have to notice that all the algorithms that Zlib supports, are lossless.
In all the examples bellow I’m going to use big.txt file and you can download it from here :)
1. Brotli
A lossless algorithm that is developed by google, commonly used for text-based data. Brotli-compressed files are usually identified by .br extension.
const fs = require('node:fs');
const zlib = require('node:zlib');
let resource = fs.createReadStream(__dirname + '/big.txt');
let destination = fs.createWriteStream(__dirname + '/big.br');
let compress = zlib.createBrotliCompress();
resource.pipe(compress).pipe(destination);▶️ Input file size: 6.3MB
▶️ Output file size: 1.7MB
2. Deflate
This is also a lossless algorithm like Brotli, and it is used for various types of data. Files compressed with the Deflate algorithm are usually identified by the .gz extension, while ZIP archives, which often use Deflate, are identified by the .zip extension.
const fs = require('node:fs');
const zlib = require('node:zlib');
let resource = fs.createReadStream(__dirname + '/big.txt');
let destination = fs.createWriteStream(__dirname + '/big.gz');
let compress = zlib.createDeflate();
resource.pipe(compress).pipe(destination);▶️ Input file size: 6.3MB
▶️ Output file size: 2.2MB
3. Gzip
A lossless compression utility that uses the Deflate algorithm. It is commonly used for compressing various types of data, especially in Unix and Linux environments. Compressed files with Gzip are typically identified by the .gz extension.
const fs = require('node:fs');
const zlib = require('node:zlib');
let resource = fs.createReadStream(__dirname + '/big.txt');
let destination = fs.createWriteStream(__dirname + '/big.gz');
let compress = zlib.createGzip();
resource.pipe(compress).pipe(destination);▶️ Input file size: 6.3MB
▶️ Output file size: 2.2MB
Conclusion
Alright! I believe this covers everything needed. If you found this article useful, please support me. Happy coding!
Resource
Wiki —lossy compression
Wiki — lossless compression
Adobe — lossy vs lossless compression
Nodejs — Zlib Documentation
