Hi everyone! Recently, I’ve been working with Minio, an open-source and S3-compatible object storage solution. It boasts numerous features that can fulfill your needs in most cases. That’s why I’ve decided to discuss these types of storage more and explore their practical utility.
Object storage, which is a relatively new technology, plays a critical role in our world. Nowadays, nearly every website has at least one object storage on the back-end side, and there are many well-known cloud-native providers, such as AWS, Dropbox, or Google Drive, which you likely interact with daily. We have different types of data storage, such as object storage, file storage, and block storage. Each has its use case, but now, we’re going to dive into object storage in more detail.
💡The idea of storing data as objects
Object storage keeps your videos, images, backup files, or any kind of unstructured content in buckets and processes them in a fast way. Think of a flat environment where our data stores as objects. Each object includes three main parts:
- ID (a unique ID instead of path)
- Data (actual data)
- Metadata (more information about the object)
Each file is assigned to a unique ID. So you can point to an object with the help of IDs. The Data is the actual content that you stored. Each object also has metadata that you can filter objects based on it or attach useful information to use in your program later.
🗣️How to talk to an object storage?
Well, this is another advantage of object storage that is also the main reason why cloud providers typically offer it. With the support of HTTP-based APIs, object storage is really good for web apps which enables them to easily integrate. In that way, by just calling API you can manager your files and access to them from everywhere.
⚒️ Comparing to File storages
Hierarchical storage which is known as file storage, organizes your files in a hierarchy of directories and sub-directories which you can see on computer hard drives.
Obviously you can just access to your files with paths and there is no flat environment anymore. File storage just supports NFS and SMB protocols and is limited in terms of scalability.
🛡️Bucket Versioning
A beautiful concept in common object storages is versioning. When you write to a bucket with the same unique ID, two things can happen:
- New object takes the place of the previous one and the previous object is destroyed
- A new version is created, and the previous version of the object is automatically preserved
So with this feature you have always an “undoing” option for write operations.
Deleting a versioned object (without specifying the version ID) is not permanent and the latest version is just marked as deleted. To permanently delete you have to also specify the version ID.
⚠️Keep in mind that based on your provider, the number of versions that is preserved from an object can be different.
🔒Object locking
This option level-up the immutability of objects and enforce write-once read-many (WORM) to protect them from deletion. In Minio and AWS S3, bucket versioning is also required to be enabled. Deleting an object (without specifying the version ID) is possible but as you may guess is not permanent. But deleting a specific version is not possible until the locking expires.
🚀Scalable for ever!
Object storages come with a lot of scalability for multiple reasons.
- With using a flat address which eliminates the need for a hierarchical structure in tradition ways, each object is assigned to a unique identifier. So each object is accessible directly with just an ID.
- With adding more nodes to your system, you can scale you system horizontal. With distributing objects, the load is evenly balanced across these nodes.
- With a rich metadata that each object can have, efficient data organization and retrieval can be leveraged. When distribution is combined with metadata indexing, allows for quick access to specific objects.
- Many object storages support geographic distribution, allowing data to be stored across multiple locations. This not only enhances data durability and availability but also enables organizations to scale their storage infrastructure globally.
These characteristics make object storages well-suited for cloud environments where scalability is a fundamental requirement.
I hope it was helpful and you enjoyed it, goodbye! 👋
Resources
Object storage — Wikipedia
What is file storages? | IBM
Object Storage: An Introduction | IBM
Minio — Bucket Versioning
