Streaming Images from MongoDB using Golang

Photo by Suganth on Unsplash

Recently I had a client that has a fairly large image library stored in MongoDB using GridFS. I was building them an application which provided an interface between their front-end website and their new back-office system, which provides accounting services among other things. Part of this application needed to retrieve images from this MongoDB system and stream them to the browser so their front-end website could use them.

Now, if you aren’t familiar with MongoDB, it is a document-based database with powerful querying capabilities. GridFS is a system designed for storing files larger than the 16MB limit imposed on documents in MongoDB, and it is a part of MongoDB. So, for example, if I need to define an image, it may look something like this in JSON format, which just happens to be how one works with MongoDB data (well, BSON, but let’s not get technical here).

We’ll use this sample moving forward to represent an image record for demonstration purposes. This sample should be pretty familiar looking for anyone familiar with the JSON format. Most of this would be expected information in an image record in a database. What is more interesting is the fileID key and value. This is an ID that points to a record in the GridFS system. We’ll use this to get the actual image contents when it comes time to stream them out to the caller.

So let’s get started with some code! For these samples I’ll be using the Echo framework. Although Go has an awesome standard HTTP library for building web applications, I find Echo helps me reduce some of the boilerplate I find myself doing over and over. We’ll also use a library for interacting with MongoDB. I won’t go into great detail on how any of these frameworks work specifically, as there are plenty of resources out there which do a far better job than I can!

The first thing we need to do is define a structure to hold our image. This structure will describe the image with useful metadata, but more importantly it will have a reference to the ID where the file is stored in GridFS.