Writing an Interval-based Worker Pool using Generics in Go

Adam Presley
4 min readJun 14, 2023
Photo by Yk K on Unsplash

I recently needed the ability to pull data from a database at a specific interval and process that data. Not only that, but I also wanted to spread the processing to different goroutines (workers) in case the other goroutines were busy, much like a worker pool. This article will describe the code I wrote to achieve this task.

The Basics

First, let’s cover the basics by defining what we are building, then talking about the most basic data structures we need.

To start, I knew that I wanted to process work from a database table at an interval. To do this, I knew I’d probably use Go’s time.Ticker struct. For every N interval, the ticker will write to a channel that we can consume and do what we want.

Now I knew that I needed to define a basic unit of work. At a minimum, we’ll need to know what data we are working with and a function to handle that data for processing. I decided to use generics to make this component as general-purpose as possible. Here’s what that looks like.

--

--

Adam Presley

Just a guy who loves his wife, kids, and writing software.