Introduction Link to heading
Go is a fantastic language with remarkable primitives, and after using it for a while, I’ve come to especially love the concurrency model. It’s simple, understandable, and allows one to write concurrent code that is both performant and easy to reason about. Today, we will talk about one of the core components of the Go concurrency model - channels.
What is a channel? Link to heading
From the Tour of Go:
Channels are a typed conduit through which you can send and receive values
Essentially, it’s a way to communicate between goroutines through an easy-to-use IPC (inter-process communication) mechanism. Since we can’t share memory between goroutines, channels are the primary way to pass data between them.
There are two types of channels in Go:
- Non-buffered channels
- Buffered channels
Why use channels? Link to heading
Let’s take an example of a super simple web server that receives a request and sends back an “OK”:
package main
import (
"fmt"
"net"
)
const addr = "0.0.0.0:9876"
func main() {
listen, err := net.Listen("tcp", addr)
if err != nil {
fmt.Println(err)
}
fmt.Printf("Started server on %s\n", addr)
for {
conn, err := listen.Accept()
defer conn.Close()
if err != nil {
fmt.Println()
}
buffer := make([]byte, 1024)
// Read into the buffer
_, err = conn.Read(buffer)
if err != nil {
fmt.Println(err)
}
// Echo back to the connection
conn.Write("OK")
conn.Close()
}
}
Super simple. Now, let’s add a requirement that we want to log the request body to a file, but let’s make it extremely inefficient - let’s make it write one character at a time to the file.
We’ll add a function to do this:
func stupidlyLogToFile(log []byte) {
// Stupidly log out each character one by one to the file
for i := 0; i < len(log); i++ {
// Create the file if it doesn't exist
if _, err := os.Stat("log.txt"); errors.Is(err, os.ErrNotExist) {
os.Create("log.txt")
}
// Open it for writing
f, err := os.OpenFile("log.txt", os.O_APPEND|os.O_WRONLY, 0600)
if err != nil {
fmt.Println("failed to open file")
}
defer f.Close()
// Write out the single byte
if _, err := f.Write([]byte{log[i]}); err != nil {
fmt.Println("failed to to log file")
}
}
}
And then we’ll call it in our main function, and measure the execution time, returning the execution time back to the caller:
func main() {
listen, err := net.Listen("tcp", addr)
defer listen.Close()
if err != nil {
fmt.Println(err)
}
fmt.Printf("Started server on %s\n", addr)
for {
conn, err := listen.Accept()
start := time.Now() // <-- Start the timer
if err != nil {
fmt.Println()
}
buffer := make([]byte, 2048)
// Read into the buffer
num, err := conn.Read(buffer)
if err != nil {
fmt.Println(err)
}
// Log it to a file
stupidlyLogToFile(buffer[:num]) // <-- Log the request body
// Echo back to the connection
resp := fmt.Sprintf("Time to execute: %v\n", time.Since(start)) // <-- Get the execution time
conn.Write([]byte(resp)) // <-- Write the execution time back to the client
conn.Close()
}
}
Now, let’s test it out with a simple nc command:
117ms! Yikes. That’s a long time to wait for a response for something that is meant to be asynchronous.
Let’s see how we can use a goroutine and a channel to make this non-blocking. But first, let’s talk about how we create and use channels.
Creating channels Link to heading
Channels are created with the make keyword, and they can be typed or untyped. For example, to create a non-buffered channel that can pass strings, we would do:
ch := make(chan string)
Non-buffered channels are blocking, meaning that if we try to read from a channel that has no data, the goroutine will block until data is available. This is useful for synchronization between goroutines, but it can also be a source of deadlocks if not used properly.
If we want a channel that can hold a certain number of messages before blocking, we can create a buffered channel. For example, to create a buffered channel that can hold 10 messages, we would do:
ch := make(chan string, 10)
A buffered channel will hold messages until it is full, and then it will block until a message is read from the channel.
Using a channel Link to heading
To pass data to a channel, we use the <- operator. For example, to pass the string “Hello, world!” to the channel, we would do:
ch <- "Hello, world!"
To read data from a channel, we also use the <- operator. For example, to read the string “Hello, world!” from the channel, we would do:
msg := <- ch
Closing a channel Link to heading
Channels can be closed with the close keyword. For example, to close the channel, we would do:
close(ch)
A channel should always be closed from the sender, never the receiver. Closing a channel from the receiver can cause a panic if the sender attempts to send data through the closed channel.
Now that we have a basic understanding of channels, let’s revisit our example.
Using a channel to log to a file Link to heading
Let’s start by creating a simple struct to hold the channel:
type Logger struct {
// Logging channel to receive logs from
Channel chan []byte
}
And add two methods to it, startLogger and log:
func (l *Logger) startLogger() {
// Start the logger listening on the channel
fmt.Println("Starting logging channel")
for {
toLog := <-l.Channel
stupidlyLogToFile(toLog)
}
}
func (l *Logger) log(info []byte) {
// Send the info through the channel
fmt.Println("Logging info")
l.Channel <- info
}
startLogger will run a loop listening on the channel, blocking until data is available. Once data is available, it will call our previous stupidlyLogToFile function to log the data to a file.
log will send the data to the channel, and then return, giving us a simple interface to actually log out data.
Now, let’s update our main function to use the new logger, by initializing our logger with a new channel, and, most importantly, starting the logger in a goroutine:
func main() {
listen, err := net.Listen("tcp", addr)
defer listen.Close()
if err != nil {
fmt.Println(err)
}
fmt.Printf("Started server on %s\n", addr)
theLogger := &Logger{ // <-- Initialize the logger
Channel: make(chan []byte),
}
go theLogger.startLogger() // <-- Start the logger in a goroutine
for {
conn, err := listen.Accept()
start := time.Now()
if err != nil {
fmt.Println()
}
buffer := make([]byte, 2048)
// Read into the buffer
num, err := conn.Read(buffer)
// // Log it to a file <-- We don't need this anymore
// stupidlyLogToFile(buffer[:num])
theLogger.log(buffer[:num]) // <-- Use the logger instead
// Echo back to the connection
resp := fmt.Sprintf("Time to execute: %v\n", time.Since(start))
conn.Write([]byte(resp))
conn.Close()
}
}
With this new implementation, the server should no longer block while logging the data to the file, and will instead return immediately, allowing the client to continue, and the logger to log the data in the background.
Now, let’s test it out and make sure our logs got written
33.917µs - that’s a massive improvement over 117ms. And we didn’t have to change much of our code to do it - as long as we know how to communicate data across goroutines, we can prevent blocking on asynchronous tasks and keep our application responsive.
And that’s it! In an upcoming post, we’ll dive deeper into channels by using them to create a persistence engine for our Redis rewrite, incorporating a WAL (write-ahead log) and a snapshotting system to ensure data is persisted to disk.
You can find the code for this example in this article here