Go Defer and Named Returned Values

May 12, 2020

5 mins read

In this blog post we’ll be exploring the powerful combination of using go’s builtin defer statement and named return values.

Individually defer is great almost all the time; however, name returned values in my opinion are a bit of a problem child, I rarely use them, but in certain cases as the one I’m about to show you I find them helpful.

Let’s look at the use case.

Let’s say we are working on an API that has some async tasks, for example, a speech-to-text action given an mp3 file. Instead of having the client wait for us (our imaginary API) to process the file which could take minutes if not hours we simply create a job, pass it off to a queue, and return the job in the response. The client can then poll for the job status, or, since we’re building an awesome app and we’re awesome, we actually provide webhooks to keep them updated on the job status.

Now you’re asking yourself what does this have to do with defer statements and named returned values? Well, I’m glad you asked let me show you.

Let’s say we have a function that processes our jobs. This function will start off simple, it’ll receive a job and process it. We have a fake speechToTextService that takes a *os.File to process. Yes, I know an io.Reader in this case would be better but, for the sake of this example we’re sticking with a *os.File.

type Job struct {
    ID string
    Status string 
    Payload []byte
}

func processJob(j *Job) error {
    f, err := os.Create("myfile.mp3")
    if err != nil {
        return err
    }

    if _, err := io.Copy(f, bytes.NewReader(j.Payload)); err != nil {
        return err
    }

   if err := speechToTextService.Process(f); err != nil {
        return err
    }

    return nil
}

Simple enough we create a file, copy the contents to it, and send it off to the service to process.

Now, let’s add the part where we handle notifying the user via webhooks, and update the status of the job. For this we will need 2 new fake services jobStorage and queue. So we can imagine their’s a listener on the queue waiting for job status updates, and then sending off the webhooks.

func processJob(j *Job) error {
    f, err := os.Create("myfile.mp3")
    if err != nil {
        failedJob, err := jobStorage.UpdateStatus(j.ID, "failed")
        if err != nil {
            return err
        }
        
        if err := queue.Push("update_job_status", j); err != nil {
            return err
        }

        return err
    }

    if _, err := io.Copy(f, bytes.NewReader(j.Payload)); err != nil {
        failedJob, err := jobStorage.UpdateStatus(j.ID, "failed")
        if err != nil {
            return err
        }
        
        if err := queue.Push("update_job_status", j); err != nil {
            return err
        }

        return err
    }

   if err := speechToTextService.Process(f); err != nil {
       failedJob, err := jobStorage.UpdateStatus(j.ID, "failed")
        if err != nil {
            return err
        }
        
        if err := queue.Push("update_job_status", j); err != nil {
            return err
        }
        return err
    }

    completedJob, err := jobStorage.UpdateStatus(j.ID, "completed")
    if err != nil {
        return err
    }
        
    if err := queue.Push("update_job_status", completedJob); err != nil {
        return err
    }

    return nil
}

What just happened to our simple little function?! Sad I know. How can we turn this into a happy story again? Well, you might have guessed it. Using defer and named return values! Let’s try our function again.

First, let’s just add the defer statement.

func processJob(j *Job) error {
    var err error
    var f *os.File

    defer func(){
        status := "completed"
        if err != nil {
            status = "failed"
        }

        updatedJob, err := jobStorage.UpdateStatus(j.ID, status)
        if err != nil {
            return err
        }
            
        if err := queue.Push("update_job_status", updatedJob); err != nil {
            return err
        }
    }()

    f, err = os.Create("myfile.mp3")
    if err != nil {
        return err
    }

    if _, err = io.Copy(f, bytes.NewReader(j.Payload)); err != nil {
        return err
    }

   if err = speechToTextService.Process(f); err != nil {
        return err
    }

    return nil
}

This is already much cleaner but the code doesn’t do what we expect. Within the defer statement if something fails the error won’t be returned. The way to solve this is we need to use named return values. One thing to keep in mind with this approach is we need to use error wrapping if not any error within the defer func would override the error in the inner function.

We can clean up our code even further by extracting the defer func into its own function.

func processJob(j *Job) (err error) {
    defer func() {
        if err1 := updateJob(j.ID, err); err != nil {
            err = fmt.Errorf("%v %w", err1, err)
        }
    }

    f, err := os.Create("myfile.mp3")
    if err != nil {
        return return fmt.Errorf("failed to create file %w", err)
    }

    if _, err := io.Copy(f, bytes.NewReader(j.Payload)); err != nil {
        return fmt.Errorf("failed to copy job payload %w", err)
    }

   if err := speechToTextService.Process(f); err != nil {
        return fmt.Errorf("failed to process speech %w", err)
    }

    return nil
}

func updateJob(jobID string, err error) error {
    status := "completed"
    if err != nil {
        status = "failed"
    }

    updatedJob, err := jobStorage.UpdateStatus(jobString, status)
    if err != nil {
        return fmt.Errorf("failed updating job status %w", err)
    }
            
    if err := queue.Push("update_job_status", updatedJob); err != nil {
        return fmt.Errorf("failed pushing job to queue %w", err)
    }

    return nil
}

There are some caveats to this approach as we lost error wrapping ability for the errors within the defer function. In most use cases I find it to not be an issue, but as with almost everything in programming the decision will come down to your specific use case.

We could just use the = for assignment and a simple return instead of return err since we’re using it as a named return value, I like to be explicit though as I find it makes reading the function easier. Either way it’s just preference so it doesn’t really matter.

Take it easy, and thanks for reading!