Go: init() and organizing your webapp

So you got an API written with Golang. It's getting pretty big with multiple endpoints, a datastore and a bunch of complex services. Even worse, you might have more than one dev working on it, doing necessary refactoring on the codebase, and running into painful merge conflicts. Perhaps it's time to think about how you got there and see what Go has to help architect a solution? This used to be a pain point for me and Go. It’s made it easy to hack out a web API, but keeping things organized was difficult and the thought of making things reusable felt completely out of reach.

Luckily Go was created by some seriously smart guys, and after spending a considerable amount of time working with it, I can say it is definitely the language that keeps giving with its rich and intentional feature set.

Enter the init() function and how it can help us create a familiar, reusable and approachable architecture for a Go API.

Initialization In Go: The Basics

init() is just a small part of the bigger picture of the initialization process in Go. Initialization helps us take care of two things, defining the order in which structs and variables are defined within our packages across the entire project.

The effective go docs on initialization provide a good overview of what happens in a few scenarios but really fall short of explaining the bigger picture of initialization and what it means for organization and the reduction of boilerplate code throughout a well architected Go project.

First, at compile time all constants will be evaluated; nothing new here but still important to keep in mind. At runtime, when a package is included, all the variables and structs in the package will be evaluated. This includes variables who are defined as functions.

var WhatIsThe = foo()
func foo() string {
    return "bar"
}

Next, it will run the init() functions contained in that package. init is case sensitive Init() will not be evaluated until it is called while init() will be evaluated when the package is imported and after the structs/variables are evaluated.

Keep in mind that it is usually best to not have more than one init() function per package, it is hard to determine which init function will execute first reliably which will hurt the overall stability of the package.

Initialization In Go: The Implication

With a quick understanding of initialization we can immediately start making some more informed decisions about how we approach scaffolding a scalable codebase. It starts with a separation of responsibilities and enforcing those with centralized initialization across your packages. Let's take a look at an example API application:

server.go

package main
import (
  "github.com/AuthenticFF/GoExample/controllers"
  "github.com/AuthenticFF/GoExample/db"

  "net/http"
  "os"
  "log"

  "github.com/julienschmidt/httprouter"
)

func main() {
  router := httprouter.New()
  defer db.Session.Close();

  router = controllers.Init(router)
  port := os.Getenv("PORT")
  if port == "" {
    port = "9091"
  } 
  log.Println("Authentic Form & Function (& Framework) listening on %s", port)
  http.ListenAndServe(":"+port, router)
}

The simplicity of the main function here is very deceptive. At first glance we have a pretty simple webapp, and that is exactly what we want. This file is concise and its code communicates clearly what it's doing. It looks like we have a http router with a DB and we are listening and serving connections on port 9091. Right from the get go we are trying to keep the functions of each file simple, concise and obvious to anyone who is approaching this project.

db/db.go

package db
import (
  "log"
  "os"

  "gopkg.in/mgo.v2"
    )

var Session *mgo.Session
var mongohosts = "goexample_database_1"

func init() {
  log.Println("Datastore Initialized");
  var err error
  Session, err = mgo.Dial(mongohosts)
  if err != nil {
    log.Fatalf("Error connecting to the database: %s\n", err.Error())
  }
  Session.SetMode(mgo.Monotonic, false)
    Session.DB(os.Getenv("DB_NAME"))
}

Here is the database configuration and initialization file. We can quickly see that we are using MongoDB for the datastore and can notice the init() function. Looking at the package variables we can see that "Session" is capitalized while "mongohosts" is not, therefore we know that Session is used by and exposed to packages that import the db package.

When this package was imported into server.go its init method was called before server.go's main function resulting in Session being defined allowing "defer db.Session.Close()" to be called without error. Providing our application with a fully initialized database with only a single line of code outside of our db package.

controllers/controller.go

package controllers
import (  
  "github.com/AuthenticFF/GoExample/services"

  "net/http"
  "encoding/json"
  "log"
  "fmt"

  "github.com/julienschmidt/httprouter"
)

var Example exampleController

func Init(router *httprouter.Router) *httprouter.Router {
  Example = exampleController{services.Example, services.DB}
  router = Example.Init(router)
  log.Println("Controllers Initialized");
  return router
}

/*
Helper functions/structs
*/
type httpStatus struct {
  err    error
  status int
}
func ServerError(err error) httpStatus {
  return httpStatus{err, http.StatusInternalServerError}
}

func StatusOk(status int) httpStatus {
  return httpStatus{nil, status}
}

type controllerRoute func(http.ResponseWriter, *http.Request, httprouter.Params) (interface{}, httpStatus);

func PublicRoute(r controllerRoute) httprouter.Handle {
  return func(w http.ResponseWriter, req *http.Request, p httprouter.Params) {
    result, status := r(w, req, p)
    WriteResponse(w, result, status)
  }
}

func WriteResponse(w http.ResponseWriter, result interface{}, httpStatus httpStatus) {
  var responseBody string
  w.Header().Set("Content-Type", "application/json")
  w.Header().Set("X-Powered-By", "Authentic F&F")
  w.Header().Set("Strict-Transport-Security", "max-age=31536000; includeSubDomains")
  if httpStatus.err != nil {
    w.WriteHeader(httpStatus.status)
    jsonBody, _ := json.Marshal(httpStatus.err.Error())
    responseBody = string(jsonBody)
  } else {
    w.WriteHeader(httpStatus.status)
    jsonBody, _ := json.Marshal(result)
    responseBody = string(jsonBody)
  }
  fmt.Fprintf(w, responseBody)
}

This is the largest file mostly due to a few helper functions and structs. Notice how we have an Init function with a capital "I" this means that it will not be automatically called on import and will have to be executed explicitly (see server.go). This is due to the fact that our controllers need to passed a router in order to define their routes. This could be avoided by defining our routes within the main package but that will make it a pain to look up routes and their associated functions by having to toggle between packages.

As the project grows you will quickly find yourself with a huge list of route declarations within the main package that will be awkward to work with. Define these routes right next to their functions and increase your development speed while staying modular.

Looking at our imports we can now see a new package "github.com/AuthenticFF/GoExample/services" we also appear to be passing variables from that package into some of our controller structs. Lets see whats going on in there.

services/service.go

package services

import (
  "github.com/AuthenticFF/GoExample/db"

  "log"
)

var DB DBService
var Example ExampleService

func init() {
  Example = ExampleService{}
  DB = DBService{db.Session}
  log.Printf("Services Initialized");
}

We have two services defined, Example and DB. We are importing "github.com/AuthenticFF/GoExample/db" and passing db.Session to the DB service. Notice how that the init function in the db package will not be called twice, how cool is that?! Let's quickly look at the two services that we are initializing here.

services/exampleservice.go

package services
import (
    "github.com/AuthenticFF/GoExample/models"
)

var exampleService IExampleService
type IExampleService interface {
    GetData() (models.Result, error)
}
type ExampleService struct {}

func (s *ExampleService) GetData(result models.Result) (models.Result, error){
  /* do something neat */
  result.Data = "Lorem Ipsum"
    return result, nil 
}

services/dbservice.go

package services
import (
  "github.com/AuthenticFF/GoExample/models"

  "gopkg.in/mgo.v2/bson"
  "gopkg.in/mgo.v2"
)

var dbService IDBService
type IDBService interface {
  SaveResult(newResult models.Result) (models.Result, error)
}
type DBService struct {
  session *mgo.Session
}
func (s *DBService) SaveResult(newResult models.Result) (models.Result, error) {

  // Add an Id
  newResult.Id = bson.NewObjectId()
  s.session.DB("goExample").C("result").Insert(newResult)
  return newResult, nil

}

Both of these are pretty straightforward and because we have handled initialization and broadcasting of these interfaces in our services.go file we are free to focus on the actual functionality within their files. We also import a new package "github.com/AuthenticFF/GoExample/models" where we will be defining the structs that will be manipulated accross the application.

models/result.go

package models

import (
    "gopkg.in/mgo.v2/bson"
)

type (  
  Result struct {
    Id          bson.ObjectId       `json:"id" bson:"_id"`
    Data        string    `json:"data" bson:"data"`
  }
)

Models are kept separate from the controllers services and db to provide them protection from being changed without consideration. Isolating them in their own package clearly indicates that they are used in many other packages. For this reason only define a struct in "github.com/AuthenticFF/GoExample/models" when it is used by two or more packages. If it isn't then put its definition in the package that it is used.

Now we are going to head back to the controllers package and see how actual routes are defined.

controllers/example.go

package controllers

import (
  "github.com/AuthenticFF/GoExample/models"
  "github.com/AuthenticFF/GoExample/services"

  "net/http"

  "github.com/julienschmidt/httprouter"
)
type exampleController struct {
  exampleService services.ExampleService
  dbService services.DBService
}

func (c *exampleController) Init(router *httprouter.Router) *httprouter.Router {
  router.GET("/api/example", PublicRoute(Example.Analyze))

  return router
}
/*
Main API method
*/
func (c *exampleController) Analyze(writer http.ResponseWriter, req *http.Request, params httprouter.Params) (interface{}, httpStatus) {
  var result models.Result;
  var err error;

  result, err = c.exampleService.GetData(result)
  if err != nil {
    return nil, ServerError(err)
  }

  result, err = c.dbService.SaveResult(result)
  if err != nil {
    return nil, ServerError(err)
  }


  return result , StatusOk(http.StatusOK)
}

Again we are keeping things as modular as possible. This file defines a single route and its associated handler method. The Analyze method doesn't actually do much analysis and instead relies on our services to do any significant processing. Our architecture allows anyone adding or modifying routes to touch as few packages as possible and also keeps a rigid separation of responsibilities in the app.

Initialization In Go: The Conclusions

Using initialization and packages effectively allows you to enforce architecture patterns through the initialization process and also by the code conventions you initially establish in your project. Leaving you with a modular and understandable codebase that relies on good naming/documentation to understand the app from a high file structure level. This avoids needing someone to have spent days(or weeks) working on the project to be familiar with how things work and what happens where.

This architecture enforces a strict separation between http routing and database interactions. Why do this from a technical point of view? So that if you need to you can easily apply different patterns of concurrency for processing(services), storing(db) and serving(controllers) data.

Go provides you with the toolset to build really powerful and fast webapps. It is up to developers to make sure they are architecting their apps correctly to make sure they can reap the benefits of its feature set.

The complete project is available at https://github.com/AuthenticFF/GoExample feel free to give it a try! I have also included a dockerfile in that repo to make it easier to spin up and to let you spend more time working than configuring. = )