Managing Your $GOPATH for Multiple Go Projects

This is a cool tip I picked up from checking out other peoples' Go projects.

When you're new to Go, the documentation tells you about $GOPATH which tells Go where to install packages and where the source codes to your project and its dependencies live. A lot of people might set $GOPATH to be $HOME/go, and work on their projects out of ~/go/src/github.com/myname/myproject.

Using one global GOPATH like this has its problems though:

• You have to git clone your projects into ugly directory paths to make sure they're inside the GOPATH. I usually prefer to just have a folder like ~/git/myproject instead, and I don't want to symlink it into my GOPATH so that Go finds it.
• The third-party dependencies of all of your Go projects get intermingled in the same GOPATH. This is especially problematic if two different projects need the same third-party dependency, but they use different versions of that dependency.

So, the solution I found to this problem is to use a Makefile for your project that creates a private GOPATH for you. Here's an example Makefile from one of my projects:

CURDIR = $(shell pwd)
GOPATH = "$(CURDIR)/.gopath"
all: build

# `make setup` to set up git submodules
setup:
	git submodule init
	git submodule update

# `make run` to build and run the bot
run: gopath
	GOPATH=$(GOPATH) GO15VENDOREXPERIMENT=1 go run cmd/scarecrow/main.go

# `make fmt` runs gofmt
fmt:
	gofmt -w src

# `make build` to build the binary
build:
	GOPATH=$(GOPATH) GO15VENDOREXPERIMENT=1 \
		go build -i -o scarecrow cmd/scarecrow/main.go

# Sets up the gopath / build environment
gopath:
	export GO15VENDOREXPERIMENT=1
	mkdir -p .gopath/src/github.com/aichaos/scarecrow bin
	ln -sf "$(CURDIR)/src" .gopath/src/github.com/aichaos/scarecrow
	ln -sf "$(CURDIR)/vendor" .gopath/src/github.com/aichaos/scarecrow/src

My project's directory structure is laid out like this:

/
  cmd/
    scarecrow/
      main.go
  src/
    scaregrow.go
    utils.go
    configs.go
    logging.go
  vendor/
    github.com/
      aichaos/
        rivescript-go/...
      mattn/
        go-xmpp/...

With this project layout, the bulk of my source code is under the /src directory (to keep the root of the repo clutter-free), the actual executable program is under the /cmd directory, and third-party dependencies are under /vendor (using the Go 1.5 vendor experiment).

Instead of running go run cmd/scarecrow/main.go, I type make run. My Makefile then generates a custom private GOPATH at /.gopath, makes symbolic links to all the relevant files underneath, and runs the program from the context of that GOPATH.

The ~/.gopath directory structure looks like this:

/.gopath
  /src
    /github.com
      /aichaos
        /scarecrow
          /src -> ../../../../../src (link)
            /vendor -> ../../../../../../vendor (link)

This way the private GOPATH has all the necessary directory structures that Go needs to find my project's source code and its vendored modules, and I don't have to clutter my global GOPATH. Also, this makes it easy to just git clone my project any place I want, like ~/git/scarecrow and not worry too much about the Go configuration.

As a side note, with the Go 1.5 vendor experiment I used git submodules to add the third-party dependencies to my project. On a fresh git clone I just type make setup which initializes and clones the submodules.