Go: Packages & Modules

Keywords

go, modules, packages, go.mod, semantic versioning, dependency management, minimal version selection, internal packages, project layout

Introduction

Before 2018, starting a Go project meant negotiating with a directory. Your code did not live where you put it; it lived under $GOPATH/src, in a path that had to mirror its eventual import path exactly — ~/go/src/github.com/acme/shop or nowhere. Clone a repo into the wrong folder and it would not compile. Worse, there was no notion of a version: go get fetched whatever was on a dependency’s default branch today and wrote it into the shared src tree everyone pulled from. Two projects needing two releases of the same library could not coexist on one machine, and the build you ran on Monday could quietly differ from the one CI ran on Wednesday, with nothing recording that they had.

The pain that finally forced the issue was the diamond. A team would add library A, which depended on a logging package at one release, and library B, which depended on the same package at a different release. There was no manifest to consult, no lock file to pin, no algorithm to adjudicate — just one folder and a coin flip. Whichever version was on disk won, and if the two were incompatible the program broke in a way no source file explained. People papered over it with vendoring tools — then with three competing ones, each with its own metadata format. The ecosystem had a dependency problem and a dependency-tool problem stacked on top.

Go modules, shipped in Go 1.11 and the default since 1.16, ended all of it at once. A project now declares its own identity and dependencies in a file at its root, lives anywhere on disk, and records the exact version of everything it builds against. Two design choices — below — make this not just convenient but boring, in the sense that matters for infrastructure: the same inputs produce the same build, every time, on every machine, and an upgrade happens only when you ask. Go modules made builds reproducible and versioning boring on purpose.

The Core Insight

A Go module is a versioned unit of distribution: a go.mod file at the root of a directory tree that names the module, states the Go version it needs, and lists the other modules it requires and at what versions. Inside that tree live packages — and the package, not the module, is the unit the compiler actually works with: one directory of .go files compiled together, and also Go’s boundary of encapsulation, where what a package exports is its public API and everything else is invisible from outside. The module is what you ship and version; the package is how you organize and hide code within it.

Two decisions distinguish Go’s system from nearly every other language’s, and the rest of the chapter is downstream of them:

Semantic import versioning. A module’s major version is part of its import path — version 2 of github.com/acme/shop is imported as github.com/acme/shop/v2, a literally different path from version 1. This sounds like bureaucracy but is what makes major upgrades safe: because v1 and v2 have distinct paths, one build can use both during a migration, and a breaking change can never silently replace the API you compiled against.
Minimal Version Selection (MVS). When parts of your graph require the same module at different versions, Go selects the lowest version that satisfies all of them, not the newest available — the opposite of the “newest-satisfying” rule npm and pip apply, and the source of Go’s reproducibility. The version set is a deterministic function of the require lines; a new upstream release changes nothing until you ask for it.

Together these make the build a pure function of the files you committed.

A mental model

Picture a module as a shipped box with a manifest taped to the lid. The box is the versioned thing you publish, tag, and depend on; the manifest (go.mod) lists what is inside and what other boxes it needs, each named with an exact version; and go.sum is the tamper-evident seal — a cryptographic fingerprint of every box you depend on, so one arriving with different contents than your manifest expected stops the build.

Inside the box are packages, each a folder with a public counter and a private back room: capitalize an identifier and it goes on the counter, lowercase it and it stays in the back, and callers see only the counter. The internal/ directory is a back room the compiler enforces — not a convention you hope people respect, but a rule it rejects your build for breaking. And the dependency graph is resolved not by a solver hunting for the newest mutually-agreeable versions but by the simplest rule that works: for each module, take the highest version anyone explicitly asked for, and no higher. That floor-not-ceiling rule is MVS, and it is why a Go build does not drift. Figure 17.1 shows the whole picture.

When to split packages and when to make a module

The two structuring decisions in Go — where to draw a package boundary and when to create a separate module — are governed by different questions, and conflating them is a common mistake.

Draw a package boundary around a responsibility and its public API. A package should have one clear job and expose the smallest surface that does it. The surest sign you have the boundary right is that the name reads well at the call site — order.New, payment.Charge — and the surest sign you have it wrong is a package named utils, common, or helpers: responsibilities-in-name-only that everything imports. Organize by feature (an order package owning its handler, service, and storage) rather than by layer (a handlers package, a services package), because feature packages keep related code together and minimize the cross-package imports that breed cycles.

Reach for internal/ whenever code is an implementation detail. It costs nothing and is enforced by the compiler: anything under internal/ is importable only by code rooted at that directory’s parent. Default application code there; promote only what outsiders must depend on.

Create a separate module only for something versioned independently. A web service and its worker that always ship together belong in one module with two cmd/ entrypoints; a library you publish on its own cadence for other teams belongs in its own module. Splitting buys independent versioning at the cost of coordination, so don’t pay for it until you need independent release cycles.

What you’ll learn

How a package serves at once as Go’s unit of compilation and its boundary of encapsulation, with capitalization alone deciding what is public, and why internal/ gives you privacy the compiler enforces rather than privacy by convention
What go.mod and go.sum each guarantee, and how together they make a build reproducible and supply-chain–verifiable
Why a module’s major version lives in its import path, and what that buys you when a dependency makes breaking changes
How Minimal Version Selection resolves a dependency graph, and why choosing the lowest satisfying version is what makes Go builds deterministic
The everyday workflow (go get, go mod tidy, vendoring, private modules via GOPRIVATE) and how to lay out a project with cmd/, internal/, and pkg/

Prerequisites

Go fundamentals: syntax, types, functions, and methods; how import brings a package’s exported names into scope; comfort running go build and go run
Command-line basics and Git, including the idea that a tag can name a release
A working Go toolchain (1.21 or newer)

Packages: the unit of compilation and encapsulation

Everything in Go is in a package, the smallest thing the compiler treats as a whole. One directory is one package: every .go file in it declares the same package name and they compile together, seeing each other’s declarations without any import. Cross that boundary and you need an import, and you reach only what the other package chose to make visible. The package is therefore two things at once — the grouping the compiler builds, and the wall encapsulation is built on.

Go’s visibility rule is famously minimal: no public, private, or protected keywords. An identifier is exported if and only if its first letter is uppercase. This applies uniformly to types, functions, methods, struct fields, and constants, and the mechanism is so simple it disappears into how you read code — a capital User is part of the API, a lowercase email field is the package’s own business.

package user

// User is exported — callers in other packages can see and use it.
type User struct {
    ID    int    // exported field
    Name  string // exported field
    email string // unexported: only the user package can touch it
}

// New is the package's public constructor.
func New(name string) *User {
    return &User{Name: name, email: derive(name)}
}

// derive is unexported — an internal helper, invisible outside this package.
func derive(name string) string { /* ... */ return "" }

The discipline that follows is to start everything lowercase and export deliberately. Exporting is a promise: once derive becomes Derive, another module may import it, and removing it later is a breaking change you owe them a major version for. Unexported-by-default keeps your API small and your freedom to refactor large. Package naming reinforces the same restraint — names are short and lowercase, no underscores or camelCase, because the name prefixes every call site and carries the meaning, so the identifiers inside need not repeat it: user.New, not user.NewUser.

`internal/`: privacy the compiler enforces

Capitalization controls visibility within a module — but the moment you publish, every exported identifier in every importable package becomes part of the contract others build on, which is often more than you meant to promise. You want some packages usable across your own codebase yet off-limits to outsiders, and that is exactly what internal/ provides — uniquely among Go’s structuring tools, enforced by the compiler rather than by convention.

The rule is precise: a package under a directory named internal/ is importable only by code rooted at internal/’s parent. github.com/acme/shop/internal/payment is reachable by anything under github.com/acme/shop/ and nothing else — an external module that tries gets a compile error, not a lint warning. This lets you expose a small, stable public surface while keeping the machinery behind it free to change. Implementation details that leak into a public path are a liability: someone imports them, and then your refactor is their breaking change.

Modules, `go.mod`, and `go.sum`

A module is declared by a go.mod file at its root, short by design. It names the module (its import-path prefix), states the Go version the code expects, and lists the modules it requires and at what versions. The go tool maintains it; you rarely edit it by hand beyond the occasional pin.

module github.com/acme/shop

go 1.22

require (
    github.com/gin-gonic/gin v1.9.1
    github.com/lib/pq v1.10.9
)

require github.com/go-playground/validator/v10 v10.16.0 // indirect

Two things deserve attention. The module line is the import-path prefix for every package in the tree — the internal/order directory is imported as github.com/acme/shop/internal/order, derived mechanically from this line plus the directory path. And the // indirect marker flags a dependency you don’t import directly but that something in your graph needs, so the manifest stays a complete account of the build.

The companion go.sum is where reproducibility becomes supply-chain integrity. For every module version your build touches it stores a cryptographic hash of the contents, and on every build Go verifies that what it fetched matches; a mismatch halts the build. This is the lock that turns “the same inputs produce the same build” from a hope into a guarantee, and the defense against a dependency being swapped out from under you: a tampered or republished version cannot slip in unnoticed, because its hash will not match. You never edit go.sum by hand — the tool writes it, and go mod verify checks it.

Note

The hash in go.sum is cross-checked against a global append-only transparency log (sum.golang.org), which lets the whole ecosystem detect if a published version’s bytes ever change after the fact — the module equivalent of certificate transparency.

Semantic versioning and the import path

Module versions are semantic versions — v1.4.2, the three numbers meaning breaking, feature, fix — and Go takes the contract literally: a minor or patch bump promises backward compatibility, a major bump declares it broke something. What makes Go unusual is the consequence it attaches to a major bump: the major version becomes part of the import path. Version 1 is imported at the bare path; version 2 and beyond append /v2, /v3, with the go.mod module line changed to match.

// v1 — the original path
import "github.com/acme/widgets"

// v2 — a different import path entirely, after breaking changes
import "github.com/acme/widgets/v2"

This is the “import compatibility rule,” and the reasoning is sharp. If two incompatible releases shared a path, a build that transitively needed both would be impossible, and a major upgrade would mean silently recompiling your code against an API that may have changed under it. By giving each major version its own path, Go makes v1 and v2 coexist as if unrelated packages: a migration moves one caller at a time, and a transitive dependency on v1 plus your own code on v2 are both satisfiable in one binary. The path-as-version rule is the small friction that buys away the entire category of “a major upgrade broke my build invisibly.”

Minimal Version Selection: the centerpiece

Here is the idea that most distinguishes Go’s dependency management, and the reason its builds are reproducible. When your graph requires the same module at several versions, a resolver must choose one. Almost every other ecosystem chooses the newest version that satisfies the constraints — pip and npm both work this way. Go does the opposite. Minimal Version Selection chooses, for each module, the lowest version that still satisfies every require in the graph.

Concretely: your build pulls in module A, which requires libX v1.2, and module B, which requires libX v1.4. Both requirements are floors — at least v1.2 and at least v1.4 — so the version that satisfies both with the least change is v1.4. MVS selects v1.4: not v1.5 even if it exists, not the latest v1.x on the proxy, but exactly the highest version anyone in your graph explicitly asked for. This is the diamond at the right of Figure 17.1.

The payoff is reproducibility for free. Because the selected set is a deterministic function of the committed require lines, building today, on CI, and on a new hire’s laptop all produce the same versions. Under “newest-satisfying,” a patch released upstream overnight can change what your build resolves to tomorrow though you changed nothing — which is why those ecosystems lean so heavily on a separate lock file to recover the determinism Go has by construction. MVS also makes upgrades deliberate: a version enters your build only because some require asked for it, so you raise it by running a command, never by the passage of time. The cost — you don’t automatically get the latest fixes — is the price of predictability over freshness, exactly the trade that makes infrastructure trustworthy.

The practical workflow

Day to day you drive modules through a handful of commands, and the tool keeps go.mod and go.sum correct so you don’t have to. go get adds or upgrades a dependency, with an @-suffix naming an exact version, a branch, or a commit:

go get github.com/gin-gonic/gin@v1.9.1   # pin an exact version
go get -u ./...                          # upgrade within the current major versions

The workhorse is go mod tidy. It reconciles go.mod and go.sum with what your code actually imports — adding missing modules, dropping ones nothing imports, recording every checksum. Run it after changing imports, and treat a clean git diff afterward as a merge prerequisite, since a diff means the manifest drifted from the code. For builds that must work offline or be auditable in one place, go mod vendor copies every dependency into a top-level vendor/ the build then prefers, trading repository size for hermeticity.

Private modules need one extra step, because Go routes fetches through the public proxy and checksum database by default, and those cannot see your private repos. GOPRIVATE tells the tool which path prefixes to fetch directly from source and exempt from the public checksum database:

go env -w GOPRIVATE=github.com/acme/*
git config --global url."git@github.com:acme/".insteadOf "https://github.com/acme/"

The first line bypasses the proxy and sum.golang.org for everything under your org; the second sends Go’s git fetch over authenticated SSH. Get this wrong and the symptom is unmistakable — a build hanging while it tries to read a username for https://github.com.

Project layout

Go imposes no layout, but a pragmatic standard has settled across the community, built on three load-bearing directories that map cleanly onto the concepts above. cmd/ holds entrypoints — one subdirectory per binary, each a tiny main.go that wires dependencies together and starts the program, so a service and its worker live as cmd/api and cmd/worker in one module. internal/ holds the bulk of the code — the feature packages (internal/order, internal/payment) the compiler keeps private. pkg/, where present, holds the deliberately public packages you intend others to import; if nothing outside your module will ever import a package, it belongs in internal/, not pkg/. Below is the shape, kept to the parts that earn their place:

shop/
├── go.mod                  # module github.com/acme/shop
├── go.sum
├── cmd/
│   ├── api/main.go         # HTTP server entrypoint
│   └── worker/main.go      # background worker entrypoint
├── internal/
│   ├── order/              # feature package: handler + service + storage
│   └── payment/            # feature package, private to this module
└── pkg/
    └── client/             # the one package external callers may import

The layout is a default, not a mandate: a single small binary needs none of it — a flat package and a main.go are fine — and you grow into cmd/+internal/ as the program acquires a second entrypoint or a public surface worth protecting.

War story: the diamond that wasn’t, and the seal that caught it

A platform team upgraded one direct dependency and their service stopped compiling with undefined: SomeFunc on a function that plainly existed. The cause was a diamond: the upgraded dependency now required a shared library at a new major version while another dependency still required the old one. Because the majors had different import paths, MVS kept both — but the team’s code was written against the old API, and the new transitive path shadowed it in a way no source file explained. go mod graph | grep shared-lib showed both paths in seconds and go mod why named the requirer; the fix was to pin the shared library to the version their code expected and upgrade their call sites on their own schedule, rather than let an unrelated upgrade drag it in.

The same week, a CI job for a different service failed with a go.sum checksum mismatch on a dependency nobody had touched. The instinct — delete go.sum and regenerate — was exactly wrong: that silently accepts whatever bytes the proxy now serves. An upstream tag had been force-pushed to different content, and go.sum, the seal doing its job, refused to build against the swap. Two lessons that travel together: inspect a diamond with go mod graph/why before reaching for replace, and treat a go.sum mismatch as a security signal, not a chore.

Build it → The repo’s Go system shows this layout at production scale. Project 02: Microservice Platform organizes several gRPC services with cmd/ entrypoints, internal/ feature packages, and shared protobuf-generated code under one module — the structure of this chapter as a working system.

Practical exercise

Difficulty: Level I · Level II · Level III

Level I — Structure a program into packages. Split a small program (say, a CLI that reads orders and prints totals) into at least three packages: a feature package with a clean exported API, an internal/ package holding logic callers must not depend on, and a cmd/ entrypoint that wires them together. Then prove the boundary: from a scratch module in a sibling directory, try to import your internal/ package and show the compile error. Explain in one sentence each why one identifier you wrote is exported and one is not.
Level II — Initialize a module and add a versioned dependency. Run go mod init, then go get a real third-party library at a pinned version and use it. Read your go.mod and go.sum and note what each line means: which require is direct versus // indirect, and what the go.sum entries guarantee. Now add an import you don’t use, run go mod tidy, and explain exactly what it changed and why — then remove the import and tidy again to watch it reverse.
Level III — Design the layout for a multi-service repo. For four services that share some code, decide what is one module versus many and justify it: where does an independent release cycle force a module boundary, and where does shared code argue for one module with multiple cmd/ entrypoints? Mark where your internal/ boundaries go and what each protects. Finally, write a paragraph on how MVS keeps the build reproducible across the owning teams — what guarantees two teams building from the same commits get byte-identical dependency sets, and what a team must do to change a shared version on purpose.

Summary

Go modules ended an era of GOPATH rigidity and version roulette by making a build a pure function of committed files. A module is a versioned unit of distribution — a go.mod that names the project and pins its dependencies — while a package is the unit of compilation and the boundary of encapsulation, its visibility decided by nothing more than the case of an identifier’s first letter, with internal/ turning that boundary into a compiler-enforced rule. Two design choices give the system its character: semantic import versioning, which puts a module’s major version in its import path so breaking upgrades can never happen invisibly, and Minimal Version Selection, which resolves the graph to the lowest version satisfying every requirer — the single decision that makes Go builds reproducible by construction and upgrades deliberate rather than ambient. go.sum seals the result against tampering, the everyday workflow keeps the manifest honest, and a cmd/+internal/+pkg/ layout maps the concepts onto a directory tree you grow into as the program does.

Key takeaways

A module is the versioned box with a manifest; a package is a folder with a public/private API, and capitalization alone decides what is exported.
internal/ is privacy the compiler enforces — default application code there and promote only what outsiders must import.
go.mod records what you require; go.sum cryptographically locks what you got. Together they make the build reproducible and supply-chain–verifiable.
A major version lives in the import path (/v2), so v1 and v2 coexist and a breaking upgrade can never silently replace the API you compiled against.
MVS picks the lowest version satisfying every requirer, not the newest — which is why Go builds don’t drift and why upgrades are something you ask for, not something that just happens.

Connections to other chapters

Go: Fundamentals (prerequisite): packages and visibility appear there as language mechanics; this chapter is what they scale into — the versioned, shippable unit a codebase is distributed as. The capitalization rule you learned for one file is, at module scope, your public API contract.
Python: Advanced Language Features and TypeScript: The Node Ecosystem (contrast): both lean on “newest-satisfying” resolvers (pip, npm) plus a separate lock file to recover the reproducibility Go gets from MVS by construction, and carry richer supply-chain histories because of it. Reading them against this chapter is the cleanest way to feel why floor-not-ceiling resolution and a built-in checksum log are a different philosophy, not just a different syntax.
The Polyglot Landscape (sibling): the build-artifact and packaging row in that chapter’s comparison is this chapter in one cell — Go’s “versioned Git-tagged tree, resolved by MVS, sealed by a checksum log” beside the registry-and-lockfile models of other ecosystems.
Go: Web Services & gRPC (extension): a real service is where this layout earns its keep — cmd/ entrypoints, internal/ feature packages, and the one-module-versus-many question become concrete the moment you have an API, a worker, and generated protobuf code to organize under one go.mod.

The Go Modules Reference (go.dev) — the authoritative specification of go.mod, go.sum, version queries, and the resolution algorithm.
Using Go Modules (the Go Blog, five-part series) — the canonical walk-through of the everyday workflow: creating modules, adding and upgrading dependencies, and releasing versions.

Deep dives

Russ Cox, Go & Versioning (the “vgo” essays) — the original design rationale for Minimal Version Selection and semantic import versioning, and the clearest argument for why lowest-satisfying beats newest for reproducible builds.
Module version numbering and the checksum database documentation — how semantic import versioning and the transparency-log–backed go.sum actually work end to end.

Historical context

The GOPATH-era documentation and the third-party vendoring tools that preceded modules (dep, glide, godep) — the problem space modules were designed to replace, and why a single official answer mattered.
The standard project-layout discussion (golang-standards/project-layout and the debate around it) — useful precisely because Go ships no official layout, so the pragmatic cmd//internal//pkg/ convention is a community artifact worth understanding critically.

--- title: "Go: Packages & Modules" keywords: [go, modules, packages, go.mod, semantic versioning, dependency management, minimal version selection, internal packages, project layout] difficulty: intermediate prerequisites: [go-fundamentals] estimated_time: "2-3 hours" --- ## Introduction Before 2018, starting a Go project meant negotiating with a directory. Your code did not live where you put it; it lived under `$GOPATH/src`, in a path that had to mirror its eventual import path exactly — `~/go/src/github.com/acme/shop` or nowhere. Clone a repo into the wrong folder and it would not compile. Worse, there was no notion of a *version*: `go get` fetched whatever was on a dependency's default branch today and wrote it into the shared `src` tree everyone pulled from. Two projects needing two releases of the same library could not coexist on one machine, and the build you ran on Monday could quietly differ from the one CI ran on Wednesday, with nothing recording that they had. The pain that finally forced the issue was the diamond. A team would add library A, which depended on a logging package at one release, and library B, which depended on the same package at a *different* release. There was no manifest to consult, no lock file to pin, no algorithm to adjudicate — just one folder and a coin flip. Whichever version was on disk won, and if the two were incompatible the program broke in a way no source file explained. People papered over it with vendoring tools — then with three competing ones, each with its own metadata format. The ecosystem had a dependency problem and a dependency-tool problem stacked on top. Go modules, shipped in Go 1.11 and the default since 1.16, ended all of it at once. A project now declares its own identity and dependencies in a file at its root, lives anywhere on disk, and records the exact version of everything it builds against. Two design choices — below — make this not just convenient but *boring*, in the sense that matters for infrastructure: the same inputs produce the same build, every time, on every machine, and an upgrade happens only when you ask. Go modules made builds reproducible and versioning boring on purpose. ### The Core Insight A Go **module** is a versioned unit of distribution: a `go.mod` file at the root of a directory tree that names the module, states the Go version it needs, and lists the other modules it requires and at what versions. Inside that tree live **packages** — and the package, not the module, is the unit the compiler actually works with: one directory of `.go` files compiled together, and also Go's boundary of encapsulation, where what a package exports is its public API and everything else is invisible from outside. The module is what you ship and version; the package is how you organize and hide code within it. Two decisions distinguish Go's system from nearly every other language's, and the rest of the chapter is downstream of them: 1. **Semantic import versioning.** A module's *major* version is part of its import path — version 2 of `github.com/acme/shop` is imported as `github.com/acme/shop/v2`, a literally different path from version 1. This sounds like bureaucracy but is what makes major upgrades safe: because v1 and v2 have distinct paths, one build can use both during a migration, and a breaking change can never silently replace the API you compiled against. 2. **Minimal Version Selection (MVS).** When parts of your graph require the same module at different versions, Go selects the *lowest* version that satisfies all of them, not the newest available — the opposite of the "newest-satisfying" rule npm and pip apply, and the source of Go's reproducibility. The version set is a deterministic function of the `require` lines; a new upstream release changes nothing until you ask for it. Together these make the build a pure function of the files you committed. ### A mental model Picture a module as a **shipped box with a manifest taped to the lid**. The box is the versioned thing you publish, tag, and depend on; the manifest (`go.mod`) lists what is inside and what other boxes it needs, each named with an exact version; and `go.sum` is the tamper-evident seal — a cryptographic fingerprint of every box you depend on, so one arriving with different contents than your manifest expected stops the build. Inside the box are **packages**, each a folder with a public counter and a private back room: capitalize an identifier and it goes on the counter, lowercase it and it stays in the back, and callers see only the counter. The `internal/` directory is a back room the *compiler* enforces — not a convention you hope people respect, but a rule it rejects your build for breaking. And the dependency graph is resolved not by a solver hunting for the newest mutually-agreeable versions but by the simplest rule that works: **for each module, take the highest version anyone explicitly asked for, and no higher.** That floor-not-ceiling rule is MVS, and it is why a Go build does not drift. @fig-go-modules shows the whole picture. ### When to split packages and when to make a module The two structuring decisions in Go — where to draw a package boundary and when to create a separate module — are governed by different questions, and conflating them is a common mistake. **Draw a package boundary around a responsibility and its public API.** A package should have one clear job and expose the smallest surface that does it. The surest sign you have the boundary right is that the name reads well at the call site — `order.New`, `payment.Charge` — and the surest sign you have it wrong is a package named `utils`, `common`, or `helpers`: responsibilities-in-name-only that everything imports. Organize by *feature* (an `order` package owning its handler, service, and storage) rather than by *layer* (a `handlers` package, a `services` package), because feature packages keep related code together and minimize the cross-package imports that breed cycles. **Reach for `internal/` whenever code is an implementation detail.** It costs nothing and is enforced by the compiler: anything under `internal/` is importable only by code rooted at that directory's parent. Default application code there; promote only what outsiders must depend on. **Create a separate module only for something versioned independently.** A web service and its worker that always ship together belong in one module with two `cmd/` entrypoints; a library you publish on its own cadence for other teams belongs in its own module. Splitting buys independent versioning at the cost of coordination, so don't pay for it until you need independent release cycles. ### What you'll learn - How a package serves at once as Go's unit of compilation and its boundary of encapsulation, with capitalization alone deciding what is public, and why `internal/` gives you privacy the compiler enforces rather than privacy by convention - What `go.mod` and `go.sum` each guarantee, and how together they make a build reproducible and supply-chain–verifiable - Why a module's major version lives in its import path, and what that buys you when a dependency makes breaking changes - How Minimal Version Selection resolves a dependency graph, and why choosing the *lowest* satisfying version is what makes Go builds deterministic - The everyday workflow (`go get`, `go mod tidy`, vendoring, private modules via `GOPRIVATE`) and how to lay out a project with `cmd/`, `internal/`, and `pkg/` ### Prerequisites - **Go fundamentals**: syntax, types, functions, and methods; how `import` brings a package's exported names into scope; comfort running `go build` and `go run` - Command-line basics and Git, including the idea that a tag can name a release - A working Go toolchain (1.21 or newer) --- ## Packages: the unit of compilation and encapsulation Everything in Go is in a package, the smallest thing the compiler treats as a whole. One directory is one package: every `.go` file in it declares the same `package` name and they compile together, seeing each other's declarations without any `import`. Cross that boundary and you need an `import`, and you reach only what the other package chose to make visible. The package is therefore two things at once — the grouping the compiler builds, and the wall encapsulation is built on. Go's visibility rule is famously minimal: no `public`, `private`, or `protected` keywords. **An identifier is exported if and only if its first letter is uppercase.** This applies uniformly to types, functions, methods, struct fields, and constants, and the mechanism is so simple it disappears into how you read code — a capital `User` is part of the API, a lowercase `email` field is the package's own business. ```go package user // User is exported — callers in other packages can see and use it. type User struct { ID int // exported field Name string // exported field email string // unexported: only the user package can touch it } // New is the package's public constructor. func New(name string) *User { return &User{Name: name, email: derive(name)} } // derive is unexported — an internal helper, invisible outside this package. func derive(name string) string { /* ... */ return "" } ``` The discipline that follows is to **start everything lowercase and export deliberately**. Exporting is a promise: once `derive` becomes `Derive`, another module may import it, and removing it later is a breaking change you owe them a major version for. Unexported-by-default keeps your API small and your freedom to refactor large. Package naming reinforces the same restraint — names are short and lowercase, no underscores or camelCase, because the name prefixes every call site and carries the meaning, so the identifiers inside need not repeat it: `user.New`, not `user.NewUser`. ## `internal/`: privacy the compiler enforces Capitalization controls visibility *within* a module — but the moment you publish, every exported identifier in every importable package becomes part of the contract others build on, which is often more than you meant to promise. You want some packages usable across your own codebase yet off-limits to outsiders, and that is exactly what `internal/` provides — uniquely among Go's structuring tools, enforced by the compiler rather than by convention. The rule is precise: a package under a directory named `internal/` is importable only by code rooted at `internal/`'s parent. `github.com/acme/shop/internal/payment` is reachable by anything under `github.com/acme/shop/` and nothing else — an external module that tries gets a compile error, not a lint warning. This lets you expose a small, stable public surface while keeping the machinery behind it free to change. Implementation details that leak into a public path are a liability: someone imports them, and then your refactor is their breaking change. ## Modules, `go.mod`, and `go.sum` A module is declared by a `go.mod` file at its root, short by design. It names the module (its import-path prefix), states the Go version the code expects, and lists the modules it requires and at what versions. The `go` tool maintains it; you rarely edit it by hand beyond the occasional pin. ```go module github.com/acme/shop go 1.22 require ( github.com/gin-gonic/gin v1.9.1 github.com/lib/pq v1.10.9 ) require github.com/go-playground/validator/v10 v10.16.0 // indirect ``` Two things deserve attention. The `module` line is the import-path *prefix* for every package in the tree — the `internal/order` directory is imported as `github.com/acme/shop/internal/order`, derived mechanically from this line plus the directory path. And the `// indirect` marker flags a dependency you don't import directly but that something in your graph needs, so the manifest stays a complete account of the build. The companion `go.sum` is where reproducibility becomes supply-chain integrity. For every module version your build touches it stores a cryptographic hash of the contents, and on every build Go verifies that what it fetched matches; a mismatch halts the build. This is the lock that turns "the same inputs produce the same build" from a hope into a guarantee, and the defense against a dependency being swapped out from under you: a tampered or republished version cannot slip in unnoticed, because its hash will not match. You never edit `go.sum` by hand — the tool writes it, and `go mod verify` checks it. ::: {.callout-note} The hash in `go.sum` is cross-checked against a global append-only transparency log (`sum.golang.org`), which lets the whole ecosystem detect if a published version's bytes ever change after the fact — the module equivalent of certificate transparency. ::: ## Semantic versioning and the import path Module versions are semantic versions — `v1.4.2`, the three numbers meaning *breaking*, *feature*, *fix* — and Go takes the contract literally: a minor or patch bump promises backward compatibility, a major bump declares it broke something. What makes Go unusual is the consequence it attaches to a major bump: **the major version becomes part of the import path.** Version 1 is imported at the bare path; version 2 and beyond append `/v2`, `/v3`, with the `go.mod` `module` line changed to match. ```go // v1 — the original path import "github.com/acme/widgets" // v2 — a different import path entirely, after breaking changes import "github.com/acme/widgets/v2" ``` This is the "import compatibility rule," and the reasoning is sharp. If two incompatible releases shared a path, a build that transitively needed both would be impossible, and a major upgrade would mean silently recompiling your code against an API that may have changed under it. By giving each major version its own path, Go makes v1 and v2 *coexist* as if unrelated packages: a migration moves one caller at a time, and a transitive dependency on v1 plus your own code on v2 are both satisfiable in one binary. The path-as-version rule is the small friction that buys away the entire category of "a major upgrade broke my build invisibly." ## Minimal Version Selection: the centerpiece Here is the idea that most distinguishes Go's dependency management, and the reason its builds are reproducible. When your graph requires the same module at several versions, a resolver must choose one. Almost every other ecosystem chooses the *newest* version that satisfies the constraints — pip and npm both work this way. Go does the opposite. **Minimal Version Selection chooses, for each module, the lowest version that still satisfies every `require` in the graph.** Concretely: your build pulls in module A, which requires `libX v1.2`, and module B, which requires `libX v1.4`. Both requirements are floors — *at least* v1.2 and *at least* v1.4 — so the version that satisfies both with the least change is v1.4. MVS selects v1.4: not v1.5 even if it exists, not the latest v1.x on the proxy, but exactly the highest version *anyone in your graph explicitly asked for*. This is the diamond at the right of @fig-go-modules. ![A Go module (go.mod at its root) groups packages behind a public/internal boundary and pins its dependencies; Minimal Version Selection resolves the graph by choosing the lowest version that satisfies every requirer — so builds are reproducible and upgrades are deliberate, not "newest wins."](../assets/diagrams/rendered/go_modules.svg){#fig-go-modules .lightbox} The payoff is reproducibility for free. Because the selected set is a deterministic function of the committed `require` lines, building today, on CI, and on a new hire's laptop all produce the *same* versions. Under "newest-satisfying," a patch released upstream overnight can change what your build resolves to tomorrow though you changed nothing — which is why those ecosystems lean so heavily on a separate lock file to recover the determinism Go has by construction. MVS also makes upgrades *deliberate*: a version enters your build only because some `require` asked for it, so you raise it by running a command, never by the passage of time. The cost — you don't automatically get the latest fixes — is the price of predictability over freshness, exactly the trade that makes infrastructure trustworthy. ## The practical workflow Day to day you drive modules through a handful of commands, and the tool keeps `go.mod` and `go.sum` correct so you don't have to. `go get` adds or upgrades a dependency, with an `@`-suffix naming an exact version, a branch, or a commit: ```bash go get github.com/gin-gonic/gin@v1.9.1 # pin an exact version go get -u ./... # upgrade within the current major versions ``` The workhorse is `go mod tidy`. It reconciles `go.mod` and `go.sum` with what your code actually imports — adding missing modules, dropping ones nothing imports, recording every checksum. Run it after changing imports, and treat a clean `git diff` afterward as a merge prerequisite, since a diff means the manifest drifted from the code. For builds that must work offline or be auditable in one place, `go mod vendor` copies every dependency into a top-level `vendor/` the build then prefers, trading repository size for hermeticity. Private modules need one extra step, because Go routes fetches through the public proxy and checksum database by default, and those cannot see your private repos. `GOPRIVATE` tells the tool which path prefixes to fetch directly from source and exempt from the public checksum database: ```bash go env -w GOPRIVATE=github.com/acme/* git config --global url."git@github.com:acme/".insteadOf "https://github.com/acme/" ``` The first line bypasses the proxy and `sum.golang.org` for everything under your org; the second sends Go's `git` fetch over authenticated SSH. Get this wrong and the symptom is unmistakable — a build hanging while it tries to read a username for `https://github.com`. ## Project layout Go imposes no layout, but a pragmatic standard has settled across the community, built on three load-bearing directories that map cleanly onto the concepts above. `cmd/` holds entrypoints — one subdirectory per binary, each a tiny `main.go` that wires dependencies together and starts the program, so a service and its worker live as `cmd/api` and `cmd/worker` in one module. `internal/` holds the bulk of the code — the feature packages (`internal/order`, `internal/payment`) the compiler keeps private. `pkg/`, where present, holds the deliberately public packages you intend others to import; if nothing outside your module will ever import a package, it belongs in `internal/`, not `pkg/`. Below is the shape, kept to the parts that earn their place: ```text shop/ ├── go.mod # module github.com/acme/shop ├── go.sum ├── cmd/ │ ├── api/main.go # HTTP server entrypoint │ └── worker/main.go # background worker entrypoint ├── internal/ │ ├── order/ # feature package: handler + service + storage │ └── payment/ # feature package, private to this module └── pkg/ └── client/ # the one package external callers may import ``` The layout is a default, not a mandate: a single small binary needs none of it — a flat package and a `main.go` are fine — and you grow into `cmd/`+`internal/` as the program acquires a second entrypoint or a public surface worth protecting. ::: {.callout-warning} ## War story: the diamond that wasn't, and the seal that caught it A platform team upgraded one direct dependency and their service stopped compiling with `undefined: SomeFunc` on a function that plainly existed. The cause was a diamond: the upgraded dependency now required a shared library at a new *major* version while another dependency still required the old one. Because the majors had different import paths, MVS kept both — but the team's code was written against the old API, and the new transitive path shadowed it in a way no source file explained. `go mod graph | grep shared-lib` showed both paths in seconds and `go mod why` named the requirer; the fix was to pin the shared library to the version their code expected and upgrade their call sites on their own schedule, rather than let an unrelated upgrade drag it in. The same week, a CI job for a *different* service failed with a `go.sum` checksum mismatch on a dependency nobody had touched. The instinct — delete `go.sum` and regenerate — was exactly wrong: that silently accepts whatever bytes the proxy now serves. An upstream tag had been force-pushed to different content, and `go.sum`, the seal doing its job, refused to build against the swap. Two lessons that travel together: inspect a diamond with `go mod graph`/`why` before reaching for `replace`, and treat a `go.sum` mismatch as a security signal, not a chore. ::: > **Build it →** The repo's Go system shows this layout at production scale. > [Project 02: Microservice Platform](https://github.com/jchu0/applied-cs-projects/tree/main/02-microservice-platform) > organizes several gRPC services with `cmd/` entrypoints, `internal/` feature packages, > and shared protobuf-generated code under one module — the structure of this chapter as > a working system. --- ## Practical exercise **Difficulty:** Level I · Level II · Level III 1. **Level I — Structure a program into packages.** Split a small program (say, a CLI that reads orders and prints totals) into at least three packages: a feature package with a clean exported API, an `internal/` package holding logic callers must not depend on, and a `cmd/` entrypoint that wires them together. Then prove the boundary: from a scratch module in a sibling directory, try to import your `internal/` package and show the compile error. Explain in one sentence each why one identifier you wrote is exported and one is not. 2. **Level II — Initialize a module and add a versioned dependency.** Run `go mod init`, then `go get` a real third-party library at a pinned version and use it. Read your `go.mod` and `go.sum` and note what each line means: which `require` is direct versus `// indirect`, and what the `go.sum` entries guarantee. Now add an import you don't use, run `go mod tidy`, and explain exactly what it changed and why — then remove the import and `tidy` again to watch it reverse. 3. **Level III — Design the layout for a multi-service repo.** For four services that share some code, decide what is one module versus many and justify it: where does an independent release cycle force a module boundary, and where does shared code argue for one module with multiple `cmd/` entrypoints? Mark where your `internal/` boundaries go and what each protects. Finally, write a paragraph on how MVS keeps the build reproducible across the owning teams — what guarantees two teams building from the same commits get byte-identical dependency sets, and what a team must do to *change* a shared version on purpose. ## Summary Go modules ended an era of `GOPATH` rigidity and version roulette by making a build a pure function of committed files. A **module** is a versioned unit of distribution — a `go.mod` that names the project and pins its dependencies — while a **package** is the unit of compilation and the boundary of encapsulation, its visibility decided by nothing more than the case of an identifier's first letter, with `internal/` turning that boundary into a compiler-enforced rule. Two design choices give the system its character: **semantic import versioning**, which puts a module's major version in its import path so breaking upgrades can never happen invisibly, and **Minimal Version Selection**, which resolves the graph to the *lowest* version satisfying every requirer — the single decision that makes Go builds reproducible by construction and upgrades deliberate rather than ambient. `go.sum` seals the result against tampering, the everyday workflow keeps the manifest honest, and a `cmd/`+`internal/`+`pkg/` layout maps the concepts onto a directory tree you grow into as the program does. ### Key takeaways - A module is the versioned box with a manifest; a package is a folder with a public/private API, and capitalization alone decides what is exported. - `internal/` is privacy the *compiler* enforces — default application code there and promote only what outsiders must import. - `go.mod` records what you require; `go.sum` cryptographically locks what you got. Together they make the build reproducible and supply-chain–verifiable. - A major version lives in the import path (`/v2`), so v1 and v2 coexist and a breaking upgrade can never silently replace the API you compiled against. - MVS picks the lowest version satisfying every requirer, not the newest — which is why Go builds don't drift and why upgrades are something you ask for, not something that just happens. ### Connections to other chapters - **Go: Fundamentals** (prerequisite): packages and visibility appear there as language mechanics; this chapter is what they scale into — the versioned, shippable unit a codebase is distributed as. The capitalization rule you learned for one file is, at module scope, your public API contract. - **Python: Advanced Language Features** and **TypeScript: The Node Ecosystem** (contrast): both lean on "newest-satisfying" resolvers (pip, npm) plus a separate lock file to *recover* the reproducibility Go gets from MVS by construction, and carry richer supply-chain histories because of it. Reading them against this chapter is the cleanest way to feel why floor-not-ceiling resolution and a built-in checksum log are a different philosophy, not just a different syntax. - **The Polyglot Landscape** (sibling): the build-artifact and packaging row in that chapter's comparison is this chapter in one cell — Go's "versioned Git-tagged tree, resolved by MVS, sealed by a checksum log" beside the registry-and-lockfile models of other ecosystems. - **Go: Web Services & gRPC** (extension): a real service is where this layout earns its keep — `cmd/` entrypoints, `internal/` feature packages, and the one-module-versus-many question become concrete the moment you have an API, a worker, and generated protobuf code to organize under one `go.mod`. ## Further reading ### Essential - *The Go Modules Reference* (go.dev) — the authoritative specification of `go.mod`, `go.sum`, version queries, and the resolution algorithm. - *Using Go Modules* (the Go Blog, five-part series) — the canonical walk-through of the everyday workflow: creating modules, adding and upgrading dependencies, and releasing versions. ### Deep dives - Russ Cox, *Go & Versioning* (the "vgo" essays) — the original design rationale for Minimal Version Selection and semantic import versioning, and the clearest argument for why *lowest-satisfying* beats *newest* for reproducible builds. - *Module version numbering* and the *checksum database* documentation — how semantic import versioning and the transparency-log–backed `go.sum` actually work end to end. ### Historical context - The *GOPATH*-era documentation and the third-party vendoring tools that preceded modules (`dep`, `glide`, `godep`) — the problem space modules were designed to replace, and why a single official answer mattered. - The *standard project-layout* discussion (`golang-standards/project-layout` and the debate around it) — useful precisely because Go ships no official layout, so the pragmatic `cmd/`/`internal/`/`pkg/` convention is a community artifact worth understanding critically.