Every serious software developer or system administrator must have at least a basic grasp of Bash commands, because they can often help you to manage computer systems much more efficiently and effectively than if you were only to use a point-and-click GUI.

In this post we’re going to write Go code that will aim to replace twelve of the most commonly used Linux Bash commands.

The goal here isn’t to reproduce all of the functionality of the commands that we look at, since many of them have a large number of options that tweak the output in all kinds of arcane ways. It’s simply to see how we can deal with some routine system-administration tasks within our Go code, without relying on any external Bash commands that aren’t strictly necessary.

You may need superuser privileges (using the sudo command) to run some of the Bash commands and Go programs that we will write — for example, when deleting files from the temporary directory.

If you want a fun challenge, look at the headings below before reading the full article and see if you can write the code to complete the various tasks yourself from scratch. Then come back to this page and compare your work with what I produced. Let me know if you think you’ve improved on my code!

(Please note that not all of the commands listed below are Bash builtins; some are part of GNU/Linux coreutils. Also note that the code below is not necessarily aiming to be as elegant or efficient as possible: it is simply intended as an interesting exercise to think about how certain functionality that commonly occurs in sys-admin scripts could easily be reproduced in Go.)

Printing a Line of Text

Bash Shell (Linux)

echo "Go"   "is great!"

The echo command simply prints out the strings that are passed to it, after concatenating them, joining each of them together with a single space character.

Golang

package main

import (
	"fmt"
	"strings"
)

func echo(text ...string) {
	var builder strings.Builder

	for i, t := range text {
		if i > 0 {
			builder.WriteByte(' ')
		}

		builder.WriteString(t)
	}

	fmt.Println(builder.String())
}

func main() {
	echo("Go", "is great!")
}

We first create a strings.Builder variable in order to allow us to store the concatenated text. Then we iterate through the input strings, adding each of the strings to the builder; for every string except the first one, we also start by adding a single byte of space. Finally, we simply call fmt.Println, which write the string generated by our builder to stdout, as well as a newline character.

Generating a Random Number

Bash Shell (Linux)

echo $RANDOM

The echo command can also print environment variables, which are prefixed with a dollar sign. However, the $RANDOM variable refers to a native function of the Bash command interpreter. So when we pass it to the echo command, we are, in fact, calling the function and passing the result to be printed.

The $RANDOM function generates a pseudo-random number between 0 and 32,767 (which is 2¹⁵−1). It does not produce cryptographically secure results, but it is useful for simple scripts and games.

Golang

package main

import (
	"fmt"
	"math/rand"
	"time"
)

const (
	randomUpperBound = 1 << 15 // two to the power of fifteen
)

func random() int {
	return int(rand.Int31n(randomUpperBound))
}

func init() {
	rand.Seed(time.Now().UnixNano())
}

func main() {
	fmt.Printf("%d\n", random())
}

In the random function, we use the Go standard library to generate a random number with our predetermined upper bound, stored in a constant. It is important to remember to seed the generator, as we do in the init function, otherwise we would always get the same result whenever we run the program, which is clearly not what we want.

Printing the Contents of a File

Bash Shell (Linux)

cat -n test.txt

This is one of the simpler Bash commands. It just reads a file and prints out its content. The -n option ensures that line numbers are printed before every line. The command can concatenate multiple files (hence the name cat), but in our implementation we'll just handle a single file.

Golang

package main

import (
	"fmt"
	"os"
)

func cat(fileName string, printLineNumbers bool) {
	fileData, err := os.ReadFile(fileName)
	if err != nil {
		panic(err)
	}

	if printLineNumbers {
		outputData := make([]byte, 0, len(fileData))

		var lineCount int

		for i, d := range fileData {
			if d == '\n' || i == 0 {
				if i != 0 {
					outputData = append(outputData, d)
				}

				lineCount++

				startOfLineString := fmt.Sprintf("%6d  ", lineCount)

				outputData = append(outputData, startOfLineString...)

				if i == 0 {
					outputData = append(outputData, d)
				}
			} else {
				outputData = append(outputData, d)
			}
		}

		fmt.Printf("%s\n", outputData)
	} else {
		fmt.Println(string(fileData))
	}
}

func main() {
	cat("test.txt", true)
}

Most of the code above is involved in printing the line numbers, since we can simply use fmt.Println to display the contents of the file as it is. We use the fmt.Sprintf function to create the prefix at the start of every line, containing the number, which is right-aligned, just as in the cat Bash command.

If you do want to make the Go code handle multiple files, that's trivial to implement, so I'll leave it for you to work out.

Printing the Contents of a File With Its Lines in Reverse Order

Bash Shell (Linux)

tac -b test.txt

The name of the tac command is a pun: it's just cat backwards. Fittingly, it prints the lines of a file in reverse order. The -b option prints the line separator at the beginning of each line.

Golang

package main

import (
	"os"
)

func printReversedLine(lineData []byte) {
	l := len(lineData)
	mid := l / 2

	for i := 0; i < mid; i++ {
		j := l - i - 1

		lineData[i], lineData[j] = lineData[j], lineData[i]
	}

	if _, err := os.Stdout.Write(lineData); err != nil {
		panic(err)
	}
}

func tac(fileName string, fixNewlineBug bool) {
	file, err := os.Open(fileName)
	if err != nil {
		panic(err)
	}
	defer file.Close()

	endLocation, err := file.Seek(0, 2)
	if err != nil {
		panic(err)
	}

	fileData := make([]byte, endLocation)

	if _, err = file.Seek(0, 0); err != nil {
		panic(err)
	}

	_, err = file.Read(fileData)
	if err != nil {
		panic(err)
	}

	if fileData[endLocation-1] == 0 { // EOF
		endLocation--
	}

	var line []byte

	for i := endLocation - 1; i >= 0; i-- {
		d := fileData[i]

		line = append(line, d)

		if d == '\n' {
			if len(line) > 0 {
				printReversedLine(line)

				line = []byte{}
			}
		}
	}

	if len(line) > 0 {
		if fixNewlineBug {
			line = append(line, '\n')
		}

		printReversedLine(line)
	}
}

func main() {
	tac("test.txt", false)
}

As you can see the code for this is much more complicated than for the cat Bash command, but it's not difficult to understand. I've created a helper function, printReversedLine, which simply prints the contents of a byte slice in reverse order: I use a simple algorithm that I have previously discussed in the post I wrote about the various ways to reverse slices.

The Seek method sets the offset for following reads or writes to a file: if the second argument is 0, then the first argument provides an offset from the start of the file, and if the second argument is 2, then first argument provides an offset going backwards from the end of the file. So file.Seek(0, 2) simply takes us to the end of the file, allowing us to work out its size.

We then read the entire contents of the file into memory and iterate backwards through each byte and print out a reversed line (which, in effect, is a line in the correct order, because we're storing each line's bytes in reverse order) when we meet a separator.

There is something of a problem with the original Bash command, which I have replicated in my Go code: the last two lines of the file to be printed are shown together. If you set the fixNewlineBug option to true, the lines will be properly separated.

Listing the Names of Files in a Directory With Reverse Sort

Bash Shell (Linux)

ls /bin | sort -r

The ls Bash command prints each of the file names within the given directory. We use the pipe character to send the output to the sort command, which orders the lines in reverse alphabetical order.

Golang

package main

import (
	"fmt"
	"os"
	"sort"
)

func ls(path string) {
	file, err := os.Open(path)
	if err != nil {
		panic(err)
	}
	defer file.Close()

	list, err := file.Readdirnames(0)
	if err != nil {
		panic(err)
	}

	sort.Strings(list)

	for i := len(list) - 1; i >= 0; i-- {
		name := list[i]

		fmt.Println(name)
	}
}

func main() {
	ls("/bin")
}

The Readdirnames method performs the main functionality here: the argument that it takes sets a limit on the number of filenames to read at once, but if the argument is less than or equal to zero, it simply reads everything that it can.

It's important to remember that just because we want to print out the lines in reverse order, that doesn't mean that we actually need to sort them in that order: we can use the sort.Strings function from the standard library to sort them in ascending alphabetical order, then we just need to iterate over the slice in reverse when we print them out, so that they're displayed in descending order.

Printing the Path to the Current Working Directory

Bash Shell (Linux)

pwd

The pwd command is easy to understand: its three letters stand for the phrase "print working directory", and it does just that.

Golang

package main

import (
	"fmt"
	"os"
)

func pwd() {
	path, err := os.Getwd()
	if err != nil {
		panic(err)
	}

	fmt.Println(path)
}

func main() {
	pwd()
}

The "os" package provides access to operating-system libraries. The Getwd function returns, alongside an error value, the absolute path (i.e. starting from the root) of the current directory.

Printing the Lines of a File That Match a Given Pattern

Bash Shell (Linux)

grep local /etc/hosts

The grep command is so useful that it's entered into the English language as a verb: people talk about grepping (not to be confused with grokking) a file, when they mean that they're searching it for pattern-matches.

The command above searches the /etc/hosts file, which is present on all Linux systems and contains a mapping of IP addresses to URLs. We are searching for the string "local", presumably because we want to know the IP address of the localhost.

Golang

package main

import (
	"bufio"
	"fmt"
	"os"
	"strings"
)

func grep(pattern string, fileName string) {
	file, err := os.Open(fileName)
	if err != nil {
		panic(err)
	}
	defer file.Close()

	scanner := bufio.NewScanner(file)
	for scanner.Scan() {
		text := scanner.Text()

		if strings.Contains(text, pattern) {
			fmt.Println(text)
		}
	}
}

func main() {
	grep("local", "/etc/hosts")
}

This code reads the file line-by-line, using the buffered IO functionality provided by the standard library. It then simply searches each line, to see if the pattern matches, and if it does, the line is printed out.

Overwriting a File With Random Data

Bash Shell (Linux)

shred -n 5 test.txt

Sometimes you don't necessarily want to delete a file, but you do want to securely remove its contents. Maybe you've saved private data, like your banking details, and you don't want anyone else getting hold of them.

The shred command will overwrite a file with random data, so that its original contents are no longer accessible. The -n option determines the number of iterations: in other words, how many times it will overwrite the file with random data.

Golang

package main

import (
	"crypto/rand"
	"io/fs"
	"os"
)

func shred(fileName string, iterations int, roundUp bool) {
	file, err := os.OpenFile(fileName, 0666, fs.FileMode(os.O_RDWR))
	if err != nil {
		panic(err)
	}
	defer file.Close()

	endLocation, err := file.Seek(0, 2)
	if err != nil {
		panic(err)
	}

	if roundUp {
		roundedEndLocation := endLocation / 1024
		if endLocation%1024 != 0 {
			roundedEndLocation++
		}
		roundedEndLocation *= 1024

		endLocation = roundedEndLocation
	}

	data := make([]byte, endLocation)

	for i := 0; i < iterations; i++ {
		if _, err := rand.Read(data); err != nil {
			panic(err)
		}

		if _, err := file.WriteAt(data, 0); err != nil {
			panic(err)
		}

		if err := file.Sync(); err != nil {
			panic(err)
		}
	}
}

func main() {
	shred("test.txt", 5, true)
}

We use the Seek method again to judge the size of our file. If the roundUp option is set, we will round the file size up to the nearest multiple of 1,024, since this corresponds to Linux's default block size. Finally, we perform the necessary number of iterations, reading random data and writing it to our file. We use the "crypto/rand" package, rather than "math/rand", since it provides us with cryptographically secure data. The Sync method ensures that the writes are actually performed on disk.

Removing All of the Files in the Temporary Directory

Bash Shell (Linux)

rm -rf /tmp

It can be good practice to clear the temporary directory at regular intervals, so it doesn't fill up with too many historic files. The example above does just that. The -r option makes the rm Bash command recursive, deleting directories as well as files, while the -f option ignores any errors.

Golang

package main

import (
	"os"
	"path/filepath"
)

func rm(path string) {
	dir, err := os.ReadDir(path)
	if err != nil {
		panic(err)
	}

	for _, dirEntry := range dir {
		childPath := filepath.Join(
			path,
			dirEntry.Name(),
		)

		if err := os.RemoveAll(childPath); err != nil {
			panic(err)
		}
	}
}

func main() {
	rm("/tmp")
}

This example is fairly simple: we just get the name of every entry within a directory, and then we iterate through them, using the os.RemoveAll function to recursively delete each file or directory and any subfiles or subdirectories.

Downloading a File From the Internet

Bash Shell (Linux)

wget https://file-examples-com.github.io/uploads/2017/10/file-sample_150kB.pdf example.pdf

It's useful to be able to download a file hosted online without having to load up a web browser. The wget Bash command does just that. Its name derives from a combination of World Wide Web and get.

As you can see, we're downloading a PDF file from a website hosting example files that contain dummy text.

Golang

package main

import (
	"fmt"
	"io"
	"net/http"
	"os"
)

func wget(url string, path string) {
	file, err := os.Create(path)
	if err != nil {
		panic(err)
	}
	defer file.Close()

	resp, err := http.Get(url)
	if err != nil {
		panic(err)
	}
	defer resp.Body.Close()

	if resp.StatusCode != http.StatusOK {
		err := fmt.Errorf(
			"status code: %d [%s]",
			resp.StatusCode, resp.Status,
		)

		panic(err)
	}

	_, err = io.Copy(file, resp.Body)
	if err != nil {
		panic(err)
	}
}

func main() {
	wget(
		"https://file-examples-com.github.io/uploads/2017/10/file-sample_150kB.pdf",
		"example.pdf",
	)
}

We first create an empty file (which we could also have done at the command line with the touch Bash command). Then we connect to correct URL using the "net/http" package from Go's standard libary.

If we get a response with a status code of 200, meaning everything's okay, we simply copy the data we received in the HTTP body into our open file. The file will automatically be closed when we end the function, because we had earlier used the defer keyword.

Printing the Current Time and the System Uptime

Bash Shell (Linux)

uptime | cut -d , -f 1 | sed -e 's/^[[:space:]]*//'

The uptime Bash command prints the current time (using the 24-hour clock) and the amount of time that the computer has been running in hours and minutes.

It also gives some other information about the number of users logged into the machine and the amount of load the CPU is under, but we don't need that, so we use the cut command to print the output only up to the first comma.

The final sed command simply uses a regular expression to remove any initial whitespace that may be present.

Golang

package main

import (
	"fmt"
	"os"
	"strconv"
	"strings"
	"time"
	"unicode"
)

func formatTime(totalSeconds int, includeSeconds bool) string {
	seconds := totalSeconds

	minutes := seconds / 60
	if minutes > 0 {
		seconds -= minutes * 60
	}

	hours := minutes / 60
	if hours > 0 {
		minutes -= hours * 60
	}

	var builder strings.Builder

	if hours > 0 {
		builder.WriteString(fmt.Sprintf("%02d:", hours))
	}

	if minutes > 0 || hours > 0 {
		builder.WriteString(fmt.Sprintf("%02d", minutes))

		if includeSeconds {
			builder.WriteByte(':')
		}
	}

	if includeSeconds {
		builder.WriteString(fmt.Sprintf("%02d", seconds))
	}

	return builder.String()
}
 
func uptime() {
	currentTime := time.Now().UTC()
	currentTimeTotalSeconds := currentTime.Second() +
		currentTime.Minute()*60 +
		currentTime.Hour()*60*60

	uptimeFileData, err := os.ReadFile("/proc/uptime")
	if err != nil {
		panic(err)
	}

	uptimeSecondsData := make([]byte, 0, len(uptimeFileData))

	for _, d := range uptimeFileData {
		if d != '.' && !unicode.IsDigit(rune(d)) {
			break
		}

		uptimeSecondsData = append(uptimeSecondsData, d)
	}

	uptimeSecondsFloat, err := strconv.ParseFloat(string(uptimeSecondsData), 10)
	if err != nil {
		panic(err)
	}

	fmt.Printf(
		"%s up %s\n",
		formatTime(currentTimeTotalSeconds, true),
		formatTime(int(uptimeSecondsFloat), false),
	)
}

func main() {
	uptime()
}

We have created another helper function that formats a given time or duration expressed as a cumulative total of seconds in hours, minutes and seconds, separated by colons. The "%02d" specifier tells the fmt.Sprintf function to turn the given integer into a string that is zero-padded to two places (so if there are, for example, 4 minutes, it will be displayed as "04").

Getting the current time is easy with the Now function in the Go standard library. We just need to remember to multiply the minutes by 60 and the hours by 3,600 (60×60), so that they're expressed as seconds.

The /proc/uptime file on Linux contains two values, separated by a space. The first value is the total time that the system has been running. The second value is the sum of how much time each processor core has spent idle. Both values are given in seconds with two decimal places.

We only need to access the first value, so we simply iterate through each byte until we find a non-numeric character (in this case, a space), only using the bytes up to the point at which we stop. We then convert these to a float64, printing out the hours and minutes with our formatTime function.

Converting Windows Line Endings to Unix

Bash Shell (Linux)

dos2unix test.txt

The Windows and Linux operating systems traditionally use different ways to represent the end of a line in a file: Windows uses a carriage return followed by a newline character ("\r\n") to signify a line break, whereas Linux simply uses a single newline character ("\n").

This can cause real-world problems: for example, I've had the containerization application Docker refuse to read my Dockerfile simply because it was encoded with Windows line endings. Quite frankly, the developers should have foreseen that issue and handled it themselves — but, for whatever reason, they didn't. So it always helps to use the style of line endings that is appropriate to the operating system you're running.

The Bash command dos2unix simply converts the line endings from the Windows format. Likewise, the unix2dos command performs the opposite operation.

Golang

package main

import (
	"os"
	"regexp"
)

var (
	windowsNewlineRegexp = regexp.MustCompile(`\r\n`)
)

func dos2unix(path string) {
	fileData, err := os.ReadFile(path)
	if err != nil {
		panic(err)
	}

	fileData = windowsNewlineRegexp.ReplaceAllLiteral(
		fileData,
		[]byte{'\n'},
	)

	if err := os.WriteFile(path, fileData, 0666); err != nil {
		panic(err)
	}
}

func main() {
	dos2unix("test.txt")
}

You could read a file and iterate through the runes that make up its content, manually removing every carriage return that is followed directly by a newline character.

However, I've chosen the easier option, which is to use a regular expression. We simply call the ReplaceAllLiteral method and it will replace all of the Windows line breaks with the simpler Linux version. Then we overwrite the file with the updated contents. We do all this using the byte slice that we're originally given — we never even need to convert it to a string.