Golang Project Structure

Tutorials, tips and tricks for writing and structuring code in Go (with some additional content for other programming languages)

Should Go Be Used for Machine Learning?

Language

  • unknown

by

Go is widely known for its simplicity, concurrency model and performance in large-scale distributed systems.

But when it comes to machine learning (ML), the most dominant programming languages have traditionally been Python, R and C++.

These languages have become almost synonymous with machine learning due to the wide range of mature libraries, frameworks and communities built around them.

Given this background, the natural question arises: Is it practical to use Go for machine learning?

The short answer is that yes, it can indeed be done. However, that answer does come with certain caveats and conditions.

An computer rendering of a brain made from metal with wires and lights attached. It is supposed to symbolize artificial intelligence and machine learning.
One of the ultimate goals of machine learning is to build computer systems that can surpass the abilities of the human mind.

Whilst Go may not yet be as mature or well-supported in the field of machine learning as Python, it is certainly very capable of being used for a range of ML tasks.

In this blog post, we will explore the current state of machine learning in Go and some of the most popular ML libraries available in Go. We will also consider the language’s strengths and limitations, and when it makes sense to choose Go over other more popular ML languages.

What Is Machine Learning?

Machine learning is a subset of artificial intelligence (AI) that focuses on the development of algorithms and statistical models that enable computers to perform tasks without explicit instructions.

Instead of being programmed to perform a specific task, a machine learning model learns from data, identifying patterns and making decisions based on the information it processes.

This capability allows systems to adapt and improve over time as they are exposed to more data.

How Does Machine Learning Use Datasets?

At its core, machine learning involves training a model on a dataset, which is a collection of examples that represent the problem the model aims to solve.

This dataset is typically divided into two parts: a training set, which the model uses to learn, and a test set, which is used to evaluate the model’s performance.

What Are Some Specific Types of Machine Learning?

Various techniques — such as supervised learning, unsupervised learning and reinforcement learning — are employed based on the nature of the task and the data available.

Supervised learning is one of the most common approaches, where the model learns from labelled data, allowing it to make predictions or classifications.

In contrast, unsupervised learning deals with unlabelled data, where the model identifies hidden patterns or groupings without prior knowledge of the categories.

Reinforcement learning is based on the concept of an agent learning to make decisions by interacting with an environment, receiving feedback in the form of rewards or penalties.

How Is Machine Learning Used in Practice?

The applications of machine learning are vast and span numerous industries, including finance, healthcare, marketing and technology.

From recommendation systems that suggest products based on user behaviour to advanced diagnostic tools that analyze medical images, machine learning is transforming how we approach problem-solving in the 21st century.

As the field continues to evolve, the potential for machine learning to enhance our daily lives and drive innovation is boundless.

Go’s Suitability for Machine Learning

To answer the question of whether Go can be used for machine learning effectively, we need to understand its capabilities in comparison to more popular ML languages like Python and C++.

Speed and Performance

Go is a statically typed compiled language. This means that it offers superior runtime performance compared to interpreted languages like Python.

For computationally heavy tasks, such as matrix operations, neural network training or optimization algorithms, Go’s speed can be a valuable asset.

Furthermore, Go’s concurrency features, powered by goroutines, provide a natural advantage in parallelizing tasks, which is crucial in modern machine learning workloads.

When dealing with large datasets or complex operations, the ability to parallelize computations efficiently can make a substantial difference in terms of performance.

Go’s standard library provides great support for parallel computing, without the complexity found in languages like C++.

This makes it easier to implement data processing pipelines and distributed computing tasks within machine learning workflows.

Development Speed

Machine learning workflows require a balance between experimentation — i.e. rapid prototyping — and performance.

In this area, Python excels due to its high-level abstractions and dynamic typing, allowing for really fast prototyping of machine learning models.

However, Go’s simplicity, strong typing and fast compilation times also allow developers to build robust systems quickly, especially in production environments.

Go’s simple syntax and design philosophy reduce the cognitive overhead for developers. The language prioritizes simplicity, which makes it easier to reason about code, maintain large codebases, and catch errors early in the development process.

Ease of Integration

One of Go’s best features is how well it integrates into existing systems.

This is particularly valuable in production ML environments where models need to interact with other systems.

Go’s efficiency in building web servers and APIs make it a strong candidate for deploying machine learning models at scale.

In contrast, while Python is excellent for model development, it’s not always the best language for deploying ML models at scale, especially in high-performance environments.

Python-based models often need to be wrapped in frameworks like Flask or FastAPI, which introduces additional overhead.

On the other hand, with Go, you can embed machine learning models directly into web services with little friction.

Community and Ecosystem

One of the main drawbacks to using Go for machine learning is the lack of mature libraries and frameworks compared to Python’s ecosystem.

Python has well-established libraries like TensorFlow, PyTorch, Scikit-Learn and others, which are battle-tested and come with robust documentation and support from a large community.

While Go’s ML ecosystem is growing, it is fair to say that it’s still in its infancy.

Several Go libraries for machine learning have emerged, but they are not as feature-rich or widely adopted as their Python counterparts.

This can be a challenge for developers who are used to the convenience of Python’s mature ML libraries.

Machine Learning Libraries in Go

Despite its still relatively nascent stage, Go does have a growing set of libraries designed to support machine learning tasks.

Let’s take a look at some of the most prominent ones:

Gorgonia

Gorgonia is one of the most promising libraries for machine learning and deep learning in Go.

It provides a framework for building and training machine learning models by creating computational graphs, similar to TensorFlow and PyTorch.

Gorgonia supports GPU acceleration via CUDA, making it possible to perform large-scale computations on GPUs.

It also supports automatic differentiation, which is crucial for training deep learning models.

And, like TensorFlow and PyTorch, Gorgonia allows you to define computations symbolically.

While Gorgonia is still under active development, it has a growing community of contributors, and its design makes it one of the most powerful Go-based libraries for machine learning.

Here’s a simple example of creating a neural network using Gorgonia:

package main

import (
	"fmt"
	"gorgonia.org/gorgonia"
	"gorgonia.org/tensor"
)

func main() {
	g := gorgonia.NewGraph()
	x := gorgonia.NewTensor(g, tensor.Float32, 2, gorgonia.WithShape(1, 2), gorgonia.WithName("x"))
	w := gorgonia.NewTensor(g, tensor.Float32, 2, gorgonia.WithShape(2, 1), gorgonia.WithName("w"))
	y := gorgonia.Must(gorgonia.Mul(x, w))

	machine := gorgonia.NewTapeMachine(g)
	defer machine.Close()

	gorgonia.WithValue(x, tensor.New(tensor.Of(tensor.Float32), tensor.WithShape(1, 2), tensor.OfSlice([]float32{1, 2})))
	gorgonia.WithValue(w, tensor.New(tensor.Of(tensor.Float32), tensor.WithShape(2, 1), tensor.OfSlice([]float32{0.5, 1.5})))

	err := machine.RunAll()
	if err != nil {
		fmt.Println("Error:", err)
		return
	}

	fmt.Println(y.Value())
}

The example code above demonstrates the basic idea of how Gorgonia works with symbolic expressions and tensors, but the library also supports more advanced operations like convolutional layers and recurrent networks.

Gonum

Gonum is another important library for numerical computations in Go.

It provides the building blocks for scientific computing, similar to how NumPy serves Python.

Gonum includes tools for matrix manipulation, optimization, statistics and linear algebra, which are essential for many machine learning algorithms.

Gonum is not a machine learning framework per se, but it is a crucial part of the ecosystem as it provides the numerical backbone for implementing algorithms from scratch.

For example, you can easily perform matrix operations, which are fundamental to most ML algorithms, using Gonum:

package main

import (
	"fmt"
	"gonum.org/v1/gonum/mat"
)

func main() {
	// create a matrix
	a := mat.NewDense(2, 2, []float64{1, 2, 3, 4})

	// perform matrix multiplication
	var result mat.Dense
	result.Mul(a, a)

	// print the result
	fmt.Printf("Matrix multiplication result:\n%v\n", mat.Formatted(&result, mat.Prefix(" ")))
}

GoLearn

GoLearn is one of the most user-friendly libraries for machine learning in Go.

It aims to be a simple, clean and intuitive library akin to Scikit-Learn in Python.

It provides implementations of basic machine learning algorithms — including decision trees, Naive Bayes, k-nearest neighbours and many others.

GoLearn is great for beginners who want to perform classification or regression tasks in Go, without needing to delve into the low-level implementation.

Here’s a basic example of how you can use GoLearn to load a dataset and train a k-nearest neighbors classifier (which works by looking at the closest known datapoints to figure out what group a new datapoint belongs to):

package main

import (
	"fmt"
	"github.com/sjwhitworth/golearn/base"
	"github.com/sjwhitworth/golearn/evaluation"
	"github.com/sjwhitworth/golearn/knn"
)

func main() {
	// load the dataset
	rawData, _ := base.ParseCSVToInstances("iris.csv", true)
	trainData, testData := base.InstancesTrainTestSplit(rawData, 0.5)

	// initialize k-nearest neighbours model
	cls := knn.NewKnnClassifier("euclidean", "linear", 2)

	// train the model
	cls.Fit(trainData)

	// make predictions
	predictions, _ := cls.Predict(testData)

	// evaluate the model
	confusionMat, _ := evaluation.GetConfusionMatrix(testData, predictions)
	fmt.Println(evaluation.GetSummary(confusionMat))
}

Some of the Best Books to Learn Machine Learning in Go

Don’t worry if you didn’t understand everything about the code examples we’ve just looked at, since they already assume some familiarity with machine learning.

If you’re a complete beginner to the subject — or just want to understand more about how to implement ML algorithms in Go — there are some great books available to help you learn much more.

I discuss a couple of the best books in the sections below.

Front cover of the book "Machine Learning With Go" by Daniel Whitenack.

Machine Learning with Go by Daniel Whitenack

This book is one of the most popular resources for learning ML in Go.

Daniel Whitenack, a data scientist who specializes in Go, walks readers through practical implementations of machine learning techniques.

The book focuses on building, training and deploying models using native Go packages, covering everything from simple linear models to more complex deep learning systems.

It is an excellent resource for those who already have a foundation in Go and want to apply it to machine learning.

The author takes a very practical approach, ensuring that readers can not only understand ML concepts but also implement them efficiently.

If you’re looking to build scalable and production-ready ML systems in Go, this is a really good starting point.

Some readers have said that the book can be a bit technical for complete beginners, but the degree of depth is well suited for those serious about mastering ML in Go.

(Please note that a second edition of this book has now been released, co-authored with Janani Selvaraj.)

Front cover of the book "Go Machine Learning Projects" by Xyanyi Chew.

Go Machine Learning Projects by Xyanyi Chew

For developers who prefer project-based learning, this book is a standout option.

It takes a really practical and hands-on approach to building machine learning applications in Go.

Each chapter introduces a new project, such as building recommendation systems, predictive models or even fraud detection tools.

By the end of the book, readers will have built a collection of fully functional ML applications using Go.

This book is perfect for developers who want to skip the theoretical deep dives and focus on practical applications of machine learning in Go.

The projects are well chosen, representing scenarios that can be easily applied in the real world.

While the book doesn’t provide exhaustive coverage of ML algorithms, it makes up for that by focusing on how to get working projects off the ground.

This book may be best suited for intermediate developers who are comfortable with Go, since it does assume some prior knowledge of the language.

Comparing Go to Python for Machine Learning

To understand when Go could prove a better choice than Python for machine learning, let’s compare the two languages across several dimensions:

FeatureGoPython
EcosystemSmaller, fewer ML librariesRich ecosystem, many mature ML libraries
PerformanceCompiled, faster executionInterpreted, slower but often sufficient
ConcurrencyBuilt-in support with goroutinesThreading is possible but limited by GIL
Prototyping SpeedSlower due to static typingFast, great for experimentation and research
DeploymentSimple and efficient, good for productionNeeds additional frameworks (Flask, FastAPI)
Community SupportGrowing, but smallerMassive community, extensive documentation
Ease of UseSimple but more verbose than PythonSimple and expressive, especially in data science

Python is undoubtedly the leader when it comes to machine learning due to its vast libraries and tools like TensorFlow, PyTorch and Scikit-Learn, along with a massive community and ecosystem.

Python excels in fast prototyping and experimentation.

However, Go is not without its merits. When performance and concurrency are critical, especially for real-time ML systems or distributed computing, Go’s compiled nature and efficient handling of concurrent tasks make it an attractive option for production deployment.

Moreover, Go’s ecosystem is growing, with libraries like Gorgonia and Gonum steadily improving.

So Can Go Really Be Used for Machine Learning?

Yes, Go can be used for machine learning, but its suitability depends heavily on the specific use case.

While it doesn’t yet rival Python in terms of libraries and community support, it shines in production environments where performance, concurrency and scalability are crucial.

Go’s ability to efficiently handle large-scale systems, deploy ML models in production and work with distributed computing make it a valuable tool in the machine learning landscape.

For developers already working in Go or for organizations that need to optimize their ML pipelines for performance, Go offers several key advantages, including speed, concurrency and ease of deployment.

While Python remains the language of choice for research and prototyping, Go is a powerful complement in scenarios where scalability and efficiency are paramount.

The future of Go in machine learning looks promising, with continued growth in its ecosystem and the potential for more robust libraries and frameworks tailored to ML workflows.

If you are considering building machine learning systems with Go, especially for production deployment, it’s worth doing some further research into libraries like Gorgonia and Gonum to take full advantage of Go’s performance and concurrency features.

Leave a Reply

Your email address will not be published. Required fields are marked *