Golang Project Structure

Tutorials, tips and tricks for writing and structuring code in Go (with additional content for other programming languages)

Interview With the Go Developer: Jon Rafkind

Language

  • unknown

by

This is the first in an upcoming series of interviews with professional software developers who use Go in their work and amateur coders who use it in their personal projects.

Headshot of the software engineer and Go programmer Jon Rafkind, who is the subject of this interview.
Jon Rafkind is a strong supporter of the Go programming language.

This first interview is with Jon Rafkind, who is a senior software engineer with around 8 years of experience using Go.

He has an undergraduate degree in computer science and a PhD specializing in the design of programming languages, but he has been coding, in one form or another, since he was a child.

Jon currently works as a consultant to startup companies in the Bay Area of San Francisco, California.


You say you have over three decades of programming experience, so how did you very first get interested in and involved with computing?

I first got involved with computing due to having access to my father’s computer at home in the 1980s. My father worked as an engineer at IBM, and he introduced me to PCs and DOS.

The first computer we had was either a 286 or a 386 with a VGA monitor. My older brother began programming on the computer before I did, and I became enthralled with the programs he was able to make.

I initially started using Borland Turbo Pascal 7, and would write some simple games with it. Pascal was an excellent introduction to programming for me because the syntax is fairly readable, and the compiler is very fast.

After a few years of Pascal, I started to use C and C++. Then I went to college and learned a few more languages, such as Java and Scheme.

I saw that you have a PhD in computer science from the University of Utah. Could you explain in very general terms the subject of your dissertation and what you were trying to achieve with it?

My dissertation was about applying the idea of syntactic extension to languages other than Lisp/Scheme.

Lisp/Scheme are well known for having a powerful feature called a macro, which is a function that runs at compile time that produces more syntax/code, and which lets the programmer generate code rather than having to write it themselves.

Code generation has been around since the dawn of computing of course, but Lisp/Scheme macros are highly reliable and understandable.

The reason that macros work so well in Lisp/Scheme is due to the homogenous nature of the syntax, in that every statement and expression is wrapped inside parenthesis. This property enables the compiler to know where an expression begins and ends, which allows the macro function to know exactly what syntax it is operating on.

In other languages — like C, C++, Java, Python, Javascript, etc. — the syntax is more free form, so it is less clear where the beginning and ending of a macro form should be.

In my dissertation I presented a parsing solution called enforestation, which allows the macro to consume just the right amount of syntax to allow the macro to run but without consuming syntax beyond the extent of the macro.

Ultimately the goal was to build a powerful macro system that could be implemented in a language like Java or Python.

Do you ever regret the academic route that you took into software engineering or do you feel that it gave you a more rigorous framework that was important in your subsequent work?

My undergraduate degree certainly endowed me with a great deal of formal training in computer science that I most likely never would have acquired without going to school.

I considered myself a pretty good programmer before entering undergraduate school, but to my surprise I actually knew quite little about the theoretical concepts of computer science.

I felt reasonably comfortable with programming and computer science by the time I entered graduate school, but I learned a great deal about doing research and about ideas that have not yet made their way into mainstream programming languages.

Overall I am quite satisfied with my decision to attend and complete a PhD. Many of the concepts I learned about in school are directly applicable to issues that arise in industry, so there was an obvious benefit to improving my formal foundation via school.

The financial aspect of doing a PhD can be a large question in that perhaps I could have made more money working in industry rather than studying and earning a small stipend. After graduating, my salary has benefited from the degree I earned so it might be a wash in the end.

What are the earliest programming languages that you worked with? How did using those languages shape your thinking about coding?

I started with Pascal, mainly because it was the only thing we had on the computer and I had a book on it.

I found the concepts involved with programming quite confusing at first — loops, switch statements and pointer values. I did not even understand what a record/struct was, so I was creating complex objects purely out of arrays.

Eventually I came to understand how all those building blocks worked, and when I moved on to C/C++ I did not have much trouble.

When I first started to learn Java in undergraduate school, I was very surprised to learn about automatic memory management — in other words, garbage collection. In Pascal/C/C++ I had always been managing memory by hand, so I thought it was incredible that the computer could do this for me.

I also learned Scheme in undergraduate school, which introduced me to the concept of lambdas — a first-class function value that could be passed around like any other variable.

I very quickly grasped the power of lambdas in that I could write a function with some local state, and capture that state in the closure of a lambda that could then be passed around elsewhere in the program. This is technically achievable in C/C++, but quite cumbersome and inconvenient.

Scheme introduced the concept of functional programming to me that, along with the lambda expression, also prioritizes immutability and recursion.

Immutability is an excellent property to use when programming as it makes reasoning about the program much easier than using mutation. Recursion is interesting and I use it from time to time, but I largely just use a traditional for loop to get most work done. Of course I also learned about macros in scheme, which ultimately led me to my research in graduate school.

In graduate school I was introduced to languages with very strong type systemsOCaml, Haskell and a few others. Those languages were really eye-opening in that their typing was so powerful: if the compiler would compile a program without error, then I could be very sure the program would work with minimal or no testing.

I started to look disfavorably on languages with weak compile-time type systems, such as C, and completely untyped languages like Python.

A weakly typed program inevitably carries bugs in it for months/years that only appear at the most inopportune time — something I believe can be avoided using proper static analysis tools (such as a static type system).

What motivated you to begin using Go? I saw on your GitHub profile that you’ve worked with other C-based languages, so was Go’s emphasis on simplicity of syntax and practicality something that attracted you to the language?

Go has many properties that I think make it a fantastically pragmatic language to use. It has a fast compiler, good type checking, produces single binary executables, first-class functions, integrates with C libraries easily, has low latency garbage collection, and a strong ecosystem of libraries and other users.

In retrospect, I probably would have been happy to use C++ with garbage collection, which is what the D system supposedly is, but I never really put much effort into using D.

Golang has a handful of language warts, as all languages do, but overall I find that I am highly productive in it, and I have high confidence that the resulting program will work correctly (due to the type system) and efficiently (due to the optimizing compiler).

You said that Go, in common with all programming languages, has its warts. That’s a good way to put it, and I wonder if there are any aspects of the language that you really dislike or, at least, feel could be improved?

Some aspects that I would improve in Go, from highest priority to lowest in my opinion are:

  • Allow named arguments in a function call, similar to what Python has, as in f(data=xyz, num=3). This might seem minor, but in a large codebase it can be difficult to remember what all the arguments are for, and chasing them down is exhausting. A partial solution is to use a custom type with named fields, f(FArgs{Data: xyz, Num: 3}), but this is clunky, and the compiler won’t help you if you forget to set one of the fields.
  • I wish that every Go file was its own module. Currently a set of files in a directory comprise the same module, where each file can freely access identifiers in any of the other files within the same module. This creates a tension where I would prefer to keep most identifiers in the same file so they are easy to find, but then my files grow very large. I guess it is unlikely this change will ever be made.
  • Error handling could use some syntactic sugar to make the handling of errors simpler. This is a very contentious issue in the Go community, and I know there are a handful of issues on the Golang issue tracker on GitHub. The essence of the problem is callers have to write the highly repetitive a, err := foo(); if err != nil { return err } or some variation of that, where a syntactic solution using the question mark operator could simplify syntax: a := foo()?. The downside to this is that error handling becomes somewhat “hidden” from the user, which is why it has not been accepted into the language so far.
  • Have some keyword that denotes immutability. In theory this should have been const, but Go does not let you declare a const struct type.

Okay. Error handling is something that gets raised a lot, but I like your suggestion of introducing named arguments to Go. Now, on the other hand, what would you say that some of the best features of the Go programming language are?

My top 5 features of Go are:

  • Garbage collection. Other languages have this too, but in Go collection is very fast and low latency.
  • The "context" library is extremely useful when writing servers. Being able to cancel a context and have that cancellation cascade down through any number of components makes the resulting system easy to reason about. As a small example, stopping a program by pressing Ctrl-C and having the program cleanup properly can be challenging in a lot of systems. Occasionally programs will simply give up and call exit(1) rather than do any proper cleanup.
  • Green/virtual threads (i.e. goroutines) are the right model for concurrency. This is as opposed to async/await that is gaining popularity in a few languages. Async/await has the issue of function coloring, where a function that is not async has problems calling functions that are async. This can lead to a fractured code base with many duplicate methods in it, one that is async and one that is not. Boring old threads are simpler to use, so long as the thread does not map to an OS thread. Java’s virtual threads are a similar feature.
  • Error handling via explicit return values is easier to deal with than automatically propagating exceptions. It is nice to write nested function calls like f(g(h(3))) where any function might throw an exception, but avoiding exception handling just leads to errors being handled in poor ways. The Go way is a bit uglier in that each function call has to be separately checked for an error, but ultimately I care about knowing exactly where and why errors are being handled.
  • Integrated package management. C/C++ still does not have a package manager, so users must manually download and install dependencies of a project. Java has Maven, but I always struggle to get Maven to work properly. In Go, installing the dependencies of a project is a simple go get command.

Do you have any opinions about the proposals for Go 2 and how you’d like to see the language evolve in future releases? Should the second version of Go be an opportunity for major breaking changes or should the conservative ethos that Go has stuck to so far prevail?

I’m not sure if Go 2 will ever come out, or if new features will just keep being added into Go 1.x.

Generics were added to Go 1 without too much of an issue. Perhaps cleaner error handling can be added as well.

I am strongly in favor of the compatibility goals that Go adheres to. It is very nice to download the latest version of Go and have my code work without issue.

I have used other languages in the past that broke backwards compatibility, and in some cases rather than fixing my code I simply abandoned the entire project/language altogether.

If there ever is a breaking change in a newer version of Go, then users should be able to opt-in via some go.mod flag rather than have the change be forced upon them.

Is there a project that you’ve worked on in Go that you’re really proud of (perhaps the NES emulator you built)? Were there interesting challenges you overcame in building it?

Of the personal projects I have written in Go I am fairly proud of the NES emulator for two reasons:

  • I got the system to work, and able to play a few games within 3 months of starting the project. Initially I had thought it would take me 6-9 months of work. I probably overestimated the difficulty of the project, but the ease of use of Go certainly helped my productivity a great deal.
  • I made heavy use of passing values via channels in goroutines, something that is fairly difficult to get right in C/C++. Writing concurrent code in Go feels much more manageable than other languages.

The emulator uses the SDL2 library implemented in C, and while Go does have a good FFI layer to call C functions, I discovered that a single FFI call is quite expensive performance-wise. I had to rearchitect my code a little bit in order to minimize FFI calls, but I was able to do this and achieve good performance in the end.

I am also writing a couple of games in Go. I used to write games in C++, but dealing with crashes and memory corruption was very frustrating. In Go those issues rarely occur, so I am able to write complete programs with much less issues in a shorter time frame.

At previous jobs I have written internal services in Go, and those services were always rock solid. In some cases, I left the services running unattended for months with no issues whatsoever.

There’s a lot of friendly rivalry between Go and Rust developers, so was Rust a language that ever appealed to you?

Rust’s appeal is that it provides strong correctness guarantees, and is usually very efficient. Those goals are admirable, but when I have tried Rust in a few small projects I found myself fighting with the compiler too often.

There are some use cases that warrant the high degree of difficulty that Rust has, such as writing an OS in Rust, but for most projects Go is good enough.

You said that Go’s fast and efficient garbage collector is one of its best features, so what do you think about Rust’s model of memory safety, which is very different to the more traditional idea of a mark-and-sweep GC?

Programming in C++ forces you to think about object lifetimes, or pay a price when you get it wrong via a crash or memory corruption.

Rust formalizes object lifetimes so that they can be checked by the compiler, which is nice in a way, but in Go I can mostly avoid thinking about the issue entirely.

Of course I pay some amount of attention to allocation and storage of objects in Go in order to prevent memory leaks, but at least I don’t have to resolve puzzles where object A must live within the lifetime of objects B, C and D.

As you’ve moved into more senior roles in software engineering, do you find that you’re still pretty hands-on with code or do you think a lot more about higher-level issues like system architecture and how to get the best out of people?

At previous roles where my seniority increased to the point that I was managing other programmers, I found myself writing less code on a day to day basis. Instead, I would spend a lot of time reviewing code from my teammates, and try to keep the system stable by disallowing exotic features.

I enjoy writing code and working on systems more than I do dealing with other people, so I took up more personal projects at home.

I also tutor college-age computer science students over the internet, so I am frequently looking at all kinds of code of various degrees of quality.

As a kid I just wanted to spend most of my time working on software, and even now I still feel that way.

Do you have any advice for programmers who are looking to move up the career ladder, maybe into management or even launching their own startup?

I would suggest that people try to get as much education as possible. After working with dozens of other programmers, there is a pretty clear delineation between people with an undergraduate degree vs those with a masters or a PhD.

The difference between them is their critical-thinking skills, and their ability to think about a system as whole as opposed to just the narrow slice of code they may be interacting with at the moment.

I probably sound biased in this regard, but I noticed this change in myself. After attending graduate school my reasoning abilities increased a great deal, and I am grateful for that.

It doesn’t even matter what field someone achieves an advanced degree in — I have worked with talented people from chemistry and physics backgrounds.

Moving into management for the first time can be a bit overwhelming at first. I would suggest that new managers get feedback from their bosses, peers and people they manage as well. That may mean directly asking for feedback rather than waiting for it to be provided.

Specific questions about feedback are usually better than general open-ended ones. For example, “How could I have improved my last team meeting?” as opposed to “How am I doing?”

I am still working on trying to launch a startup myself, so I don’t have any advice on that front yet.

Finally, what do you do to relax when you’re not working?

I mainly enjoy working on personal software projects, especially writing games.

Other than that, I like to get some physical exercise. I play competitive racquetball, and compete in local and national tournaments.

Leave a Reply

Your email address will not be published. Required fields are marked *