A Taxonomy of Toolchains

I recently came back to C++ development for the first time in 7–8 years; in many ways, for the first time ever. What follows is something I wrote when I finally began to understand the development workflow for C++. I hope it will help other students and newcomers to C++ pick out their preferred toolchain for comfortable and productive development.

What is a Toolchain?

There are two major uses of the word toolchain. Its use in reference to C/C++ typically signifies a collection of tools used to build a program; a chain of transformations which takes the C++ source code to binary machine opcodes (you thought that this was what a compiler does, but in fact the compiler is only one piece of this process). More generally, the term is also used to refer to the set of software tools and utilities a programmer chooses to assist them in development. Here, we will refer to this broader category as one's development environment. So when I say toolchain, I mean the choice of compiler and build tools, which is a subset of your development environment.

When you begin C++ development, you're thrown into the deep end. You need to choose what tools you're going to use for development, before you even know what the tools are used for. This choice is strongly affected by which operating system you use, which operating system you want your program to run on, whether you choose up-front setup cost in return for easier development, and other deeply personal preferences. The goal of this tutorial is to give you the background you need to choose wisely, plus some sensible recommendations to get you started.

A toolchain, then, is a chain of tools: the major components of a toolchain are builder, compiler, linker, and debugger. This is how they work together:

toolchain

To make this example concrete, here are the GNU equivalents of the above components:

Builder make
Compiler g++
Linker ld
Debugger gdb

The GNU toolchain is generally the de facto option for C++ development on any operating system and any processor. It's generally a good recommendation for anything you need to accomplish. When in doubt, use GNU.

What is a Development Environment?

As I'm referring to it, a development environment is nothing other than your tool belt as a programmer. It is the entire corpus of software utilities you use to get stuff done. Primarily what we'll focus on in this tutorial are text editors, compiler toolchains, build tools, and debuggers. Often these are all wrapped up in a single package, called the Integrated Development Environment (IDE).

IDE

There are two main choices when it comes to development environment. From what I've seen, seasoned software practitioners in industry are pretty equally split between these options. I think either of them would be a great choice for a newcomer. They are:

  • An integrated development environment.
  • Emacs or Vim, combined with make or CMake, plus a debugger.1

I hope to educate you in the various nuances of these choices, and give you just enough information to start your own quest for your C++ holy grail. Here is the sequence of tasks I put before you:

  1. Determine what OS you'll be doing development on (your host machine).
  2. Determine what OS(es) you'd like your program to run on (your target machine).
  3. Pick a text editor (for the host machine) (ideally works on the target as well).
  4. Pick a compiler (for the target machine).
  5. Pick a build management tool (for the target machine).
  6. Pick a debugger (for the target machine).

It's often the case that you're concerned with supporting only one OS, and you don't need to worry about the difference between host machine and target machine. But sometimes, you'll want to support all the major operating systems, right? You want the whole world to adopt your program! This can become difficult with a language that compiles to machine code, like C++. You'll need to make sure to pick a set of build tools that works on all platforms. This usually boils down to the GNU toolchain.2

Notes on Text Editor Choice

Your text editor is your main weapon in the mean streets of programmerdom. In many ways it will define your style, your strengths, and your weaknesses. Will you become a master of the timeless emacs-fu? Will you artfully slice and dice a bit of source code with sublime text kendo? Or will you employ the secrets of IntelliJitsu to amaze your friends and confound your enemies? An editor is not the full martial art—it's only a weapon—but it will do much to influence your style. Just as Bruce Lee rejected studying a single martial art in favor of borrowing techniques from all over, you should eventually try a few of these tools.3

My advice is to try a couple different ones in the next few weeks, then commit to one for the next year or so. If you continue down the path of software development, you will inevitably master many of these tools, so don't worry about making the perfect choice.4

As stated above, your main choice is to decide between a full-blown IDE on the one hand, or a text editor + build tool + debugger combo on the other. We can safely assume you'll use make + gdb as your build tool + debugger, so the only real choice is this: IDE or text editor? The main trade-off here is setup time vs. feature richness.

An IDE will ultimately allow you to be more productive, but involves an initial setup cost in terms of configuring the project.5 If you can get past that point, you have access to a world of features that are not afforded to text editors, or can only be had through a hodgepodge of plugins. These features include indexing, autocompletion, refactorings, in-editor documentation, code templates, and more. An IDE does more than simply unite the basic pieces of a development environment—it adds many utilities on top of that.

A text editor is easier to get started with, especially if you are working on someone else's project. In this case you don't have the pain of figuring out how to get your IDE to build their project; you just use whatever build process they used. However, there are some text editors which are not so easy to get started with, yet are more powerful, so I should make some further distinctions.

There are two types of text editor. One is a graphical editor like GEdit, Notepad++, or Sublime Text. The other is a command-line editor like Emacs or Vim. Most graphical editors don't bring the amazing range of functionality that an IDE provides. So these are the easiest to learn, but also the most limited. They are generally not used by folks in industry.[citation needed] In contrast, the CLI editors can be as feature-rich as an IDE. But it takes a long time and a lot of customization to make them that way. This is both good and bad. Many people get off on this.6 An IDE is ultimately easier to master than one of these editors, but once again, involves more initial setup time.

The CLI editors have one major advantage that should not be missed: they are already installed on pretty much every Unix/Linux distribution you'll ever use. Whether you need to ssh into a Linux server to edit some config files for a website, or you need to telnet into a robot to edit its source code, being proficient with one of these editors is a huge boon to your programming life. Even if you use an IDE for most programming tasks, you will probably need to use one of the CLI editors eventually.

Bottom Line: Whichever way you choose, the main point to remember is that IDEs will require more babysitting, but give you more bang for your buck. They are likely to be more difficult in the outset, but much easier after the initial setup.7

Notes on Compiler Choice

Just like choosing a debugger (below), you'll either use the default supplied by your IDE, or you'll use the GNU toolchain. GNU provides gcc for C development and g++ for C++ development. At the end of the day, g++ is just a wrapper around gcc.

There is also a relatively new toolchain (new as in 2003) called LLVM. The compiler for LLVM is called Clang, and is rapidly gaining popularity (in fact, clang has replaced g++ on OS X). Clang gives nicer error output and may actually be very helpful for a beginner. As far as I know, it accepts most of the same options as g++ so you can start using it as a drop-in replacement. You might want to check it out.

Bottom Line: Use g++ (or whatever your IDE provides).

Notes on Build Management Choice

Here are your choices when it comes to build management:

  • IDE builder
  • CMake
  • Autotools
  • Make
  • your own blood, sweat, tears, etc.

One of the biggest advantages an IDE typically brings is automated build management. Without an IDE, you need to compile your code yourself, with handwritten calls to a compiler. The good folks at GNU have come up with a terrible, barely-working solution to this problem. Instead of tediously writing compiler commands yourself, you tediously write them into a Makefile. That way, at least you only have to do it once. But you do still have to do it yourself. And you need to change it every time you add a new source file. And it often varies from machine to machine.

The state of affairs in C++ build management is so bad that there is yet another tool that runs on top of make, called Autotools, that attempts to make the process bearable. In my opinion, it doesn't really achieve this goal. Here's how it works. You write configure.ac files and Makefile.am files for automake. automake generates the files needed for autoconf, which generates the makefiles you need, which are used by make to generate compiler commands, which is what you actually wanted in the first place.

There is another build manager which does somewhat better, called CMake. It directly generates makefiles, and has very nice formatted output, though it is still rather difficult to use.8 When you start a larger project, you may want to check it out. For now, you can leave it.

Bottom Line: Choose an IDE or lightweight makefiles ... for now. Oh, and here's another diagram:

autotools

C++ build management: infinite recursion

Notes on Debugger Choice

The debugger situation can get a bit confusing, because there's many debugger front ends but few actual debuggers. There's essentially only three debuggers out there, and they are:

  • GDB (GNU toolchain)
  • LLDB (LLVM toolchain)
  • CDB (Windows toolchain)

It can also be confusing because each C++ toolchain provides its own debugger. So do you have to use the debugger that matches your compiler? As it turns out, no. Debuggers are completely compiler-agnostic; they depend on the binary format, which is determined by the OS. For example, all compilers that work on Unix produce binaries in the same format, the ELF format, and most of them provide debugging flags in the same format, the DWARF format (you can tell these things were invented by a bunch of geeks). All Unix debuggers know how to interpret these formats. So nothing prevents you from compiling with g++ and debugging with lldb, or compiling with clang and debugging with gdb.9

If you use an IDE, the choice of debugger is already made. The IDE serves as a GUI for one of the above debuggers. Otherwise, you'll generally choose GDB, or a GUI front-end for GDB (GNU wins again). It's been around for a long time and has wide support. LLDB is a new, improved choice which is almost ready for adoption, but it doesn't yet have any GUIs that support it. Still, if you plan to work command-line-only and GUI-free, you may want to give it a look.

Bottom Line: Use what your IDE provides, or use GDB.

Summing It Up

It's my recommendation that you try Emacs, Vim, or an IDE. Any one of these will be a valuable part of your toolbelt in a career in software engineering, and even in further computer science courses. Still, it can be a lot to learn a whole new toolset all at once, while at the same time learning a new language. You may want to find a powerful text editor with a more gradual learning curve, so that's okay too. I won't make any specific recommendations here (although I did in the footnotes). You can find many opinions online about which text editors are most popular.

In terms of a builder, you should probably use Make for now. There are other options to look into when your projects become bigger.

In terms of everything else, either use your IDE's default toolchain, or use the GNU toolchain. But look out in the near future for the LLVM toolchain; I think its tipping point is upon us.

All the best on your quest, and Happy Coding!


  1. You could replace the use of "Emacs or Vim" here with any text editor of your choosing. Sooner or later, I think you will have to learn one of the choices I've given (Emacs, Vim, or IDE). Your choice, then, is whether you want that to be "sooner" or "later."
  2. At the very least, it means you need to be careful not to use any platform-specific libraries in your source code. Other times, you'll only have one target machine but it will be different from your host machine, because the target is a robot, or an airplane, or a DVR. You can't exactly do development on those machines; you need to cross-compile from host to target. As a beginner, you don't need to worry about these more complex setups too much—but now that I've mentioned them, hopefully they won't confuse you if you see them mentioned on a website somewhere.
  3. That advice applies tenfold to programming languages.
  4. I myself hold black belts in both Eclipse and IntelliJ, and something like a blue belt in Vim and Sublime Text.
  5. If you are creating a project from scratch, then using the IDE's "hello world" template will avoid most of this pain and have you started very quickly. Trying to build some existing source code can be trickier.
  6. See also: yak shaving.
  7. Let me reemphasize my opinion that you should jump right in and use Emacs, Vim, or an IDE. I think you will end up learning these at some point before you graduate. Still, learning a whole new development environment while also learning a new programming language is daunting. If it's too much right now, may I recommend you try Sublime Text. At the time of writing, it's pretty new to the scene, yet already a favorite among those in the know.
  8. Believe it or not, I actually use CMake to generate my Eclipse project files, and then I use Eclipse for development. And I'm not alone. I gather that most people in the robotics industry use this toolchain. Just goes to show how messed up C++ development is.
  9. Supposedly. Theoretically. In an ideal world.

Installing GDB on OS X Mavericks

With its new OS release, Apple has discontinued the use of GDB in OS X. Since 2005 Apple has steadily been moving away from the GNU toolchain in favor of LLVM. This means that Xcode now uses LLDB instead.

LLDB looks to be a very nice replacement for GDB, and I hope to use it in the future, but currently Xcode is the only graphical front-end that supports its use; pretty much every other debugging GUI uses GDB under the hood, including Eclipse. So, if you want to debug C/C++ code in Eclipse CDT on the Mac, you must install GDB.1

Here is the procedure that worked for me. Others have reported issues with this, so please do let me know in the comments if it doesn't work for you.2

Installing & Certifying GDB

  1. Install Xcode and Homebrew, if not already installed.
  2. Now install GDB from Homebrew:
    $ brew tap homebrew/dupes
    $ brew install gdb
  3. If that worked, then lucky you! Getting it compiled is where most people seem to have trouble. Now you just need to sign it to give it permission to control OS X processes. Open up the Keychain Access application (/Applications/Utilities/Keychain Access.app). Navigate via the menu to Keychain Access > Certificate Assistant > Create Certificate...

    create-cert-menu

    Enter a name for the certificate. For this how-to, I'll call it "gdb-cert". Set the fields exactly as shown below.

    create-cert-1

    The maximum validity period is 999 days. I don't really want to deal with this again, so I'm going to max it out.

    create-cert-2

    Keep clicking the "Continue" button until you are asked for a location. Set it to "System".3

    create-cert-3

    Success!

    create-cert-4

    Now make sure the cert is always trusted. Right-click the new certificate and select Get Info. Under the Trust section, set Code Signing to Always Trust.

    cert-get-info

    cert-always-trust

    Now that we have a certificate, we need to use it to sign GDB. First, we'll restart the taskgated process to make sure it picks up the new certificate. Quit Keychain Access (you must quit Keychain Access!) and return to the Terminal for these final commands.

    Find the taskgated process.

    $ ps -e | grep taskgated
    56822 ??         0:03.11 /usr/libexec/taskgated -s
    60944 ttys002    0:00.00 grep --color=auto taskgated

    The first number in the above output is the PID. Use this to kill the process (it will immediately restart itself).

    $ sudo kill -9 56822

    Now you can finally code sign GDB.

    $ codesign -s gdb-cert $(which gdb)

    Now you should be all set! The OS X Keychain may ask for your password the first time you attempt to debug a program, but it should work!

Getting it to Work with Eclipse

There's one more step for Eclipse users. You need to specify where Eclipse can find the new GDB. Specify the path to GDB in Preferences > C/C++ > Debug > GDB:

eclipse-gdb-pref

If you already have some debug configurations, you may need to edit them individually to point to the correct place (under Run > Debug Configurations...):

eclipse-gdb-debug


  1. The CDT developers are planning to support LLDB, but they will have to write a whole new interface, and I think most of them only work on Eclipse in their spare time, so it will likely be at least some months before LLDB support is there.
  2. The procedure is derived from this StackOverflow post and this GDB Wiki page.
  3. If you are unable to save it to the System keychain, then save it to the login keychain. You can later export the cert, and then import it into the System keychain. I didn't have to do this, so comment if you have any problem.

Freedom's Just Another Word for Nothing Left to Work For

Try this idea on for size.

Just as the atom bomb [was] supposed to render war obsolete, the Internet seems like capitalism’s ultimate feat of self-destructive genius, an economic doomsday device rendering it impossible for anyone to ever make a profit off anything again.

—Tim Kreider in Slaves of the Internet, Unite!

In a world with Groupon hemorrhaging millions of dollars a year and Facebook reaching a market cap of $104 billion while making under 0.2% that amount each year, this is all too easy to believe. It seems like the profit models of most major tech companies involve living off research grants and investor funds. Facebook exists simply because the public values it enough to keep it afloat. In a world of BitTorrent, open source, freeware, ad-funded, and "freemium" revenue models, it really is starting to look like the beginning of the end of capitalism.

But is this really all that bad? Is capitalism the only working model? Here's another great excerpt from the same piece:

“Let us not kid ourselves,” Professor Vladimir Nabokov reminds us. “Let us remember that literature is of no practical value whatsoever. ... ” But practical value isn’t the only kind of value. Ours is a mixed economy, with the gift economy of the arts existing (if not exactly flourishing) within the inhospitable conditions of a market economy, like the fragile black market in human decency that keeps civilization going despite the pitiless dictates of self-interest.1

As I sit here sharing my own free, unsolicited2 opinions with the world, I'm prompted to think of the implications of recession and academic inflation in my own industry. Interestingly, this is even a problem in the research sciences, with today's glut of PhDs. The blog linked there hints at my own opinion: students and other hopefuls cannot stand up for themselves. Employers must refuse to let people work for free. It hurts the whole economy, and it is not the way forward.

Just the other day I was talking to a friend about the possibility of entering a post-scarcity economy in the next few decades. Sometimes the dearth of jobs and the neverending recession make me think that post-scarcity might almost be upon us. After all, we are becoming ever more efficient with food production.3 There is already enough food produced each year to feed everyone in the world (and there has been since 2002). What's the problem with not everyone having a job, as long as we can continue to feed them?

More and more tasks are becoming automated, putting humans out of menial jobs. This creates more demand for jobs that cannot be automated because they depend on human creativity, such as the jobs in art and writing that are now struggling for pay. It also creates more demand for jobs that require true human intelligence, which involves more than basic training; these are jobs in science, law, medicine, and engineering that demand years of education. These jobs cannot be filled by robots in the near future, or possibly any future.

So as labor jobs experience a shortage, knowledge work experiences growth. But these jobs will be less numerous as the economy becomes more productive. As a rule, a single person will be able to achieve more and more in an increasingly technological economy. Their job will become more automated. They will acquire more sophisticated tools that can accomplish the same tasks in less time. The demands of the world simply will not be able to keep up with this new level of productivity. Thus, there will be less demand for people to work. There will still be jobs to be done, but the few that are left will require a lifetime of intensive training, and cannot (nor need not) be filled by everyone.

Perhaps Karl Marx's day is approaching at last, though not quite in the way he imagined it. If everyone could have basic living expenses paid for, then we could all be free to do what inspires us. I could stop hearing artists complain about how much I'm pirating their music, and patent trolls could release their stranglehold on the tech industry. Perhaps most surprising is that these aren't just the whacky thoughts of some out-of-touch roboticist with a soft spot for science fiction.4 In actual fact, Switzerland is already voting on it.


  1. He certainly has a way with words, doesn't he? Maybe he should get paid after all.
  2. Surprisingly, no one asked me to write this (unlike some of Kreider's work).
  3. We're also having trouble managing the effects of big agro on the environment, but I view that as a separate issue.
  4. Well, they are also that.

(See the Exercises)

From a single section of a single chapter of an actual textbook:

... it can easily be shown that the boundary of the specularity is defined by (see exercises)  1 - \epsilon = \mathbf{V} \cdot \mathbf{P} .

Okay, fair enough.

It is easily shown (see exercises) that the normal to a parametric surface ...

Easy, my ass.

There are two obvious possibilities. We explore the consequences of these models in the exercises.

I obviously knew you were going to say that.

We explore the minimization problem in the exercises.

Are you guys bored writing this chapter?

... (you can convince yourself of this with the aid of a spoon).

Definitely bored.

The recovered surface does not depend on the choice of curve (exercises).

Okay, now you're just getting lazy.

... (it is complicated, but not difficult, to build more elaborate models).

Complicated, but not difficult... complicated... not difficult... until just now, I thought those two words were the same thing.


By the way, here's a couple of typical exercises:

2.10. Read the book Colour and Light in Nature, by Lynch and Livingstone, published by Cambridge University Press, 1995.

2.12. Make a world of black objects and another of white objects (paper, glue, and spraypaint are useful here) and observe the effects of interreflections.

New Grad Student Insights

Grad students love “pourovers.” No, that’s not some French Canadian word… see here. (Personally, I don’t know what’s so difficult about using a French press.)

College is still worth it. A post-secondary education is still a worthwhile investment, if we are safe in making certain assumptions. (This data is from my home country, so it might not even be applicable here in Canada.)

This concrete 'E' gets graffitied about once a week.

Nerds can tag, too.

The major sources of funding in Canada are: The Canadian Institutes of Health Research (CIHR); the Natural Sciences and Engineering Research Council (NSERC); and the Social Sciences and Humanities Research Council (SSHRC).

  • If you’re in medicine or biology, you’re probably funded by CIHR.
  • If you’re in other sciences, you hope to have NSERC money coming your way.
  • If you’re in the humanities, you look toward SSHRC.

Calhoun’s internet tickets don’t expire. If you don’t use the internet when you’re there, your ticket will still be valid the next time you visit. I used a week-old ticket today. (Calhoun’s is a 24-hour café/bakery where an endless torrent of UBC students come to study.)

Featuring: Canadian Tuxedo!

Tater tot eating contest! (Huh?)

Being a(n adjunct) professor can kill you. Literally.

Blackboard is still just as crappy as it was 7 years ago. (Yes, the last time I was in school was almost 7 years ago.)

The 99 B-Line is really loud... at 1 a.m. on a Saturday on the way back to UBC, that is!

Computer science professors at UBC prefer the term “computational intelligence” to “artificial intelligence.” Because after all, when does it cease to be artificial? What is artificial? Whoa, that's deep...

The Stanford Prison Experiment is not all that remarkable. It seems like every pop psychology article I read cites the famous Stanford Prison Experiment in one way or another, as an example of the inherent brokenness of justice systems, or something like that. But now that I’ve actually been forced to read about it as part of research ethics training,1 it’s clear to me that the experiment was designed to produce draconian behavior from the get-go! We cannot safely conclude anything from this study.2

UBC campus is an ecosystem. There are skunks and raccoons and giant seagulls everywhere! And these are my coonfriends. These guys aren't the least bit phased by humans.

There are amazing free apps to manage libraries of PDF documents. Though I have far from explored the space yet, the main ones seem to be:

These are objectively awesome. I have even started to use them for note-taking in class, instead of paper. I might write a more detailed blog post about this in the future, depending on how well it works out.

It’s hard for students to ask questions in class. Some instructors get annoyed when students get lost in class and don’t ask for help. But now that I’m a student again, I can see that it’s nearly impossible to ask good questions. I’m not going to say any more about this now; I’ll write a full post on this later.

Finally, don’t feed the grad students.

A peek into the Computer Science Grad Student Bullpen.

 


  1. Look out for this if you’re a new grad, too. You must take one of these if you are going to be involved in any kind of human or animal trials! Ask your advisor if you need it.
  2. There was no clear experimental design, no independent variable, no controls: just an environment designed to produce maximum psychological effect. There is next to nothing to be learned from the Stanford Prison Experiment; at best, it is an anecdote, not an experiment. And here we have entire theories of social psychology and humanity built upon it. In fact, the only really interesting observation to be made is how out of 50 outside observers of the experiment, only a single person ever objected to what was being done. Disturbing, indeed, but not conclusive of anything in particular.