Category Archives: Programming

On C++ Build Systems

by

0

I’ve been thinking about C++ build systems a lot recently.  I like programming in C++ a lot (relatively speaking), but I don’t think it’s news to anyone that the build system leaves a lot to be desired (dependency management, header + .cpp files, etc.).

It all started back when I started watching Handmade Hero.  Rather than using some complex build systems or even an IDE like Visual Studio, Casey Muratori just used a simple build.bat file to call the compiler.  I’m certainly no fan of batch as a language, but the simplicity of the build script along with the ease of understanding exactly how your program is compiled/linked were appealing.  He also basically just compiled 1 single .cpp file (which obviously simplifies things) – often called a “unity” or “single translation unit” build.

The next major piece came when watching videos for Jonathan Blow’s new “Jai” language.  One critical point Jon makes is that we probably should be writing our build scripts in the language we’re programming in – why bother learning some other (and often inferior) language just to build your program?

There is one point where I differ from Jon and Casey – I actually like using Visual Studio a lot.  I don’t necessarily like everything about it (notably the project files), but on the whole, I feel like the Visual Studio IDE makes me more productive compared to other alternative I’ve tried.

Anyway, it’s probably going to be some time before “Jai” or another language supplants C++.  So I was wondering…why not try and get most of the benefits mentioned above in C++?

Goals

Before we go any further, let me explicitly state some of the goals for my idea for a better C++ build system/IDE integration:

  • Build via the command line with minimal options to remember.
  • Build via Visual Studio.
  • Not specify a bunch of files when building.
  • Easily understand exactly how a program is built.
  • Easily tweak how a program is built.
  • Minimize dependencies on non-native tools (outside of the language, operating system, and Visual Studio).
  • Not manually manage Visual Studio solution, project, etc. files.
  • Easily find all files for a project within Visual Studio.
  • Edit files within Visual Studio.
  • Debug within Visual Studio.
  • Simplicity.

Prior to work put in for this blog post, I played around some with the build.bat and unity build ideas, both from the command line and within Visual Studio (generating solution/project files manually using the IDE).

The Build Script

From the above, the simplest idea for meeting the “build” requirements is the build.bat idea.  Batch files are built into Windows, and the complexity needed to call the compiler isn’t so large that it drives me crazy using the batch language.

All we really need to do to generate such a build script is have a program write out a basic text file, with a few possible parameters tweaked.  Very simple – the text could be easily embedded as strings in a program.

The basic build.bat file Casey Muratori created for Handmade Hero can be used as a starting point.  On tweak I might like to make (haven’t fully settled on this yet) is automatically calling the VCVARSALL batch file in the build script – that way, I don’t have to worry about making sure the appropriate settings are always set.  Unfortunately, one downside to this from my experience is that sometimes you’ll get an error about a command/path being too long if you repeatedly run the build.bat file in the same command prompt Window.  I haven’t run into any problems with this in Visual Studio yet though.

Deciding exactly what compiler flags/settings or other parameters to include in the build.bat file is tricky.  I’m not sure exactly what I personally want yet, but it might be different from what Handmade Hero has.  For now, punting and just having a basic build script for building a debug build is fine.

I might want to actually wrap a lot of the options in the build script in a separate C++ program – that wouldn’t certainly make supporting advanced options/features easier.  Of course, that has the trade-off of being another program people would have to download/build.

On “Unity” Builds

Before moving on, just a few comments on “unity” or “single translation unit” builds.  They’re definitely an interesting idea, with a nice benefit of speeding up build times.

One thing I definitely like about the idea is that you only have to specify 1 file when passed to the compiler – no need to manually manage a bunch of them.

However, I’m not completely sold on the idea, as it sort of requires restructuring how you right code in an unconventional manner.  And while I have no concrete evidence for it, it seems like structuring code/builds this way might complicate re-use in large code bases where you want to use code across multiple projects.  I’d like to still write my code in a more “conventional” C++ way (many separate files, including the less-than-ideal header + .cpp files) to avoid risks in going all-in with a unity build.

How can we get around these conflicts?  Well, Jonathan Blow’s idea of specifying your build in your regular programming language provides some hints.  We can create a separate file that just #includes all of the .cpp files we want to compile – no need to write the rest of your code in an odd way.

This actually works pretty well from my testing, but it isn’t perfect.  In particular, I was hoping that this file with just a bunch of #includes could end with a file extension other than .cpp, but the Visual Studio compiler doesn’t seem to be able to compile files with other extensions (if you know a way around this, let me know).  The idea of not modifying your “other” .h/.cpp files almost may not be 100% feasible – from my testing, I’ve had to make a few modifications/re-order certain things – but the modifications have been relatively minimal so far to a point where it wouldn’t be much of a burden (or maybe no burden at all) if I wanted to return to a “conventional” C++ build model.

It might be nice to generate this .cpp file with just a bunch of #includes automatically one day, but there might be some challenges, and it’s certainly not at the top of my priority list.

Visual Studio Project Files

As mentioned above, I’ve manually created Visual Studio solution/project files for C++ projects using a build.bat + unity build approach.  But I don’t like having to manually add all files in my project or tweak various build settings.

How to solve this problem?  Well, for the most part, the Visual Studio solution/project files can basically just be copied, meaning that we can embed most of the text directly in a simple program.  There are some unique IDs in the files, but at least from my testing, re-using them doesn’t seem to be a problem.

What remains then?  Well, basically just getting a list of files/folders to include in the Visual Studio project and project filters files.  Recursively getting a list of files/folders in a directory…that doesn’t seem too hard, right?  While the Windows API for doing this isn’t super ideal in my opinion, it’s not too complicated.  So doing this should be pretty easy.

Let’s Do This…Well, Already Done…

I was planning to originally write this blog post before I went and tried to implement any of it.  But feeling extra motivated about this recently, I actually went and recently implemented a lot of the basics for what I want (as described above).  And it didn’t take that long – maybe around 3-5 hours total (at night after work).  Given a folder of code and using the quick program I wrote, I can automatically generate a Visual Studio solution/project and build.bat script that actually works (minus project-specific build tweaks).

It’s far from “complete” – I’ve only tested it on one specific project so far, the build.bat script is most definitely insufficient, and there aren’t a lot of options.  I’ll add improvements as I find I need them, but it’s “good enough” now to at least put out there.

The code is available here on GitHub.  I hope it will be helpful to some others out there.  If you have any suggestions, feel free to forward them to me.  Although, since I’m making the code itself public domain, you’re certainly more than welcome to make your own changes to suit your needs.

Software Estimation Resources

by

0

I’ve recently been looking to improve my software estimate/scheduling/management abilities. Below are a few resources that I’ve found helpful in one way or another:

This is largely intended to be a quick reference to things I might want to review at later times, so I’ll update it if I find anything new.

Game Audio Programming Resources

by

0

I’m not really musically inclined at all, and I recently did some basic audio programming work for the Noah’s Ark game that involved low-level audio samples and SFML.  Not sure what (if anything) might ultimately come out of that, but I did learn some things about audio within the context of game programming.  Some of my favorite resources that were helpful during this time are below:

Video

Books

Notes from Noah’s Ark Game Refactoring

by

0

Recently, I mostly finished a fairly large scale refactoring of code for the Noah’s Ark game.  I’m pretty pleased with how it turned out – the code seems simpler, and I think I’ll be able to more rapidly add features.

Part of the refactoring was motivated by some perceived complexity introduced by some architectural ideas I was trying out – namely, “entities”, “components”, and “systems” – that I’d roughly borrowed from other game engine designs that I’ve read about through the years.

Needless to say, those ideas didn’t seem to be working very well for the game I’m working on.  From various things I’ve learned over the last year, it seemed like there were better solutions to things I was dealing with that would allow replacing a lot of complexity with simplicity.

Will the game’s architecture ultimately return to code similar to what it looked like before the refactoring?  Maybe.  But for now, I think the code is in a much better state than it used to be.  Miscellaneous notes from the refactoring are below.

Making main() Meaningful

Originally, I had a very small amount of code in main():

int exitCode = NoahArkGame::RunGame();
return exitCode;

The bulk of the game logic was contained in a NoahArkGame class.  Part of this was a holdover from when I was using Haaf’s Game Engine before switching over to SFML, but even then, it was largely unnecessary.

So I eliminated this NoahArkGame class and basically moved its logic directly into main().  Why?  Well, using this class (along with some others – “state” and “system” classes) resulted in more indirection that was making things a bit harder to understand, force-fitting certain things into a structure that might not actually be appropriate.  Putting the core game logic in main() meant I could more clearly see the connections necessary between different parts of the code and would make it easier to modify as I discovered new things.

Another interesting idea is that many people often describe a program has having a “main loop.”  Well, if you’re someone new to a code base, where would you expect to find the code for the “main loop”?  Without introducing new constructs that someone would have to learn, the first obvious answer that comes to mind is main().  Anything else is introducing a layer of indirection that has to be jumped through.  I should note that I may not 100% agree with this philosophy, but at the very least, there’s nothing inherently bad about putting a lot of code for the Noah’s Ark game directly in main() for now.  Don’t be afraid of main().

One final note – putting logic directly in main() does have the downside of making it harder to unit test (not that I was planning to do so with the NoahArkGame class either).  But a lot of the pieces orchestrated by the main loop (i.e. actual user input, rendering to screen, etc.) are inherently harder to unit test anyway, so trying to “unit test” the could be a near futile effort anyway.  Just a valid trade-off to consider; there may be more effective ways to test the main loop than unit testing too.

Cleaning Up Tile Map Code

I had a ton of code/classes related to dealing with “tile maps” in the game.  Some of this was intentionally duplicated as I was refactoring smaller parts of the game, but even accounting for that, it was still a messy sea of complexity.

I ultimately eliminated several classes – OverworldMap, ScrollableOverworldMap, TileMapBuilder, TileMapObjectLayer, TileMapTileLayer, ITileMapData, ITileMapLayer, OverworldMapSpecification, SurroundingTileMapLoader, TileMapLayers, TileMapLoader, TiledMapJsonFile – that didn’t really have the best abstractions for useful and intuitive “real-world objects” (in the context of the problems of the game) and thus made the code as a whole more difficult to understand.  That left what seems like a more intuitive collection of classes.

A ton of code was also moved/removed from the TileMap class – it either wasn’t needed anymore or seemed better to place elsewhere for now (to avoid making the TileMap class harder to understand).   I should note that the name of the TileMap class may not best fit the abstraction that it represents, but it isn’t quite clear to me yet exactly what the best class name would be.

Loading All Assets At Startup

I was previously loading assets (i.e. tile map/image files) from disk on-demand, only when needed.  That resulted in some jitteryness/slowdowns during gameplay that required additional complexity to solve.

I did some calculations, and it seemed reasonable to fit basically all of my game assets in memory at once (i.e. 50-60 MB of total usage; my “personal limit” is to keep the game under 100 MB of usage).  Furthermore, most of the assets would be needed in memory basically for the entirety of gameplay anyway, so there isn’t going to be much of a reason to try and free up memory for them.

Therefore, I changed things to just load all assets into memory at startup.  This has the downside of taking about 6-8 seconds (it was 12-14 seconds before I added a tiny bit of multi-threading for tile map loading).  But I’m not too concerned about this since I think I can ultimately have most of this go on in the background during an intro sequence that will be added later.  The good news is that the rest of the code generally doesn’t have to worry about asset loading times.

Making Member Variables Public

Another significant change I made was to make most member variables public instead of private.  This is a common “no-no” in encapsulation/object-oriented guidelines, but I’ve been questioning it lately.

First off, nothing is truly “private” – all there really is is data in the computer’s memory.  Various programming languages do somewhat good jobs at trying to protect these private variables from external access, but this is often just a compilation error.  In C++, you can still easily access private members using pointer offsets.  And some languages don’t even properly support “private” member variables – everything is public, and you have to jump through extraordinary hoops to emulate privacy.

So privacy either doesn’t exist in some programming languages and can be subverted using [often bad] techniques in others.

What are the supposed benefits of private members?  Primarily, the intent seems to be to encapsulate/hide implementation details.  There’s certainly some value in that, but just because a piece of data can be thought of as a member of a class, that doesn’t necessarily mean it is an “implementation detail.”  Sometimes, a data member can intuitively be thought of as part of the abstraction presented by a class.  In these cases, it seems to make more sense for that data member to be public – users expect that data to be in the class by virtue of the abstraction it is trying to present.  Trying to hide this and provide public getter/setter methods just results in more code to read/write/maintain and often makes a class harder to use.

Furthermore, most of the “problems” I tend to see in code aren’t actually caused by lack of “private” members – they’re more fundamental design problems.  As code has to get more complex to solve more complex problems, the impact of a public/private member seems to get tinier in the context of these larger design issues.  And if we’re trusting our developers to solve these much more complicated problems, shouldn’t we be trusting them to make responsible decisions when it comes to data access?  In short, the usage of a data member is more of the issue rather than its public/private accessibility.  Good developers should be able to notice code that is potentially accessing a private implementation detail in a negative way and avoid that code entirely.  Rather than treating a symptom of the problem, I’m wondering if perhaps we should focus more on addressing what seems to be the root cause of these problems – mentorship and growth for programmers.  We want programmers to become better people, and not just so that they can craft simple yet exceptional software.

All of that said, I think there is still probably a case for using private members and having public member functions in public API design – you don’t want to cause frustration or unnecessary work for your users if you need to change some implementation detail or data representation.  But for “internal” code that you don’t intend to heavily reuse throughout tons of projects, I question the value of private members.  It’s your code, and you can change it however you see fit.  And if you do need to change something, the compiler (at least in languages like C++) can easily point you to all of the places that need to be changed.

Not Using Classes For Everything

Last but not least, I deleted some classes that were just serving as cruft and making it harder to use/modify code.  In particular, the GraphicsSystem and CollisionSystem classes were eliminated, with the code now existing as free-floating functions in separate files.

I still find [at least what I consider] “object-oriented programming” to be valuable in certain places.  Primarily, allowing a human to work intuitively with “objects” he/she is familiar with is powerful for solving certain kinds of problems.  Classes can also provide a nice home for functionality that can make it easier to find code in a large code base.

However, object-oriented programming certainly is not a silver bullet.  Not every concept is necessarily best represented as an object.  Too many classes can make a code base harder to navigate.  And object-oriented programming can result in too much indirection and structure that could make code harder to understand or change (which can be costly if an object-oriented design is flawed).

Might the free-floating functions eventually end up in a class?  Perhaps.  If I can find a good home for them eventually, that’s something I’d kind of like to do.  But I don’t yet know what that home would be, and right now, free-floating functions makes it easier for me to shape the code into a proper design later.

 

Anyway, that sums up a lot of the major changes I recently made for refactoring code for the Noah’s Ark game (and why).  It’s entirely possible that something I’ve mentioned above is horribly wrong or stupid.  Time will tell, but I think the code is better now than it was before.

Happy coding.

Mini Book Review: Design Patterns Explained

by

0

I recently finished reading Design Patterns Explained.  I’m not really going to do a full review of the book, but there were some helpful parts that seemed worth pointing out.

In general, most of the book didn’t seem all that useful.  In particular, the chapters focusing on specific design patterns were the least appealing.  I can see situations where the general approach of a design pattern might be applicable, but specifics of a particular problem at hand likely would dictate a simpler variation of the pattern that could be derived without needing to specifically think about patterns.

However, there were two particular chapters that I liked a lot and thought were really valuable.

The first is chapter 8 “Expanding Our Horizons”.  There is lots of good discussion in this chapter, but the diagram showing how commonality and variability analysis relate to conceptual, specification, and implementation perspectives was a big highlight.  Thinking about a problem from the “conceptual” perspective helps me stay more focused in the problem domain when programming, and the other two perspectives help bridge the gap between the problem domain and crafting solid code.

The next useful chapter was chapter 16 “The Analysis Matrix”.  Extending the ideas behind commonality and variability analysis, this chapter describes a practical, concrete tool that can easily be used to more clearly see commonalities and variations within a problem.  One particular benefit of this approach that I like is that I think it can be helpful to make sure you don’t overdesign flexibility into your program – the variations you actually need are right there in the matrix (table).

There are other chapters in the book that may be useful.  However, most of the topics they cover were covered in the two chapters outlined above, and those two chapters do a great and better job of explaining the topics.

So while most of Design Patterns Explained didn’t seem too useful to me, I highly recommend checking out the two chapters mentioned above if you get the chance.  Definitely at least worth contemplating to see if they might be useful in certain situations.

Handmade Hero and a New Programming Language?

by

0

Relatively recently, I found and started watching the Handmade Hero series.  In the series, Casey Muratori is basically coding a complex game from scratch (i.e. using no libraries beyond what is required by an operating system).  You can read more by following the link above.

I’m far behind (and probably will never catch up) due to limited time, but it’s been a great series.  While I have different opinions on some things Casey brings up (and certain items may only apply to different types of software), there is no doubt that he knows his stuff and is far more experienced than me.  I’ve learned some neat tricks from watching, and he has encouraged me to think more critically about certain aspects of programming.  If you’re interested in learning lowish-level programming in a language like C, I highly recommend checking it out.

Watching Handmade Hero also led me to find Jonathan Blow’s videos where he talks about a potential new programming language for games.  His insight into flaws in current programming languages and proposals to fix some of these issues were fascinating, and Jonathan even has a prototype compiler for his new language that seems to work pretty darn well.  I’m looking forward to seeing that new language progress.

Anyway, just wanted to say thanks to both of these guys.  If those topics seem interesting to you, I recommend looking into their programming video series.  Lots of neat learning opportunities.