Graphics for Games 101. Asset Pipeline

	Author:	“No Bugs” Hare  Follow:
	Job Title:	Sarcastic Architect
	Hobbies:	Thinking Aloud, Arguing with Managers, Annoying HRs, Calling a Spade a Spade, Keeping Tongue in Cheek

To navigate through the book, you may want to use Development&Deployment of MOG: Table of Contents.]]

In Chapter XIV, I am about to write about the thing which is traditionally very dear for a vast majority of gamedevs out there; of course, I am speaking about graphics. However, please keep in mind that

“there are quite a few Really Good Books on graphicsWhile graphics (especially 3D one) is an endless topic (very well worth several volumes just about it), I am very cautious about going into too much details about graphics here. First and foremost, the subject of this book is about Multiplayer Online Games (and graphics, while obviously important, is only one of the components of an MOG). Second, there are quite a few Really Good Books on graphics (especially on 3D graphics), and TBH, I won’t be able to match authors of those books on that turf anyway, so instead I will just provide pointers to those Really Good Books.

What you will find in this chapter, is the very very basics of the graphics, just enough to start reading the other books on the topic, AND (last but not least) to understand other things which are essential for networking programming and game development flow.

Bottom line:

This Chapter is more oriented towards those developers who are coming from radically different fields such as, for example, webdev or business app development (and yes, switch from webdev into gamedev does happen).

Asset Pipeline

Unless our game is graphics-less (which MAY happen, but is not too likely), there is one all-important thing which we’ll need to deal with: it is “assets” (sometimes referred to as “content”, as in “content is the king” 🙂 ).

From our perspective, pretty much any image, 3D model, or video, is considered an “asset”. And as a Big Fat rule of thumb, assets are not created by developers, but by artists, 3D modelers, and game designers.

In turn, it immediately raises a bunch of questions:

• how asset creators interact with the rest of the team
• which tools and formats they use to provide data for the game engine, and
• how they can see what they’ve just changed, in the context of the game?

Simple Asset Pipeline

These questions got answered as soon as we define our “asset pipeline”. A simple example of an “asset pipeline” may go as follows:¹

In this example (which describes a flow for a game with “cutout animation”), your asset workflow (which essentially describes how asset moves along the asset pipeline), goes as follows:

• Artist draws “body parts” in Krita. While working on the “body parts”, she’s using with default Krita file format (.kra).
• When “body parts” are ready, she exports the results of her work from.kra into .png (so that next stage may understand it).²
• Then, animator takes these “body parts” and makes animations out of them in Spriter. At this point, he uses incoming .pngs and keeps results of his work in Spriter’s native .scml.
  • If he sees that body parts are not good for the animation he makes, he comes back to the artist, saying “Houston, we have a problem”. This, in turn, may lead to redrawing body parts (and to redoing animation)
• When animations are ready, he exports them as .png “sprite sheets”.
• Bingo! Your 2D game engine uses these “sprite sheets” as a part of your game
  • However, after looking at it within the engine, there may be questions either to animator, or to artist (or to both). Which means going back to the drawing board. Actually, most of the time it is animator himself who tests his animations within the game engine and fixes whatever-is-necessary-to-fix.

It needs to be noted that the example above is a REALLY simple “asset pipeline”. However, we can already see certain properties of the “asset pipeline”:

• “Asset workflow” is an inherently iterative process
• “you MUST NOT edit any of intermediate files directlyHowever tempting it might look, you MUST NOT edit any of intermediate files directly. THE ONLY way to change things should be via source files (and associated tools).³
• Changes in the beginning of the chain (“upstream”) usually mean some work for later stages in the chain (“downstream”)
• Moving of the assets along the “asset pipeline” can also be seen as moving along the “toolchain” consisting of different tools (with each of the tools in the chain preparing the data for the next one – and in the format which can be used by the next step too).
• There are multiple formats involved, both formats which are “native” to the tools, and “exchange” formats.
  • those “exchange” formats (in the example above – simple .pngs for bodyparts, and sprite sheet .pngs), are important. In particular, as soon as you fix .png as THE format for your “body parts”, you don’t need to care whether your artist uses Krita (or she prefers Photoshop), and so on. In short: keeping exchange formats open and commonly supported (opposed to proprietary and tool-specific) improves flexibility of your “asset pipeline” (and it IS important).

Enter Game Editor

The asset workflow above actually ends at the point when we’ve got our animations; in other words, there is no such thing as “level” within our workflow – and this is detrimental for quite a few games out there. Let’s consider a bit more complicated workflow (for the purposes of this example – Unity-based, but actually pretty much any game engine will look similar).

The asset flow shown on Fig XIV.2 is more complicated for several reasons:

• First of all, our artists are working with vector graphics to start with. Animation is also vectorized. The idea here is to support our game in two different resolutions (and at the same time to avoid scaling or raster images, which is almost inevitably visibly inferior to vector conversion to the raster).
• To achieve this, after animators has finished their work in Animate CC (former Adobe Flash Pro) – we’re exporting the sprite sheets from Animate CC into two different resolutions; these are the same sprite sheets, but in different resolutions.
• Then, we feed one of these sprite sheets to Unity Editor, so Game Designer can design levels of our game. And as soon as game levels are ready – both SD and HD sprite sheets can be used by Unity Game Engine, to provide experience.

With this kind of workflow, the following additional considerations become apparent:

• There is a new person in the picture – Game Designer
• And a new class of tools too – Game Editor
  • Sometimes, Game Editors have an ability to edit other things than game levels too; however, you should be Very Careful to make sure that you don’t edit those intermediate files (which is still a Big No-No, as mentioned above)
• Certain export steps can become complicated; in particular, settings during different types of export can become complicated/non-obvious/etc.
  • For anywhere sizeable projects, it usually leads to exports (or imports) being performed not manually, but by automated scripts.
    • “How will I implement automation?In turn, this means that whenever choosing your tools, one of the criteria you usually should consider is “How will I implement automation?”. As a rule of thumb, the more elaborated automation is provided by the tool itself – the better; however, sometimes even simple command-line export can save your bacon. On the other hand – if the only way to automate the tool is via external scripts like AutoIt – it often qualifies as a Significant Disadvantage 🙁 . Out of those tools mentioned above – all Adobe tools support very powerful ActionScript scripting, Krita has that minimal command-line export, so it MIGHT be ok, but I don’t know of an automated way of exporting from Scripter, and for anywhere sizeable projects it IS a problem 🙁 .

Enter 3D: Pre-rendering Pipeline

3D pre-rendering was pretty much the only way to have 3D-looking images in mobile games for a while; still, while these days your mobile phone can run quite a bit of 3D, pre-rendered stuff is still abundant (especially for Web-based and mobile-based games).

Moreover, even full-scale 3D games tend to have a significant amount of pre-rendered stuff; the most obvious examples include transitory and very computational-intensive things such as explosions and fire. These can be pre-rendered and then just inserted into a genuine 3D scene.

More or less typical pre-rendered pipeline is shown on Fig XIV.3:

Let’s note a few peculiarities of such asset pipelines:

• “Typically, most of the 3D work is done using one or two 3D packagesTypically, most of the 3D work is done using one or two 3D packages (in the example on Fig XIV.3 it is 3D Max and Maya 3D). This is related to the observation that 3D formats between different 3D packages are rarely fully compatible; i.e. even if a format is supposedly supported by both tools, in practice it often happens⁴ that significant information is lost during the transfer. For example, while meshes may transfer correctly, information about associated materials (texture mapping and especially shaders) is often lost during the inter-tool transfer :-(.
  • Unfortunately, this also applies to COLLADA interchange format. While COLLADA is theoretically supported by almost all the 3D tools, its interpretation by different tools is different, and as a result, it cannot provide necessary level of interoperability [StackOverflow.COLLADA].
  • One exception to this is 3D meshes (in Wavefront .obj format), which are generally compatible (as long as they’re merely meshes with no UV mapping, no materials, etc.). This is one of the reasons why if two tools are used, the line between them often goes via .obj format (as it is shown on Fig XIV.3).

Full 3D

Full 3D pipeline is quite similar to the pre-rendered one:

“Despite all the visual similarities of the two diagrams, in practice full 3D is quite different from pre-renderingThe only (and quite obvious) difference on the diagram is that we’re not pre-rendering sprites, but instead are feeding information into Unity engine. However, it needs to be kept in mind that despite all the visual similarities of the two diagrams, in practice full 3D is quite different from pre-rendering.

Most importantly, while with pre-rendering 3D artists (all of them, from 3D modeler to animator) may use pretty much whatever-features-of-software-they-want (as rendering is done within the same software – it is unlikely to be a problem), for 3D import it tends to be quite restricted by capabilities of the game engine importer. Among other things, it means that

Otherwise, you can easily end up in a situation when you’ve got models and animations which look ideally in Maya (or whatever-else-tool-you’re-using), but which look Really Ugly in your game engine (most nasty examples can include such things as unusual meshes looking just plain wrong, to losing all the materials).

TODO: DIY tools and versioning

Asset Pipeline and Source Control

It is usually beneficial to keep your source assets within your source control system. However, there are certain caveats to be kept in mind in this regard:

• while you generally SHOULD put your source assets (those which are edited by people) under your source control, a question whether intermediate assets (those obtained from source assets by purely automated means) belong there, is a not-so-obvious one.
• “you should forget about merging of your assetsmost of the time, you should forget about merging of your assets
  • even if your tool uses a text format, most of the tools out there don’t care about producing “canonical” version of the asset, and two different saves which are pretty much equivalent, may still look VERY different at text level
    • in theory, it may be possible to write a “canonicaliser” for such a text format, and to check in “canonicalised” versions. While I didn’t see such things myself, I’ve heard about them; feel free to try, but don’t blame me if it doesn’t work out ;-).
  • as a result – try to make your artists to split assets into as small files as possible.
• Files (especially intermediate ones) can be LARGE. Make sure that your source control system can handle it.
• More often than not, you may face resistance from your artists.
  • Try to make their life as easy as possible and avoid stuff which is familiar to software developers but are foreign for everybody-else. In particular, trying to explain gitflow to artists is usually detrimental; simpler things (usually along the lines of using-git-as-svn or something even simpler) are usually preferable for artists.
• If necessary – you may want to consider two source control systems – one for code, another for assets.

Asset Pipeline Summary

Of course, we didn’t even scratch a surface of asset- and toolchain-related issues; however, I hope I’ve managed to convey the message of how important and how complicated asset pipeline and toolchain can easily become. In fact, it is unclear what is of more importance for the games – game engine itself or the associated toolchain.

A few observations to summarise discussion above:

• “Asset pipeline and associated toolchain are DAMN importantAsset pipeline and associated toolchain are DAMN important
  • in particular, available tools often dictate which game engine to use 🙁
  • at the very least, DON’T try to consider engines without considering tools
• Asset pipeline IS interactive
  • in particular, round-trip times (from changing certain thing to trying it in the engine) are VERY IMPORTANT
• Asset pipelines are often ridden with Extremely Serious interoperability problems 🙁
  • If your tool says that it supports files from some other tool – DON’T rely on it working, until you try it.
    • try EVERY feature you need, before relying on it
    • get your assets into your game engine ASAP, to make sure that the whole pipeline works as intended
    • keep testing things within your engine all the time

On Specific Tools

To avoid being beaten hard (and this time for no reason ;-)), I want to re-emphasize that any discussion of specific tools is NOT within the scope of this book. In particular, there are several 3D tools which weren’t mentioned in the text above (probably most importantly – Blender and MODO); this doesn’t mean that I don’t like these tools or something.

I just don’t feel qualified to provide any advice on selecting graphics tools; moreover, I think that the choice should be left to artists (well, within the budget you have ;-( ).

Further Reading

As mentioned above, we haven’t even started to discuss asset pipelines anywhere seriously. In case if you want to know more (and at least somebody on your team should), here goes the list of reasonably good sources for reference:

• [Carter04] – a bit old, but IMO still a good starting point (and comprehensive too). A short excerpt can be found in [Gamasutra.Carter]
• [Green09] – discussing “import pipelines”
• [Arnaud11] – discusses “pull” pipelines and COLLADA; as for the latter – keep in mind that since that point, it became apparent to most of the industry that COLLADA is not going to work in real world 🙁 [StackOverflow.COLLADA].

[[To Be Continued…

This concludes beta Chapter 17(a) from the upcoming book “Development and Deployment of Multiplayer Online Games (from social games to MMOFPS, with social games in between)”. Stay tuned for beta Chapter 17(b), where we’ll start discussing 2D graphics…]]

Don't like this post? Comment↯ below. You do?! Please share:

Acknowledgement

Cartoons by Sergey Gordeev from Gordeev Animation Graphics, Prague.