Graphics for Games 101. Asset Pipeline

 
Author:  Follow: TwitterFacebook
Job Title:Sarcastic Architect
Hobbies:Thinking Aloud, Arguing with Managers, Annoying HRs,
Calling a Spade a Spade, Keeping Tongue in Cheek
 
 
Asset assembly

#DDMoG, Vol. V
[[This is Chapter 17(a) from “beta” Volume V of the upcoming book “Development&Deployment of Multiplayer Online Games”, which is currently being beta-tested. Beta-testing is intended to improve the quality of the book, and provides free e-copy of the “release” book to those who help with improving; for further details see “Book Beta Testing“. All the content published during Beta Testing, is subject to change before the book is published.

To navigate through the book, you may want to use Development&Deployment of MOG: Table of Contents.]]

In Chapter XIV, I am about to write about the thing which is traditionally very dear for a vast majority of gamedevs out there; of course, I am speaking about graphics. However, please keep in mind that

in this book you will NOT find any advanced topics related to graphics.

Hare thumb up:there are quite a few Really Good Books on graphicsWhile graphics (especially 3D one) is an endless topic (very well worth several volumes just about it), I am very cautious about going into too much details about graphics here. First and foremost, the subject of this book is about Multiplayer Online Games (and graphics, while obviously important, is only one of the components of an MOG). Second, there are quite a few Really Good Books on graphics (especially on 3D graphics), and TBH, I won’t be able to match authors of those books on that turf anyway, so instead I will just provide pointers to those Really Good Books.

What you will find in this chapter, is the very very basics of the graphics, just enough to start reading the other books on the topic, AND (last but not least) to understand other things which are essential for networking programming and game development flow.

Bottom line:

if you’re a gamedev with at least some graphics experience – it is probably better to skip this Chapter to avoid reading about those-things-you-know-anyway.

This Chapter is more oriented towards those developers who are coming from radically different fields such as, for example, webdev or business app development (and yes, switch from webdev into gamedev does happen).

Asset Pipeline

Unless our game is graphics-less (which MAY happen, but is not too likely), there is one all-important thing which we’ll need to deal with: it is “assets” (sometimes referred to as “content”, as in “content is the king” 🙂 ).

From our perspective, pretty much any image, 3D model, or video, is considered an “asset”. And as a Big Fat rule of thumb, assets are not created by developers, but by artists, 3D modelers, and game designers.

In turn, it immediately raises a bunch of questions:

  • how asset creators interact with the rest of the team
  • which tools and formats they use to provide data for the game engine, and
  • how they can see what they’ve just changed, in the context of the game?

Simple Asset Pipeline

These questions got answered as soon as we define our “asset pipeline”. A simple example of an “asset pipeline” may go as follows:1

Fig XIV.1

In this example (which describes a flow for a game with “cutout animation”), your asset workflow (which essentially describes how asset moves along the asset pipeline), goes as follows:

  • Artist draws “body parts” in Krita. While working on the “body parts”, she’s using with default Krita file format (.kra).
  • When “body parts” are ready, she exports the results of her work from.kra into .png (so that next stage may understand it).2
  • Then, animator takes these “body parts” and makes animations out of them in Spriter. At this point, he uses incoming .pngs and keeps results of his work in Spriter’s native .scml.
    • If he sees that body parts are not good for the animation he makes, he comes back to the artist, saying “Houston, we have a problem”. This, in turn, may lead to redrawing body parts (and to redoing animation)
  • When animations are ready, he exports them as .png “sprite sheets”.
  • Bingo! Your 2D game engine uses these “sprite sheets” as a part of your game
    • However, after looking at it within the engine, there may be questions either to animator, or to artist (or to both). Which means going back to the drawing board. Actually, most of the time it is animator himself who tests his animations within the game engine and fixes whatever-is-necessary-to-fix.

It needs to be noted that the example above is a REALLY simple “asset pipeline”. However, we can already see certain properties of the “asset pipeline”:

  • “Asset workflow” is an inherently iterative process
  • Judging hare:you MUST NOT edit any of intermediate files directlyHowever tempting it might look, you MUST NOT edit any of intermediate files directly. THE ONLY way to change things should be via source files (and associated tools).3
  • Changes in the beginning of the chain (“upstream”) usually mean some work for later stages in the chain (“downstream”)
  • Moving of the assets along the “asset pipeline” can also be seen as moving along the “toolchain” consisting of different tools (with each of the tools in the chain preparing the data for the next one – and in the format which can be used by the next step too).
  • There are multiple formats involved, both formats which are “native” to the tools, and “exchange” formats.
    • those “exchange” formats (in the example above – simple .pngs for bodyparts, and sprite sheet .pngs), are important. In particular, as soon as you fix .png as THE format for your “body parts”, you don’t need to care whether your artist uses Krita (or she prefers Photoshop), and so on. In short: keeping exchange formats open and commonly supported (opposed to proprietary and tool-specific) improves flexibility of your “asset pipeline” (and it IS important).

1 Note that the fact that I’m using ANY of the tools in examples, doesn’t mean that this particulartool is a good fit for your project; recommendations on specific tools are EXTREMELY out of scope of this book, and this is one of those things which you’ll need to figure out yourself
2 If we’re speaking about Krita and Spriter, there are other ways to skin this cat, but for this example, we’ll stick to the pattern which doesn’t rely on relations between specific tools
3 If you feel that there is resemblance of the asset pipeline to the IDL process as was described in Vol.1 Chapter III – you’re right, the flow is somewhat similar

 

Enter Game Editor

The asset workflow above actually ends at the point when we’ve got our animations; in other words, there is no such thing as “level” within our workflow – and this is detrimental for quite a few games out there. Let’s consider a bit more complicated workflow (for the purposes of this example – Unity-based, but actually pretty much any game engine will look similar).

Fig XIV.2

The asset flow shown on Fig XIV.2 is more complicated for several reasons:

  • First of all, our artists are working with vector graphics to start with. Animation is also vectorized. The idea here is to support our game in two different resolutions (and at the same time to avoid scaling or raster images, which is almost inevitably visibly inferior to vector conversion to the raster).
  • To achieve this, after animators has finished their work in Animate CC (former Adobe Flash Pro) – we’re exporting the sprite sheets from Animate CC into two different resolutions; these are the same sprite sheets, but in different resolutions.
  • Then, we feed one of these sprite sheets to Unity Editor, so Game Designer can design levels of our game. And as soon as game levels are ready – both SD and HD sprite sheets can be used by Unity Game Engine, to provide experience.

With this kind of workflow, the following additional considerations become apparent:

  • There is a new person in the picture – Game Designer
  • And a new class of tools too – Game Editor
    • Sometimes, Game Editors have an ability to edit other things than game levels too; however, you should be Very Careful to make sure that you don’t edit those intermediate files (which is still a Big No-No, as mentioned above)
  • Certain export steps can become complicated; in particular, settings during different types of export can become complicated/non-obvious/etc.
    • For anywhere sizeable projects, it usually leads to exports (or imports) being performed not manually, but by automated scripts.
      • Inquisitive hare:How will I implement automation?In turn, this means that whenever choosing your tools, one of the criteria you usually should consider is “How will I implement automation?”. As a rule of thumb, the more elaborated automation is provided by the tool itself – the better; however, sometimes even simple command-line export can save your bacon. On the other hand – if the only way to automate the tool is via external scripts like AutoIt – it often qualifies as a Significant Disadvantage 🙁 . Out of those tools mentioned above – all Adobe tools support very powerful ActionScript scripting, Krita has that minimal command-line export, so it MIGHT be ok, but I don’t know of an automated way of exporting from Scripter, and for anywhere sizeable projects it IS a problem 🙁 .

Enter 3D: Pre-rendering Pipeline

3D pre-rendering was pretty much the only way to have 3D-looking images in mobile games for a while; still, while these days your mobile phone can run quite a bit of 3D, pre-rendered stuff is still abundant (especially for Web-based and mobile-based games).

Moreover, even full-scale 3D games tend to have a significant amount of pre-rendered stuff; the most obvious examples include transitory and very computational-intensive things such as explosions and fire. These can be pre-rendered and then just inserted into a genuine 3D scene.

More or less typical pre-rendered pipeline is shown on Fig XIV.3:

Fig XIV.3

Let’s note a few peculiarities of such asset pipelines:

  • Surprised hare:Typically, most of the 3D work is done using one or two 3D packagesTypically, most of the 3D work is done using one or two 3D packages (in the example on Fig XIV.3 it is 3D Max and Maya 3D). This is related to the observation that 3D formats between different 3D packages are rarely fully compatible; i.e. even if a format is supposedly supported by both tools, in practice it often happens4 that significant information is lost during the transfer. For example, while meshes may transfer correctly, information about associated materials (texture mapping and especially shaders) is often lost during the inter-tool transfer :-(.
    • Unfortunately, this also applies to COLLADA interchange format. While COLLADA is theoretically supported by almost all the 3D tools, its interpretation by different tools is different, and as a result, it cannot provide necessary level of interoperability [StackOverflow.COLLADA].
    • One exception to this is 3D meshes (in Wavefront .obj format), which are generally compatible (as long as they’re merely meshes with no UV mapping, no materials, etc.). This is one of the reasons why if two tools are used, the line between them often goes via .obj format (as it is shown on Fig XIV.3).

Full 3D

Full 3D pipeline is quite similar to the pre-rendered one:

Fig XIV.4

Hare pointing out:Despite all the visual similarities of the two diagrams, in practice full 3D is quite different from pre-renderingThe only (and quite obvious) difference on the diagram is that we’re not pre-rendering sprites, but instead are feeding information into Unity engine. However, it needs to be kept in mind that despite all the visual similarities of the two diagrams, in practice full 3D is quite different from pre-rendering.

Most importantly, while with pre-rendering 3D artists (all of them, from 3D modeler to animator) may use pretty much whatever-features-of-software-they-want (as rendering is done within the same software – it is unlikely to be a problem), for 3D import it tends to be quite restricted by capabilities of the game engine importer. Among other things, it means that

YOU SHOULD TRY THE WHOLE THING WITHIN THE GAME ENGINE AS SOON AS POSSIBLE

Otherwise, you can easily end up in a situation when you’ve got models and animations which look ideally in Maya (or whatever-else-tool-you’re-using), but which look Really Ugly in your game engine (most nasty examples can include such things as unusual meshes looking just plain wrong, to losing all the materials).

TODO: DIY tools and versioning

Asset Pipeline and Source Control

It is usually beneficial to keep your source assets within your source control system. However, there are certain caveats to be kept in mind in this regard:

  • while you generally SHOULD put your source assets (those which are edited by people) under your source control, a question whether intermediate assets (those obtained from source assets by purely automated means) belong there, is a not-so-obvious one.
  • Wtf hare:you should forget about merging of your assetsmost of the time, you should forget about merging of your assets
    • even if your tool uses a text format, most of the tools out there don’t care about producing “canonical” version of the asset, and two different saves which are pretty much equivalent, may still look VERY different at text level
      • in theory, it may be possible to write a “canonicaliser” for such a text format, and to check in “canonicalised” versions. While I didn’t see such things myself, I’ve heard about them; feel free to try, but don’t blame me if it doesn’t work out ;-).
    • as a result – try to make your artists to split assets into as small files as possible.
  • Files (especially intermediate ones) can be LARGE. Make sure that your source control system can handle it.
  • More often than not, you may face resistance from your artists.
    • Try to make their life as easy as possible and avoid stuff which is familiar to software developers but are foreign for everybody-else. In particular, trying to explain gitflow to artists is usually detrimental; simpler things (usually along the lines of using-git-as-svn or something even simpler) are usually preferable for artists.
  • If necessary – you may want to consider two source control systems – one for code, another for assets.

Asset Pipeline Summary

Of course, we didn’t even scratch a surface of asset- and toolchain-related issues; however, I hope I’ve managed to convey the message of how important and how complicated asset pipeline and toolchain can easily become. In fact, it is unclear what is of more importance for the games – game engine itself or the associated toolchain.

A few observations to summarise discussion above:

  • Assertive hare:Asset pipeline and associated toolchain are DAMN importantAsset pipeline and associated toolchain are DAMN important
    • in particular, available tools often dictate which game engine to use 🙁
    • at the very least, DON’T try to consider engines without considering tools
  • Asset pipeline IS interactive
    • in particular, round-trip times (from changing certain thing to trying it in the engine) are VERY IMPORTANT
  • Asset pipelines are often ridden with Extremely Serious interoperability problems 🙁
    • If your tool says that it supports files from some other tool – DON’T rely on it working, until you try it.
      • try EVERY feature you need, before relying on it
      • get your assets into your game engine ASAP, to make sure that the whole pipeline works as intended
      • keep testing things within your engine all the time

On Specific Tools

To avoid being beaten hard (and this time for no reason ;-)), I want to re-emphasize that any discussion of specific tools is NOT within the scope of this book. In particular, there are several 3D tools which weren’t mentioned in the text above (probably most importantly – Blender and MODO); this doesn’t mean that I don’t like these tools or something.

I just don’t feel qualified to provide any advice on selecting graphics tools; moreover, I think that the choice should be left to artists (well, within the budget you have ;-( ).

Further Reading

As mentioned above, we haven’t even started to discuss asset pipelines anywhere seriously. In case if you want to know more (and at least somebody on your team should), here goes the list of reasonably good sources for reference:

  • [Carter04] – a bit old, but IMO still a good starting point (and comprehensive too). A short excerpt can be found in [Gamasutra.Carter]
  • [Green09] – discussing “import pipelines”
  • [Arnaud11] – discusses “pull” pipelines and COLLADA; as for the latter – keep in mind that since that point, it became apparent to most of the industry that COLLADA is not going to work in real world 🙁 [StackOverflow.COLLADA].

[[To Be Continued…

Tired hare:This concludes beta Chapter 17(a) from the upcoming book “Development and Deployment of Multiplayer Online Games (from social games to MMOFPS, with social games in between)”. Stay tuned for beta Chapter 17(b), where we’ll start discussing 2D graphics…]]

Don't like this post? Comment↯ below. You do?! Please share: ...on LinkedIn...on Reddit...on Twitter...on Facebook

[+]References

Acknowledgement

Cartoons by Sergey GordeevIRL from Gordeev Animation Graphics, Prague.

Join our mailing list:

Comments

  1. Christoffer Green says

    Hello, slight error: “As mentioned above, we didn’t even started to discuss asset pipelines”

    should be: “we didn’t even start to” or better yet “we haven’t even started to”

    • "No Bugs" Hare says

      Yep, that’s what happens when I’m making last-minute changes after the proofreader has already made her pass :-(. Fixed, thanks!

  2. Wanderer says

    One thing I’d like to add is about any DIY tools and formats. Based on my experience, it’s crucial to have versioning in both tools and formats. Even if it’s something simple as 10-lines JSON or CSV file, without versions things get pretty ugly soon.

    People tend to forget old versions of a tool on their machines. Some assets files “magically” reappear after several months, for example, when it is decided by lead artist that 3-months old “blue magical fog” is actually better than current “purple magical lighting” and this asset rolled back.

    Since it’s usually a huge unnecessary overhead to implement backward compatibility in the assets toolchain, those “outdated” tools and assets can create a major headache for team members and produce all kinds of weird results.

    Sometimes, when new format doesn’t have “breaking changes”, such things can happily float around for some significant time. Imagine something like:
    – a mesh file that has inverted normals, which do work fine in most of the cases because pixel shaders use different shading technique in most of the game;
    – or i18n CSV file coming from professional team of translators in the form of huge Excel sheet that has Romanian and Hungarian columns swapped out (and nobody actually remember that the column order was changed for some practical reason);

  3. Christoph Flick says

    Hey, I Just have a short question on the “intermediate Assets”. Are these all the assets between the start and end point, like the first and last artist? Or are these only the generated things like the .png from .kra?
    Why is it so bad to change then?
    Greets and thanks you for your great articles!

    • "No Bugs" Hare says

      Usually, it is just those generated things. And changing them is bad, because if your artist will EVER need to change the original – all the changes to the generated asset will be lost 🙁 – and you’ll need to reapply them again, which is known to be a Really Big Headache :-((.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.