Game Graphics 101: Rendering 2D on GPU. Shaders

	Author:	“No Bugs” Hare  Follow:
	Job Title:	Sarcastic Architect
	Hobbies:	Thinking Aloud, Arguing with Managers, Annoying HRs, Calling a Spade a Spade, Keeping Tongue in Cheek

To navigate through the book, you may want to use Development&Deployment of MOG: Table of Contents.]]

As it was noted in the beginning of this Chapter, please keep in mind that

What you will find in this chapter, is the very very basics of the graphics, just enough to start reading the other books on the topic, AND (last but not least) to understand other things which are essential for networking programming and game development flow.

Bottom line:

This Chapter is more oriented towards those developers who are coming from radically different fields such as, for example, webdev or business app development (and yes, the switch from webdev into gamedev does happen).

Rendering 2D Using 3D Engine

Why 3D Rendering is Important for 2D?

“Why do things using a 3D engine, when we can do the same using simpler - and more cross-platform - methods?After describing that ultra-simple “Minimal DIY 2D engine”  (the one which needed nothing but BitBlt()), and me starting to speak about rendering 2D using 3D engine, you may ask – “Hey, what’s the big idea?? You were arguing for reducing dependencies all the time, so what is the reason to introduce 3D engine as a Big Fat dependency? Why do things using a 3D engine, when we can do the same using simpler – and more cross-platform – methods?”

The answer is simple – using 3D rendering (in particular, shaders), we can get visual effects such as lighting (see, for example, [DesLauriers6] and [Carlin]), dynamic outlines (see [Germanunkol]), and various post-processing effects (from blur to gamma correction, from sepia to motion effects such as waves).

The key point here is that using 3D engine can improve your 2D picture dramatically. Sure, to draw simple sprites you don’t need a 3D engine (and having that minimal DIY 2D engine is often still a good idea, see below). However, using a 3D engine enables visual improvements, which are way too significant to be ignored; in particular, shader-based effects (including my own favourite – lighting) are enabled by using 3D engine.¹ The visual difference of using such effects can (depending on specifics of your game) make it standing out of the crowd really easily – and this is without taking the development cost of going into full-scale 3D pipeline (which is a Really Big effort, which we’ll discuss very briefly a bit later).

As we have a significant improvement in player space at the cost of that (indeed rather nasty) dependency of using 3D engine – it becomes a judgment call whether obtained benefits from those visual effects are worth the dependency, and quite often they are.

Graceful Degradation and Backup non-3D Engine

Graceful degradation Graceful degradation was supposed to allow the page to 'degrade', or remain presentable even if certain technologies assumed by the design were not present, without being jarring to the user of such older software.— Wikipedia —It should be noted that, even if you find a Really Good Use for your 3D engine, I am often still arguing for keeping that simple pure 2D engine – as a second (“backup”) graphics engine.² This way, on those devices with 3D engine, you’ll be able to run your game using 100% of the graphics effects, but on those devices with only 2D capabilities, you’ll still be able to run it in “gracefully degraded” mode.

In addition, if by any chance your 3D rendering happens to fail on 1%³ of devices (which is really difficult to avoid in real world) – your support will be able to say “hey, try unchecking this checkbox”, solving the problem and saving you from quite a lot of disgruntled customers (at least until you figure out the problem, which may take a while); this tends to relieve the pressure on both support and development A LOT.

And last but not least, an ability to switch off that nice-looking but battery-expensive shaders when I am away from my wall socket, brings you as a developer additional brownie points from me as a player :-).

Sprites as Polygons

“To use 3D engine to render 2D graphics, you'll need to move your 2D sprites into 3D spaceAs a very first step for going towards using 3D engine to render 2D graphics, you’ll need to move your 2D sprites into 3D space. Each sprite is usually represented by a plain square in 3D space (with planes of all the squares being parallel to each other).

Most of the time, you’ll be using “orthographic projection” to render your 2D sprites. In this case, one of the way to represent your 2D sprites in 3D world will look as follows:⁴

• you camera will be viewing at Z axis
• each sprite will be represented as a rectangle, placed in a planes which is normal to Z
• this rectangle will have an associated texture corresponding to sprite image
• screen coordinates (X,Y) of your sprites will translate into proportional (X’,Y’) in 3D world
• Z coordinate of your sprites (the one which determines “how far your sprite is from player”) will be a faithful Z’ coordinate in 3D world

It should be noted that “orthographic projection” is not the only one possible for 2D (perspective projections are usable too), but for the “Graphics 101” purposes we’ll assume you’re using orthographic one.

Shaders for 2D

Ok, we’ve got our 2D sprites into 3D space, but simply doing it won’t give you much. On the other hand, having implemented this, you will be able to start using 3D capabilities of your engine, such as setting up lighting in your 3D world.⁵

“Probably the most visually interesting capability of 3D engine when applied to 2D graphics, is shaders.Probably the most visually interesting capability of 3D engine when applied to 2D graphics (as well as probably the most “cool” one in terms of technology behind 😉 ), is shaders. Let’s note that within this book, we won’t go into details of shaders, but rather will just give a very cursory description; moreover, in the context of “shaders for 2D”, we will restrict our discussion to the bare minimum. In particular, at this point we won’t discuss stuff such as rendering pipeline, or UV mapping; these things, while being all-important for real 3D, become rather trivial for our 2D case, and their complexities can be ignored for the time being. A very brief overview of these (and other 3D-related) things will be provided later in this Chapter; for now, we’ll concentrate on the very simple representation of sprites as polygons described above, and will see what can be done with it in mind (which is apparently, “quite a lot” 🙂 ).

What is Shader?

“Strictly speaking, “shader” is a program which is executed for each “vertex” or each “pixel”Strictly speaking, “shader” is a (micro) program which is executed for each “vertex” or each “pixel” of each of your frames.⁶ Let’s forget about “vertex” shaders for the time being, and concentrate on “pixel shaders” for a while.

So, we have a (micro) program which needs to be executed for each of our million or so on-screen pixels, and at least 30 times per second; it means that the program will be executed like 30M times per second. If we try to execute it on a CPU (like 6-core running at 3GHz), we’ll have only like 3e9*6 CPU clocks per second, so we’ll have only like 600 CPU clocks per our shader program (that is, not doing anything else but running shaders). And 600 CPU clocks is not much to put it mildly. Which means that

Of course! These days, even the cheapest GPU will bring you at least 300 or so cores (with monster cards having over 3000 of them), and performing the same thing across all the cores, is what GPUs are really good with. Those 30M shader runs per second, when distributed across 300 GPU cores running @ 1GHz each, will mean that we have 10000 GPU clocks to run our shader. And while GPU cores are weaker than CPU ones⁷ for generic calculations, they have hardware support for those things which matter for graphics and shaders. Just as one example, GPUs tend to have sin/cos operations⁸ at mere 8x of the cost of integer addition operation, while for CPU the ratio is more like 50-200x.

GPU: DirectX/OpenGL

“At the very moment when we take a turn towards GPU-land, we’re pretty much bound to use one of the interfaces which are usually associated with 3D – DirectX or OpenGL.At the very moment when we take a turn towards GPU-land, we’re pretty much bound to use one of the interfaces which are usually associated with 3D – DirectX or OpenGL. Of course, it is possible to have a library which will hide them from view, but deep inside, if we’re using shaders, we’ll still call either one or another.

This, in turn, means that adding shaders to your 3D implies quite significant restrictions on your platforms (and problems for support and bugs-which-occur-on-one-in-a-million-devices too – bugs in obscure 3D implementations tend to be Really Sneaky 🙁 ).

As a result, if possible at all and if non-PC/non-console platforms are involved, I usually suggest to have a “backup” 2D engine as discussed above; this way, you’ll be able to have a Nice-Looking graphics on those devices which can run your shaders etc., and Gracefully-Degraded-but-Still-Playable graphics on those devices which cannot.

Language to describe shaders: HLSL vs GLSL vs Cg

“As GPUs are quite different from CPUs, no wonder that shaders are usually written in their own programming language.As shader is executed on GPU, and GPUs are quite different from CPUs, no wonder that shaders are usually written in their own programming language.  At the moment, there are three languages in this field: HLSL, GLSL, and Cg. I am not going to discuss their pros and cons, as most of the time, you won’t be able to choose your shader language as such; rather, shader language is usually dictated by your underlying platform/library:

• HLSL is for DirectX
• GLSL is for OpenGL
• Cg will work for both, BUT was reported to have severe problems outside of nVidia world [StackExchange.HLSLvsGLSLvsCg]

Pick your poison, guys and gals, pick your poison…

Pixel Shaders

For 2D graphics, most of the time you’ll be interested in pixel shaders. Most of those effects mentioned above (including lighting) are usually implemented by pixel shaders.

“A pixel shader is a (micro)-program which returns the color of the pixel.A pixel shader (also known as “fragment shader”) is a (micro)-program which returns the color of the pixel. The simplest possible pixel shader is “identity shader”, which does nothing and merely returns the original color. A somewhat-useful pixel shader will manipulate colors (for example, doing some gamma correction); going into more complicated field, you will be able to work with textures (including using some of them as normal maps to implement lighting).

I’d like to speak about pixel shaders a bit longer (especially about those lighting shaders I love ;-)), but going further into specifics would deviate me too much from the subject of MOGs, so (with some regret) I refer you to an excellent tutorial in [DesLauriers]. However, be warned: the world of shaders is so fascinating, that it will be difficult to get out of there and to start working on the other parts of your MOG ;-). Examples of other good tutorials include: [Carlin],  [Shehata] and [Germanunkol] (the last one is not actually a tutorial, but is IMO still quite a useful and interesting exercise in shaders).

Oh, and last but not least – quite a few of pixel shaders which you will find on sites discussing “3D shaders” will actually work for 2D graphics too. Actually, all the post-processing effects will work (though whether a specific effect will look good for your 2D game – is a different story).

Vertex Shaders

Vertex shaders for 2D graphics are not really obvious – “hey, what vertices we’re speaking about?”. However, as we’re running on GPUs, and GPUs are all about vertices, we may have uses for vertex shaders too.

“as soon as we've got vertices - they can be manipulated in our vertex shader, allowing to implement such effects as scaling, skewing, etc., quite easily.As noted above, we’ll be representing our sprites as rectangles, and those rectangles can be represented by 4 vertices. And as soon as we’ve got vertices – they can be manipulated in our vertex shader, allowing to implement such effects as scaling, skewing, etc., quite easily.

Impact of shaders on a MOG data flow

One thing which needs to be mentioned, is that while shaders are routinely used for MOGs, they usually do not change the game mechanics. In other words – effects of shaders on most of the games out there, are purely visual and do not affect gameplay etc. (unless very indirectly, via providing player with better visual experience).

Looking at the same thing from a bit different perspective, we can say that data normally flows only from CPU (and Reactors) towards GPU (and shaders), and not in the opposite direction. In turn, it means that we don’t need to think about potential interdependencies and race conditions which might be caused if stuff-computed-by-GPU would have affected state of our Reactors running on CPU, phew 🙂 .

That being said, it needs to be noted that it IS possible to use shaders beyond pure visuals (for example, for collision detection and/or for rigid body physics/…) – but within the games, GPUs are usually too busy with graphical stuff to spend time on physics. So as a rule of thumb, physics is done by CPUs, and visuals are done by GPUs.

[[To Be Continued…

This concludes beta Chapter 17(d) from the upcoming book “Development and Deployment of Multiplayer Online Games (from social games to MMOFPS, with social games in between)”. Stay tuned for beta Chapter 17(e), where we’ll start speaking about 3D…]]

Don't like this post? Comment↯ below. You do?! Please share:

Acknowledgement

Cartoons by Sergey Gordeev from Gordeev Animation Graphics, Prague.