Game Graphics 101: Rendering Pipeline & Shaders

	Author:	“No Bugs” Hare  Follow:
	Job Title:	Sarcastic Architect
	Hobbies:	Thinking Aloud, Arguing with Managers, Annoying HRs, Calling a Spade a Spade, Keeping Tongue in Cheek

To navigate through the book, you may want to use Development&Deployment of MOG: Table of Contents.]]

As it was noted in the beginning of this Chapter, please keep in mind that

What you will find in this chapter, is the very very basics of the graphics, just enough to start reading the other books on the topic, AND (last but not least) to understand other things which are essential for networking programming and game development flow.

Bottom line:

This Chapter is more oriented towards those developers who are coming from radically different fields such as, for example, webdev or business app development (and yes, switch from webdev into gamedev does happen).

Rendering Pipeline

By now, we’ve described lots of the different concepts which are necessary to perform 3D rendering. Now it is time to put all of them into perspective, discussing how the whole thing works. As always within this Chapter, we’re not going into details – and are going to leave out many optional things (notably – geometry shaders), but this should be sufficient to start understanding the whole picture; for good and much more detailed description – see, for example, [Gregory].

Modern GPUs process things along the following lines (NB: we’ll be using terminology from [Gregory]; keep in mind that in some cases, essentially the same things may be described using different terms):

Of course, it is a greatly simplified diagram, but for our purposes it will do. Let’s take a closer look at the stages of the pipeline:

• Originally, all we have is our 3D scene in a system of coordinates known as “world space” – i.e. in meters or yards of the physical world we’re dealing with.¹
• Then, we’re doing Vertex Processing; at this stage, we’re at the very least performing:
  • some projection transformation from world space into a “homogeneous clip space” (see [[TODO]] section above for description of the “homogenous clip space”). Very shortly – it is a camera-centered 3D space.
  • per-vertex lighting
  • In addition, optionally a lot of stuff can be performed here, including, but not limited to:
    • per-vertex texturing calculations
    • procedural animation (such as moving leaves on trees)
  • “After Vertex Processing, we usually have our vertices in 'homogeneous clip space'After Vertex Processing, we usually have our vertices in “homogeneous clip space”
  • Rasterization stage takes those vertices in “homogeneous clip space” and rasterizes them into “fragments” (usually pretty much the same as “pixels”). Rasterization stage includes:
    • Clipping (which is trivial to do in “homogeneous clip space” 🙂 )
    • Screen Mapping (3D to 2D coordinate mapping, also trivial in “homogeneous clip space”)
    • Triangle Traversal – generating fragments (usually “fragment”=”pixel”) from triangles; each fragment has a bunch of attributes, which attributes are simply interpolated from appropriate vertex attributes.
    • Optionally it may also include “early Z-test” – hiding some of invisible pixels
  • At this point we’ve already got rid of 3D vertices and are working with 2D pixels/fragments. However, instead of color, at this point each pixel/fragment still has a bunch of other attributes (such as lighting).
  • Pixel Processing stage. At this stage, we’re working with those 2D fragments which have associated attributes – and need to calculate color out of those attributes. It is at this point where textures are usually applied. In addition, lots of other stuff can happen here, including, but not limited to:
    • Lighting (bump mapping/shadows/…)
    • Anti-aliasing
    • All kinds of post-processing effects
  • Merge stage. In general, it (a) filters some fragments out (based on number of potential criteria, including Z-test if it hasn’t been performed earlier), and (b) blends/merges all the fragments/pixels into the frame buffer.
  • Phew, we’ve finally got our next frame – and can start the whole process again for the next frame 🙂 .

Highly Parallel Nature of Rendering Pipeline

“each of the stages of the rendering pipeline is operating on a large set of items (vertices or fragments/pixels)One thing which needs to be noted with regards to rendering pipeline, is that each of its stages is operating on a large set of items (vertices or fragments/pixels). When coming to specific numbers for real-world scenarios, we’re speaking at least about hundreds of thousands of vertices/fragments, and more likely about millions. Which means that splitting this load over several thousand cores is not a problem; and this is exactly what modern GPUs (aided by APIs) are doing.

We’ll need this observation a bit later, when we start speaking about shaders.

Fixed-Function Pipeline vs Programmable Pipeline. Shaders

The pipeline shown on Fig. XIV.5, is very generic; this is the way how 3D was processed 15 years ago, and the way it is processed now. However, over the time, there were quite significant changes to this process.

Originally, rendering pipeline (such as the one shown on Fig. XIV.5), was a fixed-function pipeline – each stage was doing exactly what was pre-programmed by hardware guys, and while stages were configurable – and were able to produce quite good graphics too – 3D developers were craving for more 😉 .

In early 2000’s, hardware manufacturers started to produce GPUs with programmable pipelines; as a first step, Vertex Processing stage and Pixel Processing stage were made programmable.² In short – we’re supplying a (micro-)program (known as “shader”) to do all the necessary work to transform one single vertex (for Vertex Processing) or one single pixel (for Pixel Processing) – and then run this program millions of times for each frame (on all those hundreds and thousands of GPU cores).

Traditionally, vertex shader is a (micro-)program which takes a vertex and produces a vertex – with no ability to create additional vertices (this is left to Geometry shaders). The vertex can have quite a few attributes – which will be then interpolated by Triangle Traversal (sitting within Rasterization) while it converts vertices to fragments; then these attributes will be fed to the pixel shader.

Pixel shader is another (micro-)program, which takes a pixel/fragment with these interpolated attributes, and produces a pixel with color (which color will be written to the frame buffer, subject to some restrictions).

“These days, even if the program uses fixed-pipeline, that fixed-pipeline is simulated over shaders anyway 😉Adding shaders has allowed to make tons and tons of new effects, and these days it is difficult to find a serious 3D game which doesn’t use shaders. Actually, even if the program uses fixed-pipeline stuff these days, that fixed-pipeline is simulated over shaders anyway ;-).

What should you use? Fixed-pipeline vs Shaders

If you have a question about using fixed-pipeline vs using shaders – probably you are in the same camp as myself (i.e. not exactly coming from 3D ;-)). It means that dealing with shaders can become overwhelming.

Still, current state of things is that you’d better start with shader-based pipeline – even if with the simplest possible ones (effectively mimicking fixed-based pipeline for the time being). The reason for it is two-fold.

The first reason to go with shaders from the very beginning lies with the current implementation of fixed-pipeline processing: these days, all fixed-pipeline implementations are merely simulated on top of shaders anyway – and sometimes there are rather weird restrictions. In one example, Mac OS X implements OpenGL “compatibility profile” only for outdated “legacy” OpenGL 2.1 – and OpenGL 4.1, while available, doesn’t implement “compatibility profile” which implements fixed-pipeline stuff. In another example, OpenGL ES (the one intended for non-PC devices), starting from OpenGL ES 2.0 is not supporting fixed-pipelines at all.

The second reason to use shader-based pipeline is that more likely than not, by the end of the day you will probably need shaders anyway; and migration from simplistic-shaders to more-complicated ones is much more straightforward than migration from fixed-pipeline to shaders.

“The only advantage of using fixed-pipeline is to support really ancient devices – but as of 2016, such devices are extremely rare.The only advantage of using fixed-pipeline is to support really ancient devices (which don’t have shaders at all) – but as of 2016, such devices are extremely rare. With shaders available on smartphones starting from iPhone 3GS and Samsung Wave (that’s around 2009-2010), and also on Raspberry Pi and as a part of WebGL, it is indeed pretty difficult to find a GPU without at least some shader support. As a result, even I myself³ cannot really argue for developing a game specifically aimed for non-shadered GPUs.⁴

Bottom line: these days, if going 3D, I would suggest to keep away from fixed-function pipelines, and start with shaders right away. Take the simplest shader which you can find on the Internet – and go from there. Alternatively, tools such as [ShaderGen] or [DirectXTK] may help.

On the other hand,

(and make an isolation Logic-to-Graphics layer around it, as described in Chapter VI); otherwise, you’re going to spend LOTS of time until you have something presentable 🙁 . While I am a kind of guy who’s known for being on “DIY everything in sight” side of things – 3D engines is one thing which is complicated enough to make it simpler to integrate (which won’t be the picnic either(!)), than to develop it ourselves 🙁 .

If going this way – you will still need to understand the basic concepts we’ve discussed here, so your time spent reading this Chapter, wasn’t wasted ;-).

Rendering Pipeline and MOGs

As this book is about MOGs, I’m trying to tell about relations of all the 3D stuff with multi-player aspects of the game. With rendering pipeline, one thing helps – it is that, as we can see, all the data is moving from the left side (logic/physics) to the right side (screen), with no information coming back.⁵ It means that, as a rule of thumb, we do not need to care about 3D rendering affecting our physics; in other words – we can simulate our physics world⁶, and feed the results of this simulation to the rendering pipeline. Phew 🙂 .

[[To Be Continued…

This concludes beta Chapter 17(g) from the upcoming book “Development and Deployment of Multiplayer Online Games (from social games to MMOFPS, with social games in between)”. Stay tuned for beta Chapter 17(i), where we’ll close our very cursory discussion of 3D with a very brief discussion on animation.]]

Don't like this post? Comment↯ below. You do?! Please share:

Acknowledgement

Cartoons by Sergey Gordeev from Gordeev Animation Graphics, Prague.