2D Scaling Basics for Games and Web Design

	Author:	“No Bugs” Hare  Follow:
	Job Title:	Sarcastic Architect
	Hobbies:	Thinking Aloud, Arguing with Managers, Annoying HRs, Calling a Spade a Spade, Keeping Tongue in Cheek

With mobile games and social games, it seems that 2D graphics has made a comeback to gaming industry. How long it will last is not entirely clear, but for the time being 2D is back into the game (pun intended). And when it comes to web design, 2D is (and has always been) there. It means that as developers, we  need to deal with 2D graphics and associated issues, including scaling of 2D images.

When scaling 2D images (which happens all the time with 2D games and with web design too), don’t forget quite a few important considerations which are well-known to graphics professionals on one side [Gordeev], and are quite difficult to find about on the other side. Moreover, due to distribution in perception between different people, these issues often become a contentious point between graphics artists (who tend to see these issues easily), and developers (with quite a few of them – though certainly not all – not seeing the problem at all). I can assure you that even if you don’t see the difference, it doesn’t mean that the difference doesn’t exist; it is all about the percentage of people who can see it, and usually a reasonable compromise can be found.

The whole problem is that

While Scaling is Easy, Visually Appealing Scaling is Not

The whole thing we discuss here is all about scaling of 2D images. Very often we don’t know for sure what the resolution is of our graphical image that our end-user.will see. Actually it happens all the time. And yes, it applies to web design too (and pre-scaling to 100% size pixel-wise doesn’t guarantee that the user will see it without scaling, see section on Web Design below).

“So far so good, but the real problem is about 'HOW to resize'.So, what usually happens is the following: we’re shipping one image with our game (or web page) and asking the game (or a web browser) to resize the image to the size which we are about to display. So far so good, but the real problem is about “HOW to resize”. And don’t think “Hey, we’re using bicubic (Lanczos, whatever) scaling, which is the best thing out there, so we are fine” – while it may be the case, quite often, it is not.

It should be noted that for the purposes of this article we will speak mostly about scaling bitmap images; scaling vector images is not a subject to most of the scaling problems (though interestingly enough, scaling to really small sizes affects vector images too, see discussion of problem #2 below).

Problem #1: Upscaling Images Sucks

The first problem with 2D scaling is that while downscaling, images MIGHT work well (subject to further discussion below), upscaling (scaling from a smaller size to a larger one) almost universally looks rather poor. Let’s compare:

NB: please make sure to set your browser’s zoom to 100% before comparing the images

NB2: if you don’t see the difference – it means that you fell into the percentage of people who have a high tolerance to this kind of deficiency; you’ll need to believe our poll that the difference exists (or to ask a dozen of your colleagues to look at it instead)

“Most of the people tend to see this difference, and it arises from a quite obvious fact that 250x182px image simply doesn't have enough information for a much larger amount of pixels in 312x227px. The first image is our logo, first scaled down to 550x400px and then again scaled down to 312x227px, the second one – the same logo, first scaled down to 250×182 px and then scaled back up to 312x227px (why we’ve chosen these sizes for comparison, will become clear when we start discussing web design issues for those users who’re using browser zoom). Most of the people tend to see this difference, and it arises from a quite obvious fact that 250x182px image simply doesn’t have enough information for a much larger amount of pixels in 312x227px. Whatever method we will use for scaling – the difference will still be there.

Based on this observation (which applies pretty much across the board, though effects on photographic images tend to be less pronounced), we can come up with the following

Problem #2: Scaling to Really Small Sizes Sucks Big Time

Another problem which is typical for 2D scaling, is that whenever you scale an image to a really small size pixel-wise, quality tends to degrade much worse than it would degrade if scaling to a not-so-small size. Worse than that, even if you’re scaling a vector(!) image to a really small size (such as 16×16 px), the result will look quite bad.

Font hinting Font hinting is the use of mathematical instructions to adjust the display of an outline font so that it lines up with a rasterized grid. At low screen resolutions, hinting is critical for producing clear, legible text.— Wikipedia —This is a well-known phenomenon, and it also has at least two real-world consequences which can be easily observed. The first consequence of this “scaling to really small sizes sucks” phenomenon, is that high-quality TrueType fonts (which are essentially vector graphics(!)), have so-called “hints”, which are essentially “fixes” to be used when glyphs are rendered at small sizes. Think about it: with all the algorithmic and CPU power available to the TTF rendering engine they still need a manually-adjusted version for small sizes¹.

favicon A favicon is a file containing one or more small icons, most commonly 16×16 pixels, associated with a particular website or web page.— Wikipedia —A second easily-observed consequence of problems when scaling goes into hundreds-of-square-pixels range, is favicons. It is extremely difficult to find a site which uses a simple scaled-down version of their logo (part of their logo, etc.) as a favicon; most of commercial-grade favicons are drawn by hand by so-called “pixel artists”, who draw it pixel-by-pixel to achieve the desired effect.

Which brings us to

As these images are small, adding them at pixel-art sizes won’t hurt your app size much, but if you find a good pixel artist, the difference you can get in quality can be drastic.

---

 

Problem #3: Downscale from Smaller Size Usually Looks Worse than Downscale from Larger Size

The third problem with visual appearance of scaling is not that bad as the first two and in many cases can be lived with. However, this problem does exist and it would be incorrect to ignore it.

In short – as a rule of thumb, when downscaling, the higher the resolution of the original you have, the better the result you get after downscaling². Below there are two examples, scaling a picture (with original resolution of 3055x2222px), down to 312px width. The first image has been scaled to 800px width, and then to 312px width (i.e. is equivalent to storing 800px image within your game and scaling it in-game further), and the second one has been scaled to 350px, and then to 312px:

While the difference is much more subtle than that of the example in “Problem #1” section, it does exist, and it MAY justify the increase of the size of graphics shipped with your app. Of course (as most of the things in life), it should be done within reason and with moderation, which leads us to

---

 

Problem #4: Your Mileage May Vary

Another issue with scaling (fortunately enough, usually quite subtle so it’s not the end of the world if you’re making a mistake), is related to the interpolation method you’re using for scaling. There are way too many developers out there who think along the lines of “bicubic is better than linear just because the power of 3 in ‘cubic’ is bigger than the power of 1 in ‘linear’”, and that “Lanczos is better because, well, it is the New Great Thing”; unfortunately, it is not the case. In practice, Lanczos and bicubic tend to work pretty well for photos, and bilinear tends to work better for thin-line graphics, but strictly speaking, you need to check it visually for your type of graphics before making any kind of decision.

Differences in Perception

As noted above, there is a chance that you weren’t be able to see the difference even between the pictures in “Problem #1” section. It doesn’t mean that your vision is deficient in any way; instead, it is all about individual tolerance to this kind of imperfection, and as with most such things, there is a statistical distribution of such tolerances. Personally for me this tolerance is quite low, so I tend to see it quite well for a developer (though much worse than, for example, a pixel artist).

“For us (as developers) it is important to know that the difference does exist, and that we should be ready to store graphics with larger sizes even when we personally don't see the need to do it.Still, even if you personally don’t see the difference (or don’t care about it), you can be sure that there are people out there who will notice it; whether the portion of the end-users who will dislike your game (or website) because it is large enough to justify an increase in app size – is a business decision, and we as developers should try to avoid making any business decisions (well, unless one of us is also a CEO or something). For us (as developers) it is important to know that the difference does exist, and that we should be ready to store graphics with larger sizes even when we personally don’t see the need to do it.

Aliasing, Nyquist, Shannon, and Fourier

As we can see, the whole scaling issue is very messy; for those of us who’re interested in “Why it is so?”, there is a tiny bit of theory in this section; if you’re not interested in the reasons behind this mess – feel free to skip this section. Those who really know what the whole thing is about – please skip this section too 😉 , as the description here is going to be really sketchy (I don’t want to overload developers with details here, just to provide some pointers on “where to start digging if the topic looks interesting”).

Aliasing Aliasing is an effect that causes different signals to become indistinguishable (or aliases of one another) when sampled.— Wikipedia —The Big Reason behind scaling problems is usually described as aliasing, which is a very common problem in the signal processing field. Let’s consider (very briefly) the problem from this point of view.

The whole original image (before it has even entered the camera lens) is essentially a function of two coordinates: f(x,y)³. This function is a continuous (and not a discrete) function (we can safely put quantum effects aside for our purposes). After the camera has done its job, however, we’re obtaining an image file, which is substantially a spatially discrete representation of this function f.

This directly leads us to a Nyquist-Shannon’s sampling theorem, which basically says that we can preserve all the data about the original continuous function in a discrete representation, as long as sampling frequency is at least twice as high as all the frequencies in the signal which we want to preserve (for the purposes of graphics, as we’re speaking about spatially discrete functions and spatial sampling, these frequencies are related not to wavelength frequencies, but to the resolution of details we need to preserve). And as we cannot possibly get infinite sampling frequency (i.e. cannot get infinite number of pixels), it means an inherent loss of detail when performing sampling in general and re-sampling (for re-scaling purposes) in particular. Further details are beyond the scope of this article, with Wikipedia [WikiFourier][WikiNyquistShannon] and [Efros] being reasonably good points to start reading about it.

This interpretation of aliasing for graphics leads us to the following scaling algorithm:

a) take a bitmap;
b) re-create the original continuous function (in a 2D Fourier form)
c) perform discretization (sampling) of this continuous function again to a new basis

In a sense, this scaling algorithm should be optimal from an information theory point of view (though technically, it is obviously optimal only with a few assumptions, including the assumption that discretization has an infinite precision, which won’t happen in practice). On the other hand, being theoretically optimal doesn’t guarantee optimal visual perception.

In practice, for the purposes of scaling in most cases a local interpolation (bilinear/bicubic/Lanczos) is used instead of full-scale Fourier reconstruction, and I haven’t seen any direct 2D-Fourier-based scalers available (except for those discussed in uni courses) and JPEG exactly-2x downscaling⁴. If you know about practical applications of 2D Fourier to image scaling – let me know, if you dare experiment with it – please-please let me know, with a cherry on top 🙂 .

---

 

Consequences for 2D Games

Coming back to Earth, let’s repeat practical consequences of all scaling issues, when applied to 2D games:

1. DON’T upscale images

2. When having small but important details which need to be shown in hundreds-of-square-pixels resolutions, think about having them as a pixel-art for these sizes

3. Try to find the optimal balance between graphics size and quality-after-scaling.

4. Don’t rely on one scaling filter to work well for all the types of graphics.

While it’s certainly not rocket science, it is yet another thing to remember 🙁 .

Scaling Problems. Now for Web Design Too!

For web design, it has long been considered that scaling is not a problem. Common perception is that you should just pre-scale an image to the size (in pixels) in which the image will be rendered, and then to specify this size in HTML or CSS, and all the problems will be gone. However, there are two more or less recent developments which make it not that obvious a choice.

Responsive Web Design Responsive web design (RWD) is an approach to web design aimed at crafting sites to provide an optimal viewing and interaction experience across a wide range of devices (from desktop computer monitors to mobile phones)— Wikipedia —One quite obvious scenario which requires images to be resized, is if your design is not only “fluid” or “responsive”, but is also based on resizable graphics. In this case, most of the considerations listed above for scaling, do apply. On the other hand, resizable graphics is still not too common these days. However, even if your page is fixed-width, you might want to provide a larger-size image in some cases.

Web Design: A Case for Having Larger-Size Image Even for Fixed-Width Pages

During recent years, quite a few people have been using browser zoom feature (NB: here we’re not speaking about “text size change”, but about more recent “scale the whole page” feature). It implies that browser needs to apply image scaling too. According to [MarshallRoch], in 2012 a whole 10% of Facebook users were using the browser Zoom-In mode (I wasn’t able to find more recent data, but my guess would be that the number has increased , in particular, due to larger-resolution monitors on Windows).

What happens with an image, pre-scaled to the exact number of pixels, which is specified in HTML/CSS (this is what everybody and their dog is doing these days), and then the page is viewed in a Zoom-In mode? Well, if the Zoom-In is 125%, and the image is 250x182px, the browser needs to re-scale the image from 250px width to 312px width, which indicates “upscaling”. And as we’ve seen above (with an example exactly for these sizes in “Problem #1” section above), “upscale” is a Bad Thing which needs to be avoided.

To deal with this problem, you can use the trick which has been used for some of the pictures on this very blog.

“To make sure that the site is seen without too much degradation even when scaling is enabled, some of the pics on this blog, have been pre-sized to a size (in pixels) which is larger than that specified in HTML/CSS.To make sure that the site is seen without too much degradation even when scaling is enabled, some of the pics on this blog, have been pre-sized to a size (in pixels) which is larger than that specified in HTML/CSS. In such a case, all the major browsers (at least Chrome/FF/IE) scale this larger-image directly to the desired size=zoom*CSS_size, which leads to better image quality. The difference in image quality when zoomed in, greatly depends on the nature and size of the image, being much easier seen for smaller images. That’s why for this site we’ve chosen to serve images of a larger-than-100% size for only a handful of images, including the site logo (the one at the top-left of every page), which stays crisp regardless of browser zoom being 100%, 110%, or 125%. The first comparison above (in “Problem #1” section) demonstrates exactly the difference which users-using-125%-Zoom-In will see, between image pre-scaled to “excessive” 550px width, with the same image being pre-scaled to optimal-for-100%-zoom 250px width.

While this approach is not without drawbacks (slightly larger size to be downloaded by the end-user, and minor loss of clarity at exactly 100%), we’ve decided that this is the most reasonable compromise for logo and a few other images, but it is not worth it for the rest of the images on the site.

Some people may (and will) argue that this is kind of punishing mainstream users to make life of not-so-mainsteam users better; however, it is not all black-and-white, and taking into account a very limited amount of such a punishment and a number of people who’re using browser zoom being around rather high at 10+%, IMHO there are cases when such trickery is a positive thing overall.

Conclusions

Image scaling has always been, and still is, quite a can of worms. Moreover, now it has started to apply to web design too.

However, via (a) understanding why it happens, and (b) following a few simple guidelines described above, related complexity can be kept in check.

Acknowledgement

Cartoons by Sergey Gordeev from Gordeev Animation Graphics, Prague.