Bot Fighting 101: Don’t Feed the Hacker

Author:  Follow: TwitterFacebook
Job Title:Sarcastic Architect
Hobbies:Thinking Aloud, Arguing with Managers, Annoying HRs,
Calling a Spade a Spade, Keeping Tongue in Cheek
Don't Feed the Hacker

[[This is Chapter 29(c) from “beta” Volume VIII of the upcoming book "Development&Deployment of Multiplayer Online Games", which is currently being beta-tested. Beta-testing is intended to improve the quality of the book, and provides free e-copy of the "release" book to those who help with improving; for further details see "Book Beta Testing". All the content published during Beta Testing, is subject to change before the book is published.

To navigate through the "1st beta" of the book, you may want to use Development&Deployment of MOG: Table of Contents.]]

In previous instalment of this loooong chapter on Bot Fighting, we discussed how attackers are likely to approach your game, and categorized all the attacks on MOGs into “Attacks on Silly Deficiencies”, “Specific Attacks”, and “Brute-Force Attacks”.

Now, we know enough to start discussion on “what MOG devs can do to fend off attackers”. First, and most obviously, we have to avoid those “silly deficiencies” which quite a few of those attacks are based on.

Our operational phrase for our Bot Fighting 101 crash course is “don’t reveal information beyond absolutely necessary”. In practice, while each of the efforts to stop providing unnecessary information isn’t rocket science, making sure that you covered all of them, will require quite a bit of diligence.

The following is a (non-exhaustive) list of things to be done in this regard:[[TODO: convert from text to prose]]

  • Judging hare:DO use Authoritative ServersDO use Authoritative Servers
    • It was discussed at length in Vol. I’s chapter on Authoritative Servers; in short – if you’re not doing Authoritative Servers, your game is very likely to become a cheater-fest, with the only way to handle it being moved towards Authoritative Servers. Quite a few companies already learned it in a painful way, and are in the middle of a cumbersome-and-error-prone migration of their game logic from Clients to Servers.
  • DO use Interest Management (see Vol. I’s chapter on Communications for details). As discussed in Vol. I, without Interest Management you will be distributing all the information to all the Clients, which in turn enables the whole families of attacks, including wallhacks (=”seeing through walls”) and maphacks (=”lifting fog of war”).
  • [[TODO: Linux]]
  • When you’re using system calls, consider all the information fed to the system calls, as “information already in the hands of the attacker”.1 To minimize negative impact of such leaks on the game, make sure that you DON’T reveal more than absolutely necessary, in system calls and their parameters.
    • Sure, we DO need to use system calls. In particular, there is no way to avoid the following:
      • Network calls (most of the time – socket calls)
      • Graphics calls
      • User input calls
    • Inquisitive hare:Some of the system calls are not absolutely necessary, and using them will significantly simplify life of the attackerOTOH, the following system calls are not absolutely necessary, and using them will significantly simplify life of the attacker:
      • System-level encryption calls.
        • Whatever encryption we’re doing, can be lifted trivially (revealing our bare protocol and enabling all kinds of bots) – as soon as we’re using system-level calls to encrypt.
        • Mind you, encryption is a Really Good Thing™ to deal with hackers, it is just that it should be done by our own monolithic executable, and not by system-level DLLs.
      • System calls which do both encryption and sockets
        • The reason is the same as for pure encryption.
      • Any system-level call which deals with character-based output – at least, for any non-constant data which is related to current game state.
        • This includes such stuff as using standard OS controls for mere display purposes (Windows controls are particularly nasty in this regard), and goes all the way up to using functions such as DrawText() and ID3DXFont::DrawText().
        • The reason is that whenever you call DrawText() to show something related to current state of the game, you have already leaked significant information about current game state to the attacker.
          • The worst real-world case I know about, was a whole bunch of poker games, using standard Windows controls to show “chat window” – which, by tradition, for poker apps has all the history of the hand up-to-now. It means that attackers can get current state of the game in an easily-parseable form, by merely sending WM_GETTEXT message to the “chat window”. And as soon as they have it – they can write all kinds of “data mining” programs (which are usually prohibited by ToC as upsetting the balance of the game and scaring players), automated advisors (also prohibited by ToC for the same reasons), and just need to add sending WM_MOUSEUP messages at fixed positions within the game window to enable writing a full-scale automated bot player.
        • Kerning Pair In digital typography, kerning is usually applied to letter pairs as a number by which the default character spacing should be increased or decreased— Wikipedia —To address this class of vulnerabilities – the only way I know is to owner-draw everything.
          • You need a control? Write it yourself (or at least use a 3rd-party static library which does it without using system-level calls which take character input).
          • You need to draw text (for the control above or otherwise)?
            • At least – use your own bitmap fonts2 to combine your bitmapped characters into a line of text. BTW – for quite a few games, simplistic bitmaps-with-per-character-width (but without kerning pairs) often look “good enough”.
            • At most – you can use something like FreeType library to render your TTF/OTF/…-based text into bitmap – and then render the bitmap to screen via the system-level API call.
            • In any case, all the relevant calls to Windows API will look as a bunch of BitBlt() calls, which are much more difficult to reverse engineer than reading parameters of a simple call to DrawText().
          • Hare with While using system DLLs is inevitable (to the extent discussed above), using non-system DLLs within your Client is a Big No-No™.While we’re on a subject of 3rd-party calls: DON’T reveal information by using 3rd-party DLLs. While using system DLLs is inevitable (to the extent discussed above), using non-system DLLs within your Client is a Big No-No™. Very briefly, the reasoning goes along the following lines:
            • There is no reason to use DLLs on the Client-Side (except for official 3rd-party plug-ins for your app). For detailed discussion – see, for example, [NoBugs2010].
            • with each DLL, we’re splitting our monolithic executable, and they are providing lots of information to the attacker. And as using DLLs in 2017 (beyond some very narrow cases) doesn’t make any sense – replacing all-DLLs3 with static libs clearly qualifies as a Very Good Thing™.
              • I cannot even count the number of games which are using openssl.dll – and had their protocols hacked using this attack vector as a result.
            • Note that with F.L.I.R.T., even 3rd-party static libraries are vulnerable – but dealing with F.L.I.R.T. is a subject of a separate discussion in [[TODO]] section below.
              • For the time being – just make sure that you’re compiling 3rd-party libraries from source; in addition, when doing so, you SHOULD:
                • Use your own compiler settings (different from those used in default compile)
                • Use all the library-provided #defines to switch off all the features you don’t need within the library.
                  • This tends to play very nicely with TLS libs such as OpenSSL: as for MOG, we’re controlling both sides of communication – we can (and SHOULD) disable all the algorithms except for one we’re using (for discussion on which-exactly-TLS-algo-to-use – please refer to Vol. IV’s chapter on Basic Security); and the best way to disable unnecessary stuff is via library-provided #define
    • DO encrypt your traffic. If you’re not doing it – you just make your game vulnerable to fundamentally-undetectable proxy bots. While you’re implementing encryption:
      • DO check the certificate on the Client-Side (and DON’T use Anonymous Diffie-Hellman for encryption)
      • DO store certificate-for-checking within your executable (and DON’T use system-level storage of root certificates for this purpose)
        • DO obfuscate the certificate within your executable.
      • Along the lines discussed above – DON’T use system-level libs for encryption; neither use 3rd-party DLLs for encryption (though statically-linked OpenSSL-or-whatever-TLS-lib-you-fancy – is fine).
      • Hare thumb up:Scrambling will help to protect your protocol even if the attacker manages to F.L.I.R.T. with your TLS libraryDO scramble your traffic right before feeding it to encryption library; DO it using your own method. We’ll discuss more on scrambling (and more generally – on obfuscation) starting from [[TODO]] section below, but for the time being – let’s note that such scrambling will help to protect your protocol even if the attacker manages to F.L.I.R.T. with your TLS library,4 and to get your unencrypted traffic.
        • It doesn’t matter how insecure your scrambling is (=”don’t bother to use an anywhere-standard encryption library”), the only real requirements for scrambling/obfuscation are the following:
          • It MUST be reversible for obvious reasons.
          • It SHOULD be non-standard (though it MAY be constructed out of standard primitives, more on it in [[TODO]] section below).
            • If you’re using a standard library for scrambling – it can in turn be F.L.I.R.T.-ed with, which goes against the goal of scrambling discussed right above.
          • It SHOULD be convoluted (~=”simple XOR with a constant byte is not good enough”).
            • Think of it as if you’re designing a very lightweight crypto algorithm yourself; in this process, there is a 99.(9)% chance that whatever-you5-design, won’t be secure – but it can easily be a good scrambler (and a unique one too, which is important).
          • It SHOULD have some means of integrity checking (such as “message checksum calculated in your own way and/or scrambled”). If such a checksum doesn’t match on the receiving side – it is a serious indication of being hacked,6 so it has to be reported, and probably have corresponding player flagged.
          • Ideally – your scrambler SHOULD be randomly generated for each new build out of reversible primitives (for a long discussion on it, including versioning – see [[TODO]] section below).
  • In C++, DO disable RTTI. RTTI is a nice feature of C++; however – it turns out to be  nicer to hackers than to gamedevs. If you don’t disable RTTI, than for each and every object of every-class-that-has-at-least-one-virtual-function, hacker will be able to know not only the VMT pointer for this class (and VMT pointer identifies the class7),  but also class name. Actually, this is the only case when information about C++ source-code names leaks to the executable, and it happens to be of enormous value for the attacker. Without RTTI, all attacker has is merely a “it is an instance of some class” (and has to guess about what-the-class-does); with RTTI, for the same class attacker can see “hey, it is an instance of the class which author has thoughtfully named ‘PlayerHealth’ or ‘UnitPosition'”.
    • To deal with it, three approaches are possible:
      • disable RTTI (“/GR-” switch in MSVC). This comes at a cost of losing ability to make dynamic_cast<> (which are rather easily replaced with DIY kinda-dynamic-cast based on DIY virtual functions); as for all the other goodies of RTTI such as typeid() (which might be usable to make maps of object types), I yet to see any real-world uses for them (and even if they do exist – they’re very few and far between, so replacing them with DIY virtual functions won’t be a problem).
      • in your production build, run a pre-processor which replaces all the class names with their hashes (or something similar). TBH, I don’t really like this approach (it is cumbersome, doesn’t deal with 3rd-party libraries, etc.); also – it may be interpreted as an insult to the attacker (and believe me, we don’t want to make attacker a personal enemy). If you still decide to go this route – make sure to randomize names on every build.
      • Obfuscate VMT pointers as discussed in [[TODO]] section below; as RTTI is inherently based on VMT pointer – hiding it will make debugger rather unhappy.
      • Personally, I would argue to both disable RTTI, and obfuscate VMT pointers. My rationale goes as follows: (a) hassle due to disabled RTTI happens to be very limited (at significant anti-hacker benefit); and (b) obfuscating VMT pointers is certainly a Good Thing(tm), but it can’t be applied across the board, so there going to be non-obfuscated VMT pointers in our program (and that’s when disabling RTTI at least won’t make the job of attacker easier).

Phew. The list above is certainly non-exhaustive, but following all the advice above is a de-facto prerequisite to any realistic protection. Otherwise – according to the weakest-link principle – attackers won’t even bother with hacking your other defenses (so they won’t even know how clever your other defenses are <sad-wink />).

1 As it will be discussed later – there is no good way to hide system calls anywhere reliably, which means that “we should treat them as easily-interceptable”; in other words – we should treat these calls as residing beyond-our-defence-perimeter.
2 Of course, properly obfuscated as we’ll discuss in [[TODO]] section below
3 except for those which are system-level-only; still, we SHOULD statically link msvcrt.lib etc. etc. For Mac OS, make sure to read [Kulesza]
4 =”identify functions within the library using F.L.I.R.T. feature of IDA Pro”
5 or me for that matter
6 errors in outermost encryption can be just due to packets being corrupted in transit over the Internet; however, for the data protected by outermost encryption, chances of any problem at packet level propagating below, are 2-128 ~= 3e-39. Even if we consider all the packets for all the players for a million-player game running for 10 years, we get merely 1e6 players * 20 packets/player/second * 86400 seconds/day * 365 days / year * 10 years ~= 6e17 packets, so chances of crypto randomly failing even for one single packet during this time is miniscule 2e-27 (~=”failures due to bugs etc. are much more likely”).
7 in [[TODO]] section below, we’ll discuss how to deny attacker easy access to information whether two objects are of the same class, or of different classes


[[To Be Continued…

Tired hare:This concludes beta Chapter 29(c) from the upcoming book “Development and Deployment of Multiplayer Online Games (from social games to MMOFPS, with social games in between)”.

Stay tuned for Chapter 29(d), where we’ll get to real real obfuscation <wink />]]

Don't like this post? Comment↯ below. You do?! Please share: ...on LinkedIn...on Reddit...on Twitter...on Facebook



Cartoons by Sergey GordeevIRL from Gordeev Animation Graphics, Prague.

Join our mailing list:


  1. says

    “it looks as an insult to the attacker”

    Isn’t all of these steps an insult to the attacker? How is “running a pre-processor which replaces all the class names with their hashes” an insult where other steps to stop the hacker is not?

    • "No Bugs" Hare says

      > Isn’t all of these steps an insult to the attacker?

      Depends on point of view, but in general, “defending against” doesn’t equal to “insulting”. If I remove RTTI information, I am defending – and when a serious attacker sees it, the reaction is like “oh, those defending guys know enough to remove RTTI, good for them”; it tends to build respect, not a being-insulted feeling. OTOH, if I am leaving RTTI in place, but making it effectively useless – it is often more insulting, because when attacker can see that RTTI is present, an anticipation of “hey, I can use RTTI” builds, but then there is a quick disappointment; such creating-false-pretense with quick-disappointment-afterwards is known to cause “being insulted” feelings (regardless of real intentions of whoever-creates-them).

  2. klaim says

    I read the first volume and it doesn’t clearly answer the question related to my main multiplayer project: in the case where you need to provide at least one server implementation that will run on the client’s PC (that is, like MineCraft or Neverwinter Nights, more correctly).
    Of course the server implementation you provide to the user is then not trustable. However it is still possible to make a different implementation in authoritative servers that would be available only when playing “online”. This is basically the case where the game is thought with both solo and multiplayer (and maybe merged together since Demon’s Souls/ Dark Souls showed that it works).

    My main question is in this case is it helping a lot the attacker to provide that server impl, even if you have a different (maybe with more protections) server implementation on your authoritative servers?

    • "No Bugs" Hare says

      If I understand correctly, you’re speaking about having an Authoritative Server for multiplayer mode (let’s name it Real-Authoritative-Server), and a Client-Side copy of more-or-less the same server for single-player mode (let’s name it Server-on-Client).

      In such a case, it _will_ make your protection efforts more complicated .

      However, (in contrast with Authoritative Clients) IMO you WILL have a fighting chance. To deal with cheaters in such worse-than-usual conditions, depending on the size of your player base, you may want to do one or more of the following: (a) use obfuscation-techniques-discussed-over-the-course-of-this-chapter to obfuscate Code+Data over the critical path from user inputs to network and back from network to the graphics (which is a good idea regardless of your project using Server-on-Client), (b) obfuscate _protocols_ between your Client and Server (part on protocol obfuscation coming in a few weeks on this site) – again, a good idea anyway, (c) use _different_ protocol obfuscations for your Server-on-Client and Real-Authoritative-Server, and (d) obfuscate your Server-on-Client when you compile it for Client-Side releases (while keeping it non-obfuscated for Real-Authoritative-Server).

      Fortunately, if your game is not out yet, there is not _that_much_ risk in deploying your very first version without this obfuscation stuff, and it usually can be added later (when your game becomes popular enough to attract serious cheaters – which tends to happen around tens-of-thousands of simultaneous players) without _too_much_ negatives-due-to-previously-revealed-info. OTOH, as soon as your project starts to gain popularity – it is better to start these obfuscation efforts ASAP (_before_ cheating starts to hurt, as serious obfuscation does take time and effort).

      Hope it helps :-),
      ‘No Bugs’

      • says

        Thank you very much, this completes my understanding of the problem.
        At least there is hope to have some protections for fairness in multiplayer, once the game is successful indeed.

        To clarify: these kinds of games (Neverwinter Nights and MineCraft) have indeed a server running locally for solo game but also for multiplayer game. A player can make other player come play in his game world (which is the whole purpose of Neverwinter Nights for example). So let’s say some players in the same LAN, they can play in a persitent world that is just available to them.
        In exchange, the developer cannot, of course, give any protection, the player is on his own and should not try to expose the server to internet.
        However that allows some group or people to still provide servers online, for a limited number of players (as it does not scale). That’s what happened with both games. The people providing these servers are on their own so they have to do the protections themselves, as much as they can, but in the same time most will not because the number of players is under a hundred for each server. This can cause issues, as we have seen recently with people trolling MineCraft servers put online by young people. But at least it does not break companies.

        So MineCraft also have “Worlds” that is a service of safe server instances that you kind of loan to get the high protection done by people at Mojang/Microsoft.

        I am (and my future team will be) small so I have to plan what could be done for all that time the game isn’t successful enough to pay for more people in the team doing protection. From the book and these articles and what you say here, things can be done in this situation, so it helps to know that in advance and have some idea of what to prepare to be able to get there sometime.

        • "No Bugs" Hare says

          > In exchange, the developer cannot, of course, give any protection, the player is on his own and should not try to expose the server to internet.

          However strange it may sound – developer CAN provide some protection even in this case ;-). IF developer has his stuff with protocols randomly obfuscated during the build (as it will described soon, maybe even next week – though building blocks for it are already discussed over last 2-3 weeks, including my own obfuscation library) – he CAN provide a service which will compile random-but-compatible-with-each-other (and incompatible-with-anything-else) Client+Server _pairs_. This way, breaking one such pair WILL NOT allow breaking any other pair – which means that cost of the attacks on the game will go through the roof (in particular, because sharing attack experiences between different attacking teams won’t fly anymore).

          Heck, using this technology it even _seems_ possible to protect 100%-open-source projects (while the _source_ code itself is 100% open, the _binary_ is heavily randomized using build-time random number which is not known to anybody-except-for-builder-himself; in a sense – it is similar to having crypto algorithm public, and only the key private…).

          • says

            Thank you very much for these information, it helps a lot! I noted all that and will try probably start the work on obfuscating as suggested after I get some players.

          • "No Bugs" Hare says

            Sure (however important anti-cheating is, getting the game running with lots of players is more important by definition). Good luck!

Leave a Reply

Your email address will not be published. Required fields are marked *