Bot Fighting 102: System-Specific Kinda-Protection. Anti-Debugger, Anti-DLL-Injection, VM Detection.

Author:  Follow: TwitterFacebook
Job Title:Sarcastic Architect
Hobbies:Thinking Aloud, Arguing with Managers, Annoying HRs,
Calling a Spade a Spade, Keeping Tongue in Cheek
Wizard of OS

[rabbit_ddmog vol=”8″ chap=”Chapter 29(d) from “beta” Volume VIII”]

After we discussed the very basics in “Bot Fighting 101”, we can go a bit further and discuss an activity which is traditionally very popular among anti-reverse-engineers: using certain system-specific tricks to deter attackers.

TBH, I think that while such system-level protections are somewhat useful, their importance is grossly overstated; in particular, if you think “hey, we’ll use this neat trick and we’re done protecting” – you’d better change your mind, or your game (if successful enough) will become a cheater-fest.

The problem with abusing system-level stuff as a protection is that whatever-system-level-trick-we’re-using, has a corresponding system-level-trick-which-counters-our-trick. So, at the very best, even if we find some-completely-new-trick-nobody-knows-about-yet (which is BTW extremely non-trivial) – even in this case all we can hope is to deter attackers once – and at the very most, for a few months (usually – more like a few weeks). Moreover, all specific trickery becomes utterly useless at the very moment when it is published, so writing anything new in this section wouldn’t make any sense (and for pretty-much-everything-which-isn’t-new, counter-tricks are already long-found and long-deployed by hackers).

Nevertheless, there are several reasons to use system-level protections (even well-known ones which are discussed in this section):

  • To deter novice hackers.
  • To show a notice “hey, we know you’re trying to hack us” (without any banning1). While we do know that this notice will be disabled easily, it will make sure that we won’t ban those who accidentally launched our program under debugger (or gave up really quickly); also, such a notice might strengthen position in case of legal disputes with players.2 As we’ll refer to this notice a few times in this section, let’s give it a name – GotchaNotice.
  • To integrate them with other protections (while for the time being, such integration is not in scope, we’ll discuss it [[TODO]] section below). In general, tightly-integrated different layers of protection DO have a potential to make a life of attacker signigicantly worse than otherwise (though we should be careful not to give up one protection layer when adding another one).

In addition, let’s note that:

  • For the purposes of this book, and for system-level protection, we’ll concentrate on Windows. Similar stuff exists on other platforms too, but sorry, you’ll need to find references to anti-reverse-engineering-techniques for the other platforms yourself.
  • we will not consider time-based protections as system-level ones; in the context of MOGs with servers having an independent timer, time-based protections can be made very viable and very-difficult-to-bypass (and they’re not that much system-specific either); we’ll discuss time-based stuff in [[TODO]] section below.

With all these disclaimers in place, we can start discussing some of the most popular system-level protection techniques.

1 But mentioning that it is a violation of ToC, that in case of repeated offenses user will be banned, etc. etc. – make sure to discuss details with your legal team.
2 Once again – make sure to discuss with your legal team whether they want it or not.


Debugger Detection/Prevention

Hare with omg face:it is fundamentally impossible to prevent (or detect) debugging, at least as long we’re staying on one single box.Arguably, a technique which is most-popular-among-novice-anti-reverse-engineers, is an attempt to detect (or prevent) debugging of our program. Very often, reasoning goes along the lines of “hey, if we can prevent debugging our program – we’re done, nobody will be able to reverse-engineer it”. There is one problem with such a naïve logic, and it is that it is fundamentally impossible to prevent (or detect) debugging, at least as long we’re staying on one single box. In other words –

For each and every debugger detection/prevention trick, there exists a counter-trick, disabling our protection.3

BTW, most (though not all) of the common anti-debugging trickery can be found in [Ferrie]; while it is a bit outdated, for our purposes (as discussed above, we’re not aiming to list that-not-yet-known-to-attackers-way-of-messing-with-Windows4) it will do.

Let’s note that [Ferrie] takes over 140 pages, which means it is a lot of material; as such – we won’t discuss all the trickery in detail in this book, rather providing a very short outline – and referring to the source where a more detailed description can be found (mostly [Ferrie], but there will be some other references too). Also, for code examples for some of the techniques, you may want to look at [al-khaser]5

3 The only exception I know at the moment, is time-based detection on the Server-Side; we’ll discuss it in [[TODO]] section below.
4 by the time you read this book, it will become known to hackers anyway
5 Actually, al-khaser can be seen as a ‘sample malware’, which illustrates my earlier point on “we’ll be using quite a few techniques which are typical to malware guys” <sad-face />


Are You Trying to Hack Me? Honestly??

One family of extremely-popular-and-never-working anti-debugging tricks revolves around using one of the OS-level functions which are designed to tell you whether you’re being debugged. There is one problem with using these functions: when using them, we’re merely asking OS “whether we’re being debugged”, and moreover – as OS is under control of the hacker,6 we’re essentially asking the hacker “are you hacking us (please be perfectly honest when you’re answering)?”.

This means that none of those “protections” can really work in the wild; in fact,

all the protections in this subsection are disabled in bulk by easily-available-to-everybody [ScyllaHide].

Even with this in mind – we have to note that some of these non-protections are even worse than the others. Let’s discuss some outright silly stuff – an attempt at protection, which is sooo obvious, sooooo well-known, and soooo easy to disable, that it’s practically useless (except for showing GotchaNotice discussed above). This wannabe-anti-debug-protection is to call IsDebuggerPresent(), check return code and pray really hard that nobody got past this “defence”:

if( isDebuggerPresent() )

This “protection” is in fact soooo silly that it can be bypassed literally within 5 minutes (see [DutchTechChannel@YouTube] for a demonstration). Very briefly – it is a good example of a really bad protection because of the following deficiencies:

  • Call to IsDebuggerPresent() in the executable is very obvious.
    • To mitigate this problem,7 instead of calling IsDebuggerPresent(), we MAY read “BeingDebugged” field from “Process Environment Block” a.k.a. PEB directly (see [Ferrie] for details)
  • That check after calling IsDebuggerPresent() exists only in one place, and can be trivially disabled (just by replacing ExitProcess() with NOPs in your .exe).
    • To mitigate this problem, instead of making an explicit check, we may assume that the “correct” value of the flag (which is 0 for BeingDebugged) is a constant, and use this memory location as a supposed-constant for our obfuscation routines, as discussed in [[TODO]] section below; the very basic idea here goes along the lines of “if we XOR8 our data with the value of BeingDebugged, then if we’re not being debugged – obfuscation will work as expected, otherwise – it will produce wrong result which will crash the program sooner rather than later”. This would allow us both to have this de-facto check spread all over our executable, and to get it less obvious
  • Unfortunately, even if we read BeingDebugged flag from memory directly instead of calling IsBeingDebugged(), and spread its use all over the executable, this “protection” can be still disabled by merely writing 0 to PEB <sad-face />.
    • Even worse, there is a standard open-source tool which does exactly this [ScyllaHide] <very-sad-face />
  • As a result – while we MAY use this kind of checks either to improve our obfuscation just a tiny bit, or to issue GotchaNotice discussed above – real value of such protections is negligible <very-sad-face />.

Pretty much the same reasoning goes for other similar techniques, including, but not limited to (for discussion of these techniques, unless specified otherwise, please refer to [Ferrie]):

  • NtGlobalFlag
  • Heap Flags
  • NtQuerySystemInformation().{SystemKernelDebuggerInformation|SystemProcessInformation}
  • NtQueryInformationProcess().{ProcessDebugPort|ProcessDebugObjectHandle| ProcessDebugFlags|ProcessBasicInformation|ProcessBreakOnTermination| ProcessHandleTracing} (the last two options are briefly mentioned in [ScyllaHide.doc])
  • NtQueryObject().{ObjectAllTypesInformation|ObjectTypesInformation|ObjectTypeInformation}
  • NtSetDebugFilterState() being successful (NB: if trying to play with it – make sure to read [Ferrie], as it is probably overbroad; anyway, [ScyllaHide] defeats it).
  • EnumWindows, EnumThreadWindows, and NtUserBuildHwndList [ScyllaHide.doc]
  • FindWindow(), FindWindowEx(), NtUserFindWindowEx()
  • GetWindowThreadProcessId(), NtUserQueryWindow()

Once again – I’d rather not bother with doing these kinda-protections, except for (a) showing GotchaNotice, and (b) if direct memory read is possible instead of issuing a system call (such as for IsBeingDebugged(), NtGlobalFlag, or Heap Flags) – there is some value in mixing data-directly-read-from-memory as a supposed-constant in our obfuscation routines (which will be discussed in [[TODO]] section below).

6 well, unless it is a not-hacked-yet-console
7 that is, unless we DO want it to be very obvious just to show GotchaNotice
8 any other mixing function will do too


Using System Calls to Mess with Debugger

In addition to reading stuff from the OS, there are also certain ways to make some system-level calls, aiming to make a debugger’s life more difficult. Unfortunately, not only all of them can be bypassed, but most of them can be disabled by the same [ScyllaHide]. Such easily-disabled kinda-protections include:

  • All kinds of messing with PE/PE headers (one example I still remember, was about marking executable as “big-endian” in PE headers – and for a few months, it even did help). As of 2017, I don’t know of any such technique which is still not thwarted by a serious reverse-engineering-oriented debugger.
    • As a rule of thumb, we still DO want to remove relocation table from our Client-Side executable (just to avoid giving up information which we can hide); also, as discussed above – we DON’T want to use DLLs where removing relocation is much more controversial
      • One downside of removing-relocation-table-from-.exe is that messing with PE MIGHT count as “suspicious” by some AVs heuristics, but as of 2017, simple removing relocations still seems to pass.
      • NB: on the Server-Side, you SHOULD leave relocation tables intact, as they enable ASLR, which is important from security perspective.
  • NtSetInformationThread(ThreadHideFromDebugger), and related NtCreateThreadEx(THREAD_CREATE FLAGS_HIDE_FROM_DEBUGGER) (the latter mentioned in [Kulchytskyy]).
  • NtSetInformationProcess().{ProcessBreakOnTermination|ProcessHandleTracing} [ScyllaHide.doc]
  • BlockInput()
  • Floating-point exceptions which used to crash OllyDbg.

In general, those system calls mentioned above are well-known, and are easily disabled by ScyllaHide, so generally I don’t recommend using them. OTOH, the following system calls are trickier and are still reported to work to confuse at least for some of the debuggers:

  • SwitchDesktop()
  • RtlProcessFlsData()
  • Disabling/patching functions ntdll.dll!DbgBreakPoint() and/or ntdll.dll!DbgUiRemoteBreakin() [Assar] (see also BlockAPI() from [] for a more generic example of system-DLL-patching code)
  • Hare pointing out:at least in Windows, for any process there can only be one user-mode debuggerSelf-debugging (at least in Windows, for any process there can only be one user-mode debugger). Reportedly can be disabled via setting of EPROCESS->DebugPort to 0 [Tully], but it requires messing with kernel mode, which is not that easy. In any case, self-debugging can be disabled by debugging parent executable, and running target process pretending that your debugger is that process; OTOH, to the best of my knowledge, this is really messy.
    • BTW, to make life of hacker-trying-to-hijack-parent-executable, more difficult, in conjunction with self-debugging I suggest to work on active (i.e. with messages exchanged on regular basis) heavily-obfuscated protocol between parent (debugger) process and a child (debuggee) process to complicate debugger-pretending-to-be-parent class of attacks
    • As a side benefit, the same parent-child construct can be used to measure timings in a cross-process manner (i.e. if timings between two processes on the same box are off by several seconds – something is certainly wrong in this picture – such as ScyllaHide trying to hide time-based intra-process debugging).

Trying to Detect Debugger Side-Effects

Another bunch of techniques revolves around detecting side effects of debuggers. As these side effects are significantly less obvious than a simplistic call to IsDebuggerPresent(), some of them may stay around for a while (and even better-for-us – some of them may be pretty difficult to fix for an automated tool such as ScyllaHide).

Still, the following techniques are either automagically disabled by ScyllaHide (and therefore, aren’t good):

  • NtClose()
  • OutputDebugString()
  • SwitchToThread(), NtYieldExecution()

Some of side-effect-based techniques-which-still-reported-to-work at least for certain debuggers, include:

  • Debugger detection in TLS callback.
    • In particular – it is possible to detect debugger thread via NtQueryInformationThread() in TLS callback (see [Ferrie]).
  • SetUnhandledExceptionFilter() and causing an unhandled exception.
  • LoadLibrary() call leaving the file open (this technique essentially relies on bug in the debugger, but such buggy debuggers are reportedly still in existence).
  • Guard pages via VirtualProtectEx()/VirtualAllocEx()
  • RaiseException() and/or issuing INT 3H and/or issuing strange instruction such as LOCK CMPXCHG8B [halsten@OpenRCE]. The idea is to raise various exceptions and to see whether debugger consumes them.
  • Arguably the most interesting and subtle ones – PAGE_EXECUTE_WRITECOPY (see  [] for details, and don’t forget those #pragma comment(linker,”/SECTION…”) and #pragma code_seg(“xyz”) from the demo source code), and its close cousin, QueryWorkingSet().ShareCount [].

Still, it is IMO a matter of time until ScyllaHide includes protection from these (or debuggers fix their issues which allow for detection in the first place).

Messing with Binary Code to Cause Debugger Issues

One more bunch of techniques tries to get our binary code unusual enough so that debuggers will behave strangely when running over it. For x86/x64, these techniques include:

  • Adding REP prefix to non-string instructions. Efficiency of this technique relies on bugs in debuggers, which are rare now.
  • Using INT 2D. Efficiency of this technique relies on bugs in debuggers.
  • Using long-form INT 3 instruction (“CD 03”) to disrupt breakpoints. Current efficiency – unknown.
  • Using undocumented opcode 0xF1 [Falliere]
  • PUSH SS/POP SS to detect single-stepping. Current efficiency – unknown.
  • Using “nanomites”. Very briefly – “nanomites” use (single-byte) INT 3 instruction to cause an exception, then all INT 3 exceptions are caught, then checked whether the exception is one from legitimate exceptions in the table-stored-in-executable, and then jumping to the destination-from-the-table. Still a rather efficient technique.

Femida hare:these techniques (especially nanomites) are rather powerful, and do provide significant protection. On the other hand, as any single protection measure, on its own they're certainly not sufficient to protect.Overall, these techniques (especially nanomites) are rather powerful, and do provide significant protection. On the other hand, as any single protection measure, on its own they’re certainly not sufficient to protect. On the third hand (yes, we, hares, have three hands <wink />) – it is usually done as a post-compile processing by 3rd-party tools, so normally it won’t interfere with our source-level efforts.


One popular kinda-anti-debug-protection measure is to scan our own code for INT 3 (0xCC), which is normally used by debuggers to set breakpoints [Falliere]. IMNSHO, it is a pretty flawed technique (both causing false positives due to legitimate occurrences of 0xCC in the code, and false negatives – as debuggers can use something such as 0xFA for software breakpoints [Falliere]). I suggest to leave CC-scanning alone, replacing it with much-more-reliable checksum-based integrity checks (which will be discussed in [[TODO]] section below).

Production Bot Detection/Prevention

After we discussed debugging, we can (and SHOULD) take a look at how the real (production) bot will work while our game is running.


Under Windows, one of the major ways for bots to mess with our programs is known as “DLL injection.” I won’t go into detailed discussion on DLL injection, just noting that:

  • To know what we’re dealing with, make sure to read [Darawk@Blizzhackers] (especially “Code Cave” method, which is particularly nasty).
  • TLS callbacks (discussed above) will allow to detect new threads being launched within our process, including those ones injected. In general – while this protection can be disabled, it is going to be quite an effort.
  • Disabling LdrLoadDll() (via writing RET instruction to the beginning of it in your own virtual space) has been reported to work against some of the methods of DLL Injection []. OTOH, as not all of injections rely on LoadLibrary() (in particular, as we’ll see below, we can kinda-LoadLibrary ourselves), so it is not a really bulletproof solution.
    • On the plus side, this technique is going to play nicely with our previously-stated “noDLLs” rule, but still may require quite a bit of tweaking to make sure that all the necessary stuff is loaded before we invoke this really-nuclear option

Also, let’s keep in mind that it is perfectly possible to manipulate our program without DLL injection (reading its memory from outside, and sending input events as if they’re sent by player). Still, anti-DLL-injection can be seen as a tool important-enough to bother about it a bit.


An ultimate way of bot running undetectable is to run a VM hypervisor on the Client PC, then run our Game Client within guest, and to run the bot directly in the host. With such bot configurations, it is extremely difficult to detect bots, however:

  • Blue Pill Blue Pill is the codename for a rootkit based on x86 virtualization.— Wikipedia —At least in a theoretical Blue-Pill-vs-Red-Pill sense, with MOGs we do have an upper hand in this battle (because we have an MOG Server as an external reference, it is possible to find out whether the observed reality is simulated or real, at least in theory).
    • This logic, however, doesn’t apply to bots which DON’T modify Client-Side reality by illicit means (i.e. VM-based bots which merely read RAM of our Client, and issue mouse clicks to our VM, in theory can be made perfectly undetectable).
  • In practice, anti-VM bot fight happens to be much more down-to-earth.
    • One popular way of detecting VMs is by looking at device drivers (all the VMs I know have rather specific drivers, which can be identified rather easily); also such things as registry keys, adapter names, files, and processes (the latter – mostly if VM-supporting tools are installed into guest) tend to give VM away.
      • A whole bunch of VM detection techniques based on this kind of stuff, can be found in [al-khaser]. Hint: when using techniques from [al-khaser], make sure to obfuscate all the constants – and to integrate results of your checks into obfuscation too; otherwise, finding and disabling your protection won’t be too difficult.
    • A CPUID-based technique for x86/x64 virtualization detection can be found as a part of virt-what [virt-what]; while the code is GCC-oriented, converting it into MSVC isn’t too difficult.
    • We can also try to communicate with VM host, using tools-provided-by-VM for this purpose (see [Bachaalany] for Virtual PC and VMWare examples)
    • What to do when you DO find that Client runs under VM, is a different story. Some games simply prohibit running Client under VM (issuing a relevant message). Alternatively, you may allow running your Client under VM – but raise a “red flag” (which in turn can be used to run more checks – both Client-Side and Server-Side ones; more on it in [[TODO]] section).

In general, VM is a major threat which bot writers use to circumvent our defences, so they SHOULD NOT be left without attention.

Summary on System-Specific Anti-Debugging Measures

My recommendations for system-specific anti-debugging measures would go as follows:

  • NB: I do NOT consider time-based protection and integrity checks a part of “system-specific anti-debugging measures”, so summary below doesn’t apply to them. Time-based protection will be discussed separately in [[TODO]] section below, and integrity checks – in [[TODO]] section below.
  • DON’T assume that any of these measures will prevent debugging; at most – we’re speaking about obtaining some kind of delay.
    • To make this delay matter – we have to generate something new for each new build; as one example – we can integrate not-so-obvious debug detection with our obfuscation as discussed in [[TODO]] section below.
  • Judging hare:DON’T spend more than 10% of your overall anti-bot-fighting time budget on system-specific protections.DON’T spend more than 10% of your overall anti-bot-fighting time budget on system-specific protections. I know first-hand that delving into these things can be very tempting (and can easily become a story of never-ending improvements), but their efficiency in the wild tends to be pretty low, so I strongly suggest you concentrate most of your anti-bot-fighting efforts in other fields (which will be discussed at length below).
  • DO use something outright silly simple such as IsDebuggerPresent() to issue a GotchaNotice (though make sure to discuss it, and exact wording, with your legal folks first)
    • DON’T consider it as a protection – not at all.
  • DO consider removing relocation tables from your Client-Side .exe files. NB: it is not 100% black-and-white, see discussion above.
  • DON’T spend time on anything which is disabled by ScyllaHide (well, unless you happen to know how to detect Scylla <wink />).
  • IMO, most of the techniques described above are not worth the trouble; however, there are a few notable exceptions:
    • TLS-based callbacks are not that difficult to implement – and may allow to deal not only with debuggers, but also with DLL-injection used by production-level bots
    • Disabling LdrLoadDll() (to make DLL Injection much more difficult)
    • Some serious efforts to detect running under VM (though what-to-do-when-VM-is-detected, is a business-level/GDD-level decision)
    • nanomites are rather powerful (translation: when looking for your 3rd-party-protection-for-already-compiled-executable – generated randomized nanomites all over your code is certainly a Big Plus™9). Moreover, nanomites can be implemented as a post-processing of already-compiled executable, and by a 3rd party (so they won’t eat into your anti-bot-fighting time budget).

9 OTOH, if a 3rd- party protection uses nanomites only internally (to protect its own “encryption”), their value is greatly diminished; and if it uses nanomites only internally and without any randomization-on-each-build – the value of nanomites for bot fighting becomes pretty much zero (as all non-mutable 3rd party protections have already been hacked long ago).


[[To Be Continued…

Tired hare:This concludes beta Chapter 29(d) from the upcoming book “Development and Deployment of Multiplayer Online Games (from social games to MMOFPS, with social games in between)”.

Stay tuned for Chapter 29(e), where we’ll try to discuss another popular-and-not-really-working technique, known as “code encryption” (though “code scrambling” is IMO a better name for it)]]


Don't like this post? Comment↯ below. You do?! Please share: ...on LinkedIn...on Reddit...on Twitter...on Facebook



Cartoons by Sergey GordeevIRL from Gordeev Animation Graphics, Prague.

Join our mailing list:


  1. Dahrkael says

    Theres another method to avoid debugging:
    hook and modify ntdll.dll!DbgBreakPoint: and ntdll.dll!DbgUiRemoteBreakin calls, which as of now is not prevented by Scylla as far as I know.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.