Contents of “Development and Deployment of Multiplayer Online Games: from social games to MMOFPS, with stock exchanges in between”

 
Author:  Follow: TwitterFacebook
Job Title:Sarcastic Architect
Hobbies:Thinking Aloud, Arguing with Managers, Annoying HRs,
Calling a Spade a Spade, Keeping Tongue in Cheek
 
 

Cover for the upcoming book
[[This is a Table of Contents (and also “About” and “Scope” sections) from the upcoming book, which is currently being beta-tested. For details of this Beta Testing, please refer to the post on it. All the content published during Beta Testing, is subject to change before the book is published.]]

The book is WIP; new content (with links below) is usually added on weekly basis (no warranties of any kind, batteries not included).

As an alternative to links below, you may download PDFs of Beta Chapters (for offline reading, essentially the same as the links below)

Contents

Table of Contents

About

  • About the Author
  • About the Translator
  • About the Illustrator

Scope

  • Whether Writing Your Own Engine or Using Existing One, You Do Need This Book
  • Focused on Massively Multi-Player Games, but Applies to LAN-based Games Too
  • Genres: From Social Games to MMOFPS, with Stock Exchanges in Between
  • Topics: All But Gameplay/AI/Monetization
    • Too Large to Fit and Therefore Sketchy: 3D and Physics
  • Prerequisites
  • CD NOT included
  • BYOS (Bring Your Own Salt)

Volume 1. Architecture and Pre-Development

WIP: First few chapters have already been rewritten to “RC1” status from “beta” ones; for these Chapters the most up-to-date ToC is provided.

Chapter I. Game Design Document from an MOG perspective

NB: RC1 of Chapter I is about 80% rewrite of “beta” Chapters I and II

  • Are you Passionate about Your Game?
  • 3000-word Crash Course for Indie Developers and First-Time Game Developers
    • On GDD
      • Surprised hare:Subject to Change, Seven Days a WeekSubject to Change, Seven Days a Week
      • The Over-Generic Fallacy
    • On Project Stakeholders
      • On Focus Groups
      • On Monetization
      • On Stakeholder Availability
      • Summary on Project Stakeholders
    • On a Typical non-MOG Team Structure
    • Time-To-Market, MVP, and Planning
    • On Importance of Holding Your Horses
  • Three All-Important GDD Rules
  • Limited-Life-Span vs Undefined-Life-Span Games
  • Client-Driven vs Server-Driven Development Workflow
  • On Matchmaking and Social Aspect of your MOG
  • Your GDD Requirements List
  • Additional MOG-Specific Teams
    • Network Team
    • Back-End Team a.k.a. Server Team
    • Both Network Team and Back-End Team MUST be First-Class Citizens
  • Running Costs Breakdown
  • Game Entities and Interactions
    • Game Entities: What are You Dealing With?
    • Interactions Between Game Entities
    • What Should You Get? Entities&Interactions Diagram
    • Examples of Entities and Interactions
      • Social Farming and Farming-Like Games
      • Casino Multiplayer Games Stock Exchanges, Sports Betting and Auction Sites
      • Virtual World Games (Arenas[[TODO]]/MMOTBS/MMORTS/MMORPG/MMOFPS)
      • Tournament-Oriented Games
    • Entities&Interactions Diagram as a Starting Point to Architect Your Game Bibliography

Chapter II. On Cheating, P2P, and [non-]Authoritative Servers

NB: RC1 of Chapter II is about 50% rewrite of  “beta” Chapter III

  • If you’re popular enough, they Will find Reasons to Cheat
  • The Big Fat Hairy Difference from e-commerce
  • Dealing with Cheaters
    • Attacks: The Really Big Advantage of the Home Turf
      • Published Attacks: Higher Impact, but Home Turf Advantage is Regained
      • Low-Impact and High-Impact Attacks
        • Game Rule Violations
        • DB Attacks
        • Information Leaks
        • Reflex Augmentation
        • Grinding Bots
        • Attacking Another Player’s Device
        • Multiple Player Accounts
        • DDoS
        • MMOG Attack Type Summary
      • Authoritative Client: Pretty Much Hopeless against Cheaters
        • Code Signing – Strike 1
        • Web of Trust – Strike 2
        • Cross-Checks – Strike 3
        • Homomorphic Encryption – doesn’t even start to fly
        • Authoritative Client MMOG Summary
      • Deterministic Lockstep: No Game Rules Violations, but Wide-Open to Information Leaks
      • Authoritative Server: As Cheater-Proof As They Go
        • Authoritative Servers: Scalability is Imperfect, But is Workable
          • A Very Example Calculation
        • Summary: Authoritative Server is not ideal, but is The Only Thing Workable
      • Assertive hare:Yes, It Is Going to Be Client-ServerBottom Line: Yes, It is Going to Be Authoritative Server

Chapter III. Protocols.

NB: RC1 of Chapter III is about a 30% rewrite of  “beta” Chapter VII. More changes are planned for RC1, including adding description of the common technique for overlapped-splitting the Game World for MMOs

  • RTT, Input Lag, and How to Mitigate Them
    • Data Flow Diagram, Take 1
    • Input Lag: the Worst Nightmare of an MOG developer
      • Input Lag: User Expectations
      • Input Lag: How Much We Have Left for MOG
      • Accounting for Packet Losses and Jitter
        • Internet is Packet-Based, and Packets can be Lost
        • Cutting Overhead
        • Client-Side and Server-Side Buffering
      • Input Lag: Taking a Bit Back
    • Data Flow Diagram, Take 2: Fast-Paced Games Specifics
    • RTT
      • LAN RTT vs Internet RTT
      • On CDNs and Geo Server Distribution
      • RTT and Players
    • Back to Input Lag
    • Data Flow Diagram, Take 3: Client-Side Prediction and Interpolation
      • Client-Side Interpolation
      • Client-Side Extrapolation a.k.a. Dead Reckoning
      • Running into the Wall, and Server Reconciliation
      • Client-Side Prediction
      • Take 3 Diagram
      • Lag Compensation a.k.a. Server Rewind
      • There Are So Many Options! Which ones do I need?
  • Game World States and Reducing Traffic
    • Server-Side, Publishable, and Client-Side Game World States
      • Client-Side State
      • Server-Side State
      • Publishable State
      • Why Not Keep them The Same?
      • Non-Sim Games and Summary
    • Publishable State: Delivery, Updates, Interest Management, and Compression
      • Interest Management: Traffic Optimization AND Preventing Cheating
      • Before Compression: Minimizing Data
      • Compression
        • Delta Compression
        • Dead Reckoning as Compression
        • Classical Compression
        • Combining Different Compression Mechanisms and Law of Diminishing Returns
      • Traffic Optimization: Recommendations
  • Point-to-Point Communications and non-blocking RPCs
    • RPCs
      • Implementing Non-Blocking RPCs
      • Void vs non-Void Non-blocking RPCs
        • Non-void RPCs
    • Client-to-Server and Server-to-Client Point-to-Point communications
      • Inputs
        • Input Timestamping
      • “Macroscopic” Client Actions
      • Server-to-Client
    • Server-to-Server Communications
      • Seamless Handling of Transient Disconnects
        • Option 1. Separate Caller/Callee Handling
        • Option 2. Two Guaranteed Delivery Streams
      • Going Further: Inter-DB Transactional Integrity
      • Server Object Addressing
      • Using Message Queues for Server-to-Server Communications
        • MQs and Transactional Integrity
        • Uses for MQ on the Server-Side
        • Brokered vs Brokerless
  • IDL: Encodings, Mappings, and Backward Compatibility
    • IDL Development Flow
    • Developing your own IDL compiler
      • IDL into AST using YACC/Lex (or Bison/Flex)
        • Other Uses for YACC-based Code Generators
    • IDL + Encoding + Mapping
    • Example: IDL
      • Sanitizing Input Data
    • Example: Mapping
      • Mapping to Existing Classes
    • Example: Encoding
    • Backward Compatibility
    • On Specific Encodings

Chapter IV. DIY vs Re-use: In Search of Balance

NB: RC1 of Chapter IV goes along  “beta” Chapter IV pretty closely.

  • Business Perspective: DIY Your Added Value
  • Engine-Centric Approach: an Absolute Dependency a.k.a. Vendor Lock-In
    • Implications of Vendor Lock-In
    • Engine-Centric Approach: Pretty Much Inevitable for Indie MMORPG/MMOFPS
    • Engine-Centric Approach: You Still Need to Understand How It Works
    • Engine-Centric Approach: on “Temporary” dependencies
  • “Re-Use Everything in Sight” Approach: An Integration Nightmare
  • “DIY Everything”: The Risk of Never-Ending Story
  • “Responsible Re-Use” Approach: In Search of Balance
    • “Responsible Re-Use” Examples
    • “Responsible Re-Use”: on “Temporary” dependencies
  • Summary

Chapter V. Reactor-fest. I got Event Loop. I got Event Loop. I got All Event Loop

RC1 OF Chapter V IS CURRENTLY BEING WORKED ON (based on parts of “beta” Chapter V and parts of “beta” Chapter VI).

ToC BELOW IS FOR “BETA” CHAPTERS , and will be updated as the work on “RC1” goes ahead.

Chapter V. Modular Architecture: Client-Side

  • Graphics
    • On Developers, Game Designers, and Artists
    • On Using Game Engines as Pure Graphics Engines, and Vendor Lock-In
    • Games with Rudimentary Graphics
    • Games with 2D Graphics
    • On pre-rendered 3D
    • Games with 3D Graphics
    • Logic-to-Graphics Layer
    • Dual Graphics, including 2D+3D Graphics
  • Programming Language for Game Client
    • One Language for Programmers, Another for Game Designers (MMORPG/MMOFPS etc.)
    • A word on CUDA and OpenCL
    • Different Languages Provide Different Protection from Bot Writers
    • Hare asking question:Big Fat Browser ProblemResilience to Reverse Engineering of Different Programming Languages
      • Compiled Languages
      • Languages which compile to Byte-Code
      • Interpreted Languages
      • Summary
    • Language Availability for Game Client-Side Platforms
    • Sprinkle with All The Usual Considerations
    • Default Option: The Least-Commonly-Denominated C++
    • Big Fat Browser Problem
  • Distributed Systems: Debugging Nightmare
    • The Holy Grail of Post Mortem
  • Portability: Platform-Independent Logic as “Nothing But Moving Bits Around”
  • Stronger than Platform-Independent: Strictly-Deterministic
    • Strictly-Deterministic Logic: Benefits
    • Strictly-Deterministic Logic: On User Replay
    • Implementing Strictly-Deterministic Logic: Definitions
    • Implementing Inputs-Log
    • Implementing Strictly-Deterministic Logic: Original Non-Strictly-Deteministic Code
    • Implementing Strictly-Deterministic Logic: Strictly-Deteministic Code via Intercepting Calls
    • Implementing Strictly-Deterministic Logic: “Pure Logic”
    • Implementing Strictly-Deterministic Logic: TLS-based Compromise
    • Implementing Strictly-Deterministic Logic: Which Model to Choose?
    • Implementing Strictly-Deterministic Logic: Which system functions we’re speaking about?
    • Strictly-Deterministic Logic: Non-Issues
    • Strictly-Deterministic Logic: No Access to Globals
    • Strictly-Deterministic Logic: Pointers
    • Strictly-Deterministic Logic: Cross-Platform Issues
    • Strictly-Deterministic Logic: Implementation summary
    • Strictly-Deterministic Logic: Overall summary
  • Event-Driven Programming and Finite State Machines
    • Relation to Finite Automata as taught in Uni
    • Implementing Deterministic Finite State Machines
    • EventProcessor Variations: Circular Buffers
    • Deterministic Finite State Machines: Nothing Too New But…
    • Finite State Machines: Summary
  • Client Architecture Diagram
    • Queues-and-FSMs (QnFSM) Architecture: Generic Diagram
    • Migration from Classical 3D Single-Player Game
      • Interaction Examples in 3D World: Single-Player vs MMO
        • MMOFPS interaction example (shooting)
        • MMORPG interaction example (ragdoll)
        • UI interaction example
    • FSMs and their respective States
      • Game Logic FSM
      • Animation&Rendering FSM
        • Game Loop
      • Communications FSM
      • Sound FSM
      • Other FSMs
    • On Additional Threads and Off-Loading
    • On Latencies
    • QnFSM Variations
    • On Code Bases for Different Platforms
    • QnFSM Architecture Summary
    • Relation to Actor Concurrency
    • Relation to Erlang Concurrency and Akka Actors
  • Bottom Line for Chapter V

Chapter VI. Modular Architecture: Server Side

  • Deployment Architectures, Take 1
  • Don’t Do It: Naïve Game Deployment Architectures
  • Web-Based Game Deployment Architecture
    • Web-Based Game Deployment Architecture: How It Works
    • Taming DB Load: Write-Back Caches and In-Memory States
    • Write-Back Caches: Locking
      • Optimistic Locking
      • Pessimistic Locking
      • Transactions
    • Web-Based Deployment Architectures: FSMs
    • Web-Based Game Deployment Architecture: Merits
  • Classical Game Deployment Architecture
    • Game Servers
      • Game World Servers
      • Matchmaking Servers
      • Tournament Servers
      • Payment Server and Social Gateway Server
    • Implementing Game Servers under QnFSM Architecture
      • Game Logic and Game Logic Factory
      • TCP Sockets and TCP Accept
      • UDP-related FSMs
      • HTTP-related FSMs
      • CUDA/OpenCL/Phy FSM
      • Simplifications
    • QnFSM on Server Side: Flexibility and Deployment-Time/Run-Time Options
      • Threads and Processes
      • Communication as an Implementation Detail
      • Moving Game Worlds Around
      • Online Upgrades
      • On importance of Flexibility
    • DB Server
      • DB API and DB FSM(s)
      • Meanwhile, at the King’s Castle…
      • Single-Connection Approach
      • Multiple-Connections Approach
    • Failure Modes and Effects
      • Communication Failures
      • Server Failures
        • Containment of Game World Server failures
      • Server Fault Tolerance: King is Dead, Long Live the King!
        • Fault-Tolerant Servers: Damn Expensive
        • Fault-Tolerant VMs
          • Virtual Lockstep: Not Available Anymore?
          • Checkpoint-Based Fault Tolerance: Latencies
          • Complete Recovery from Game World Server failures: DIY Fault Tolerance in QnFSM world
    • Classical Game Deployment Architecture: Summary
  • Enter Front-End Servers
    • Front-End Servers: Benefits
    • Front-End Servers: Latencies and Inter-Player Latency Differences
    • Client-Side Random Balancing and Law of Big Numbers
      • DNS Round-Robin
      • Client-Side Random Balancing
      • Client-Side Random Balancing: a Law of Large Numbers, and comparison with DNS Round-Robin
      • Server-Side Load Balancers
      • Balancing Summary
    • Front-End Servers as a CDN
    • Front-End Servers + Game Servers as a kinda-CDN
    • On Affinity
    • Front-End Servers: Implementation
      • Front-End Servers: QnFSM Implementation
        • Routing&Data FSMs
        • Routing&Data Factory Thread
        • Routing&Data FSMs in Game Servers and Clients
    • Front-End Servers Summary
  • Operating Systems
    • New Generation Chooses Cross-Platform! Well, at least it SHOULD…
    • Hare with omg face:The Eternal Windows-vs-Linux DebateEternal Windows-vs-Linux Debate
      • Open-Source
      • Stability/Reliability
      • Security
      • Technical Differences (kernel scheduler, network stack, etc)
      • Is it Enough to Decide?
      • Free as in “Free Beer”
        • TCO wars
        • On ISPs and Windows-vs-Linux Cost
      • Time To Market: Familiarity to your Developers
      • It is All about Money :-(
      • Mixed Bags
      • Linux-vs-Windows: Time to Decide
    • Things to Keep in Mind: Windows
    • Things to Keep in Mind: Linux
    • Things to Keep in Mind: All Platforms
  • Asynchronous Processing for Finite State Machines/Actors: from plain event processing to Futures (with OO and Lambda Call Pyramids in between)
    • Take 1. Naïve Approach: Plain Events (will work, but is Plain Ugly)
    • Take 2. OO-Style: Less Error-Prone, but Still Unreadable
    • Take 3. Lambda Continuations to the rescue! Callback Pyramid
      • On Continuations
      • Exceptions
      • Limitations
    • Take 4. Futures
      • Similarities and Differences
        • All the different takes are similar
        • Differences from std::future etc.
        • Similarities to Node.js
    • On serializable lambdas in C++
    • TL;DR for Asynchronous Communications in FSMs
  • FSMs and Exceptions
    • Validate-Calculate-Modify Pattern
    • Enforcing const-ness for Validate-Calculate-Modify (C++-specific)
    • Exceptions before Modification Stage are Safe, including CPU exceptions
    • Exception-based Determinism
    • FSM Exception Summary
  • Programming Languages
    • Going Cross-Platform
    • Cross-Platform C++
    • Cross-Platform Languages
      • Pros (compared to C++)
      • Cons (compared to C++)
      • Personal Preferences and FSMs
    • Scripting Languages
    • On Languages as Such
    • Which Language is the Best? Or On Horses for Courses
    • Supporting ANY language/compiler/JIT: Is It Worth the Trouble?
    • Supporting Different Environments
      • Line-to-Line Translations: “1.5 code bases”
      • Line-to-Line Translations: Are They Practical?
      • Inter-Language Equivalence Testing: FSM Replay Benefits
    • On Code Generators and YACC/Lex (or Bison/Flex)

Chapter VII. Modular Architecture: Protocols

  • Data Flow
    • Data Flow Diagram, Take 1
    • Input Lag: the Worst Nightmare of an MMO developer
      • Input Lag: User Expectations
      • Input Lag: How Much We Have Left for MMO
      • Input Lag: Taking a Bit Back
      • RTT
      • Back to Input Lag
    • Data Flow Diagram, Take 2: Fast-Paced Games Specifics
      • Internet is Packet-Based, and Packets can be Lost
      • Cutting Overhead
      • Accounting for Losses and Jitter
      • Take 2 Diagram
    • Data Flow Diagram, Take 3: Client-Side Prediction and Interpolation
      • Client-Side Interpolation
      • Client-Side Extrapolation a.k.a. Dead Reckoning
      • Running into the Wall, and Server Reconciliation
      • Client-Side Prediction
      • Take 3 Diagram
    • Lag Compensation
    • There Are So Many Options! Which ones do I need?
  • World States and Reducing Traffic
    • Server-Side, Publishable, and Client-Side Game World States
      • Client-Side State
      • Server-Side State
      • Publishable State
      • Why Not Keep them The Same?
      • Non-Sim Games and Summary
    • Publishable State: Delivery, Updates, Interest Management, and Compression
      • Interest Management: Traffic Optimization AND Preventing Cheating
      • Before Compression: Minimizing Data
      • Compression
        • Delta Compression
        • Dead Reckoning as Compression
        • Dead Reckoning as Compression: Variations
        • Classical Compression
        • Combining Different Compression Mechanisms and Law of Diminishing Returns
    • Traffic Optimization: Recommendations
  • Point-to-Point Communications and non-blocking RPCs
    • RPCs
      • Implementing Non-Blocking RPCs
      • Specifics of Non-blocking RPCs
      • Non-void RPCs
      • Same-thread operation
    • Client-to-Server and Server-to-Client Point-to-Point communications
      • Inputs
      • Input Timestamping
      • “Macroscopic” Client Actions
      • Server-to-Client
      • Server-to-Server Communications
  • IDL: Encodings, Mappings, and Backward Compatibility
    • IDL Development Flow
      • Developing your own IDL compiler
    • IDL + Encoding + Mapping
    • Example: IDL
    • Example: Mapping
    • Mapping to Existing Classes
    • Example: Encoding
    • Backward Compatibility
    • On Google Protocol Buffers

Chapter VIII. Unity 5 vs UE4 vs Photon vs DIY

  • Unity 5
    • Event-driven Programming / FSMs
    • Unity for MMOG
      • Communications: HLAPI
        • State Synchronization
        • RPCs (a.k.a. “Remote Actions”)
        • HLAPI Summary
      • Communications: LLAPI
      • Unity 5/UNet Summary
  • Unreal Engine 4
    • Event-Driven Programming/FSMs
    • UE for MMOG
      • UE Communications: very close to Unity 5 HLAPI
  • Photon Server
    • Photon Server SDK: Communications
    • Photon Cloud/PUN: Communications
  • Summary
  • Engine-Centric Development Flow
    • Server-Driven Development Flow
      • Server-Driven Development Flow: Personal Suggestions
    • Client-Driven Development Flow
      • Client-Driven Development Flow: Personal Suggestions
    • Important Clarification: Development Flow vs Data Flow

Chapter IX. Pre-Coding Things Everybody Hates, but Everybody Needs Them Too

  • Arguing hare:Everybody Hates These, but Everybody Needs Them TooSource Control System
    • Git
      • Git and unmergeable files
      • Git and 3rd-party libraries
      • Git branching model
  • Continuous Integration
  • 3rd-party Libraries: Licensing
  • Development Process
  • Issue Tracking System
    • Issue Tracking: No Bypassing Allowed
  • Coding Guidelines
    • Naming Conventions
    • Project Peculiarities
    • Per-Subproject Guidelines
    • Enforcement and Static Analysis Tools

ITEMS BELOW ARE STILL WIP, UPDATES ARE USUALLY PUBLISHED EVERY WEEK

 

Volume 2. Development

Chapter X. Things To Keep In Mind

  • Writing for Cross-Platform
  • Error Handling
  • Writing for Debugging
    • asserts, and then assert some more
      • Multi-level asserts
    • FSM Recording
    • Text Logging
    • Post-mortem
  • Writing for Security
  • Writing for i18n
  • Ongoing Testing
    • Unit Testing
    • Replay Testing
    • Simulation Testing
      • Dumb User Simulation
      • Latency/Packet Loss Simulation
        • Wireshark what’s going on!

Chapter XI. To TCP or to UDP? That is the question.

  • TCP vs UDP
  • TCP Disadvantages: Latencies and Traffic
  • UDP Disadvantages: Firewalls and Complexity
  • Making TCP Interactive
  • Making UDP Reliable
  • Reliable UDP Libraries

Chapter XII. Protocols and Marshalling

  • Hare pointing out:Endianness and AlignmentsMarshalling
    • Endianness and Alignments
    • Marshalling Formats
    • Strings: UTF-8 from the Very Beginning
  • Point-to-Point Communications
  • Subscriptions/Publications
  • Guaranteed vs non-Guaranteed
  • Keep-Alives
  • Disconnects and Mass Disconnects
  • Compression
  • NAT traversing

Chapter XIII. Graphics

  • 2D
    • Double Buffering
    • Scaling
    • Animation
    • Assertive hare:Things you Won't be Dealing With: Polygons/Vertices/etc.Physics (sort of)
    • UI
  • 3D
    • Things you Won’t be Dealing With: Polygons/Vertices/etc.
    • Meshes/Textures/UV-Mapping
    • Lights/Camera/Action
    • Objects/Cells
    • Animation
    • Post-Processing
    • Physics
    • UI

Surprised hare:The Tricky Part: Client-SideChapter XIV. Cross-Platform C++ Programming

  • Quite Simple One: Server-Side
    • IPC
    • Sockets
    • Multiple-Writers-Single-Reader Queues
  • The Tricky Part: Client-Side
  • Sample C++ Coding Guidelines
  • On Memory Management
    • locality
    • NUMA

Chapter XV. Random Number Generation

  • What is a Good RNG
    • Marsaglia/NIST tests
  • PRNG
  • Hardware RNG
  • Hardware-assisted (P)RNG
  • RNG-Critical Environments
    • Beyond Bit-Stream: Basic Chi-Square Analysis
  • Separating Game RNG from Transport-Security RNG

Chapter XVI. Basic Security

  • Inquisitive hare:Security by Obscurity is NOT Necessarily BadTLS
  • OpenSSL
  • UDP: DTLS, or TLS over Reliable UDP
  • Server-Side
  • Heresy: Security by Obscurity is NOT Necessarily Bad

Chapter XVII. Logins

  • Dreaming hare:ID or email?On Observers
  • Back-End/Player logins: Separate or Common Login Space?
  • 3rd-Party Logins
  • ID or email?
  • Visible IDs: Censorship
  • Password Hashing
  • Password Recovery

Chapter XVIII. Self-Update

  • State-Based vs Version-Based
  • Signed
  • Differential Updates
  • Online Updates
    • Integration with a Network Library
  • Handling Protocol Updates

Threatening hare:No Backdoors WhatsoeverChapter XIX. Testing, Debugging, and Logging

  • No Backdoors Whatsoever
    • Limited Logging at Sensitive Points
  • Automated Testing
  • Logging for Post-Mortem

Chapter XX. Payment Processing

  • In-Client vs On-Web
  • Credit Cards
    • PCI DSS
  • Electronic Cheques
  • Bitcoins

Chapter XXI. Avatars

  • 2D: Uploadable vs Library-Based
  • Uploadable Avatars: Censorship

Chapter XXII. Deployment Architecture – Take 1

  • Femida hare:To Cloud or Not To Cloud?Once again on MVP
  • Don’t Count on In-House Servers
  • To Cloud or Not To Cloud?
    • Cloud vs Traditional ISP
    • Choosing Cloud Provider
    • Choosing Traditional ISP
    • Choosing Servers
  • CDNs
  • Remote Management (Cloud/iLO/DRAC/IMM)
  • Firewalls
  • On Storage
    • SSD, NVMe, and RAID
    • RAID-5 vs RAID-0 for OLTP loads
    • BBWC

Volume 3. Deployment

Laughing hare:Seating Idle? Forget it!Chapter XXIII. Life after Birth

  • Seating Idle? Forget it!
  • Fighting Backlog From Both Ends
  • Prioritization
  • Release Cycle
    • Deployment without Stopping the World
    • On Continuous Delivery

Chapter XXIV. Back-End Tools

  • Control
  • Monitoring
  • Reporting
  • CRM

Hare with hopeless face:Rules: Enforceable vs Non-enforceableChapter XXV. Player Abuses

  • Rules: Enforceable vs Non-enforceable
  • Attacks
    • Brute Force
    • DDoS
    • App-Specific
  • Disconnect Abuses
  • Payment Abuses
  • Multiple Accounts
    • Identifying Computer/User
  • Promotions and Promotion Abuses
  • Items Traded on Secondary Markets (eBay etc.)

Hare with hopeless face:Being... ReactiveChapter XXVI. Protecting from Internal Abuses

  • Being Active vs Reactive
  • Attack Vectors
  • Need-to-Know/Need-to-Do
  • Logging Everything and More
  • Roles and Hierarchy: Limited-Depth Trees

Chapter XXVII. Optimizations and Scaling

  • Other-than-DB Optimizations & Scaling
  • Hare wondering if you are crazy:When to Violate 3NFDB Optimization & Scaling
    • Indexes and Double Indexes
    • Physical DB Layout
    • Caching, More Caching, and Even More Caching
    • When to Violate 3NF
    • Replicas
      • Aggregates
    • Scaling
    • Federated Databases
    • Split Databases
      • Inter-DB transactions

Chapter XXVIII. i18n

  • Phrase-Level Translation
  • Translation Engine
    • Separating Translators from Developers
  • Dealing with Layouts

Chapter XXIX. Multi-Currency

  • Multi-Currency Payments
  • Wtf hare:Currency-Exchange AbusesCasinos and Stock Exchanges: Multi-Currency Accounts
    • To Margin or Not To Margin
    • Currency-Exchange Abuses

Chapter XXX. Security, Take 2

  • Two-Factor Authentication
    • Login Recovery for 2FA
  • The Big Problem of Remote Admin’s Laptop
  • BYOD for Internal People in General
  • Hardening
  • Security Perimeters
  • IDS

Sarcastic hare:'Money is no Object': A Curse in DisguiseChapter XXXI. Deployment Architecture, Take 2

  • “Money is no Object”: A Curse in Disguise
  • On Clusters
  • On SANs
  • On Reserve Datacenters
    • Remote Backups

Chapter XXXII. Themes, Modding and 3rd-Party Tools

  • Intentional/Unintentional Abuses
  • Advantages of Explicit Support for Modding/3rd-Party Tools
  • DLLs vs Processes

Chapter XXXIII. Basics of Bot Fighting

  • Security by Obscurity
  • Difference Between Programming Languages
  • Judging hare:Good Development & Deployment cannot make your game successful, but bad ones can easily kill itPassive Methods
    • No Sensitive Text as Windows Controls
  • Server-Side Statistics
  • Client-Side Detection
  • Captcha and Others

Chapter XXXIV. Conclusion

  • Good Development & Deployment practices cannot make your game successful, but bad ones can easily kill it

Appendix. Applicability to LAN-based games


[[Now, we will interrupt our beta book, and will ask you a question:

How do you want this book to be posted on this site during Beta-Testing?

  • Start from the very beginning (Chapter I) (67%, 43 Votes)
  • Skip all the preliminaries in Part A, start right from coding (Part B, Chapter IX) (20%, 13 Votes)
  • I don't care about Business Requirements! Skip Chapter I, start from Chapter II (11%, 7 Votes)
  • Forget it, I won't read it anyway (2%, 1 Votes)

Total Voters: 64

Loading ... Loading ...

EDIT: POLL IS CLOSED NOW

Back to the book…]]


About

'No Bugs' HareAbout the Author. The author of this book is a ‘No Bugs’ Hare from the warren of Bunnylore. He is known for being a columnist for Overload journal (ISSN 1354-3172) and for his significant contributions to software development blog ithare.com. As ‘No Bugs’ is a rabbit with his mother tongue being Lapine, he needed somebody to translate the book into some human language. And of course, as the book is highly technical, to translate technical details with the highest possible fidelity, he needed a translator with substantial development experience.

'No Bugs' HareAbout the Translator. This book has been translated from Lapine language by Sergey Ignatchenko, a software architect since 1996. He is known for writing for industry journals since 1998, with his articles appearing in CUJ, Overload, C++ Report, and (IN)SECURE Magazine. His knowledge of Lapine is quite extensive, and he is routinely translating the column which ‘No Bugs’ writes for Overload. During Sergey’s software architecting career, he has been leading quite a few projects, including being a co-architect of a G20 stock exchange (the same software has been used by stock exchanges of several other countries too), and a sole original architect of a major gaming site (with hundreds of millions user transactions per day, and processing hundreds of millions of dollars per year). As a kind of paid hobby, he’s also inventing things: he’s an author and co-author of about a dozen of patents (unfortunately, owned by his respective employers :-( ).

'No Bugs' HareAbout the Illustrator. Illustrations for this book are made by Sergey Gordeev from gagltd.eu. He is a professional animator with 10+ awards from various animation festivals, and is known for being a director of some of Animated Mr. Bean series.

Scope

Whenever you open a book for the very first time, you naturally have two questions: “Is this book for me?” and “What is this book about?”. First, let’s see whether this book is for you if you want to use a 3rd-party game engine.

Whether Writing Your Own Engine or Choosing an Existing One, You Do Need This Book

Assertive hare:One thing you should never do, is blindly believing that the engine will be a perfect fit to your specific gameIn many cases it may be very tempting to use an existing (usually 3rd-party) game engine rather than to develop your own one. And quite often it may be a Really Good Idea to do so. However, one thing you should never do when developing anything-more-complicated-than-two-player-tic-tac-toe, is blindly believing that the engine will be a perfect fit to your specific game. Even when the game engine is used by dozens of highly successful games – there is no guarantee that it will work for your specific requirements (unless, of course, you’re making a 100% clone of one of these games).

Instead of assuming “The Engine Will Do Everything Exactly As We Want”, you should try to understand all the relevant implications of the engine-you’re-about-to-choose, and see if its limitations and peculiarities will affect you badly down the road. This is where this book will be useful. It won’t compare existing engines, but it will describe (and explain) principles which are necessary to build a technically successful multi-player game. It means that you will be able to make an informed decision about the game engine (and maybe will decide to write your own parts to complement the existing engine).

So, if you’re going to choose an existing game engine – you need to understand how game engines work (so you can choose the game engine which is suitable for your needs even when you grow to millions of players). And if you’re going to write your own game engine – you certainly need to understand how game engines work. It means that

Whatever you do while developing your own Massively Multi-player Game,
You DO Need This Book
:-)

That’s all what matters for now; we’ll discuss pro- and contra- arguments for DIY over reusing 3rd-party stuff in Chapter [[TODO]].

Focused on Massively Multi-Player Games, but Applies to LAN-based Games Too

Hare pointing out:If you're about to write a LAN-based game – most of the this book still applies, but make sure to read AppendixNow let’s try to answer the question “What is this book about?” As the title suggests, this particular book is all about massively multi-player games working over the Internet. It means that single-player games are certainly not our focus; however, for LAN-based multi-player games with only a few players this book is relevant (to a certain extent). While you certainly can build a LAN-based game along the lines described in this book, and it will work, doing it in a way-optimized-for-massively-multiplayer-environments may involve quite a few overkills, which in turn may lead to unnecessary delays with releasing your game. If you’re about to write a LAN-based game – most of the this book still applies, but make sure to read Appendix “Applicability to LAN-based games” for discussion of potential simplifications which you may apply. [[Beta notice: at this point Appendix is tentative and not guaranteed]]

Whenever saying “massively multi-player” game, I mean that the game should be able to reach really massive numbers; if following the advice in this book (including that of in Part D), millions of the players should be perfectly doable; I’ve personally architected a site which has hundreds of thousands of simultaneous players and handled hundreds of millions of user transactions per day.1


1 for our purposes, “user transaction” is an interaction of a user with the server, which causes server-side changes, where these changes are visible to at least some other players

 

Genres: From Social Games to MMOFPS, with Stock Exchanges in Between

When it comes to the game genres, this book aims to cover all of them (as long as multiple players are involved). While these games vary in terms of requirements imposed on the engine, these requirements are still interrelated; moreover, these requirements represent a more-or-less continuous spectrum, so breaking it at any given point would be quite arbitrary. Let’s take a look at the Latency Tolerance a.k.a. Acceptable Input Lag (which is one of the most important requirements from the multiplayer game engine perspective):2

  • Social Games (usually Facebook/Phone). Typical Latency Tolerance: usually minutes are ok
  • MMO Turn-Based Strategies. Typical Latency Tolerance: seconds
  • Multi-Player Casino Games (bingo, blackjack, poker). Typical Latency Tolerance: single-digit seconds
  • Stock Exchanges and Sports Betting. Typical Latency Tolerance: as fast as possible, though most important is “being fair” requirement.
  • Massively Multiplayer Online Real-Time Strategies (MMORTSs). Typical Latency Tolerance: low triple-digit milliseconds
  • Massively Multiplayer Online Role-Playing Games (MMORPGs). Typical Latency Tolerance: < 200-300 milliseconds
  • Massively Multiplayer Online First-Person Shooters (MMOFPSs). Typical Latency Tolerance: high double-digit milliseconds

As you can see, there is a whole spectrum of latency requirements, from minutes for social games to high double-digit milliseconds for MMOFPS, with pretty much everything in between. However, we will see that as soon we take latencies into account, further differences between genres become much less pronounced from the multiplayer engine perspective.


2 only typical values are provided, your mileage may vary, batteries not included. Also note that here we’re speaking about end-to-end latency/input lag (from controller button pressed, to player seeing results on screen); lots of different factors will eat milliseconds out of this allocation even before we have a chance to look at the data, see further discussion in Chapter [[TODO]]

 

Topics: All But Gameplay/AI/Monetization

Inquisitive hare:While we're not about to answer the question “WHAT do you want to do?”, we will try to answer the question of “HOW to do whatever-you-want-to-do”Game development and deployment is a huge task, so it is important to realize what exactly we want to cover. This book is very ambitious in this regard: it aims to cover all the aspects of game development and deployment, with one (though all-important) exception. In this book, we won’t try to answer questions such as “What should your game be about?”, “How should your game look?”, and “How to make money out of your game” ; these are all-important business questions which you need to answer yourself. When starting development you should know exactly how you want your game to be played, how you want it to look, how your AI (if applicable) will work, and how you’re going to monetize it.3 These questions are completely out of scope of this book.4

On the other hand, as soon as you have answers to these questions – this book has got you covered. We’ll discuss pretty much everything you will need to release your game and keep it running, from overall architecture to deployment and post-deployment issues.

In other words: while we’re not about to answer the question “WHAT do you want to do?”, we will try to answer the question of “HOW to do whatever-you-want-to-do” in as much detail as we can fit into one single book.

Too Large to Fit and Therefore Sketchy: 3D and Physics

While we’re aiming to answer all the questions related to “HOW to do whatever-you-want-to-do”, there are a few issues which are very large on their own, and just wouldn’t fit into one single book. As a result, these issues are admittedly sketchy in this book; while we will have a chapter on 3D graphics and sound (and will discuss physics engines too), please don’t expect this to be as detailed on these topics as those books which are dedicated to 3D engine design.

What is included, should be sufficient to choose your 3D engine (which can often work as a physics engine too). However, if you want to write your own 3D/physics engine, you will need to go beyond this book (I will provide necessary pointers within appropriate chapter).


3 of course, your vision will change as development goes on, but at any point you should have a clear vision of “What you want to achieve”
4 Note that while business question of monetization is not covered, a technical question of payment methods is covered to some extent

Prerequisites

Surprised hare:If your game project is your very first programming project – you're likely to have difficulties understanding this bookThis book is targeted towards at least somewhat experienced developers (in other words, it is NOT “how to develop your first program” book with IDE screenshots and copy-paste examples). If your game project is your very first programming project – you’re likely to have difficulties understanding this book5. For example, I am not going to explain what the “function” (“database” etc.) is; also I expect you to know what “source control” is and why it is necessary, and why encapsulation is a Good Thing(tm).

On the other hand, this book doesn’t rely on in-depth knowledge in any specific area. You don’t need to be a network guru who knows every tiny detail of RFC 793 by heart, neither you need to have hand-on experience with shaders and/or CUDA; even less I expect you to be a C++ guru who is capable of writing boost-like templates (to be honest, at the moment this last one is beyond my own capabilities too). Of course, 3D graphics experience may be helpful for 3D games, and knowledge of network basics and sockets won’t hurt in any case, but whenever discussing the issues which go beyond “things which every developer out there should know anyway”, I will try to provide pointers “where to read about this specific stuff if you happen to have no idea about it”.


5 Feel free to read the book even in this case, but don’t complain too much if it turns out to be too difficult

 

CD NOT included

The book is not going to have an associated CD. First, any attempt to provide CD-with-sample-code almost inevitably ends up in a very platform-specific code, and as one of the main points of the book is that your platform depends on your needs, providing heavily platform-specific code to illustrate this point will be a kind of oxymoron. Second, as mentioned above, this book is not intended as a book-where-you-can-copy-paste-your-very-first-game-from. The reason behind this approach is that in the Internet Age it is easy to find specific information such as “how to write your first Direct3D program”; such tutorials are abundant and can be found (if necessary) elsewhere.

On the other hand, 50’000-feet view with architectural concepts and practical observations such as “what are the advantages of TCP over UDP and vice versa depending on your particular game” or “why exposing multithreading to game logic level is harmful” are relatively difficult to find; this is what this book intends to cover.

BYOS (Bring Your Own Salt)

Judging hare:In the practical world (especially in game development), for each and every “Do THIS_THING this way” statement there exists a counter-exampleOne last thing I would like to mention before we proceed to more practical matters. There is no one single sentence in this book (or any other book for that matter) which is to be taken as an “absolute truth”. In the practical world (especially in game development), for each and every “Do THIS_THING this way” statement there exists a counter-example, illustrating that sometimes THIS_THING can (or even should) be done in a different manner.

Every advice out there has its own applicability limits, and so does any advice within this book. When I know certain game-related scenarios when these limits are likely to be exceeded (and the advice will become inapplicable) – I will try to mention it. However, it is extremely difficult to predict all the usage scenarios in such a huge industry as game development, so you should be prepared that some of the advice in this book (or any other book) is inapplicable to your game without warning.

Therefore, take everything you read (here or elsewhere) with a good pinch of salt. And as salt is not included with the book, you’ll need to bring your own one. In more practical terms –

For each and every decision you make based on advice in this book, ask yourself:
Does This Advice Really Apply to My Specific Case?

[[To be Continued…

Tired hare:

First (actually, first, second, or ninth – depending on the poll above) chapter from the book coming soon… Stay tuned!

EDIT: beta Chapter I, Business Requirements, has been published. All the beta chapters will be shown on this page in ToC as they are published.]]

Don't like this post? Comment↯ below. You do?! Please share: ...on LinkedIn...on Reddit...on Twitter...on Facebook

Acknowledgement

Cartoons by Sergey GordeevIRL from Gordeev Animation Graphics, Prague.

Join our mailing list:

Comments

  1. says

    Something I didn’t see mentioned in your TOC is latency simulators. For doing online development with UDP, it is the easiest tool I’ve used for finding bugs. Take what you’ve made, crank the latency up to 1000ms or so and see if it still works. Then set it to reorder and/or drop packets, you’ll find even more bugs. The one I used was a regular Linux box bridging two subnets, I can’t remember the name of the program it used.

    Really interested in the book BTW. New Riders “Developing Online Games” is also an excellent resource for MMO development.

  2. Ian Smith says

    Put i18n somewhat sooner than Volume 3. It is part of UI planning.

    Don’t forget my favorite bot fighting technique, Game Design. I played WoW back shortly after beta. My attack sequence was: tab, 122322422 90 repeat. ( 9 and 0 to replenish health and mana.) By contrast, the game I had played just before that involved too much aiming and risk to effectively bot. The game was too complex to reduce to easily reproducible behaviors regardless of area. A context sensitive game is inherently bot resistant, and more fun to play. The game I played before the fun one? Matrix Online, where I was essentially forced to bot for crafting. Because I was the only player with a decent crafting supply, I ended up making a few hundred dollars selling the crafting supplies.

    If fishing is boring, it will be botted almost immediately. Context, player feedback, and in some cases, less clues. Trove fishing is botted by looking for the sound playback function call. If the line just twitched and it was an obvious message or giant function call, it would be harder to bot. Including memory addresses, fishing bots were released 2 days after fishing was released.

Leave a Reply

Your email address will not be published. Required fields are marked *