HomeVideos

Protecting Your Online RPG’s Economic Integrity

Now Playing

Protecting Your Online RPG’s Economic Integrity

Transcript

302 segments

0:00

Hi, I'm Chris Wilson. I was going to  start the video by joking "if you're  

0:04

an average viewer of mine, you're  probably running an online RPG and  

0:07

have questions about how to maintain the  integrity of its economy", but honestly,  

0:11

looking at the emails I get, I'm beginning  to suspect this might actually be true.

0:15

Don't you hate it when  you're playing an online RPG,  

0:18

trying your hardest to level up and  find good items, but it feels like  

0:20

all the other players are miles ahead of  you because they're not playing fairly?

0:24

In this video, I'm going to talk about economic  integrity. Put simply, this is the ability for  

0:29

players to fairly earn progress and items through  in-game actions, on a level playing field, without  

0:34

outside influence or cheating. Basically, that the  game doesn't let people unfairly get ahead of you.

0:39

I'll discuss why it's important for developers  of online RPGs to prioritise economic integrity,  

0:44

what policies they should consider,  how they should deal with cheating,  

0:47

real money trading, and even their  own customer support departments.

0:51

As some of you may know, I was one of  the original creators of Path of Exile,  

0:54

and ran it until late 2023. From  the very first day of development,  

0:58

we prioritised the economic integrity of its  online realm. We wanted to make sure that  

1:02

progression and items were meaningful. Players  needed to know that other people couldn't just  

1:06

cheat their way to the top. I think we did  a good job with this overall, and learnt  

1:10

a lot along the way. As more and more games  launch with secure online servers these days,  

1:14

I felt it was important to make a video about what  I've learned over my career. The suggestions in  

1:19

this video are my personal opinions after a couple  of decades of thinking about this every day.

1:23

When some people hear you talking about  competitive games, they assume you mean games  

1:27

like Overwatch or League of Legends, but online  RPGs can also be highly competitive, just in a  

1:32

different way. In these games, the economy is the  competitive part. Players compete to make faster  

1:37

progress, get to higher levels, find better items  and complete harder content before everyone else.

1:42

Each Path of Exile league launch is highly  competitive. A lot of players schedule  

1:46

considerable time off work and stay up overnight  to play as much as they can while the economy is  

1:50

fresh. Players come out of the woodwork, bringing  their friends, whenever a new league starts. The  

1:55

seasonal ladder reset model pioneered by  Blizzard North after the launch of Lord of  

1:58

Destruction is an absolutely critical part of the  way that many modern Action RPGs retain players,  

2:03

but it hinges on players feeling that the  economy they're playing in has integrity.  

2:08

These players are trading a lot of time and  effort to get the feeling that they're making  

2:11

real progress that matters, and finding real  items that have value. If something happens to  

2:15

the integrity of the economy that causes them to  doubt this, then they quickly stop playing. This  

2:20

is why it is so catastrophic when a game launches  and then there's some bad exploit soon after  

2:24

launch, wrecking the economy and everyone's  interest in continuing to play that season.

2:28

As a developer, if you're paying the significant  cost of server-authoritative multiplayer for your  

2:32

game, it’s worth finishing the job: protect  the economy those servers exist to validate.

2:37

Online RPGs are also an escape from societal class  issues. Players expect a level playing field where  

2:43

wealth doesn't matter. Playing an online RPG  lets you outplay and outsmart the other players  

2:47

to become more successful than them regardless of  their real world wealth. And the economy needs to  

2:52

have integrity for you to feel this way. You don't  want to be playing a game where some millionaire  

2:56

can buy his way in and get better items than  you just because he's rich in real life. 

2:59

Now sure, if the game doesn't have an  online mode and is just single player,  

3:03

there is no server-validated competitive  economy and hence no integrity considerations  

3:07

are needed. But people in single player  games can just arbitrarily cheat as all  

3:10

the data is local. Many players want  to play on secure multiplayer servers  

3:14

because it validates that progress is real. Some games have a Solo Self-found mode,  

3:18

where players are unable to party with or trade  with each other. In this mode, their progression  

3:22

is still validated serverside so they have  bragging rights about what they have achieved,  

3:26

but there's no way for them to trade with a  greater economy so they aren't affected by  

3:29

many integrity issues. While that's exactly what  some players are looking for, an overwhelming  

3:34

majority of other players like to play together  and trade. If you limit or remove trade from your  

3:38

game to sidestep economic integrity issues,  you certainly miss out on a lot of potential.

3:42

In this video, I'll be primarily talking  about economic integrity for games with  

3:46

trade, but many of the issues still  apply to online RPGs without trade.

3:50

The first step is treating your game’s  economy with respect inside your studio.

3:55

For example, let's say that a celebrity  approaches you and wants to do a promotion  

3:58

for your game. They're going to play it  on stream alongside your lead developer  

4:01

and it'll get a lot of attention. The  problem is, they're famous and busy and  

4:04

aren't going to play through the game to  get a character ready. They want you to  

4:08

pre-create a high level character for them to  play. This could be really good for marketing,  

4:12

but it could also completely undermine your  community's trust in the game economy if you're  

4:15

just going around making characters and items for  people. In a situation like this, you should play  

4:19

with them on a test server and to communicate to  the players that it's on a test server and not  

4:23

the live servers. I know it seems like it's just  one character and won't impact the larger economy,  

4:27

but it's about optics. If you are willing to break  the seal and start interfering with what's on the  

4:31

live servers, players will worry that's going on  to a much larger extent they don't know about.

4:36

In a similar way, you should be very careful  with who gets new information that isn't  

4:40

publicly disclosed in the patch notes.  For example, if you add a secret recipe  

4:44

that players are meant to go looking for,  and your development team knows about it,  

4:47

what's to stop them using it on the live servers  to get a large economic advantage themselves?  

4:51

You need to make sure there are policies  and rules in place that make them very  

4:54

aware which information can't be used  on the live servers until it's public.

4:58

The same applies to your alpha testers.  They may learn secret information that  

5:02

gives them an economic advantage on your live  servers. Make sure they not only understand  

5:06

what they're not allowed to do, but are given  an opportunity to skip testing a particular  

5:09

patch entirely if they want to be eligible  for things like winning prizes in events.

5:14

Economic integrity is about making sure  that players have equal opportunities  

5:17

to earn progression and items in your game.  You enforce this fairness with a combination  

5:22

of policies that you post and decisions that you  make in the cases where the policies don't cover  

5:26

a situation. When you have to make a decision,  be consistent and communicate your reasoning. If  

5:31

you make a decision for the sake of fairness  and players don't understand what happened,  

5:35

they may perceive it as unfair, which undermines  the entire point. If you explain the pros and  

5:40

cons of different approaches and why you  picked the one you did, then players will  

5:43

hopefully respect your transparency and that you  were trying to find the best overall solution.

5:47

Players remember precedents. If there's  compensation for an outage or a problem,  

5:51

then they will expect a similar outcome the next  time it happens. If you are giving compensation,  

5:55

you should do it in the form of something that  doesn't affect the progression economy, like  

5:59

credit for in-game cosmetics. I strongly recommend  not compensating players with actual game  

6:04

progression or game items, as while you may make  those players happy, everyone else will see this  

6:08

as interfering with the game's economic integrity. The next topic I want to talk about is players  

6:13

getting unfair economic advantage by just  outright cheating. This can come in many forms,  

6:17

from exploiting bugs in game logic to running  third party programs that reveal information  

6:21

they don't normally see, to running bots  that play for them while they're not there. 

6:25

Ideally you've constructed your game so that no  form of cheating is possible, but this is quite  

6:29

hard in practise. Online RPGs are typically run  as live services with new features and code going  

6:34

live frequently. These are often developed  under tight deadlines and may contain subtle  

6:38

logic mistakes if they haven't been fully tested.  I have previously posted a video on game protocol  

6:43

exploitation that is worth checking out for more  information. In addition, the architecture of the  

6:48

games themselves may make it necessary for the  game client to know certain information that  

6:52

you'd really rather the player didn't see, like  the entire map for the current area for example,  

6:56

so that's sitting in memory ripe for nefarious  programs to read it out and display it. Finally,  

7:01

stopping botting is fundamentally extremely hard  due to the nature of the bots being that they  

7:06

pretend to be real players. So as good as your  intentions are, you will likely need to have plans  

7:10

and policies about how to deal with cheaters. You should work out a legal definition of cheating  

7:15

for your terms of service, and a plain English  one for your community guidelines. Note that  

7:19

bad actors will do their best to "technically  not cheat", so make sure to specify that the  

7:24

guidelines will be updated over time and that  it's all at the developer's discretion anyway.  

7:28

There's often a fine line between efficient play  and cheating. Make sure to clearly define where  

7:32

the line is between a bannable exploit and just  regular smart play, especially for games that  

7:37

encourage players to think outside of the box. In terms of punishments for cheating,  

7:41

I recommend a zero-tolerance policy, with  permanent bans for intentionally exploiting  

7:45

or cheating. It may sound harsh to permanently  ban customers, but non-cheating players will  

7:50

greatly appreciate this as a signal that you  are putting the integrity of the game first.  

7:55

There's a study on anti-cheat enforcement  which is linked in the video's description,  

7:58

that shows that non-cheating players engage with  a game much more if they see that cheating players  

8:03

are being punished. This sounds like common  sense of course, but it's good to see it backed  

8:06

up scientifically. On a side note, if you're  operating in certain regions like Taiwan, get  

8:12

legal advice before permanently banning players. If you ban someone for gaining economic advantage  

8:17

through cheating, they may have used multiple  accounts, so you should detect and punish the  

8:21

associated accounts also. You should be  careful about banning innocent players who  

8:24

triggered the exploit unintentionally and didn't  purposefully gain advantage from it. You may need  

8:29

to define a threshold where guilty players are  defined as people who did it more than once,  

8:33

or who did it after a certain time threshold.  This can be very tricky to make fair. You will  

8:38

also need a robust procedure for handling  appeals, when players believe they have  

8:42

been banned unfairly. You need to be firm with  players who did cheat, so that they can't talk  

8:46

their way out of it, while being forgiving with  innocent players who were flagged by accident.  

8:50

This is very difficult to get right, but is  worth the time and effort to investigate fully.  

8:54

Players need to feel that this process is fair. Make it easy for players to report exploits,  

9:00

ideally with a command or window that is tied  to their play session so that you can examine  

9:04

logs for relevant information about where they  are and what they were doing. You should contact  

9:08

players who reported useful information to thank  them for their report. There's nothing worse than  

9:12

going to the effort of writing up a report of  a live exploit and then hearing nothing back.  

9:16

Consider a bug bounty program where you reward  them with a consistent and communicated reward,  

9:20

potentially including real money, for reporting  exploits. This would hopefully drive more people  

9:25

to take the path of disclosure rather than trying  to exploit a vulnerability for personal gain. 

9:30

If you decide to have a policy where you just  warn or temporarily suspend players for minor or  

9:34

first-time offenses, you still need to remove the  items or progress that they gained by cheating. If  

9:40

you permanently ban someone, it's tempting to just  leave their account as-is with the cheated items  

9:43

or progress intact, because they're permanently  banned and can never access it again. This comes  

9:48

back to hurt you when a few years later, they  manage to appeal the ban and get their account  

9:51

back, full of all their illegal wealth. It's  worth taking the time to clean up every account,  

9:55

even the ones you think you're banning forever. Cleaning up accounts after economic incidents or  

10:00

bugs is an extremely manual and fiddly process. It  usually involves a bunch of senior developer time,  

10:05

because even if you have good tooling in place,  the primary characteristic of a game exploit  

10:09

is that it involves abusing something new and  unexpected, so you'll likely need new or updated  

10:14

tools to repair the damage each time. This is all  worth it though, because if players see you taking  

10:18

it seriously, they will trust the game more and  value their progress and time spent playing it. 

10:23

Sometimes when an economic exploit is going on,  the exploiters will give the cheated items to  

10:27

random players, in an attempt to muddy the  water and make it less likely that you'll  

10:31

just ban everyone involved. This can involve  a lot of manual cleanup, but you don't want to  

10:35

let those exploited items get into the economy.  When you're cleaning up something like this, you  

10:39

should consider taking the game servers down so  that things don't get worse while you're working  

10:43

on it. Players will be disappointed that they  can't play, of course, but they will respect that  

10:47

you're putting the integrity of the game first. If you're in a situation where the economic damage  

10:52

is widespread and exploited items have reached  too many players, your only option may be a full  

10:56

database rollback. It's very important that this  is a game-wide rollback rather than a partial one  

11:01

that just affects some players. That way there  is no chance that any of the exploited items  

11:05

have gotten out into the wider economy. It's also  fairer that everyone suffers a progress setback  

11:10

rather than a subset of accounts that you may have  incorrectly associated with the exploit. Remember,  

11:16

you're doing the rollback because the situation  is not perfectly well-understood and fixable. 

11:21

You're hopefully using a database system  with write-ahead-logs, which means that  

11:24

you can do a point-in-time rollback to just  before the damage started to occur. If not,  

11:29

you may have to restore one of your backups, which  may be hours or days before the damage occurred,  

11:33

dramatically increasing the amount of  progression loss that regular players  

11:36

suffer as a result of the incident. While  game-wide progression loss is very bad,  

11:40

it is less bad than ruining the integrity of the  economy, which is potentially the entire reason  

11:44

why people are playing in the first place.  If you do roll back significant progression,  

11:48

players will expect a very detailed  explanation of what happened and why. 

11:52

Rollbacks aren't always bad though. A quarter  of a century ago, I was playing Diablo II online  

11:57

and I accidentally sold my favourite bow to the  vendor in town while their inventory was full,  

12:01

so I couldn't buy it back. It was permanently  gone, and I felt terrible. An hour later,  

12:05

there was a realm-wide rollback to fix some  unrelated problem, and I got my bow back. 

12:10

You should be aware that rolling back the database  may have unintended side effects, like allowing  

12:14

players to profit from deterministic unknown  information they had revealed after the point the  

12:18

server is being rolled back to. For example, if  a player identifies an item, then it gets rolled  

12:23

back to unidentified, they know what that item  is now and may gain a small amount of economic  

12:27

advantage from this. There are of course ways to  mitigate this, such as scrambling this type of  

12:31

unknown information, or lazily evaluating it in a  non-deterministic way, but this requires thinking  

12:36

about it in advance when developing those systems. In addition to the types of cheating we just  

12:40

discussed, some players may cheat by botting,  either to farm additional items for themselves,  

12:44

or to sell for real money. You should absolutely  take this seriously and ban all accounts involved  

12:49

also. The subtleties of how to detect and root out  bot networks are outside the scope of this video,  

12:54

but you'll likely end up with a multi-person  department inside your studio doing this fulltime.  

12:59

Let me know in the comments if you'd like me to  talk about this area in more detail in the future. 

13:03

Next up I'd like to talk about keeping players'  economic progression contained in the economy  

13:07

they're playing in, free from influence from  outside economies such as the real world.

13:11

This is going to sound pretty obvious,  but any pay-for-advantage features in your  

13:15

game are going to have an effect on economic  integrity. If players can pay real-life money  

13:19

to alter something that alters the speed  of their progression or item acquisition,  

13:22

then this undermines the level playing field that  you are offering as your core value proposition.  

13:27

While selling pay-for-advantage stuff clearly  makes money in the short term, you are choosing a  

13:31

long-term play by focusing on offering a fair game  that players feel good about their progress in.

13:36

What counts as pay-for-advantage  isn't always clear. For example,  

13:39

it's common these days for games to offer a head  start of a few days or a week for players who  

13:43

purchase a more expensive pre-order package.  Players getting to reach the game's highest  

13:47

content before people who paid less is a huge  violation of the game's economic integrity.

13:52

If you do operate some kind of early access  period for your game, make sure to wipe player  

13:56

progress at the end of it. Also consider that  early access is meant to be about iterating on  

14:00

and smoothing out the game experience for the  real launch. In this bad funding climate, a lot  

14:05

of studios are using early access as a cash grab  to get the funds they need to finish the game's  

14:09

development. If this comes at the expense of the  game's economic integrity, that's a big problem.

14:14

The fantasy of playing an online RPG is to win  based on skill, intelligence and time spent,  

14:19

and if other players can just leverage  their real world situation and just buy in,  

14:23

that's absolutely not fair and will  undermine players' interest in playing  

14:26

the game properly. Also, the tolerance of real  money transactions encourages account theft,  

14:31

botting and exploitation, which damage the game  in other ways. You should explicitly disallow  

14:36

real money trading game-wide. Players should  not be allowed to trade items or progress for  

14:40

things outside of the economy they're in.  But that doesn't just include real money!

14:44

What many developers don't realise initially  is that trading between separate economies  

14:48

in game is actually very similar to real  money trading. If you start a new season,  

14:52

and a player arranges a trade where  they give up some of the best items  

14:56

from the previous season in exchange  for getting ahead in the new season,  

14:59

that's them getting an unfair advantage in this  fresh season based on something they did in the  

15:03

past. Exactly the same as working longer  hours at work last week and spending the  

15:07

money on those same items. Your level playing  field needs to apply on a per-economy basis.

15:13

But if you keep thinking about this,  you realise there are other things that  

15:16

commonly happen in online RPGs that have the same  consequences as real money trading. For example,  

15:21

if a popular streamer gets donations of in-game  items or assistance from their community,  

15:25

that's basically the same thing as  them paying money for that advantage,  

15:28

but in this case they're paying in public  recognition because they're famous. This type  

15:32

of transaction is super hard to police,  of course. Where do you draw the line?  

15:35

Are you going to outright ban players giving  items or favourable trades to their friends?

15:39

Speaking about streamers and free items, this  can flow the other way, too. If a streamer has a  

15:44

surplus of valuable items in game and wants their  stream to become more popular, because it's their  

15:48

career, they might choose to run lotteries or  giveaways, where people watching them can get free  

15:53

items. This also undermines the economic integrity  of the game because the viewers are getting  

15:57

economic progression in exchange for something  they did outside of the game world itself. 

16:02

One thing that people don't consider is that  one of the greatest potential threats to the  

16:06

economic integrity of an online RPG can  be the company's own customer support  

16:10

department. It's often their job to step in  and fix problems that have occurred, and that  

16:14

can result in unfair economic consequences for  other players if there aren't strict policies. 

16:18

I'll tell you a story. A number of years ago,  there was a popular online RPG that had a  

16:23

policy where if a player's account was hacked,  they could contact customer support to request  

16:27

that their items be restored. At face value, this  seemed like a helpful policy, because it wasn't  

16:32

the player's fault that they lost their items, and  restoring them would entirely solve their problem. 

16:36

The problem is, players worked out that they  could pretend to have their account hacked by  

16:39

loading it up with masses of valuable stuff,  having a friend log in and take the stuff,  

16:43

and then write in to support and report all  their items were stolen. Customer support  

16:48

would then helpfully restore all the items,  duplicating them. They didn't have the time  

16:52

or resources to track down where the potential  hackers had hidden all the original items,  

16:55

so the fake hacker friend went unpunished and the  conspirators successfully duplicated their gear. 

17:00

So, how should you go about dealing with  stolen items from a policy point of view? 

17:04

Well you could empower customer support with  heaps of tools that can track items between  

17:08

accounts and try to work out where the stolen  items have been laundered to, but note that  

17:11

these items may have been legitimately traded  between players. If a valuable sword was stolen,  

17:15

and it's currently held by someone who traded for  it fair and square without being able to tell that  

17:19

it was contentious, then it's very unfair on  them if it's ripped away to be given back to  

17:23

the original victim. That would undermine players'  trust in the trade system. It's also disastrous  

17:28

economically if there's a policy for the sword  to just be duplicated to solve the problem. 

17:33

As we saw in the duplication story before, the  issue isn't just that legitimately hacked items  

17:37

are duplicated, but that if restoration  is possible, then players have massive  

17:41

incentives to try to trick customer support  with fake stories. While it's possible that  

17:46

customer support could be given access to logs  of what occurred in-game, it's unlikely those  

17:49

logs contain enough granular detail to be able to  reconstruct exactly what occurred in all cases.  

17:54

It may also require specialist knowledge of  exactly how the game's architecture works,  

17:58

which customer support workers may not have. And  because a policy allowing restorations would cause  

18:03

the number of fake reports to go through the  roof, it'd certainly take far too much time to  

18:07

properly investigate each one with the depth  that would be required to make a fair call. 

18:11

Really the only practical policy for a game  that prioritises its economic integrity is  

18:16

to state that there are no character or item  restorations under any circumstances. This  

18:20

completely eliminates any chance of accidental  duplication through customer support. It does mean  

18:25

that players have to be responsible for securing  their own accounts, of course, with unique  

18:28

login credentials on their game account and any  associated login systems such as email addresses  

18:33

or platforms like Steam. A policy like this shows  utmost respect to the integrity of the economy,  

18:38

though it can of course be unfair if a theft  occurs that was not preventable by the player. And  

18:42

that's why, as the team running the game, you need  to also work to keep accounts secure on your end  

18:46

and prevent unauthorised access by other players. But preventing customer support from restoring  

18:51

characters or items isn't the end of the story.  Their job still involves helping players with  

18:55

other issues, and even those areas can touch on  economic integrity. For example, let's say that  

19:00

a player writes in and wants to change the  name of their character. That seems like a  

19:04

completely normal thing for customer support to  help with, but they still need to be careful. A  

19:08

potential reason for changing your character's  name is to escape a bad reputation in game. If  

19:13

someone scams other players, reneges on promises,  and generally becomes known as untrustworthy,  

19:17

then changing their name to a new identity  provides an economic advantage to them because  

19:21

they are able to resume their profitable bad  behaviour with a fresh slate. It's very important  

19:25

that customer support policy fully considers  the outcome of each possible intervention.

19:30

Your support team is effectively an admin  interface to the economy. Treat it like one.

19:35

I'd like to finish this video with an  example of where I personally screwed  

19:38

up regarding economic integrity and  fairness. Around four years ago,  

19:41

we had just launched a new Path of Exile  expansion, and were having some severe  

19:44

technical problems with the servers that limited  the number of players who could log in per  

19:48

minute. The result was that a couple of hundred  thousand players were stuck in a login queue,  

19:52

where they were forced to wait up to two  hours before they could start playing.

19:56

As you can imagine, an hour or more behind other  players in a race to the new content and its  

20:00

rewards is a massive economic disadvantage, so  this situation really sucked. Given the server  

20:05

instability, the players stuck in the queue  were essentially a random selection of players.

20:10

The problem was, we had spent a  lot of money marketing this launch,  

20:13

and that marketing campaign involved  a number of streamers who had been  

20:15

booked to play for a couple of hours were  sitting there streaming a login queue.  

20:19

I made the decision to bypass the queue  for these streamers, thinking that it's  

20:23

better for everyone if those streamers could be  playing and entertaining people in the meantime.

20:27

I didn't consider the economic advantage  that this would give those streamers,  

20:30

putting them ahead of some other players  because of a real-world privilege,  

20:32

and how that would be unfair. We were rightly  called out on this, and I instantly realised  

20:36

the mistake I had made. This was a valuable  lesson in how even an hour of headstart was  

20:40

an inexcusably large advantage to give people  who had not done an in-game action to earn it.  

20:45

It also shows how even if you structure an  entire live service around economic integrity,  

20:50

you still have to be so careful and fully  think through every decision you make. 

20:55

I hope you enjoyed this deep dive into economic  integrity. Thanks very much for watching.  

20:59

If you enjoyed the video, please like and  subscribe, and I'll see you next time!

Interactive Summary

Loading summary...