Data-driven versus intuition-driven game design

When it comes to game design, we’ll find out one of these days if artificial intelligence can beat a human being at creativity. But today, the matter is still up for debate.

Some game designers believe that intuition is the best tool for being creative while others think that data-driven design rules, particularly in an age of digital games — mobile or online — where you can collect so much data about what players want.

What’s the right balance between science and art when it comes to game design?

We discussed this topic at Casual Connect Europe in Berlin on a panel that I moderated. The panelists included Philipp Karstaedt, general manager for Europe at Gree International; Tammy Levy, director of product for mobile at GameStop’s Kongregate; Adam Telfer, product lead at Wooga; and Christopher Kassulke, CEO and owner of HandyGames.

Here’s an edited transcript of our panel.

Above: Left to right: Philipp Karstaedt, Adam Telfer, Tammy Levy and Chris Kassulke.

Image Credit: Dean Takahashi

GamesBeat: What should game developers, at a minimum, know about data-driven game-making?

Philipp Karstaedt: There’s a place for data driven and a place for design driven. The only thing that can go wrong is when you say “this is the only way.” It depends on what you want you want to do, what stage you’re at, whether you’re a publisher or developer. There’s no right path. Everyone is still figuring this out.

Tammy Levy: It’s fine to think about both sides, data and design. If you’re going at it from the data side, get informed developers who can make design decisions. If you’re coming from the other side, learn about data. Understand the basics so you can make informed decisions. It depends on context.

Christopher Kassulke: As game makers, we’re all more coming from design in the beginning. But then, data takes over. We’ve seen that in the games and in the stores. A lot of games are becoming data driven. There’s some sense that what we’re missing is that the data guys aren’t really talking to the design guys, and the design guys don’t understand the data guys. They speak totally different languages. Most of the data guys aren’t gamers. Without any game experience, data is worthless.

Adam Telfer: My role is part game design and part data design. I’m at the meeting between the two. I see data as a tool, just like any other tool in game design’s toolbox. I go back to when I started in game design, when I worked in play testing. Data, all it is is effectively play testing your entire user base, using it as a high-level view of what’s going on.

If you’re just getting started, there are free tools available out there for collecting data, looking at things like retention and engagement numbers. That will help you take that first step toward making better decisions in game design.

GamesBeat: I sense a lot of emotion around the divide between data and design. As far back as I can remember, Zynga has been a lightning rod for this. They would talk about doing A/B tests on what color a button should be. That was a bit more of a controversy about how to design monetization into games that might be able to proceed without monetization, though. It’s not the whole topic.

Above: FarmVille 2: Country Escape is part of Zynga’s new mobile effort.

Image Credit: Zynga

Kassulke: Perhaps it’s not, but at the end of the day, ever since we’ve had free-to-play we’ve had data-driven games. Before, we’d just say, “This is a great game,” and you’d buy it at the store for $60. We had a design in our head, and we believed that was a good way of making a game. Maybe it’d be tested in small groups, but that’s all. Keep in mind that it’s always about what happens in testing.

Karstaedt: Zynga was really just the first company that used data for monetization. In the end, data is about way more than that. On the other hand, it’s also true that data has almost become too powerful in some cases. People have said, “This is my data hammer,” and suddenly, they see data nails everywhere. The art of game-making goes far beyond that. Understanding game design and how people interact with it, that’s data as well. It’s about retention. It’s not always monetization. People are unhappy about the monetization side for various reasons. But it’s a very powerful tool. Without it, it wouldn’t be possible to emulate some of the successes we’ve seen. It’s self-defeating to ignore data just because you don’t like monetization.

Levy: It’s funny that you mention Zynga because I was there at a point when it was overly optimized at times. It’s important to be aware of how to use data as a tool but not let it dictate all the decisions you’re making. Understand that it has its place and time. Not everything can be optimized to the last percentage point, the last extra cent out of each player.

It’s also important to talk about scale when you talk about data optimization. A lot of times, you don’t have enough players to optimize certain features based on data. Then, as a designer, you have to ask yourself what your instincts tell you as far as how to optimize the game and make it better at that point. If you only have 100 or maybe even 10 players getting to that point in the game, there’s no significance in your data. It’s important to understand at what levels you can use data and at what levels it can hurt you.

GamesBeat: Do you run into a lot of game designers who will not pay attention to data? I saw Warren Spector, who’s a pretty famous game developer in the U.S., put up a Facebook post after the U.S. presidential election. All the analysts were wrong about who would win, and so, he said, “If anybody comes to me and says I have to do a data-driven game design, I’m just going to say ‘Trump.’” Have you ever had to convince someone to look at data that way?

Above: Warren Spector will create a new System Shock 3 for OtherSide Entertainment.

Image Credit: Dean Takahashi

Kassulke: It’s not only Trump. It’s also Brexit, to give another example, here in Europe. Everyone said that Brexit couldn’t happen, and it did. Life is not that simple. You can’t quantify everything. Games are entertainment, and everyone is entertained in different ways. I always ask people, “What was your favorite Indiana Jones movie?” It’s always the first one. Nobody likes the last one because the marketing and analytics took over. It was just boring. Games that are only data driven will be boring.

Consumers are not stupid. You need to have some soul in your game. Data is good, but you need to make the first steps into a game as a game designer. “That’s my vision. That’s the way it will be.” Then, you can check that against data and perhaps change it step by step. But that’s a different topic.

Levy: As a publisher, we see both ends. We see developers coming to us where they’re fully design driven, and when we show them data they say, “No, I’m not interested in listening to that.” And we see the other end of the spectrum, games without a soul. “It has good retention because it has this feature that these other games have as well.” It’s just a compilation of pieces of other games that are supposed to work well, jammed together into one Frankengame.

For developers that don’t want to listen to data, one thing that I’ve found is very powerful is benchmarking, talking through the impact of making changes based on things we see out in the market. We have a lot of games out there to use as examples. “You think you’re going to be the outlier, but of 10 games out there exactly like this, nine have behaved in a particular way in this area.” Talking through the impact is important; “If you make this change, this is what could potentially happen in terms of these metrics.”

Telfer: I don’t want to talk about the Trump stuff too much, but when you think about that situation, it wasn’t just the data that said Hillary was going to win. It was also a lot of intuition from a lot of experts. I don’t think you can just assume that the data created that issue. Also, there was a lot of soft data from polls that had potential bias, which points to a problem in the way that data is gathered. In the future, we’ll hopefully have better models to gather and use polling data.

Going back to Warren Spector, he’s one of the few game designers right now who have that luxury of not being affected by data at all. Most of us don’t have that luxury, especially when you’re talking about free-to-play games, high-volume games, games that are trying to find a broad appeal. In those cases, data is very important.

When I work with designers who are resistant toward using data, it usually comes down to the idea that as soon as we start using data, that means we’re going to entirely focus on quick wins and what you said about Frankenfeatures. That’s not really true. It’s two separate things. You can look at data and use it to follow an easy path, or you can use it to try and build a more engaging part of your game.

Let’s say you want to increase retention and engagement. One way is to add a daily reward system, something that gives players a little reward every time they come in. That’s the usual way. Another way to approach it is to look at our game design loops and try to make them more engaging, so it’s inherently fun to come back. You can start both of those paths from data that says players aren’t coming back, and you need to build higher engagement.

Levy: Where we see that work best is when we point out a scenario or an improvement, and then let the designers come up with a creative way of approaching that problem, instead of dictating a way of doing it to them. It’s a hard thing to do. Say you’re testing a website, and you have trouble finding a button. The obvious solution is to make the button bigger. But it might be the color or the way the layout leads you to it. You need to let the designer fix the real problem. A lot of data people make the mistake of simply saying, “We need to increase retention, so you should add a log-in bonus.” But that’s not the problem. You need to be able to say, “I’m seeing this gap. Can you come up with a solution to fix that?”

Above: Wooga is pronounced “Vooga.” It’s like saying the old “Sega!” scream.

Image Credit: Dean Takahashi

GamesBeat: When Wooga started, I imagine your role didn’t exist. Along the way, when did data people become part of design teams?

Telfer: I wasn’t there from the founding, so I don’t exactly know. But Wooga was always a free-to-play company. We started on Facebook. If I were to speculate, I think data was embedded from the beginning. Over my own career, starting in indie games — as soon as you move into free-to-play, you see the value of data. My job as a game lead — part game design, part producer, part product manager — became part data as well. As I gained more knowledge of how to use data, it became a bigger part of my job.

Karstaedt: Going back to what I was saying about what’s hard data and what’s soft data, if you do exit polls, that gives you only so much customer insight. What people tell you they want is not necessarily what they really want, which is why it’s important to back that up with hard data. It doesn’t matter if everyone tells you they love a certain feature, or they’d play your game if it had this feature, when they don’t follow through on that. If the data shows that they’re not acting on this thing they say they love, clearly they don’t love it. That’s the value of hard data.

Sometimes, you have to abandon your own idea of what you’d like to play. A lot of indies, I think, have that problem standing in their way on the path to mass-market success. If you start working on games with a publisher or with a license, at some point, you may have to abandon some of what you personally think is good and go see what the market wants.

Even if you don’t want to design your game based on data or market research, at least … in case you want another company to come in and operate your game or have anything to do with your game, at least put in the data collection triggers. Make sure the game backend collects data, even if you don’t think you want to use it. Give others the opportunity to use it if you want anyone else to be involved with your game. Change things in a way you think is best, but see what your players actually want. With data, you can determine that.

GamesBeat: Chris, you have all kinds of people at your company. You’re the CEO. You have finance people, marketing people, game designers, and now data people as well. You may be the boss of these people, but you have to get them to work together. How do you make sure data people mesh well with designers?

Kassulke: Communication. Drinking beer [laughs]. It’s about understanding where both parties come from. We have very small teams, compared to Zynga or Gree or Wooga and many others. The team decides if they want to use the data. We always say, “Are we asking the right questions?” Data is good, but if you do games like we do — globally, on many different platforms — they have to work on PC, on console, on smartphones, on tablets. Perhaps a title isn’t the best performer in the mobile space, so maybe we should concentrate on another platform. Or, we might concentrate on a particular country. Why is a specific game not performing in this specific region? You can dive deeper and deeper into the analytics, but can you find the right answer?

With us, it’s easy enough. We just try it out and see. That’s what we’ve found is always the best option. If you look at which games are performing in the category — take Zynga for an example. Their latest titles were not huge successes. The problem is that they’re having to invest so much money and time into a new title to get it past a certain point where the analytics will work. When was the last time Wooga released a new game? Release more titles. Try new things.

Data, in my opinion, is currently holding back the game industry. I see a lot of guys who don’t care about data. They’re just trying new, cool stuff. The freshest titles I see are coming from indies who don’t care very much about data. They’ll try out something without data, and then, they’ll analyze from there. That’s a valuable experience for the game designer and the studio.

Karstaedt: In some ways, it’s a bit like building cars. There are cars that look amazing, that are unique, that are hand built, and there are maybe 10 of them in the world. They’re made without data. That’s not what their customer wants. They couldn’t care less. That’s fine. And then, there are the Toyota Camrys and VW Golfs that are maybe a bit more boring, but they sell to a mass market.

It always comes back to this question of video games as an art form versus video games as a business. Mobile free-to-play is more on the business side, but it earns a lot of money for a lot of people. I can understand that they want to make sure what they put out makes money. But in the end, the only proof that data is everything is that you can’t plan for success. You can do all the market research, but most companies that are big now started with one big game that was very successful. Almost no companies have followed the path of something like Supercell and managed to continuously bring out game after game that finds success.

At the core of it, game design is an art — molding an experience for the user, telling a story. Even if it’s a simple story, a story that’s told in five-minute play sessions. If you manage to do that well, you can start looking into data. But like Tammy said before, data can lead you to a dead end. You can overanalyze and over-optimize. With a different approach, you can find a step-function change, and it can perform much better. If you just follow optimization blindly, you get stuck over-optimizing an inferior solution. Only design can go against that.

Telfer: Getting back to the question about data and design people in a co-working space, typically, I don’t see fights breaking out between data and design. Usually, I see more fights between marketing and design. When you set up a team where there are two silos operating completely independently, that’s where you get the finger-pointing.

That’s why it’s important to get data people on the team. Not every team can afford to have a dedicated data person, but design people can take a percentage of their time to become more data driven themselves. That builds their appreciation and makes for a much healthier environment. I’ve worked with great game designers that are way better than me at analyzing data, and it’s made them better at design. They can take learnings from data and apply those numbers.

The interesting thing to me about data driven versus design driven isn’t necessarily around soft launch. When you soft launch a game, you throw up the KPIs and see how you’re doing and say, “OK, are we going to push this or not?” Most people should be doing that. The real question is at the beginning of the game — when the team’s getting together and saying something like, “We’re going to create the greatest mobile MOBA ever because we feel like that’s a game the market’s ready for.” Versus making a decision based on market research and driving the design of a game with market research. That’s a pretty hot debate there — how much of a core game, from the beginning, should be driven by the design team saying, “We think this will do great,” versus analysis and data making that decision.

Above: Smash Land from Supercell. This game died in soft launch.

Image Credit: Supercell

GamesBeat: On soft launch, this is a fairly new phenomenon, but nowadays, people will put a game into soft launch for quite a long time. Games are going into soft launch for as much as 18 months or two years. Supercell puts games into soft launch and then kills them. What are we supposed to learn from soft launch? Why do some games take such a long time?

Telfer: For Dawn of Titans, they were in soft launch for 21 months. They were looking for a two-point increase in their 30-day retention, something like that? That’s how important that was. If you increase your retention by two points, you’re looking at a pretty big difference in your overall LTE, how long that game lasts. To me, you should soft launch as long as you want. When you launch, that’s the one chance you have at feature traffic driving a critical mass into your game. Why not take the [time] and soft launch with data so you can properly evaluate your design?

GamesBeat: If it doesn’t make the charts, though, is that all just a waste of time? Dawn of Titans still tanked.

Telfer: It’s hard to say because I don’t know how much money is made in soft launch. I assume that if they hadn’t done that, it would have tanked faster. Is it worth 21 months of production? That’s pretty hard to say. But I know from looking at soft launches that it’s a very tough decision to stop a game in soft launch. It’s a game that designers have put their heart and soul into for a number of years. Are you going to tell those people to kill their game because they’re off by two percentage points on one number?

Karstaedt: That’s the epitome of games as a business. But in the end, it’s true. If you wait for almost two years, the market moves a lot. A lot of games that have been successful have excellent retention and monetization, but they also hit the taste of the market at a particular moment. Coming back to this question of design driven versus customer-insight market driven, there needs to be a balance. You can have the best game for two years ago, but at this point, people don’t want to play it anymore.

You can’t get it right by making just one decision. It’s one decision after another, and at some point in there, you’re going to have to get lucky. Data can help maximize your chances. But even Michael Jordan didn’t hit every one of his shots.

Levy: It’s also important to set out your goals for soft launch and understand what you can and cannot do with it. We’re talking about a company that had the luxury of keeping a game in soft launch for almost two years to maximize D30. Let’s say their D30 was three percent and aiming for five. How many players are going to come in over and over again to validate that D30? It’s a huge amount, thousands. That costs money. Indie developers don’t have the marketing dollars to do this over and over again.

You have to get smarter about what you’re trying to validate during soft launch. Sometimes, a small team won’t even have enough resources to validate an ARPDAU. You’re talking about one to two percent of players. You also have to get smart about the data you decide to analyze during soft launch. I hear over and over again from developers who want to do two A/B tests or five A/B tests running at the same time. I’m supposed to be the data person, and I’m fighting them on this. You need good test design, and if you want to validate all the hypotheses you have, you’re going to need a million-dollar marketing budget that you don’t have. So, get smart about how to approach this. What’s the key question you want to answer during soft launch? Go after that.

GamesBeat: Chris, are you doing any soft launches?

Kassulke: No, we’re no longer doing soft launches. The question is, is it worth two years of spending and investing time and changing the game, when something can happen out there and suddenly your concept is no longer interesting? Or your competitors catch up and pass you because you took two extra years? I know too many companies that are good at checking out what others are doing in soft launch.

Above: Ninja Spinki Challenges is the new mobile game from Flappy Bird creator Dong Nguyen.

Image Credit: Obokaiden

GamesBeat: That happened to the Flappy Bird guy. They were getting ready to launch Ninja Spinki in December, and somehow, everybody found out, and clones started flooding the market.

Kassulke: Exactly. We absolutely check out what our competitors are doing. That’s very interesting for analytics and data because they do the homework. It’s easier for us nowadays to simply release games. A game that works in Portugal isn’t automatically going to work in Brazil. A game that tests well in Austria might not work in Germany. Depending on the way you go about your soft launch, you may not get the right data.

That’s why we’ve decided to just go out and release our titles. Maybe we won’t get a feature on Apple or Google the first day, but we can bring our consumers automatically to our game, which can mean one million downloads right off. We save a lot of money, because CPIs are crazy nowadays. If you want to do test markets and all the data, you have to invest a lot. As you say, do you have a marketing budget of $1 million? That’s nothing nowadays. With that much, you’re not even in the big league. You’re playing down there with the indies.

GamesBeat: Are you significantly changing games after launch?

Kassulke: We do that later on because then, we can get real data. Everything else before that is fake in a way. The title isn’t ready. You’re changing things. You do a lot of testing while competitors are coming out. Two whole years of testing a game just to get two points higher, so Apple likes you more, or you get better investment of marketing money? It’s still a bet.

GamesBeat: I get the feeling Tammy disagrees with you.

Levy: I still feel like there’s value in test markets. But it all depends on your goals. We have games that we test market for three weeks and games that we test market for four months. We try to be conscious about this and not do an excessively long test market.

You can do simple things like testing the stability of your game. You can release in one country, get some users, make sure the game isn’t breaking. When you release it globally, at least that won’t disappoint those first players early on. You can go to some countries in Europe, spend a few thousand dollars, and still get some big wins in terms of validation.

Again, this is focusing on things you can actually change. I’m not going to focus on my D30 retention, but maybe I can focus on my tutorial and make sure that people get to see the whole game and learn the game. I can make sure that’s working, in broad strokes, during a short test market.

Above: Mark Skaggs, speaking at GamesBeat 2015.

Image Credit: Michael O'Donnell/VentureBeat

GamesBeat: A lot of this makes me think of Mark Skaggs, the co-creator of FarmVille and a lot of other big titles at Zynga. I recall that he developed a philosophy of game designers swallowing their pride, understanding that they have to rely not only on their intuition, that they have to realize what they want isn’t necessarily what the market wants. In Zynga’s case, early on, you had a lot of younger men designing games for an audience that turned out to largely consist of older women. They had no instinctive idea of what that audience really wanted.

Levy: My favorite quote from Mark Skaggs goes, “Measure five times, cut once.” Whenever there was an argument, that was something he kept saying over and over again. Create a plan but be informed about what you’re going to execute on before you do it.

GamesBeat: I also remember Andrew Sheppard saying that they’d changed from the days of launching minimum viable products, toward launching more finished products. This is a bit like what you’re saying. You’re not just putting something out there, so users can finish your design for you. You complete what you’re envisioning and then put it out.

Kassulke: We need information for every game. What we do is essentially test groups. They’re sitting at HandyGames playing the game with the game developers, and then, we can ask the right questions. We know what they see in the game and what they don’t. Are they checking out advertisements? There’s technology out there to really test-case your games. We can even tell what they don’t use and why they don’t use it. Then, you can do another test group and find out if the yellow button works better than the blue button or the red button or whatever. You can already do that.

We’ve checked out places like the Philippines as a test market. Money works much differently than in Europe. The user behaviors are different. When I see a game that’s half-ready and the developer says that they’re kicking it out of their portfolio because it’s not going through the funnel, well, of course not. That’s normal. Why would people play the game? I wouldn’t even put something like that on the market as a test case.

I often joke that I’d love to get all the titles that make it into the last third of the funnel from Supercell, from Zynga, from you guys. Those games could make a lot of money if you just did things a little differently, if you didn’t spend millions on a single title. Maybe you don’t need that. Maybe you just need to advertise a little differently. I’m still waiting on getting the discards from that end of the funnel, but I’d do it.

Karstaedt: I often hear people talking about games as if they’re any other kind of product, which I think misses the mark completely. A consumer product makes people’s lives easier. A game is not a tool. It’s a story we tell. If you put out something where you’re not convinced about it yourself, people can sense that. They don’t want to play something that’s designed by committee. They don’t want to be the person who has to help you improve this. Certainly, the Philippines and all these other soft-launch territories are getting more expensive. Australia is one of the territories where you would say the players are fairly similar, or Canada, but everyone’s doing that now.

Soft launching can help your game better, but it’s more about launching it and continuing to develop it, adding features along the way, which you should be doing with a mobile game anyway. Adding content to keep the game fresh and keep the players engaged. You can do that based on indications of how people interact with the game and what they like most about it, but that’s extra. That’s how you keep a game alive. It’s not how you bring a game back from the dead. You can’t fix a bad game with data.

Above: KPIs.

Question: Fundamental question — how do you measure fun? Any design process that requires iteration has a feedback mechanism. But if you use time in game as a proxy, that can just mean I’m incredibly frustrated. Maybe I’ve played a level nine or 10 times, and I spend a little money to get past it. In the short term, you’ve made money, but in the long term, you’ve upset me because I’m not having fun. How do you tell if I’m having fun?

Telfer: One simple way I like to see how a user base is having fun is just chatting with players in the live chat. Looking at high-level KPIs, you can’t necessarily say if a game is going to be fun over the long term. You can look at symptoms of what fun is. You can look at engagement in low-level gameplay — battles, rounds played — or in something like a match-three game you can look at how many moves it takes to finish a level, how close players get to completing the puzzle each time. That can give a sense of how fun that level is. We can look at the life span of a user, how long they stay in a game. But until we get into things like biometrics, it’s very difficult to measure.

GamesBeat: The fine line between fun and frustration is important. Are there particular KPIs that game designers should be aware of?

Telfer: One big caveat that I always find with metrics is segmenting your user base by your user-acquisition strategies. The source of a user is very important. When you’re looking at averages across all of your data, your user base includes a whole bunch of different types of players. But just segmenting by user acquisition can give you a sense of bright spots in the market, which source is giving you your best users. That should be the ultimate goal of your product marketing — not just buying enough users to keep the game going, but getting users of higher quality. We’ve seen 20 percentage point differences in things like retention just because we’re using a different marketing source.

Levy: You should also segment by concrete platform. Android and iOS will probably be different. Organics versus someone you paid six dollars for will probably be different. Sometimes, you’ll have to group. You can’t get every single country because some countries will have very few users to let you get significant data, but making sure that you’re considering these differences is important. You might be optimizing for an average that doesn’t speak to the true appeal of the game.

Kassulke: When you’re talking to Apple or Google, for example, just one platform, they’ll want to see their retention rates. If you want to have any promotion, check that out. If it’s not correct, they’ll probably not feature you, and that’s very cheap marketing. As you say, it’s important to check out different target groups. Concentrate on tier-one markets and then take care of others.

Karstaedt: Talking about platforms and what they want to see and combining that with issues around soft launch — one really important thing is finding stability. When you load up the game and interact with it, does it work for everyone? Even if on average one percent of users is affected, on some platforms that might be a large group of users. The propensity of people to leave you a one-star review if a game crashes is so high, and then, Apple will come at you and say, “Well, you have 3.5 stars. Your game isn’t a good experience for a lot of people. We’re not going to feature you.” Not dealing with that is simply shooting yourself in the foot. Monetization data is also important, but fundamentally, the most important thing is getting your game stable. Make sure it loads quick. Make sure you eliminate those simple frustrations. Then, you can move on to other things.

I totally agree about segmenting — and segmenting very specifically. When you’re doing that, think about what you want to do with the game. You can’t design something that appeals to everyone. If you want to design something that appeals to people who like short sessions and then you see that people are playing your game for a long time, that might not be a good thing. The data can help make those decisions. First, understand what you want to give the market and then optimize toward that.

Above: The Walking Dead.

Image Credit: Telltale

Question: You can soft launch for a short time, but that will only give you data on the initial phase of your game. How do you stress test something like economy design? If that breaks down in the long term, how can you test for that?

Telfer: It’s a very tough problem to solve. Do effective play testing internally. Make sure you have ways to model your economy and project something like a year or two years down the line. Play your game as if it was two years later. Implement a way to load it up on a device that plays the game as if it’s been running into the long term.

Levy: We have models that stress test different situations. You’ll know where players are at seven days. From there, you can develop models that look at what players are spending and how fast they’ll get through levels. If players aren’t spending quite that fast, will they get through levels at a different rate? You need to get into your economy design and test it, see how it pans out when you change key variables. Does it end up where you thought it would?

Kassulke: It’s one of the hardest things to understand, that kind of long-term social behavior among players. From the beginning, it’s important to make sure that there’s a good way for new players to reach out to old players. You can run simulations of player behavior, though it’s hard to do — run a game very quickly with 10,000 simulated agents that interact and form guilds. If you see a point where they stop interacting, that might be a problem. It might simply be easier to trust to your instinct — “I’m going to do everything I can sensibly and hope for the best.” If a game is active for two years, after all, that’s already a win. I’d take that to the bank.

Levy: Once you’re live, you can add more things to your game. Your game is going to be very different after two years of life in ways that you can’t necessarily predict. This is really a place where you need to trust your design intuition to make the right choices. As he was saying, if you’ve kept players for two years, you already have a fantastic game.

Above: Call of Duty: Infinite Warfare publisher Activision gave fans a chance to play the game early.

Image Credit: Activision

Karstaedt: I would simply say you can’t do it. That’s my personal opinion. We have a lot of titles that are out in the market. The oldest one that is still regularly played is 10 years old. People are still playing it. But that has to do with bringing new content in. We let the consumer choose some things. But can we predict that? No.

We’ve released a lot of games. All of our titles are normally built for three years. That’s a minimum. Then we go on and ask if it’s worth it to invest another year or two into the title — part two, part three, whatever. But really, planning for that is impossible. If there’s someone who can do that, good luck to them.

GamesBeat: I’ll leave you guys with one thought. If you look at The Walking Dead game from Telltale, they force you to make decisions — who you’re going to save, who you let die. Then, at the end of the game, you see percentage choices that users make. Maybe 55 percent let a certain character die, and 45 percent saved their life. That’s fun for me.

I think of Call of Duty as well, where they use heat maps of where people get shot in multiplayer to figure out the design of a particular level. Game designers use that data, but then, they also release it after a match, so you can go and look at where you died and become a better player as a result. There are ways, I would say, that data can be fun.

Disclosure: The organizers of Casual Connect paid my way to Berlin. Our coverage remains objective.