Dec 312013
MornAndMeSquareHow to spell my name, and something to remember about news sites.

My post about preserving and archiving old MMORPGs got picked up by a lot of news sites. A lot of people have written to tell me in particular about Ten Ton Hammer’s coverage of it:

Adam Clegg, who formerly worked on Warhammer at EA/Mythic before joining Mark Jacobs at City State Entertainment, has called on EA to release one final build of the game with a developer single player…

I’m not sure who Adam is; my name and mini-bio are at the bottom of every post. Now, I hate to pick on Ten Ton Hammer in particular. They’re good people, and I read their site. I could point out some sort of flaw or inconsistency in the way every gaming site covered the Warhammer shutdown, and that’s the point here. Ten Ton Hammer’s mistake just gets to be the example because you don’t need insider knowledge to spot it. Borrowing from my cousin Jason, who’s gotten himself into his share of news stories over the years:

I’m not the least bit annoyed at Ten Ton Hammer. It’s just how things are; reporters have deadlines and facts don’t always get checked and sometimes the fact-checking checks out wrong. Code ships with bugs, too. Just remember those two questions the next time you read a completely authoritative article about Syria.

Share Button
Dec 192013
warhammer_skull_white_on_black_200On the shutdown of Warhammer Online’s servers, the preservation of games as a medium, and a way forward.

Warhammer Online…is offline. Yesterday, at 6 PM EST on December 18, 2013, the servers shut down and everyone was kicked out of that world forever. Without a server, the clients on everyone’s machines were capable only of sitting forlornly at a failed login screen. It was over. To quote the blog of Josh Drescher, one of the producers on the game:

That’s sad in a lot of ways, not the least of which being that the hard work that hundreds of developers put into it will vanish, without a meaningful way to ever visit it again. It’s one of the cruel realities of MMORPG development. You can’t just load up your old work years later and show it off to your kids.

As I spend more years in game development, I think more about the evolution and legacy of games. We’re still in our infancy as creators. We’re only 41 years away from Pong. That’s not very long in the history of a medium, relatively speaking: You may think of Metropolis as an early movie, but commercial film theaters had been around for 34 years when it debuted. “Talkies” took longer to emerge than 3D acceleration did. Those early films, even the most primitive or puerile, are important in understanding the path to Skyfall and Lincoln today.

Our medium has been an ephemeral one. Each generation of hardware succeeds the last, and the ability to play back old games is lost even more than it was with old celluloid. Nothing disappears as completely as an online game, where a central server is essential to running the game at all. But for at least part of Warhammer Online, it doesn’t have to be that way. You see, I remember something Josh may have forgotten:

C:\> WAR.exe -singleplayer

In every unreleased, internal-only developer build of the Warhammer client, there was the option to run without a server. As the lead client engineer I spent a good amount of time doing that. There were no login or character selection screens. There were no NPCs or other players. There was no gameplay of any kind. It was just you and the entire world spread out before you. You could fly around like Superman, or teleport anywhere at will. You could watch the sun rise and set over Altdorf, and see the smoke rise from fires forever burning. And you could see the thousands upon thousands of hours of work and craftsmanship that went into creating a world that has now been unplugged.

If the right people at EA choose, they can put out one last build of the game client. There was one switch that said, “If this is a public build, force singleplayer mode OFF”. Change that to “ON”, hit Compile, and release the executable. It won’t have to be released with any art files or a massive download. It can run standalone, pulling assets out of the patch files that the last players will still have sitting on their drives. This won’t compete with any current or future game, because it’s not a game anymore. But it’s a place for the die-hard fans to visit by themselves, to reminisce and remember the times they had there with others. It’s something the hundreds of developers who worked on it will still be able to run for their kids someday. It’s a piece of history for Professors of Game Studies in 2113 to better understand what MMORPGs looked like before the neural implants.

It won’t be WAR; that only exists with other players. But it’s a double-clickable museum exhibiting much of what WAR was, so it won’t be forgotten completely. It’s an effort by all of us, as developers, to preserve a living record as our transient medium is created and destroyed. I can’t do this; I left behind the code when I left EA. But there are people inside EA who can easily make this happen.

Please do.

Share Button
Oct 172013
f(u)A very high-level commentary on how we’re using Multi-Version Concurrency Control and Persistent Data Structures to spread MMORPG world updates across the maximum of threads with the minimum of locks.

It’s busy here. We’ve been going all-out on all the core infrastructure of Camelot Unchained. When you’re starting with a blank sheet, you don’t have a lot of other things to build off of. It’s a lot like raising the first side of a barn. There’ve been a lot of pieces coming together at once that all rely on one another, and the dependency graph on the engineering task list is as complex as some of the actual engineering. But we’re coming out of that phase and back into the “make fun shiny things” type of work, with a very solid foundation and architecture to support it all.

There’s a lot more I’ll be writing about how that architecture works, because this is cool stuff. But to hold things for now, here’s a high-level summary pulled directly from our code comments. It brings together a lot of my thinking on functional programming and concurrency into something intensely practical and scalable. I’m excited to see all of these things working:

 *                                                                                       *
 *                            ABOUT THE WORLD UPDATE SYSTEM                              *
 *                                                                                       *
 * Our World and Entity model uses persistent data structures, immutable Entity data,    *
 * and multi-version concurrency control to allow us to update all Entities in the       *
 * world simultaneously across multiple threads. Each thread is capable of running       *
 * various update Actions to add, remove, or replace the Entities in the world. Within   *
 * each thread, update Actions see a single, constant world state while they run, even   *
 * though other threads are running other Entity updates simultaneously.                 *
 *                                                                                       *
 * All changes to the world state happen in WorldUpdateAction functions. These take an   *
 * IUpdateInterface as a parameter. The key thing for WorldUpdateActions is that they    *
 * must be entirely self-contained, other than what they see or change through the       *
 * IUpdateInterface. Most importantly, they must not modify any data other than by       *
 * calling through the interface. They also must not bring in references to things       *
 * like Entities that they've captured through a closure or other external sources.      *
 * (EntityIDs are OK, and are the "right" way to do this. But don't keep an Entity       *
 * itself; always get the current version of the Entity through the IUpdateInterface     *
 * using its EntityID.)                                                                  *
 *                                                                                       *
 * Under the hood, the IUpdateInterface doesn't immediately apply your changes to the    *
 * WorldState that's shared with other threads. Instead, it builds a trace of everything *
 * that you access through the interface during execution of your Action. The various    *
 * threads will take turns holding exclusive write access to a shared, mutable, global   *
 * world state. When it's a thread's turn, it will run through its accumulated traces.   *
 * For each Action, it will check that everything in the world the Action read or wrote  *
 * is in the same state it was in when the Action originally ran. If so, it'll commit    *
 * all the changes and present a new immutable WorldState atomically to the other        *
 * threads. If not (because some other thread modified those Entities first) it will     *
 * throw out the trace and re-run your Action against the new version. That's why it's   *
 * vital that you don't modify or depend on anything outside the IUpdateInterface. If    *
 * you do, then your Action will be hard to roll back and re-run cleanly.                *
 *                                                                                       *
 *     (and then implementation details follow...)                                       *
 *                                                                                       *

Some of that sounds out-there. Some of it sounds scary. Some of it may sound grossly inefficient if you haven’t been exposed to some of the approaches and data structures common in functional programming. But the way it’s actually working in practice is the subject of the next few posts.

Share Button
Sep 122013
WebIconFollowing up the grand hand-waving proposal about web/MMO integration, here’s an actual working demo made with real code from the City State laboratory!

Back in April, when we were in the midst of pitching Camelot Unchained on Kickstarter, I made a video and some posts about “Making a Game Out of the Web”. The main idea was that instead of putting a game in the browser, we’d embed the browser in the game and build our in-game UI out of various small web pages. There were three main goals with that: Continue reading »

Share Button
Apr 122013
Skyrim150A discussion of the ways that Bethesda’s development process differs from the less-successful RPG studios that your author has worked for in the past.

Before I started my current studio, I was very fortunate to get the opportunity to work on (the then-unannounced) Skyrim at Bethesda Game Studios. Leaving there was one of the hardest decisions I’ve made in my career, because even then it was apparent to me that they had something special on their hands. But more importantly, I was learning a lot from them about how they did it. In the past, I’ve ended up in a lot of roles where I was either working alone or winging it as leader of a team. Until working at Bethesda, I’d never felt like I’d had really good examples of how other people ran projects. I probably learned more about project leadership in one year there than in a decade of making it up as I went along.

If I were going to distill what I saw of the Bethesda’s way of building games into a few key points, it would be these four things. Continue reading »

Share Button
Apr 112013
WebIconSome people put games in browsers. By doing it the other way around, we can make better games and empower users at the same time that we increase our connection to them.

Note: This originally appeared in various forms on the Camelot Unchained site and as a Kickstarter Update, but it’s important enough, and has enough applicability beyond that one game, that I wanted to put it up on my personal blog as well. Whether we succeed or fail, this is how I believe games should be doing things: Continue reading »

Share Button
Apr 022013
CU_Logos03_transOMG KICKSTARTER!!!

Somehow, I’ve let this blog languish for over two years. But there’s excitement afoot. Today, Mark and I launched the Kickstarter Campaign for Camelot Unchained. I think everyone was expecting something like this when we founded the studio, and they were surprised when we initially went in the opposite direction with March on Oz. Since this is my personal blog, I want to take a moment to personally reflect. So what changed? Continue reading »

Share Button
Jan 112011
In which I ask for feedback on some very fundamental game mechanics.

I need your help. So far, this little game project has been running on a square grid. This wasn’t an intentional decision; it was just an artifact of the tileset I had, to put it politely, “borrowed“. Now the time has come to move to my own assets, and before doing so I want to make that a more conscious decision. I have my own thoughts, but I don’t want my preconceived ideas to color the the essential question: What feels right?

So, squares or hexes? And while you’re trying it out, diagonal moves or not? Diagonal attacks or not? Please try out some of the possibilities and let me know in the comments what feels better to you. I really do value your feedback.

Grid type

The biggest choice is between the two grid types. They both evoke a certain kind of nostalgia. The square tiles are all about classic gaming, whether on the Apple II in the form of Ultima and Deathlord and others, or on the console in the form of Zelda. The hexes point towards tactical board games and tabletop gaming, which has its own warm memories attached. Hexes have been less-used in the computer gaming world (outside of Civilization V), which might be good or bad — fewer people have an attachment to them, but the uniqueness might be a draw.

A square grid is much more naturally suited to human architecture, at least as it’s practiced in our society. The hexes seem to give me much more organic looking terrain, because there are no grid lines that continue infinitely. But do I necessarily need to reproduce our world’s architecture and/or terrain?

I think hexes have an advantage for projecting tiles up into a pseudo-3D look. On a square grid, a door in a north-south wall can’t be seen unless you tilt the entire map 45 degrees, and I’ve always felt that just made the world feel askew. (Sorry, Fallout and Baldur’s Gate.) With hexes, as long as they’re aligned east-west rather than north-south, I can literally sidestep that problem.

Diagonal movement

For the diagonal movement, I’ve set it up so that you can move diagonally if both of the cells you could potentially walk through as intermediates are empty. If not, you’ll have to take two grid-aligned steps. Visually, that means that to move from the green cell to the red cell, both yellow cells must be empty:

Timing-wise, I’ve made diagonal moves take proportionally longer depending on the length (i.e. after such a move, your character can’t move again for a longer time). That’s 41% longer for the square grid and a whopping 73% longer for the hex grid. It seems a fair choice but it tweakable.

For the square grid, diagonals feel like a win all around. For the hexes it’s less clear-cut. Being able to move along 12 different directions definitely smooths out movement, and it makes up for the grid not supporting a true north/south axis. Try moving due north the length of the screen on the hex grid, with and without diagonals and you’ll see what a difference it makes. On the other hand, on the hexes it’s a very long distance. With diagonal hex moves, does it feel like monsters can “jump” into attack position from too far away?

Diagonal attacks

On the square grid, I also experimented with adding diagonal attacks. Do they feel right to you? Does the potential for being mobbed from eight directions at once (rather than just four) outweigh the benefit in flexibility? I’ve skipped left them off the hex grid because it felt too much like a reach attack (the “diagonals” don’t directly abut), but on the subject of getting mobbed from too many directions at once, are the six possible attack directions on the hex grid a balance issue for you, as opposed to the four or eight in the square grid?


I have my own answers to all of these questions, but they were the same answers I had before I wrote a single line of code, so I don’t completely trust that I’m letting go of my preconceptions and going with what plays right. Design is an iterative process, and nobody can iterate in a vacuum. So what plays right to you? Let me know below!

Share Button
Dec 132010

Skyrim100In which your author goes on the record about what else he’s been working on for the last year.

In the 99% of my time that doesn’t go into this blog, this is what I’ve been working on for the last year. Specifically, I’ve been helping make this happen: Continue reading »

Share Button
Dec 032010
In which a peaceful world, lacking even the concept of “death”, is transformed into a brutish place of strife, suffering, and mortality — and we learn about changing the states of entities from functional code.

And now, some progress. I’ve added combat and killing and hit points and experience and all the other necessaries to the previous code. On reflection, I feel a bit of unease about this. In my last decade of professional work I’ve never had a blank sheet. There was always some body of code to begin with, a prequel to be sequelized or an engine with someone else’s game attached, and as part of that there came an existing concept of combat and conflict. Here was a world with none of those things. Not only could nobody die, but the very idea of “die” didn’t even exist within this tiny pocket universe. I changed that:

let CanAct c now =
    TimeUntilAct c now = TimeDelta.Zero && > 0.0f<hp>

With that simple addition, it became possible for the inhabitants of this world to lose whatever abstracted vitality they might have possessed, and somehow things became fun. With something to lose, there was a challenge to overcome. Try it yourself: Continue reading »

Share Button