Saturday 23 December 2017

New song: Grom

Here's my newest song: "Grom"! It's a loud cello trio! I deliberately make the strings rattle a bit to get a more aggressive sound, which was a fun thing to fool around with.



This is actually an older composition: I wrote this one in 2012 to play together with the other cellists of the Kunstorkest, which I was a member of at the time. Since at the Kunstorkest we were supposed to play subtly and rarely truly loudly, I wanted to write something that's super loud and unsubtle, just for the fun of doing something different. At the time I never actually recorded it though, so I did that now and here it is! :)

Sheet music for this one can be found at music.joostvandongen.com. I've also made arrangements for violin and viola, which can also be found there.

PS. Until I took that photo I never realised how evil a cello's bridge can look...

Monday 4 December 2017

The Daysailer and the Night Cave

I've finished a new song! It's a longer instrumental track with an extensive cello solo towards the end. :)



For this track I had a lot of fun playing around with delay (echo). The rhodes (the piano-like instrument that's heard throughout) has a rhythmic echo at every 8th note. This means that the rhodes is basically constantly playing harmonies with the notes it played just before, which is a pretty interesting effect. At 2:58 I even switch to playing triplets on the rhodes while the delay keeps going in normal 8th notes. So the echo and the original notes have a different rhythm there, which gives an even more interesting effect.

In case anyone wants to give playing it a try, here's the sheet music for the cello part, plus a recording of the song without the cello, to play along to:

This is the longest track I've finished so far and I'm really happy with the result. I hope you enjoy it! ^_^

Tuesday 21 November 2017

Lament

I've recorded a new piece! It's a short sad cello solo called 'Lament'.

The composition itself isn't really new: I wrote it way back in 2001. However, I never managed to record it to my own satisfaction and thus never published it online. After all these years I've now finally managed to make a recording that I'm happy with. It's not perfect of course, but I finally dare publish it and share the sheet music.



In case anyone wants to give it a try, here's the sheet music:

I found another recording I made of this one from 2010. I wasn't happy with it back then and indeed, listening to it now makes me realise that I did get quite a bit better at recording my cello since then. Part of that is in technique, like having a better microphone, using a clicktrack and stitching together recordings to combine the best parts from several recordings. But I can also hear that I have more control of my cello now: especially my vibrato get better and there's less unwanted noises in my recordings now. I've been playing cello since I was a kid back in 1992 and it's really nice to hear that I'm still getting better at it by practising a lot in my spare time.

Sunday 12 November 2017

The Master Waits

I've composed and recorded a new song! :) It's called The Master Waits and it's a duet for either two cellos, or viola and cello. Here's a recording:



I'd be honoured if anyone else were to play this (not that I expect many cellists read this blog ;) ), so here's the sheet music:

This composition came about when I was improvising a bit and discovered that you get a pretty cool sound if you pluck the open C string at every beat and then add various chords on top of that. I had so much fun with that that I decided to write a composition around this idea. :)

Initially I thought this one was pretty easy to play, but it turned out to be really hard to get the pitch exactly right for all of those chords. Keep in mind that unlike a guitar, a cello doesn't have frets, so if you're finger is only 1mm off, it already sounds bad. There's quite a lot of dissonance here (on 0:19 the plucking part even plays G and g# together) which is always difficult to play. On top of that comes that when plucking you can't listen and adjust the pitch as you go: it needs to be right immediately. Luckily I made different mistakes each time I recorded this piece, so with 4 recordings for each part I got every note right at least once. A somewhat sad score, but after stitching those recordings together it sounds great so it really doesn't matter.

I had a lot of fun composing and playing this one, so I hope others will have fun listening to it. Enjoy! :)

Sunday 5 November 2017

The Light Deep Down

I've made a new song! It's an instrumental track that features a lot of cello, in combination with vocals and digital instruments.



There's more cello here than you might realise: the eerie scratching sound at the beginning and ending is also cello! :) Normally as a cellist you wouldn't want to make a sound like that, but here it adds to the atmosphere nicely. This sound is achieved by slowly playing on the open G or C string, too close to the bridge while applying too little pressure.

The cello solo at 1:57 has distortion applied to it, making it sound a bit closer to an electric guitar than a cello normally would.

I've made sheet music for the cello part, and a recording of the song without the cello so that it can be played along to. The result is quite different from any other cello music I know and I find it quite fun to play along to. Here it is, in case anyone wants to give it a try:

The vocals towards the end were sung by Joyce Scheeren and the faster bass in the second half was played by Thomas van Dijk.

I've been writing music for years but have always been struggling with actually finishing anything, so I'm really glad that I've managed to finish and share two new songs in just two weeks. I've got a bunch more almost finished, so I hope I can keep up the pace.

Tuesday 24 October 2017

Never be ashamed of your process if the result is good

Over the years I've composed quite a few songs that include my cello, but I never dared actually record and release them. I did publish some recordings before, but I've always marked them as "unfinished" or "quick sketch" or some other excuse, planning to practice more and make a better recording later on. Today I finally have one that I consider "finished", but the process ended up quite different from what I expected back then. It's called Growl, and it's a loud cello duet. Have a listen!



I'd love for others to also play this composition, so I've made sheet music for it as well. Let me know if you end up playing this composition: I'd be honoured and would like to hear how it went!
I've also made a start with making a page that collects all my music and sheet music. It currently doesn't contain much yet, but I intend to record and release all my compositions. That means I have a lot of work ahead of me, as I need to record another duet, two solo cello pieces, two trios and a bunch of songs that combine cello with electronic instruments. I'll add songs to music.joostvandongen.com as soon as they're finished.

The reason I never finished recordings before is that playing cello well enough for a recording is incredibly difficult. This is difficult on any instrument and the particular challenge of cello is to play perfectly in tune. There are no frets like on a guitar, so place your fingers a millimeter too high or low and you're out of tune. Since most compositions require jumping around the fingerboard it's a huge challenge to always jump exactly the right amount.

I always figured I would solve this by practising more, and indeed I did practise quite a lot (for a hobby player, that is). However, by now I've played cello for over 25 years and I'm still not there. I'm not a professional cellist who can practice several hours every day for years, so I've finally concluded that I'm just not going to get good enough to live up to my own requirements. But I want to share my compositions! So I've reached a different conclusion: while I'm going to keep practising to get better, I'm henceforth also going to 'cheat' wherever needed to make recordings sound better.

I use two main cheats. One is that I play with a clicktrack: while recording I hear the ticks of a metronome on my headphones so that I play at a constant speed.

The other and bigger cheat is that I record the song a bunch of times. I then mix and match parts from each recording to get a complete recording that sounds acceptable. I tend to make different mistakes every time I play a song, so if I record it enough times I'm bound to get every note right at least once. Growl is a duet, so two cello parts, and each of the two cellos in Growl is a combination of six different recordings. Heck, if needed I can even use autotune to correct an individual note (this is used exactly once in Growl).

Mixing recordings together works a lot better than I had expected beforehand: if I mask the seams carefully then afterwards I can't even tell where they are. You might think you can hear them, but the lifting or shifting of the bow sounds so similar to a seam in the mix that you need to be a real specialist to be able to tell the difference.

Working like this makes me a bit cynical, since I wanted to be a good enough cellist to just record everything once and it's perfect, and clearly I'm not. However, if there's one thing game development has taught me, it's that it doesn't matter how you make something. The only thing that matters is the end result. If it's fun to play, then it doesn't matter whether the AI is truly intelligent or just pretending to be so. It's okay if everything is smoke and mirrors as long as the player doesn't notice it. The same goes for any creative endeavour: if it sounds/looks/feels good then it is good. There's no higher goal other then the end result and your creation isn't any worse or better depending on how you got there.

Sunday 15 October 2017

Working with generic room-based matchmaking

When creating matchmaking for a game you can either build it all yourself, or use a generic system provided by the platform you're releasing on. Steam, Microsoft, Sony and Nintendo all offer similar room-based systems. The basic idea is that clients can create, search for and join game rooms, and the actual decision which room to join is entirely made by the client. There's very little actual logic happening in those matchmaking servers. Today I would like to discuss how they work and what benefits and limitations they come with.

As far as I know most smallish games build their matchmaking using these generic room-based systems. The big triple-A developers seem to usually build everything themselves from scratch, but that's too much work and too much server maintenance to be feasible for most smaller developers.

Our own game Awesomenauts initially launched using these basic room-based systems. We didn't develop our own matchmaking system Galactron until years later when we concluded that the requirements for a genre as demanding as a competitive MOBA don't sit well with the limitations of such generic room-based systems.



The specifics of the systems that Steam, Microsoft, Sony and Nintendo offer aren't public and thus can't be discussed here, so I'm not going into any detail on how they each differ from the generic ideas described here. However, they share enough that I feel this post is very relevant if you ever make matchmaking for any of these platforms, or another platform that's built on similar ideas.

The basic concept is that the platform holder runs servers that handle rooms. A room usually represents a single match with a bunch of players in it. For example, in Awesomenauts a room would have the 1 to 6 players in it that are playing together in a match. If a player wants to join a match then the game will ask the server for a list of all rooms that have a spot left for at least one more player. The game then chooses which room is the best fit and requests the server to join it. Upon joining the game receives a list of the other players in that match and can make a connection with them and start playing.

It's important to realise that the room doesn't actually run the game. It's not a gameplay server. Usually one of the players is the gameplay server. All that the room does is keep a list of who's in the match and allow players to search for matches to join. The room is only intended for finding and maintaining lists of matches and can do very little else.

Since the room doesn't handle the actual gameplay it can often be closed once the match has started and the players are connected. Only if you support drop-in do you really need to keep a room up once the match has already started. Whether closing the room once the match has started is actually how it works depends on the platform though.

The server that handles the room can do some basic logic, but generally not much. One of the most important things it does is limit how many people can be in the room. You usually set a limit and if two players try to join at the same time while there's only space for one, then the server will make sure only one actually succeeds. This solves one of the more hairy race conditions around matchmaking.

Rooms aren't just usable for matches: you can also use them for some other things. A simple example is a chatroom. This is simply a room that players use to exchange chat messages, for example if you have global chat somewhere in your game's menus. Whether rooms scale to large numbers of players for a true global chat varies per platform. This case also shows another use of rooms: often it's possible to send messages to other players in the same room through the rooms server, without actually connecting to other players.

Also, if the client is in a room it will get notifications of changes to the room and of messages being sent through the room. You shouldn't expect rooms to be able to handle high numbers of messages or send them quickly, but something like a chatroom where lag isn't really an issue might be suitable, depending on the platform.

A more common use of rooms outside gamerooms and chatrooms is setting up a party to play with: this is also usually a room. This kind of room starts out with just one player in it, and if that player invites friends to play together then they join her party. The party leader then does matchmaking to find a room with enough free spots for the entire party.



Joining a gameroom with a partyroom of several people is often a hairy thing to develop. If the platform holder didn't supply anything for this particular situation then there are a lot of things that can go wrong that you need to solve. For example, what if a party of 3 players starts joining a room that has 3 empty spots, but by the time two of the players in the party have joined someone from outside also joins that room, taking up the final spot? In that case you need to roll back the joining for the two players who already joined and go back to searching. The number of edge-cases that might occur here is pretty horrible and is just one example of why developing an online multiplayer game is so much more work than developing a single player game. (For some more reasons check this blogpost on why adding multiplayer makes game coding twice as much work.)

Another interesting challenge when using generic room-based matchmaking is what supposedly happened to one of the Gears of Wars games at launch. Rumour has it that their matchmaking broke down because lots of clients all decided that the same room was the best one to join. Only a few got in before it was full, and then the remaining players all tried joining the same next room. Since each time most would fail it often took so many tries to actually get into a room that the matchmaking took forever. This particular situation is difficult to test without a large playerbase and definitely something to consider during development.

Rooms allow for additional data to be stored in the room. This data can be used in any way the developer sees fit and will probably contain information on the game mode, the skill of the players in the match and the map on which the match takes place. This way when a player is searching for a specific game mode the game can look through the list of rooms for one that's the desired game mode. This also allows the game to check whether the other players in the room are at a similar skill level as the player. Rooms can hold arbitrary data as the developer sees fit, but they generally don't allow for a large amount of data, so only the essential stuff is stored here.

While it's often relatively easy and quick to use the matchmaking systems that platform holders provide, there are some downsides to this approach. One of the most important ones is that if you make a multiplatform game then you'll need to build this again for each platform. If you're doing simultaneous launch on 3 platforms then this is going to take a serious amount of programming work.

You might think you can reuse a lot of your code since the basic ideas behind these rooms are so similar, but in our experience the platforms have just enough differences that this approach quickly becomes a horror story. The old version of the Awesomenauts matchmaker supports 5 platforms: Steam, X1, X360, PS4 and PS3. We tried to abstract away the platform-specific details as much as possible, but adding code for all the weird edge cases each platform introduces has turned our old matchmaking codebase into one big tangled mess where any change might break any of the platforms. Designing a good architecture for handling the differences between these systems is a tough challenge, especially if you're doing it for the first time.



Note that the room systems on various platforms also can't connect to each other. So if you want to do cross-platform multiplayer then you basically need to write your own matchmaking servers.

Another problem of using these generic room systems is that they allow for very little server-side logic to be run. This is a huge limitation on how well you can match players. This is also the main reason we redid the entire Awesomenauts matchmaking code and built our own, called Galactron. This launched in 2016. The limitations on matchmaking imposed by a lack of server logic is a big enough topic that I'll write a separate blogpost about that soon.

The generic room-based matchmaking systems that platform holders provide make developing matchmaking a lot easier than without. They save you all the hassle of developing and maintaining your own servers, so unless you're doing a large production, chances are they are the best option to efficiently develop the matchmaking for your game.

Saturday 23 September 2017

Taking live soundtrack improvisation to the next level

I've previously written about how Rene Derks and I did a live musical improvisation to someone playing Journey and Ori And The Blind Forest. This was a pretty unique experiment already, but recently we kicked it up a notch with an even bolder performance: instead of knowing what was coming, we let players choose whatever game they wanted to play and we had to improvise live music to that. Since this was an experiment not all of it worked well, but I think it was a success: a bunch of games turned out pretty awesome and we learned a lot. Today I'd like to explain how we approached each game and why some of them worked, and others turned out not to be a great fit for this format. I've included a couple of short videos from the performance, check 'm out!

The basic idea for these performances is that we mix live improvisation with gameplay. Instead of playing the original game's soundtrack we come up with new music on the spot and try to respond to whatever is happening in the game. The result is a truly adaptive soundtrack, by the magic of spontaneous improvisation. As always with improvisation, some of it sounds bad, a lot sounds okay, and some moments are pure magic that can only be experienced there and then.



For the previous performances we had chosen specific parts of the games Journey and Ori that would have a nice emotional arc to add music to, and we thought beforehand about the kinds of emotions and melodies we wanted to do for the various sections of those games. While that makes for better music, it's also limiting: we can't do a whole lot of games that way and experimentation on the spot is somewhat limited. So when we got a chance to try this again at Abunai in August 2017 we chose a different approach: we went in deliberately unprepared. We had a Switch and a laptop with some 40 Steam games installed and let players choose whatever they wanted to play. We improvised to whatever they did.



Beforehand we thought that if 25% of our performance was awesome, 25% was okay and 50% sucked, we would consider it a successful experiment. We may have done a little bit better than that, but not much: some games indeed didn't work, and for some games we only figured out what kind of musical approach worked after a few minutes of struggling.

Another change we made compared to the previous performance is that we played in a small hall in a corridor. During our previous performance at Animecon was had a big stage. That's great if a performance is a big success, but it creates a lot of distance to the audience and you need a lot of audience to make it work. At Animecon on average we had maybe 20 viewers, which made the hall look like a big empty failure. So at Abunai we requested something much simpler: a spot that people walk past on their way to other events at the con, and some couches. This reduces the distance to the audience and makes it easy for people to watch for a few minutes and then walk on. In total we reached way more people this way, and it was much more fun to do. There was even room for musically fooling around with the audience a bit.



The first player of the afternoon chose Limbo and this was also the most successful game for us. During the afternoon we could easily see whether we were doing well by checking whether passersby stayed to watch and listen. They did for Limbo.

The reason Limbo is so great for this kind of musical improvisation is that it has a slow pace with a strong emotional arc. First a player walks for something like 20 seconds, which is a nice duration to set a mood musically. Then he approaches a trap. We know the game so we can start increasing the tension by playing something eerie. Then the player interacts with the trap and might even die a couple of times, which are great opportunities for doing something special musically. In this case I played 2 high shrieking notes at every death, combined with a few hard hits on the djembe by Rene. By doing the same thing at every death the whole performance gets glued together.

Limbo is slow enough that Rene even saw room to play something similar to sound effects on his djembe. For example, he played taps that mimic footsteps, drum rolls during slides and a deep bass when the player lands after a slightly deeper drop.



The second game that was chosen was Ikaruga, an insane bullet hell shooter. This was very difficult for us to react to: the screen is filled with bullets practically all the time so there's isn't really any kind of arc that we could latch onto. We ended up just playing a fast beat and only really reacting during boss fights. It worked somewhat, but the whole idea of the music reacting to the game was kind of lost here and cello+djembe isn't the best possible combo for a fast beat.



The worst of the day and the only complete failure was Tekken 7. A player brought a laptop to show his skills at beating the final boss on the hardest difficulty. He was incredibly good at the game so that was impressive to see, but making music to this was practically impossible. The action was too fast to follow and the camera constantly jumped around to emphasise specific scenes. For a short bit we tried to play sound effects on the punches, but the game is so fast that we totally failed at seeing them coming.

Figuring out what to do with the next two games was really interesting. Spelunky is a game with a rather constant state during gameplay: every five seconds you kill an enemy, grab some loot and do some platforming. Individual events are too fast to react to and the flow of events is pretty much constant. However, Rene noticed that during each level the player is always going down. I tried following this progress by starting each cave on the highest string of my cello and then dropping to the next string as the player drops. While this idea is nice, I feel it was too subtle to be picked up by the audience. In hindsight it might have been better to react to how much health the player had left. Spelunky is a rogue-like game so the tension of being low on health is really high.



During The Binding Of Isaac we found a better trick. For each floor Rene chose a base rhythm that he expanded as the floor progressed, building towards the boss fight. During empty rooms he went back to the floor's basic rhythm. While Rene kept playing this beat on his djembe, I stopped playing altogether. Whenever the player cleared a room, I played a short melody; the same one for each room. I had another melody for opening a chest. And then as the player reaches the final boss of a floor, I started actually playing and driving up the tension. I think this worked really well and it shows a very different approach compared to what we did during the other games.

Next up was Broforce, in 3 player coop. Sjeez, that game is chaotic! Each level is just 40 seconds of constant gunfire and explosions. I had no idea where to look or what to respond to. This was almost as bad as Tekken 7. Ugh.

So I shivered when the next choice was Ultimate Chicken Horse: another super chaotic multiplayer game. However, here we found a fitting approach. A match of Ultimate Chicken Horse is a series of short rounds, where each round starts with everyone modifying the level a bit, and then everyone playing it. Each phase lasts around 15 seconds. Reacting musically to the actual events is impossible in this chaotic game, but instead we simply did something special for the phases. During the building phases I did some goofy plucking on my cello, deliberately playing melodies that sound a bit childish and dumb. And then during the action phase I played a faster, more aggressive variation with the bow. Rene made this contrast even stronger by not playing at all during the building phase and playing a really fast djembe rhythm during the action phase. I think this captured the back and forth of the gameplay phases quite well, and was a good fit for the wacky art of Ultimate Chicken Horse.



We ended with two slower single player games: Zelda and ABZÛ. The Legend of Zelda: Breath of the Wild is a great fit for improvising music. Most things the player does take long enough that you can react to them. Walk for half a minute, fight for half a minute, look at your loot for a bit, climb a hill, swim some water, another battle. Each of these steps takes long enough that you can really react to them without turning the music into a chaotic mess that goes all over the place.

ABZÛ is an even slower game. Like Journey it's a gorgeous game that's mostly about looking and experiencing, with very little challenge in the gameplay. The serene under water gameplay is a great fit for setting a mood through music, but I think our instruments weren't the best fit here. Djembe and cello are interesting for many things, but underwater moods are difficult with this setup. Also, after playing for around 2.5 hours the sound got a bit stale for us.



Nevertheless, during the final round of ABZÛ another interesting thing happened. The player had overlooked that he needed to free a little robot before a door could be opened. Instead, he repeatedly pressed a button on the door, hoping it would open. I played two simple notes each time to accentuate that it didn't work, and since the player kept trying, I kept playing those same two boring notes. This went on for way too long and the audience started laughing at how stupid that sounded. This was one of the nicest and most direct bits of audience interaction we had. It also taught me an important lesson: sometimes it's okay if it sounds bad musically if the interaction is fun or interesting in some other way.

Finally, after the performance we had dinner at Rene's parents. They happen to own a piano and Rene also plays that, so we started jamming some more, despite having played so long during the day already. This sounded quite wonderful and it taught me the biggest change we should do if we do this kind of performance again: we need a more diverse set of instruments. I can only play the cello, but since Rene plays both keyboards and percussion we should just bring both to diversify the sounds and moods we can play. Or maybe we should even get a third musician next time!


In the end I think this was a really interesting experiment. The quality of the music varied from really awesome to really bad and that's okay: the whole point of improvisation is that you don't know what's going to happen and the moments where it's awesome are true magic, happening right there and only then. The key lesson we learned is that different games require vastly different musical approaches and the trick is to find whatever arcs the game has going and respond to those through music.

Sunday 3 September 2017

The requirements of good matchmaking

Matchmaking is a highly important part of most modern multiplayer games. Build it badly and the user might be stomped by players way too good for them, get a really laggy connection or wait forever to get into a match. It's also a difficult topic to analyse and test before launch because there are so many unpredictable factors. For example, how are your players spread out over the globe? How much do match durations vary? How long are players willing to wait? How good are their internet connections? All of these and many other factors influence whether any given matchmaking implementation actually works well. Surprisingly few developers have written about the matchmaking in their games in detail, so I figured it would be useful to share our experiences. Over the years we've had several different approaches to matchmaking in Awesomenauts, and have iterated live on each of those, so let's have a look at what we've learned so far!



Matchmaking is such a big and complex topic that I'm going to write a series of posts about it. Today I'm going to kick off this topic by looking at the requirements of "ideal" matchmaking. In future posts I'm going to explore the actual matchmaking implementations we've used and their pros and cons. This will start with the standard matchmaking systems that the big platform manufacturers provide (like Valve, Sony, Microsoft and Sony), followed by an analysis of why we switched away from that and how we built our own. I'm also going to dive into some more controversial topics, like the psychological side of matchmaking and how we managed to massively reduce the percentage of people who leave a match before it finishes.

What defines good matchmaking? This depends on the game of course, but usually the matchmaker will try to achieve most of the following things:
  • Low waiting times to get into a match.
  • Low ping.
  • Match players of similar skill.
  • Match premades with premades.
  • All players in a match start at the same time.

Many additional requirements are possible, such as:
  • Match players who speak the same language.
  • Punish bad apples by matching them with each other (for example leavers and griefers).
  • Don't get matched with the same people too often.
  • Gameplay variation: maybe you don't want to be in the same kind of matchup every time.
  • Match players who use voice chat with others who also do so.
  • Match based on team composition (if one team has a rare team composition then maybe match them with others who also don't have the standard set of tank/healer/harasser/etc).

It's rarely possible to achieve all of these requirements simultaneously. Even if your game is a big success and 100 people or more search for a match every minute, then some requirements might still not be doable with acceptable waiting times. For example, if you want to match Australians only with other Australians, then waiting times deep in the Australian night will be extremely long for all but the biggest games. What works for a game with a playerbase as large as League Of Legends or Call Of Duty doesn't work for most other games.



I've previously written a blogpost about how many players you need for quality matchmaking and the short summary is that 1000 concurrent players isn't much from a matchmaking perspective. However, it is for most games: especially indie games are considered a hit if they reach 1000 simultaneous players every day. Even if you're convinced your game will be an indie hit you still need to think long and hard about how to make the matchmaking work with that relatively small playerbase. And this isn't limited to indies either: even a big triple-A multiplayer production like Lawbreakers currently has only a few hundred simultaneous players.

Because it's usually impossible to achieve all matchmaking goals simultaneously, you need to think about how important each of them is. Especially waiting times suffer when you make other requirements more important. The more requirements you can ignore, the better you can do on the other ones. For example, in Awesomenauts we've chosen to ignore ping with teammates: the matchmaker only looks at ping with enemies. This is because you can't hit your teammates anyway, so if there's more lag there then that will be much less of a problem. In an ideal world we would also want a low ping with teammates, but in practice we can achieve better ping with enemies if we ignore ping with teammates, and thus in our case this is a good choice to make.

One important thing to keep in mind when designing a multiplayer game is that adding more gamemodes often means splitting the playerbase and thus increasing waiting times or reducing matchmaking quality. Think long and hard before adding an additional matchmade game mode!

Reducing the matchmaking requirements isn't the only way to reduce waiting times. For example, if you can design your game in a way that allows for drop-in-drop-out, then players will usually be able to get into a match right away. Or if that's not possible, then maybe if rounds are short enough players can spectate a round while waiting for it to finish and then play in the next one. This doesn't reduce the waiting time, but it does make waiting a lot more enjoyable. I've previously written a blogpost that explores a bunch of different approaches to designing a game so that multiplayer works with a smaller playerbase.



Since this series of posts is for a large part based on our own experiences, I will mostly talk about matchmaking for MOBAs. In other words: real-time team-based games where skill is extremely important, where matches take relatively long and where the same players should play a match together from start to finish. In a sense MOBAs are amongst the most demanding games in terms of matchmaking requirements.

It's important to realise that other types of games have different requirements. For example, in shooters skill is often mostly ignored. Instead, teams are reshuffled after every round to balance things out. Shooter players usually also expect to get into a match really quickly, while MOBA players are mostly okay with waiting a couple of minutes for a better match-up. Of course this varies from game to game: not all games in the same genre have the same matchmaking requirements.

In the end the most important thing is to take matchmaking seriously. Think about the requirements of your game and don't assume they are the same as for another game. In some cases you might even have to redesign your multiplayer experience to make matchmaking work, maybe by removing some game modes or adding something for players to do while waiting for matchmaking. Matchmaking can't be an afterthought if your multiplayer experience is to thrive!

Saturday 15 July 2017

What we learned improvising a live game soundtrack

Recently I've done a couple of music performances that were quite different from any live game music performance I've ever seen. We let someone play the games Journey and Ori And The Blind Forest live on stage and we improvised a completely new soundtrack on the spot, reacting to whatever the player was doing. This was a unique experiment for us and we learned a lot about what works and what doesn't, so today I'd like to share our approach and experiences.



Usually when game music is performed live the original soundtrack is replicated by musicians on stage. We however completely ignored whatever the original soundtrack had sounded like and improvised based on what we saw. The resulting music is different with each performance and sounds nothing like the original soundtrack. The fun of improvisation is that it's entirely in the moment. You don't know what's going to happen and it will never happen in the exact same way again. A truly live experience! Sometimes we might make mistakes, but sometimes we might also improvise the most intensely awesome music. Doing this in response to a game that's being played live is really exciting! Here's a video compilation of our performance at Animecon in the Hague last month:


Some people might know my game Cello Fortress. While there are similarities between Cello Fortress and our live game soundtracks, they really are very different. In Cello Fortress I control the game by playing cello. The cello is a game controller and the game is designed around this unique input device. On the other hand, the live game soundtracks in this blogpost are for existing games: the music doesn't control the game at all. We just respond musically to whatever happens on the screen. This is a bit less extreme than Cello Fortress, but as far as I know it also hadn't been done before and it's a different, interesting approach.

Before I continue, a short word about who 'we' are. The original idea for this performance came from Bram Michielsen and Zuraida Buter, who asked us to improvise live music to Journey at the Screenshake festival in Antwerp in 2015. After that we did a couple more performances and added Ori And The Blind Forest to the mix. I did these performances together with Rene Derks, who played djembe, keyboards and pads. Rene is a lecturer in game design and production at the NHTV in Breda and previously worked on Loco Roco 2 and Gatling Gears. I myself played cello and in my daily life I'm lead programmer at Ronimo Games, mostly known for our games Awesomenauts, Swords & Soldiers and the original version of the student game De Blob. We're not professional musicians but music is a rather serious hobby for both of us.

Improvising interesting music is difficult, and doing so together with other musicians even more so: whatever you're playing needs to not just sound good on its own, but also sound good with what the other musicians are playing. This means you can't just suddenly change the rhythm or mode unless you have a lot of experience improvising together. The specific combination of djembe and cello helps a lot here. It's only two instruments and they both carry a very clear responsibility: the djembe is in charge of rhythm and the cello is in charge of melody.

That gives me total freedom to change the mode or chord without needing to worry that this will sound bad with the other instrument. This kind of freedom is extra important here because we need to respond not just to each other and the audience, but also to the game. It also leaves room for great accidentals: a fast djembe rhythm can work great with slow cello notes and vice versa, so as long as we change intensity at the same moment it's okay if we don't always agree on whether we're increasing or decreasing the intensity.

When preparing for our first gig we started by choosing specific parts of each game, looking for areas with enough of an arc to make them interesting musically. The part of Journey we chose starts relaxed in the desert, then goes down into a darker valley with machinery, then has a very fast descent and finally goes through a dark, scary area that ends in a climax as the player flees from flying monsters. This gives us plenty of room for variation and for having an emotional arc in the music.



The next step was to figure out the general mood we wanted in each area and to come up with some ideas for the kinds of things we could do there musically. We looked up some longplay videos of Journey and improvised over those. We didn't actually write any music but we did end up with a good idea of what kind of things would work were. We also decided where Rene would play djembe and where he would use atmospheric synthesizer pads. In some cases we defined rather specific musical ideas, but in most it was just general directions. For example, in the haunted caves in Journey we didn't have more than this idea: "the caves get low dark dissonant cello and the djembe keeps silent in the beginning and joins after a while."



We wanted to make it very clear to the audience that we were truly responding to the game and not just playing pre-composed music, so we looked for some player actions we could respond to. Journey turns out to be great for this: jumps can take as long as 10 seconds so they are long enough that we can respond to them musically. During a big jump I might for example play an upward flurry on cello, hold a long sliding note during the fall and then get back into the rhythm as the player lands on the ground. Musically it feels slightly childish to play an upwards line as the player jumps upward, but audiences responded very well to this. We could clearly hear in their responses that these were the moments where they truly 'got' it.

To tie it all together musically and to have some kind of theme that the audience can recognise and remember we choose a couple of events for which I wrote small melodies. This is very much like in Zelda, where a short melody plays whenever you open a chest. In Journey we chose the finishing of each level as the key moments and in Ori And The Blind Forest the opening of doors. I quite liked the melody I came up with for Journey so I've also reused it as an attack melody in Cello Fortress.


This kind of performance isn't just about the music, but also about the person who sits on stage playing the game. Figuring out what's best here was a challenge. On the one hand the most pure expression of the concept would be to have a random player from the audience play the game without any preparation. However, this is risky since the player might not know the game and get stuck or need to retry an area a dozen times. That's going to be really boring to watch. On the other hand a prepped player feels like faking it. So we ended up making sure we had a player who knew the basics of the game, but who hadn't played with us before so that we both didn't know what was going to happen exactly.

This worked well in general but during one performance it backfired. The player got stuck in a dark cave in Journey and walked around aimlessly for minutes. It's difficult to improvise anything good for that and I saw some people from the audience leave during that section. I think next time it might be better to look for a player who knows the game better and accept that the player might rush through some sections. That's probably more fun than watching someone be stuck, trying to figure out what the jump button is or looking in every nook and cranny to find every last item...



While Journey is a slow game in which we could respond to individual actions by the player, Ori And The Blind Forest is way too fast for this kind of approach. For example, combat often takes only 2 seconds. We tried switching to combat music every time an enemy was encountered, but the music alternating so much just didn't sound good. Music needs to flow. So instead we decided to respond to individual events very little and mostly just have different types of music per area. The player can quickly go from place to place but generally stays in an area for at least 30 seconds so it's okay if the music changes whenever you go in or out of an area.



In Ori We also took a more melodic and less improvisational approach: Rene played keyboards here and we defined most of the chords and arpeggios beforehand. This works, but it felt a bit too forced to our liking. Our goal with this project was to truly improvise and after doing a couple of performances especially Ori felt a bit stale.

Doing these performances was really exciting and we've learned a lot about what works and what doesn't. Our next experiment with this idea is going to have a different structure. At the Abunai convention we want to try a smaller, more cosy setting where we don't decide beforehand which games are going to be played. Instead we're just going to bring a list of suitable games and let the audience decide whatever they want to play and we'll improvise to that. If someone happens to be carrying their own Switch they could even hook that up and play their own games! This means we can't prepare beforehand, which will probably result in less polished music, but at the same time it will be much more spontaneous so I expect more fun, weird things will happen. I also hope that not being on a stage will make the interaction between the audience, the players and us more direct. I'm really looking forward to trying out this new experience!

Monday 10 July 2017

The Ronimo coding style guide

Last week I showed the coding methodology that we use at Ronimo, which describes our workflow. This week we'll have a look at what our actual code looks like, which is defined in our coding style guide. The idea behind our style guide is that if all code is formatted in a similar manner then it's easier to read and edit each other's code. For example, reading a different bracing style or naming convention usually takes some getting used to, and we avoid that altogether by having a strict style guide that all programmers at Ronimo must follow.



I haven't seen a whole lot of style guides from other companies, but from what I've heard our style guide is quite a lot stricter than what's common elsewhere. I'm not sure whether that's true, but I can totally imagine it is since I'm known for being precise (sometimes maybe overly much). Our style guide isn't set in stone though: there's an exception to every rule. If our style guide really doesn't make sense in a particular situation, then it's okay if a coder ignores it somewhere. Just as long as there's a good reason for it.

Some of the choices in this document are quite arbitrary. Sometimes alternatives would have been equally good, but without clear choices you can't have similar formatting for all programmers. This is especially true for bracing style. I know that this is a heated subject and while I have a clear preference, good arguments can be made for alternative bracing styles. (Still, it would have been nice if advocates of the other major style wouldn't have called it the One True Bracing Style... ;) )

A key element in our style guide is that I want code to read like English wherever possible. Variable and function names should be descriptive and only the most commonly known abbreviations are allowed. Brevity isn't a concern of mine, readability is.

Not all points in our style guide are about formatting though. Others are about actual language constructions. C++ is a rich language with an enormous amount of possibilities, but quite a few are too confusing or have too much risk of bugs to actually use. For example, nesting ternary operators is totally possible in C++, but the result is rarely readable so we disallow it altogether.



Our style guide also contains some rules that are intended to make cross platform development easy. On consoles you usually can't choose your compiler, so you have to work with whatever Nintendo, Sony or Microsoft have chosen, including the limitations of their compilers. We've researched what features of C++ each supports and have forbidden some of the newer C++ constructions that we think might not work on one of the consoles. Since we're not currently actively developing on some of the consoles we went by documentation only though, but I'd rather be too strict here than too lenient.

Another thing you can see in our style guide is my dislike for complex language constructions. C++ allows for some highly impressive stuff, especially using templates and macros. While I appreciate that these tricks can sometimes be really useful, I generally dislike them whenever they become too difficult to read. In the rare cases where these tricks are truly needed they are allowed, but usually I prefer if complex language constructions are avoided.



One particularly hotly debated point in coding styles is whether to mark class member variables. If the Car class has a float speed, do we call that speed, mSpeed, _speed or something else still? I've chosen to simply call this speed. Here too the reason is that I want code to be as similar to English as possible. The more prefixes and underscores there are, the further it moves away from natural language and the more difficult it becomes to just read code and understand it like you would text.

However, there's a good reason many programmers mark their member variables: it's very important in code to know whether a variable is a class member, a function parameter or a local variable. This argument is true, but I think we have that covered elsewhere: our style guide contains limitations on how long a class or function can be. If a function is short and fits on one screen, then it's easy to immediately see where variables are coming from. I think if classes and functions are short enough, then markers for member variables aren't really needed.

Note by the way that the rule for how long functions and classes can be is the one broken most internally. Sometimes it's just really difficult to split a class or function in a neat way. In the end the goal of our style guide is to produce clear code, not to hinder that by forcing clumsy splits. Still, there's real skill in figuring out how to neatly split classes and functions into smaller, more maintainable units, so if you're not super experienced yet then more often than not a neat split is possible and you just don't see it. In my opinion the ideal size of a class is anywhere between 200 and 400 lines, but a rule that strict isn't feasible so what's listed in the style guide is more lenient.

Now that I've discussed the reasoning behind our coding style guide, let's finally have a look at what it's actually like!

The Ronimo Coding Style Guide


There is an exception to every rule. However, keep to these rules as much as possible to maintain a constant layout and style for all the code. A lot of this is taste and a constant code layout requires setting aside one's own taste and keeping to these rules. Once used to it, it's easier to read such code.

When working in another language than C++, try to keep as close as possible to the C++ coding standard, but of course within reason. There are some specific notes on C# at the bottom as well.

C++


  • All code and comments are in Great Britain English. Not American English. So these are right: colour, centre, initialiser. These are wrong: color, center, initializer.
  • Every comma is followed by a space, for example doStuff(5, 7, 8)
  • Spaces around operators, for example: 5 + 7 instead of 5+7
  • Tabs have the same size as four spaces. Tabs are stored as tabs, not as spaces.
  • Functions and static variables are in the same order in the .h and .cpp files.
  • Use #pragma once instead of include guards (we recently switched to this so you will still see a lot of old include guards in our code).
  • Try to make short functions, preferably no more than 50 lines.
  • Avoid making very large classes. Split a class whenever you anticipate the class will grow a lot and you can cleanly split it. In general, try to keep class sizes below 750 lines. However, don't split a class if the split would result in messy code. Some examples of messy divorces are: too tightly coupled classes, friend classes and complex class hierarchies.
  • Don't write lines that are longer than what fits on a normal 1920*1080 screen (keep in mind that the Solution Explorer is on that screen as well).
  • When splitting a long line over several lines, make sure indentation is correct with the relevant parenthesis. For example like this:
    myReallyLongFunctionName(Vector2(bananaXPos + xOffset,
                                     bananaYPos * multiplier),
                             explodingKiwi);
  • Use forward declaration where possible: put as few includes in header-files as possible. For example, use class Bert; instead and move the #include "Bert.h" to the .cpp-file.
  • Includes and forward declarations are ordered as follows:
    • Start with all forward declarations from our own code (in alphabetical order)
    • Then includes from our own code (in alphabetical order)
    • One empty line
    • Then per library:
      • Forward declarations from that library (in alphabetical order)
      • Includes from that library (in alphabetical order)
  • The include of the .h-file belonging to a .cpp file is always at the top.
  • Don't define static variables in a function. Use class member variables instead (possibly static ones).
  • Everything must be const correct.
  • Names of variables and functions should be descriptive. Long names are no problem, undescriptive names are. Only use abbreviations if they are very clear and commonly known.
  • The first letter of classes, structs and enum type names is a capital. Variables and functions start without a capital. Each further word in a name starts with a captical. Don't use underscores ( _ ) in variable names. So like this:
    class MyClass
    {
        void someFunction();
        int someVariable;
    };
  • Member variables aren't marked with something like an m in front of them or an _ behind. Functions should be short enough to keep track of what is declared in the function and what in the class. Don't mark member-variables with this-> either.
  • Implementations of functions are never in .h-files.
  • Templatised functions that can't be implemented in the .cpp file are implemented in an extra header file that is included from the main header file. Such a class can have 3 files: MyClass.h, MyClassImplementation.h and MyClass.cpp. So like this:
    class MyClass
    {
        template <typename T>
        void doStuff(T thingy);
    };
    
    
    #include "MyClassImplementation.h"
  • Start template typenames with T. If more info is relevant, you can add words after that, like TString.
  • Several classes can never be defined in the same header file, except if a class is part of another class (i.e. defined inside the other class).
  • Between functions there are two empty lines (in the .cpp file).
  • Use empty lines to structure and group code for readability.
  • Add loads of comments to code.
  • Write a short explanation of what a class does above each class. Especially make sure to explain relations there (e.g. “This class helps class X by doing Y for him”).
  • Curly braces { and } always get their own line, so they aren't put on the same line as the if or the for. They are also never left out. The only exception is if you have lots of similar single line if-statements right beneath each other. In that case it's okay to put them on a single line. Like in this example:
    if      ( banana &&  kiwi && length > 5) return cow;
    else if ( banana && !kiwi && length > 9) return pig;
    else if (!banana && !kiwi && length < 7) return duck;
    else                                     return dragon;
  • When writing a do-while function, put the while on the same line as the closing brace:
    do
    {
        blabla;
    } while (bleble);
  • Indent switch statements like this:
    switch (giraffeCount)
    {
        case 1: text = "one giraffe";  break;
        case 2: text = "two giraffes"; break;
        case 3:
            // If it's more than one line of code
            doStuffOnSeveralLines;
            text = "three giraffes";
            break;
        case 4:
        {
            // Can add curly braces for readability
            int x = getComplexThing();
            text = "quadruple giraffe";
            break;
        }
    }
  • Function parameters have the exact same names in the .h and the .cpp files.
  • If a function parameter and a class member variable have the same name, then either think of another name, or add an _ to the end of the function parameter. For example like this:
    void setHealth(float health_)
    {
        health = health_;
    }
  • Precompiler instructions (everything starting with #) should be kept to a minimum, except of course for #include and #pragma once.
  • Don't write macros.
  • Variables inside functions are declared at the point where they are needed, not all of them at the start of the function.
  • In constructors, prefer using initialiser lists over setting variables in the body of the constructor. Each initialisation in the initialiser list gets its own line. Make sure variables in the initialiser list are in the same order as in the class definition in the .h-file.
  • Don't use exceptions (unless you're using a library that requires it).
  • Don't use RTTI (so don't use dynamic_cast). RTTI costs a little bit of performance, but more important is that RTTI is almost always an indication of bad object oriented design.
  • Use reinterpret_cast and const_cast only when absolutely necessary.
  • Don't commit code that doesn't compile without errors or warnings (and don't disable warnings/errors either).
  • Don't commit code that breaks existing functionality.
  • No global variables. Use static member variables instead.
  • Use our own MathTools::abs instead of std::abs. This is because std::abs is inconsistently implemented between platforms, causing hard-to-find bugs.
  • Always use namespaces explicitely. Don't put things like using namespace std in code.
  • Never ever even think of using go-to statements. We have a thought detector and will electrocute you if you do.
  • Don't use the comma-operator, like in if (i += 7, i < 10)
  • Don't use unions.
  • Don't use function pointers, except where other libraries (like STL sort) require it.
  • Only use the ternary operator in extremely simple cases. Never nest the ternary operator. Example of a simple case where it can be used:
    print(i > 5 ? "big" : "small");
  • When exposing a counter for an artist or designer it starts at 0, just like it does for normal arrays in code. Some old tools might still start at 1 but anything newly developed for artists starts at 0.
  • When checking whether a pointer exists, make this explicit. So use if (myPointer != nullptr) instead of if (myPointer)
  • Use RAII (Resource Acquisition Is Initialization) where possible. So instead of creating a separate function initialise(), fully initialise the class in its constructor so that it's never in an incomplete state.
  • Write the constructor and destructor together: write the corresponding delete for every new immediately, so you don't forget to do it later on.
  • If you add temporary debugging code, then add a comment with QQQ to it. Never commit code with QQQ: remove the debugging stuff before committing.
  • If you want to leave a marker for something that needs to be done later on, use QQToDo. If it's something you need to do yourself then add your letter to it, for example QQToDoJ. Only commit QQToDo if it's really impractical to finish it right away.
  • Class definitions begin with all the functions and then all the variables. The order is public/protected/private. This might sometimes mean you have to use the keywords public/protected/private several times in a single header file (first for functions, then again for variables).
  • A class starts with its constructors, followed by its destructor. If these are private or protected, then they should still be at the top.
  • When something has two options, but isn't clearly true/false, then consider using an enum class instead of a boolean. For example, for direction don't use bool isRight. Instead, use enum Direction with the values Left and Right.
  • Instead of std::string and std::stringstream, use our own classes RString, RStringstream, WString and WStringstream (these tie into our own memory management).
  • The variable float time always stands for the time since the last frame, in seconds. If time means something else, make this explicit, for example by naming it timeExisting instead.
  • Make sure all code is framerate-independent, so use the variable float time often to achieve this.
  • Make the intended structure explicit and clear. For example, avoid doing things like this:
    if (yellow)
    {
        return banana;
    }
    return kiwi;
    What was really intended here, was that depending on yellow, either banana or kiwi is returned. It's more readable to make this explicit like this:
    if (yellow)
    {
        return banana;
    }
    else
    {
        return kiwi;
    }
  • Use nullptr instead of NULL.
  • We only use auto for things like complex iterator types. For everything else we explicitly type the type.
  • Use range-based for where applicable, for example:
    for (const Banana& ultimateFruit : myList)
    (Note that it's important to type that reference when not working with pointers, since otherwise the Banana would be copied in this case.)
  • Use override and final wherever applicable.
  • If a function is virtual, the virtual keyword should always be added to it, so not only in the parent class, but also in each version of the function in the child classes.
  • Use strongly typed enums, so with the class word. Their casing is the same as for classes. For example:
    enum class Fruit
    {
        Banana,
        Kiwi,
        ApplePie
    };
  • Don't use rvalue references (&&) unless you really really really need them.
  • Use unique_ptr wherever possible. If an object isn't owned, then it's stored as a normal pointer.
  • Avoid creating instances of complex types in the initialiser list. Simple copying and setting goes into the initialiser list, while more complex code like calling new goes into the body of the constructor. Here's an example of how that works:
    FruitManager::FruitManager(Kiwi* notMyFruit):
        notMyFruit(notMyFruit)
    {
        bestFruitOwnedHere.reset(new Banana());
    }
  • Only use shared_ptr in cases where shared ownership is truly needed. In general, try to avoid shared ownership and use unique_ptr.
  • Lambda's that are too big for a single line are indented like this:
    auto it = std::find_if(myVec.begin(),
                           myVec.end(),
                           [id = 42] (const Element& e)
                           { 
                               return e.id() == id;
                           });
  • Don't use MyConstructor = delete (this doesn't seem supported on some compilers we might need).
  • Don't use the C++11 feature for list initialization (this doesn't seem supported on some compilers we might need). So we don't use this:
    std::vector<int> thingy = {1, 2, 3};
  • Don't use any features that were added in C++14.

C#

  • Variables at the top of the file instead of at the bottom. We do a strict split between functions and variables (like we do in C++), so here it's first all variables, then all functions.
  • async functions must always end their name with Async, for example eatBananaAsync.

That's it, our coding style guide! ^_^ While I imagine you might disagree with a bunch of the specific rules, I think it's useful for any company that does programming to have some form of a style guide. Our style guide can be a good starting point for creating your own. I'm quite curious: what's the style guide like at your own company and how do you like it? Do you have one at all?

Sunday 2 July 2017

The Ronimo coding methodology

I'm a strong believer in working in a structured way. As a game programmer you need to build complex systems and just going with the flow isn't good enough. That's why I've written two documents to describe how we code at Ronimo, which every programmer and intern gets to read on their first day. Our Methodology document explains the workflow while the Style Guide explains the layout of our code. Today I'd like to show our Methodology and describe the reasoning behind the rules in this document.



Note that the contents of this document aren't very original: most if it is a combination of common agile practices that I like.

Let's start by having a look at the actual methodology document:

The Ronimo coding methodology


General method of implementing a new feature:

  1. Analyse what should be made. Discuss approach with end users (usually artists and/or designers) and lead programmer.
  2. Split into small tasks of a day at most.
  3. Make a basic planning for the small tasks. Put the core of the functionality and the most difficult parts to implement at the front of the planning.
  4. Implement each small task.
  5. Evaluate result with end users and lead programmer.
  6. Explain to end user what the new functionality does, so that it's actually used.

Implementing a small task:

  1. Analyse what should be made.
  2. Verify that related existing code is bug-free (perform tests!).
  3. Research and experiment with any new technology required for this feature.
  4. Code-design for new functionality. Don't focus too much on theoretical future use, but do keep it in mind.
  5. Refactor existing code to make room for your new feature.
  6. Test whether older functionality is still intact.
  7. Implement new functionality.
  8. Test whether new functionality works.
  9. Finish code: add comments, clean up code and check the destructors.
  10. Test whether new functionality still works.
  11. Test whether older functionality still works.
  12. Evaluate result. If showable, also show this intermediate result to the end user.

Various other rules:

  • Keep a personal list of all the small things that you should do. If you come across some issue that you can't immediately take care of, or if someone asks for a feature that you promise to build later, then write that down in your list. Don't think you can always remember everything.
  • Never continue to work on something if you have encountered a crash or major bug. Always fix the crash first.
  • "Premature optimization is the root of all evil" (Donald Knuth). First implement a new feature in the simplest / clearest way possible, then analyse whether it works, then analyse the performance and only if necessary implement optimisations.
  • Don't worry too much about making your code in such a way that unknown future extensions might be easy to add. When new functionality is needed, the code can still be refactored. Of course, if it's little work to do, then do make things as generic as possible.

Most of these are quite clear, but it's interesting to discuss some of the reasons behind these rules. Often enough I've seen coding interns have loose ends everywhere in their code because they were working on five things at the same time and forgot to test, clean up and finish some of them. That's why our coding methodology requires that you finish what you were doing before you get to the next thing. This is also why I require that big tasks are split into smaller ones: a person can only remember so much and the more things you're working on simultaneously, the bigger the chance that you'll overlook something important.

At the same time I prefer the agile way: only make what you actually need and expand the codebase as you go. During early development you don't know all the features you'll need, nor do you know for all problems how you'll solve them. However, adding things one at a time will often make code bloated and without focus, and will muddy class responsibilities. That's why code needs to be refactored often. Refactoring requires discipline. Often you can hack in a new feature in just a few hours, or first spend half a day refactoring to make room for that feature in a good way. It's easy to skip or postpone refactoring, but doing so often produces unworkable code in the long run. That's why refactoring is an explicit step in our methodology.

Focusing only on one small new thing at a time shouldn't be taken too literally though. While I think it's important to not work on other things until what you're doing is finished and clean, that doesn't mean you shouldn't look ahead. When building something complex it's important to think about how you're going to make the whole system work. I once had an intern who had to rebuild a large part of a tool because he had taken our coding methodology too literally and hadn't thought ahead at all. The most complex features of the tool were not possible at all with what he had build. The key here is to find the right balance between thinking ahead and focusing on one thing at a time.



Another staple of mine is that programmers need to communicate directly with designers and artists. We believe extensive design documents are rarely a good idea, so the only way to know what's needed exactly is to talk to the designer or artist who needs a new feature. Often that person hasn't defined the exact details of the feature, so the coder needs to discuss it with them and think about the caveats, also from a design and art perspective. To smoothen this communication it helps a lot if the programmer has a little bit of experience in design and art, but even if that's not the case I think it's the programmer's job to talk the language of the designer or artist, not the other way around. It's really difficult for a designer to speak code, but a programmer should be able to talk about his work in comprehensible English (or Dutch in our case).

One thing that's surprisingly missing in our Coding Methodology is unit testing. We have a strong focus on testing our own code extensively, but the document doesn't say you need to write unit tests. This is because I think gameplay is often too chaotic and unpredictable to test well with unit tests. Certain things are testable with unit tests, but the bugs we encounter are often not things where I can imagine how a unit test would have found them. Often it's things that function fine but result in undesired gameplay.

I do realise that not making units tests makes us more vulnerable to bugs than a team that always writes unit tests, so we emphasise that if a crash or major bug is found, it needs to be fixed right away. We might have more bugs than software developers who write extensive unit tests, but at least we fix them quickly. I do think we ought to use unit tests more for things like server architecture. Unit testing isn't in our blood at all and it probably should be at least a little bit. I'm curious though: do you use unit tests for your gameplay or engine code?

Regardless of whether you agree with the particular rules in our methodology document, I think it's important that all programmers think about their workflow. Just doing whatever you feel like doing isn't good enough. Discipline and structure are important for anyone who works on larger, more complex systems. What's you're coding methodology like? If you happen to work at a company, is there an official document like the one I've shown today?

So that's it, the Ronimo Coding Methodology! Next week I'll show our Coding Style Guide, which is quite a bit more strict than most coders are used to.

Sunday 29 January 2017

The many meanings of the confusing word 'lag'

'Lag' is one of the most confusing words used in gaming: it's used to describe wildly different problems. This makes communication difficult: when a player is complaining about 'lag', what's his problem exactly? Is he talking about his internet connection, framerate or input delay? So today I'd like to list the different meanings of the word 'lag', both to reduce confusion and to explain why all of them are important. I've grouped them into three main types.

It can be difficult to know exactly what's going on, but it would already help a lot to always at least specify which of the rough groups you mean, instead of just saying 'lag'.

Note that this post is not specifically about our own games. These problems can happen in most games, although the exact impact depends on how the game was been programmed.

Internet and connectivity issues


  • High ping: ping is the time it takes for an internet packet to travel from your computer to another. In most games the gameplay still feels smooth with high ping, but you'll often start seeing oddities like getting shot by someone who isn't in view anymore, or your shots missing despite perfect aim. The faster the game, the more problematic high ping is.
  • High jitter: jitter is an often overlooked aspect of ping. Jitter is when some packets take longer than others. Jitter often causes stuttery movement in games. You might have low average ping but still experience stutters because of high jitter. (Packet loss often looks the same as an extreme type of jitter so I'm not listing it separately here.)
  • Pingspikes: this is when occasionally the connection becomes extremely slow or even drops altogether for a few seconds. In most games ping spikes result in serious problems, like your controls not working anymore for a while, getting a lot of damage at once at the end of the spike, or even being disconnected.

All of these problems can have many causes, like a slow internet connection or your roommate downloading a torrent over the same connection. There's one common cause that's relatively easily solved though: WiFi. WiFi increases ping, causes jitter and packet loss and, worst of all, pingspikes. If you can, never play online games over WiFi. Connect to your router with an ethernet cable instead. Check this blogpost from the League Of Legends devs to see how bad WiFi really is.

Delay between input and seeing the result


  • Input delay: the time between you pressing a button and the game actually processing it. Games usually process input only once per frame, so the higher the framerate, the lower the input delay. Drivers and the operating system will also cause a little bit of input delay, or a lot if it's a complex controller like the Kinect camera.
  • Input jitter: this is when the input delay varies. This can happen if you press the button just before the frame update and then just after. If the game runs at 30 frames per second, this can create a 33 milliseconds difference in input delay. The higher the framerate, the lower this kind of jitter wil be.
  • Rendering delay: once the gameplay has processed your input, it needs to be rendered. The GPU is usually at least one frame behind, and in some cases the game itself can be as well. For example, if you set the Multithreading Mode in Awesomenauts to Higher Framerate, then the graphics are always one additional frame behind on the gameplay, causing additional input delay. Again, the higher the framerate, the smaller this effect.
  • Monitor delay: this is usually the most noticeable cause of delay between pressing a button and seeing the result. Many televisions have all kinds of clever processing to make the screen sharper and smoother, but this can add over 100ms of lag. That's a lot and can be very noticeable. When using a television you should always switch it to Game Mode to solve this. Computer monitors generally don't have this problem.

Framerate problems


Low framerate is also often called "lag". Players sometimes don't realise that there are different types of low framerate. Being precise when describing your problem helps a lot if you ever ask anyone for help with how to solve your framerate problems.

  • Constant low framerate: this usually means the videocard or processor is not fast enough to run the game.
  • Occasional long stutters: the game sometimes freezes for half a second or more. This might be caused by problems reading from the harddisk, so switching to an SSD might help. In general though, a game should be programmed in such a way as to do disk reads asynchronously and thus not freeze like this, but sometimes this is really difficult to achieve.
  • Regular short stutters: every second or so the game skips a few frames. Such stutters are really short but can be very noticeable, especially during smooth camera movements. This might be caused by running something else on your computer (like a virus scan or a lot of tabs in your internet browser). It can also be caused by the game not balancing the work well enough between frames, causing some frames to take longer.

Framerate problems and input delay are both often decreased by turning off VSync. This does come at the cost of getting screen tearing though. Which you prefer is a matter of personal taste.