Designing Games for Telephones

24 October 2011 | 0 comments

Hinterland – a poem that you play – ran for two weeks at the Edinburgh festival. Over four cantos, players would seek out strangers meeting particular criteria (which varied per canto), and get them to help answer a questionnaire. Then, players would phone the Operator, and hand over their answers. The Operator wasn’t a human being, though: it was a piece of software.

I wanted to write a little about the technical design behind the game: how we wrote a game for telephones, and the design considerations that went into it.

Hinterland began in a “scratch” week of prototyping at Battersea Arts Centre in early summer. As well as exploring the concept of the game, the nature of its poetry, and the experience we wanted players to have, we also took a first run at the technology behind it, to see what was possible. This was where I got to understand the material we were working with – in this case, telephony: what was possible, what felt natural, and how we could convey the experience we wanted through it. I wrote code while the rest of the team wrote poetry and design the game.

We quickly came to a simple technical brief for the game: it had to be playable on a simple mobile phone, with nothing more than voice calls and SMS. “Designing for a Nokia 3210” became a mantra. By the end of the week, we had a working prototype: a single Canto, playable beginning to end, with a working Operator and working telephone lines.

Basic Telephony

We’re lucky to exist in a climate where writing smart answerphones for telephone-based games is remarkably easy. Hinterland is made possible by Twilio: an API for “building Voice and SMS applications”. Twilio connects a phoneline to an endpoint on a webserver. When you sign up, you get a number in the US or Canada; you specify a URL on your webserver for Twilio to post data to. And then, every time you receive a call, Twilio POSTs data to that endpoint. If you return an XML file that accords to the TwiML spec, Twilio will follow the instructions contained in that file: speaking text, playing an MP3, asking for voice or numeric input.

That’s enough to make our Operator with.

When a player calls the Operator, the software works out which Canto that player is on, and what question to ask them next. It asks them that question, waits for the answer, and repeats this process until they’ve answered all the questions in the canto.

Hinterland identifies players by their mobile phone numbers, which Twilio passes to us (making maintaining state easy). Initially, we created a new player the moment they dialed our number, but this led to people starting missions when they perhaps hadn’t planned to – if, for instance, they rang the number they saw on the website.

As with so much of the project, we fixed this with human intervention. Because starting the game required visiting Forest Fringe, we’d ask players for their mobile numbers at the venue. When they called the phone number, we’d already determined that they were playing the game. Similarly, they couldn’t proceed to the next canto until they returned to Forest Fringe, where they’d receive a new canto booklet, advance their avatar to the next part of the Hinterland, and finally, be moved onto the next canto in our system by a volunteer. This ensured that the phoneline always behaved in the manner players expected.

Twilio’s phone numbers are all based in the US or Canada – not much good for a primarily British audience. The first version of the game we tried used an 0800 Freephone number forwarding to Twilio. That sounds good on paper, but turns out to very expensive when you’re on a mobile – most mobile contracts don’t include Freephone numbers. For the Edinburgh run, we acquired an Edinburgh phone number from Soho66, which cost next-to-nothing. We had to pay a small VOIP fee for the forwarding to the US, but it meant that players would call a British landline, which made the cost of the call to most of them minimal.

Dealing with responses.

Once we’d got a player’s answers, we had to make a custom canto from them. A few of the answers were multiple choice – specified via pushing a button on the keypad. Most, however, were spoken responses.

When Twilio records speech, it includes the URL for an audio file in the data it POSTs to your server. In the Hinterland admin interface, it was possible to listen to the audio of the player’s answer, and enter a transcription of it. That text transcription appears in a version of the poem that’d appear on the website for the player (and whoever they chose to share the private URL of their page with) to see.

The text transcription enabled us to produce an audio version of the poem. The cantos are designed as performances to be listened to, first and foremost. We’d experimented with various forms of delivery at BAC, and settled on the use of a speech synthesiser for most of the cantos. It wasn’t enough to just put the whole canto into a speech synthesiser, though; that tended to sound disappointing. So Ross provided audio templates of the poem: large multitrack Garageband files, cleverly designed to allow new words to be dropped in, and branching segments to be appropriately muted. We’d synthesise the player’s dialogue, assemble their poem as an MP3, and upload it to our server.

This was a little intensive, and involved a behind-the-scenes team in London working on the audio whilst Edinburgh worked on transcription. The trade-off for using people was nuance: the poems sounded good, and there’s nothing like the good old human ear for tweaking audio until it sounds good. Still, I’m going to keep looking for ways to streamline the audio production process in future runs: the balance of “sounding good” and “returning a file to a player in a good time” was often hard to strike.

Early feedback before Edinburgh from the production team was that a clearer “inbox” of what needed doing would help them a lot, so I created a view of incoming responses grouped by player, indicating exactly how many players were waiting for transcription or upload. It meant the production team only had to watch one page to stay on top of things.

This became the heart of our workflow. Indeed, most of the fine-tuning in the weeks running up to Edinburgh came from understanding the workflow of a distributed production team – split between Edinburgh and London – and what processes would help players understand the game best, whilst ensuring everyone on the production team was clear about what had to happen next for each player, and who was doing what.

Delivering the poem

Finally, we returned the canto to the player via SMS and email. The latter told them their poem was online, and gave them a URL where they could see it as text and listen to it. However, email was a luxury, for players with smartphones: the main purpose of the email was so that players could revisit the experience when they returned home to a computer.

SMS became our primary messaging system: a short text to tell players that their canto was ready, and that they could return to Forest Fringe for the next one. The text gave them a number to call, which would play them their latest canto (again using Twilio). It meant they could get their latest poem as soon as it was ready – ideally, not too long after they’d submitted their answers – and listen to it wherever they were. We used Clickatell for our SMS gateway, which cost about a penny to send a single SMS. There are many SMS gateways, all offering similar services, and my choice came down the simplest one to integrate. Luke Redpath’s Clickatell gem made integrating text messaging into the Ruby application trivial.

SMS also proved useful for getting in touch with players in the field. Audio transcription is hard – especially given some of the questions we were getting players to answer! One afternoon in the run up to launch, I quickly added a way for the Hinterland team to send any player a custom text message, which proved handy on several occasions.

We never really had many problems with the design of the game itself, but we used the feedback and experiences of the early players in Edinburgh to constantly improve it for others. As the Edinburgh run progressed, we added clarity to some dialogue that had proved confusing, and added many more pauses to the dialogue: there were certain points where, as a phone was passed back and forth, players and their interviewees would miss crucial words. With hindsight, I’d have spent longer on the canto editing tools earlier: there was still a bit too much that only I could alter, and spreading the load of the production work would have left me more time to solve more specific problems with code.

The Hinterland web application – powering both the simple public website and the telephony back-end – didn’t gain much major functionality in the weeks between the scratch at BAC and the Edinburgh run. What it did gain was polish: a better understanding of the game’s requirements, of the workflows that made production run smoothly, of the nature of telephony, and the elements of the poem that needed clarifying to players. The project benefited hugely from how quickly we got to an end-to-end working prototype, and it was great to be able to scratch code at the same time as scratching the game design.

Picture by Maria Hägglöf

0 comments on this post.

Comments on this post are now closed.

Related Projects

Latest blogposts from Tom