Coding a Story that Makes You Move

Interactive storytelling can take many forms—and it might just make you get off the sofa.

Image for post
Photo Courtesy of Nnudoo

“The way we experience story will evolve, but as storytelling animals, we will no more give it up than start walking on all fours.”

— Jonathan Gottschall

I’m becoming progressively more obsessed with exploring different forms of storytelling. I’m admittedly a bit of an outcast in the literary fiction community since I am so intrigued by these unconventional models. I’m curious about the potential of things like interactive storytelling and the model of serialized fiction.

To culminate on a semester’s worth of coding study, I want to write a piece of flash fiction that would be advanced by the user taking on certain poses and motions. The story would give the reader prompts, just as asking them to reach up and grab something.

Once the reader or player, if you will, takes on the right pose, the story will advance to the next part. I want to get more creative than the above example, but I’ll also need to dig deeper into PoseNet to see what types of poses are in the machine learning library. It’s going to be an interesting to try and tell a unique story that isn’t too constrained by the technology while—quite frankly—still being constrained to what this machine learning library can realistically pick up on.

Most motion-based interactive games require complex camera systems or VR headsets.

Image for post
Photo Courtesy of Insta_Photos

“I think the best stories always end up being about the people rather than the event, which is to say character-driven.”

―Stephen King

And that’s all well and good—I own a VR headset and love it. But I’m curious to see if I can create something playable that only utilizes a webcam. I’m going to attempt using the ml5js library, specifically their pose-recognition component, PoseNet. I’ve had mixed results throwing myself into JavaScript libraries in the past, so I’m a bit concerned I’ll hit roadblocks beyond what web tutorials on p5 can answer for me.

Regardless, the idea of creating a story that people could play through with just a webcam is very intriguing to me. It might prove difficult to make something like this run smoothly, but if I’m able to pull it off, it could make a physically interactive story very easy for someone to play through. Webcams are by far more common than VR headsets. If this worked, it would open the door to making interactive storytelling more available to the average user.

I’m an unconfident person who likes contingency plans. If I can’t get this all working, I do have a plan B—burn everything down.

Well, all the code, at least.

Plan B will be to frantically not sleep for the last week of the semester and make a very simple game with the library where you control a cat side-scrolling looking for treats. Because when all else fails, cats are the answer.

Writer and poet from Neptune. Instructional designer in NYC. Grad student at @NYUTandon studying Integrated Digital Media.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store