Latest / The Killscreen Podcast / The Level Designer Who Went Underwater
Transcript
- This is a verbatim transcript of the conversation between Jamin Warren and Jacob Kudsk Steensen. Jamin Warren: ...Very dramatic. Very, very, very dramatic. I know. Jacob Kudsk Steensen: Well, you can begin with a laugh, you know? Jamin Warren: Yeah, yeah. Exactly. Exactly. Hello, everybody. My name is Jamin Warren, and I'm very excited to be with you today with Jacob. He's spent the last decade doing something really unusual, which is treating the video game engine as a medium of ecological fieldwork. I love all the fieldwork stuff that you've been doing over the last, you know, 15 years. All of his immersive installations have grown from years of onsite research: underwater volcanic vents near the Azores, collapsed ice caves in the Swiss Alps, that experimental forest in Minnesota... transforming them into virtual worlds that sit somewhere between scientific document and living dream. And so we're going to be focusing on some of the worxk that Jacob has been doing for the Otherworlds show at Phi Centre in Montreal, which gathers five distinct spaces and spans about 15 years, and then also talk a bit about Psychosphere, which is built from photogrammetry and newly discovered submarine volcanic landscapes, and then finally close it out with Song Trapper, which is part of an expanding video game world called Evoker. Jacob, thank you so much. Let me share this presentation... Or, let me... Jacob Kudsk Steensen: Yeah, thank you for having me. Jamin Warren: Yeah, of course. Hold on one second. Cool. Well, I think maybe to get started, do you want to tell me about your background? How did you get into working with games and game engines? And maybe also some early interest that you had in ecology, in the natural world? Because we often don't think of those things as being connected. You know, indoor kids kind of stay indoor kids. But yeah, tell me a bit about your background. Jacob Kudsk Steensen: Yeah, I mean... Do you have the picture of Facing Worlds? Is that in here? Jamin Warren: The Unreal map? Don't worry, I have it. Jacob Kudsk Steensen: I just... this is kind of a good introduction because, like, you know, I was... two things. So, I grew up in the countryside—I mean, I grew up in different places with my mom—but I spent a lot of time in a really, really small town on the West Coast of Denmark, near the water. There was, like, less than 2,000 people lived there. And so, that was very outdoorsy. And my dad is, like, very much into ornithology. And I kind of grew up around wetlands, water, brackish water... and I’ve worked a lot actually with wetlands in a lot of recent projects because I really know them from a personal level. So, I grew up very close to environments in nature. But then at the same time, you know, when this came out—this video game map—I was 12 years old. And I bought a copy of Unreal Tournament, this video game where this map is from. And this was like, kind of a game that really was part of a series of games that really exploded full-on interactive, three-dimensional space in a very, like, free-roaming, playful, organic way. And it came with a level editor. So you could open up this map or other levels and change them around. This was like a "Capture the Flag" game, and then I changed the play mode to other things, and I started changing materials. So I'm saying this because it was never like a conscious choice for me to sort of work with video game engines or natural environments. I think I'm just an exact generation where the two just really sort of happened. I mean, of course, there have been natural environments forever, but like, a sense of climate awareness, narratives about climate change, environmentalism... at the same time as there is this explosion of full-on three-dimensional virtual worlds that you can modify. When you buy a game, you can modify it. So I just grew up with that very naturally. And I've just kind of accepted it. You know, as a kid, I wanted to work in video games, and I wanted to be an animator of characters. But then I went more into level editing. So I’ve used Unreal, or Source engine for Half-Life, I used CryEngine for a little bit, a little bit of Unity... but I’ve used Unreal since it came out. So for me, it's like a musical instrument. But I very much fell in love with the spatial, environmental part of video game technology. And I still use this. Jamin Warren: Yeah. Do you feel like there are specific affordances for you using Unreal versus like other video game engines? You mentioned it's like an instrument for you. You know, if you're someone who plays guitar, they'll tell you that there's a difference between a, you know, Les Paul versus like a Stratocaster. They have very specific ways of talking about the fret action, and the weight, and the feel, and the tone... Do you feel that way about game engines as it relates to your work? Jacob Kudsk Steensen: I think... I mean, I could probably have ended up really focusing on another engine might as well, but I think there's something that Epic Games did that was a little bit unusual. Like, Source did this too—so, Source is the video game engine behind the video game Half-Life. And any game that Epic has published, even before that, came with a version of the technology used to make the game. So every time... it's like, they also make Fortnite now, and revenue from Fortnite is why they can expand on the capabilities of Unreal. If you don't have Fortnite, you do not have Unreal. Trust me, these two things go hand-in-hand. They need... but they've done that for like 40 years or something. And so, it was a little happenstance, but around the same time there was also like the Quake engine, but it was kind of difficult to use. The Unreal just came with a very, like... you know, change these little things in the map in the title, and then you go from like a multiplayer to a single player, or from deathmatch to whatever. It was very immediate. For whatever reason, also like working with textures and materials was very easy. Unreal Engine also comes with—it's built in, them the new one—something called a Brush Editor. So you could like block out a level directly in the video game software. So for if you're like a solo developer and you're just playing around, it's very helpful to have a tool where you're inside the software of the experience building the experience. So there's less friction. The other thing is that, you know, at the time of—now there's like Unreal Engine 5—but when Unreal Engine 4 came around, it was after the 3 and they made it open. So anybody can use it. And like, I was also using CryEngine a little bit. CryEngine, this German company, legally... I don't know what it is now, but when I was really starting to seriously, like, "Okay, now I'm going to start doing pretty big art projects," Source engine in its contract does not allow you to make anything but games. And I didn't really want to just make games. I mean, I freelanced on game projects, I made mods, I started making like modifications of Unreal and then building worlds with it. And I've done that since then, in different engines, but just kind of stayed with that. So there are some things for technical and IP and creativity. Jamin Warren: Yeah, yeah, yeah. Jacob Kudsk Steensen: There's also Unity, but the last thing that's really cool about Unreal is that because it comes from Epic, they made this game called Gears of War at some... I mean, this very like generic blockbuster title, whatever, but it's just like... so every time they launched a title, they would just essentially, they would just provide creators with like tools that take millions of dollars to create, and they just give them to you. And so you could just kind of pull off that a bit. And so they made this game Gears of War 1, what it did is that it allows you to have a really complex 3D world, have a timeline and a sequencer where you can control loads of different parameters at the same time. You can kind of like visualize the whole transformation of the three-dimensional world as you're in it. And that kind of clicked with me because I was like, "Oh, I can have someone work on music over there. I can start working on animation over here." And you can have all these elements and you can work on them synchronously, which adds to this instrumental music band way of working that is how I create work now. I work very kind of orchestral in a way. I mean, that's how I think about it. Jamin Warren: Yeah, yeah, yeah. I mean, it does... like, it is this transformational technology, I think, in terms of enabling small teams to do things with—and that wasn't always the case. It used to require, you know, as you know with game engines, you needed a lot more people and they were often proprietary and you needed big teams to actually work with them. And it's been interesting to kind of see we're kind of back to this moment where artists can actually put together something really profound and uniquely expressive with the same, the exact same thing that is used to basically ship these like enormous, enormous games as well. So I do think we're at a really interesting, really, really interesting inflection moment as it relates to video games and the creative process. Well, let's talk about Otherworlds. You know, Otherworlds is at Phi Centre in Montreal. If you're in Montreal or in Canada or northeast part of the United States, you can make it up to Montreal, you absolutely should. You know, it's an interesting project for you because it basically kind of treats your archive as the terrain. You've kind of been—one of the things I've liked about your work is that you sort of approach these kind of same questions, albeit with different tools, but often looking at these kind of same questions about how we interface with the natural world, trying to ask these deeper questions. And so all these past projects sort of are like chapters or levels or stanzas. When you're inside a body of work like this large, how does it feel to like look back at all the things that you've done? Jacob Kudsk Steensen: Um, yeah, I got this question a lot at the opening or we did a talk, and it's kind of difficult for me to answer, but it actually came from one of my collaborators I work with a lot now, Lou O’Neil, which is like... he joined my works five years ago and he said that it made him look a lot different at older works. And so for me, it's like I'm looking at individual pieces on their own kind of a bit the same as they're created, but the combination of them just creates a different modality. It's like you walk from documentation of underwater volcanoes to a collapsed glacier cave to songs of an extinct bird, from virtual reality to like an interactive kind of documentary-style game on an app to a purely sculptural room. And so there's something about that journey and kind of pick and choose in a bit that I really like about it, because it's like, if you're to see all of it you'd have to spend like 10 hours or something. So audiences are going to sort of drift through some worlds and then find something—a narrative, a concept—that works, or visuals or sound or whatever, that they feel like, "This is where I want to dive in." And that's the feeling I have when I do environmental fieldwork and work with these biologists and people in places. It's like you go to some of these landscapes and you can have an infinite amount of information of species—I mean, rocks, lichen, whatever—there's like an infinite amount of information from a scientific perspective you have about the place. So you have to find "What is my entry point? What can become an interesting artwork here?" So you kind of drift around, you have some knowledge, and then I dive into something. And then I go mega, mega, mega hard on that one thing. Recorded it, digitized it inside-out. I mean, like, I've spent months in some of these locations that I work with. Jamin Warren: Yeah, yeah. Was there anything that like surprised you looking back? Jacob Kudsk Steensen: Yeah, there was one thing, and that's I intentionally brought in this thing I made... actually with Cry- I did a slight modification of Far Cry 2 with the physics, so there are some island slopes you can't get up. So I put the player outside a little island, so you're kind of like swimming towards this island before you drown, but the slope is too steep. So you as a player you hit this curve and you go around it and then you drown, and then you're kind of resurrected and you have to swim to it. So it was kind of my early way of trying to be like a conceptual artist. You know, I mean, this is... I don't even... that's a while ago. 17 years ago or something. It's like, that was actually really emotional for me to see because I remember thinking at the time like, you know, I was trying to think like, what do you do as an artist? I mean, how do you make a living, what do you do, what do you... and I was like, "Oh, I'm going to try to do something really purely conceptual in here. Something beyond like production or even a game." It's not even a game—I mean, there's a little recording of this game on a monitor, right? This is like... I can't even... I don't even have the game anymore. I have a video recording of the gameplay. And it's this swimmer on this island drowning and swimming to it. It's so basic. But like, but I almost got chills, the feeling that like, "Oh, now I'm an artist doing this." And I collaborated and showed in museums and everything, and it's like at the time that would be my wildest dream. So it's very emotional seeing that, this journey. But also nice going back to some early works because I have some earlier that was more kind of story-driven and gamey. And it was really nice seeing those because I haven't done those so much the past five years. I’ve done these like big musical immersive worlds. So it was cool going back and seeing these like hybrid gaming interactive installations and maybe I'll make a new one soon or something. There was something that I felt was more like in the moment in a way—like there's lots of artists doing these installations. And I've had the feeling for some years that like, "Oh, but that's kind of weird to me to do that again because I've done that before." But now I'm kind of attracted to that playful, hacky modality gamey format. So maybe I'll make... I don't know, but maybe I'll... I was kind of inspired by that, seeing some old works too. This is not just to be self-referential, but it was... it's a cool feeling. I mean, it's a cool feeling. And also because the technology has changed so much. So literally how the work is looked and done is so different. Jamin Warren: Right, right. And also that's like a testament to, you know, for you consistently approaching games as a place where you could explore ideas. Because that's not always true. Sometimes people dip in, they do one project, and then they they move on. It doesn't become a formative part of their their practice. Jacob Kudsk Steensen: I love the expressive quality that's possible in real-time 3D. It's incredible. I mean, it's amazing. So painterly. Jamin Warren: Yeah, yeah, of course. Well, you know, along those lines, you know, some of the language around how Otherworlds is being presented uses games pretty, pretty explicitly. I believe like "levels of an unfolding video game" is in the, in some of the promotional material for the Phi description. But at the same time, you know, you're... you don't make, you know, commercial video games in that, you know, sort of in that traditional sense and at times have kind of refused some of the mechanics that you might find in games. What do you feel like you are like keeping from games? You know, like what are the things that you look at games like, "Yeah, I want to keep that," and then what are things that are like, "Actually I don't need that in my in my work right now"? It's often because you're sort of in between these spaces so you do have to make these decisions around like, you know, what you would like to keep and what you would like to let go. Jacob Kudsk Steensen: Yeah, I mean I would like to make a more interactive game again soon. But the main thing is that the process of creation is very fluid when working with a game engine. So that's one thing that attracts me a lot. You have different artists, people, perspectives, sound, music, animation, whatever, architecture, that has to work together. But if one piece of code breaks the whole thing breaks. So literally when you compile the software, all of it has to communicate. If one breaks, the entire thing will break. It will not compile. You cannot run it. And so that means that by the nature of that, you have to work quite unilateral. And there's a culture in game development of... you know, you have a hunch, you have an ambition, but you can't fully describe the experience of it. Because you have to go through kind of a human creation process to hit the feeling of playing it. And I'm very inspired by that. So if I create, I mean, there's also a VR work in the show from eight years ago, I work with this composer Michael Riesman that is the key collaborator of Philip Glass, we became friends in New York, and he built like this generative music instrument that I put inside the VR. And there's like a moment to some people, just some, where they... they almost cry, and like, I received letters from people who have experienced this VR work about this music and this extinct bird. But to get to that there's just this crazy iterative process. It's like five scenes in the work, different colors, sound, light, and there's a sense of like resonance with that between me making the 3D worlds and work with Michael on the music that had to sort of fit together to create that embodied experience. And that's really from the creation what I really like about games. It's like, you talk to people in industry, more or less, that is very playful. And the playfulness in the process is what gets you, even if you make a game with good gameplay, it's what gets you to the gameplay. I mean, unless you're a big blockbuster studio that like repeats the same gameplay all the time. At in the beginning they had to find that, you know? So you... and so that really attracts me. But making kind of interactive explicit playable game experiences at museums is something... I did some VR works that was perhaps my way of doing that like eight years ago because what I like there is that um the barrier—I mean regardless of if I like VR now or not and so on—it's like it's a virtual three-dimensional world, there's a space changing and all you need to be able to interact with that 3D world... I mean, okay you put something on the head but then you just use your body. So ironically the barrier to entry is so much smaller than a video game controller in an exhibition. This is so much harder for most people. Unless you're like gaming native and then it's like if you go into that space then you make something really niche and I do that- I am working on some projects to doing that and kind of appreciate that culture. But I took that away for some years and instead I went into this the more like orchestral and collaborative dimension of video game technology to build large-scale immersive exhibitions to the point now where we have like a custom sound system, we digitize exhibition architecture, we put it in a game engine, we move like 30 speakers around, they are triggered by virtual worlds and effects and I use like a really complicated generative almost like an open-world mechanic where everything changes that actually runs behind the scenes and then it pivot out in a local area network to multiple computers that connect screens and speakers, glass sculptures and all this stuff. So I've kind of tried to build out a very ephemeral way of experiencing something in proximity to the experiences I would have sitting in 3D software or if you know how to play games but also a little bit for people who may never have had an interactive three-dimensional experience. I mean a lot of people really when you deal with large museum crowds—I mean these are shows some of them is like 30,000, 100,000 visitors—there's some considerations to occupy that space that for me at least, I genuinely want people to enter these environments. You know, I so accessibility is actually a big thing for me. So that's that's the thing I kind of took out was more explicit like gameplay in that sense for some years at least. But that was cool seeing some older works too where it's actually there. And I was like, you know, this is actually really nice.