To Make the Dream Cube, It Took Mashing up Technologies (and Inventing New Ones)

At the upcoming Digital Signage Expo (February 26-28) in Las Vegas, there will be a lot of talk about integrating electronic media into the designed environment. On the 26th, at a full-day session called “New Design Directions: Dynamic Digital Environments,” Michael Luck Schneider, senior designer at ESI in New York will discuss, in some detail, the collaborative effort it took to create the Dream Cube in Shanghai. I asked him about how his global team worked together as they communicated between Cologne to Sydney to Beijing and points between; and the ways and means of systems design. In my previous interviews on the topic of media-rich environments, panelist Paul R. Levy, president and CEO of Philadelphia’s Center City, talked about the use of digital media in the large-scale urban environment; Jeff Kovel, AIA, principal at Skylab Architecture in Portland, Oregon, discussed his firm’s experience in building Camp Victory for Nike. Here we dig down into ESI’s interactive spectacular, designed, as Schneider says, to “demonstrate the power of collaboration in shaping a more sustainable future.”

Susan S. Szenasy: Let’s use your project, the Dream Cube in Shanghai, to discuss how media rich environments come together, and how this project relates to other, previous work at ESI. Firstly, the global communication part. How did that work? Describe the expertise on the team in NYC and Shanghai, and another location, if any. Then talk about how the team came together.

Michael Luck Schneider: At ESI we always start the design process by imagining a project from the audience’s perspective and thinking about what their experience should be. Once we defined the vision for the Dream Cube—to create an interactive spectacular that would demonstrate the power of collaboration in shaping a more sustainable future—the next step was to find the partners that help us produce it. Through an open bid process we ended up with a pretty incredible international team based in in New York, Cologne, Sydney, Hong Kong, Shanghai, Beijing, and Los Angeles. The first partner to come onboard was the lighting design group Full Flood, based in Los Angeles. As the visual content of the Dream Cube really came to life through projections, LED and LCD displays, light was a key medium to create a unified experience, and Full Flood was instrumental in the process from an early stage. Spinifex Group from Sydney then came on to translate our design into animations and develop the majority of the audio-visual content. They’re unique in having a deep understanding of the technology that drives media experiences, which enables their team to build amazing stories for unique canvases like the Dream Cube. They also worked with a team in Hong Kong to produce local Shanghai video shoots and compose the experience’s musical themes. PRG Germany joined as the systems integrator, and they oversaw the design and implementation of what was a highly complex and groundbreaking technological system. Pico in Shanghai supported our designs and mockups. They also did the final exhibit fabrication and installation. With this international group, having face-to-face meetings with everyone at the table was essential to establish a shared view of the project and common goals. Once these were defined, we were able to use digital tools to review and track process in all quarters, and we held weekly calls to ensure clear communication throughout the process. Spinifex also developed a model that enabled us to view media mapped to a 3D visualization, which turned out to be invaluable. In addition, we had two full-scale mockups built—one in Shanghai and one in Berlin—that enabled us to review and refine the project in person at critical stages.

SSS: Talk about the existing software products that you worked with. How you found them, what do they do best, what needed to be developed for the project, and how the manufacturers can help in this development. I’d like to know how you integrated various products and who did what (what kind of expertise, background do they need for this work?).

MLS: Once we’ve detailed the design for an interactive media experience, the next major step is deciding whether we can adapt existing off-the-shelf software or whether we have to create a fully custom solution. To make this decision, you have to know what’s already out there in the market, how far you can push it by making modifications, and when you have to develop custom software to deliver the experience you want. With the Dream Cube, we were able to use some commercially available solutions as basic building blocks, but in many cases we had to extend their capabilities and customize them or write entirely new software to support such a complex, interactive system. This responsibility was shared by Spinifex and PRG. Spinifex began by creating a pre-visualization tool in Flash that enabled the team to see video mapped to a rendered space. They then used open frameworks to custom build interactive applications that drive 3D animations on the façade and let visitors interact with LEDs through sound and motion. Meanwhile, PRG was creating a custom media playback system using Insynergie to deliver synchronized content from over 200 PCs throughout the Dream Cube. They developed the system to feature video tracking, motion sensors, and other triggers that enabled visitors to interact with LEDs through gestures and other actions, resulting in a highly interactive space. A custom show control interface was built both in English and Chinese so docents could customize each show to audience size.

SSS: How does working in a design firm like ESI, known for bringing together many different kinds of experts to serve a project, help in putting media-rich environments together? Is there a new kind of learning curve? I’m particularly interested in how architects respond to this kind of work, what they know really well, and what they need to learn to take advantage of such an expert team.

MLS: I think that one of the really unique things about ESI is that we have a commitment to collaboration as a core value; both in the experiences we create and the process that gets us there. We enjoy working with architectural teams to identify ways in which a space might become more dynamic or be transformed through interactive or media-rich storytelling. By collaborating at an early stage, we can share ideas about the full range of technologies available and work together to create a seamless media experience where physical elements such as LEDs, rods, or mesh become integral components that enhance the architecture. We can also be very effective in partnering with architects on projects along a continuum of developmental stages. We love partnering at the beginning of a project like the Dream Cube to co-create an integrated experience. We’ve also been able to come in very late in a site’s development to create media and content experiences that work with the system in place, like we recently did with Barclays Center in Brooklyn. Whatever stage we become involved at, though, our goal is essentially the same: to integrate storytelling, hardware, and software elements at a systems level and create authentic, interactive environments that engage, inform, inspire, and activate audiences.

SSS: Everyone talks about systems these days, but I have the feeling that none of us, including designers, understand the full impact of what systems integration can mean to a successful, sustainable project. When systems integration works, how does it work, really? And what needs to be changed in designers’ methods, in addition to the mantra of collaboration.

MLS: The trickiest thing about systems integration is that if you want to be innovative, you have to have two things. The first is a systems integrator who can be really flexible and work with you to realize your design. Part of this is just engaging the integrator in your vision and then building trust by collaborating closely throughout the process. The second essential ingredient of systems innovation is that you have to have the ability to experiment. You have to be able to test out new ideas to see how your technologies fit together, what will work reliably over the long term, and what needs to be modified as you move forward. At ESI, we have a lab that lets us mashup existing technologies, invent new ones, and then thoroughly test everything so that when we incorporate a piece of leading-edge media in a project, a client knows it will create a transformative experience and, just as important, that it will work. It is important to have a highly iterative process with R&D built in. To create a unique project like the Dream Cube, you have to have a clear starting point that the systems integrator and the designer agree on. Once the production phase begins, you also need a proof-of-concept period with numerous mockups through which the designer and the systems integrator collaboratively determine the final systems technology. With the Dream Cube, we started with a design and through this process, ended up with a final as-built system that had evolved by maybe 60 percent and delivered a truly singular and seamless media experience.

SSS: And finally, the outcomes. Describe what the most proud moment was in seeing the Dream Cube work. And what would you do differently if you were designing the Dream Cube today, knowing what you learned from the project?

MLS: In all honesty, it’s hard to pick just one. For starters, the sheer number of people who came and experienced the Dream Cube is pretty staggering. We had 4 million visitors come over six months, which meant there were 500 people moving through every 15 minutes, and as a designer, I have to say I’m proud that our system and technology performed continually with that kind of use. If I had to pick a single moment, though, it would probably be from the first day the pavilion opened and people went through recording every second of the experience with their mobile phones. There was also the moment—as I said, it’s hard to just pick one—when people went into the 360-degree theater that controls the LED façade for the first time. The client was concerned that because Chinese people tend to be more publicly reserved than, say Americans, visitors might not embrace the interaction that generates the spectacular outside. So we all kind of held our breath. Then when we saw everyone not just responding to the invitation but participating enthusiastically and authentically, it was incredibly rewarding. I think that if I were to do it again today I might add one more reveal moment into the experience. The first reveal of the Dream Path at the top of the escalator and the final reveal of the 360-degree theater always created the most Oohs and Aahs and really allowed us as designers to control how the story unfolded. I think adding an additional reveal could have made a slightly tighter experience, but overall, with the amazing team we worked with, we were able to accomplish something that made us all incredibly proud.