“From its earliest days, new media art looked to cinema as one of its major inspirations. Future Cinema heralds the next phase in new media’s evolution that will create an aesthetic space and social place of profound immersive interactive experiences.” — Jeffrey Shaw, Project Director

“Future Cinema Systems is a vital project for building the future of Art Tech in Hong Kong and it is a privilege to be able to collaborate with Professors Shaw and Kenderdine under the leadership of HKBU, and for our talented faculty and programmers here at CityUHK to make a vital contribution to its success.” — Richard Allen Co-PI

Future Cinema Systems: Next Generation Art Technologies is an unprecedented Hong Kong art tech initiative to build the next generation fully interactive and immerse cinematic architecture and applications. The project is spearheaded by Jeffrey Shaw, Chair Professor of the Academy of Visual Arts, Hong Kong Baptist University, in collaboration with Richard Allen at the Center for Applied Computing and Interactive Media at City University of Hong Kong (CityUHK) and Sarah Kenderdine at L’École Polytechnique Fédérale de Lausanne (EPFL) in Switzerland. This pioneering project has been awarded HK$35.4 million in funding from the Innovation and Technology Support Programme under the Innovation and Technology Commission of the HKSAR Government.

Technical Architectures

Future Cinema Systems will consist of three technical architectures. The first of these is a high resolution, 360 degree, LED panoramic, stereoscopic projection environment. This state-of-the-art facility, which has been built at HKBU’s new Visualization Research Centre is the first of its kind in the world. 8 metres wide and 4 metres tall, it contains 400 LED panels (26 million LED pixels) and is supported by a 32.4 channel surround sound system that allows for a fully immersive audiovisual experience. LED illumination affords unprecedented screen brightness and a seamless visual array within the 360 space of virtual reality. Furthermore, it allows participants to be fully visible to one another, and, unlike the projected image, casts no shadows. HKBU’s facility combine with a newly upgraded Gallery 360 Space at ACIM. ACIM’s new space contains a towering cylindrical screen measuring 4 meters in height and 9.5 meters in diameter, driven by five Barco F80 Projectors and a supercomputer (with cutting-edge graphic cards), enabling real-time interactive rendering at resolutions up to 12800x3200. Immersion is further heightened with integrated multiple tracking systems, including Position Tracking and Head Tracking via the HTC Vive tracking system, as well as human body recognition via Azure Kinect Sensors. Other advanced tracking systems like Eye tracking sensors and EEG sensors are in development, and will be integrated into Gallery 360 in the future.

These two arenas are complemented by a high-resolution 360 facility at EPFL to allow audiences at each location to telematically connect, sharing content and personal interactions in real time. The second architecture is a Human-Computer Interaction Engine. This consists in an integrated array of sensing inputs which will identify, correlate, and integrate a wide range of sensory cues from the participating spectators. CityU faculty members Miu Ling Lam (SCM) and Rosa Lam (EE) are involved in building the HCI engine. The final piece of the technical jigsaw is the Co-evolutionary Narrative Engine. Bespoke software allows the sensory inputs to be organized, orchestrated, and variously prioritized. It ensures that the audio-visual manifold seamlessly reacts and responds to the active and passive sensory inputs of the participants over time. In this way, the experience of the spectators will be rendered fully interactive and immersive; spectators are turned into full participants in an unfolding drama which they co-create. ACIM programmers Leoson Cheong and Sky Suen are playing a central role in the development of this architecture.

Aesthetic and Cultural Heritage Applications

To fully utilize these technical frameworks, the Future Cinema Systems’ collaborators will develop several applications or demonstrators. The first of these is the Future Experience of Archive. This project will present novel designs for browsing, navigating, and immersively experiencing data sets of digital content, such as photographs, movies, paintings, sculptures, and animations. The second, will develop tools for rendering cultural heritage, past and present, fully accessible and navigable in immersive space. The third, will facilitate the development of new forms of interactive art and interactive public participation in real-time mixed reality immersive visualizations. SCM faculty members RAY LC and Hector Rodriguez are creating interactive art for Future Cinema, along with several new media artist alumni from SCM’s programme.