The Japan News by The Yomiuri Shimbun

Home > Opinion > Science



The Stage of Co-Creation Opened Up by the Shadow Media:
Performances at the Genoa Science Festival

Yoshiyuki Miwa
Professor, Faculty of Science and Engineering, Waseda University

Awareness of the scene achieved through the body

Several years ago, our laboratory had an opportunity to experience a scene where the community consciousness of We was created. This took place at the site of the inclusive dance presided over by Ms. Yoko Nishi, Professor at Toyo Eiwa University. The inclusive dance is a physical activity by which various people—regardless of whether they are physically challenged or not, regardless of their age, gender, or dancing experience—cooperate with each other for creation, expression, and interaction through the movement of their bodies. Here evolves the world that is described in a poem by Misuzu Kaneko as "All different, all just right." And the intermediation by effects of the scene triggers cyclical creation of expression—as in wrapping while wrapped, wrapped while wrapping—leading to the generation of a story.

In order to sense the scene, the body itself needs to function as a sensor. The existing media and communication systems, however, have not succeeded in incorporating the effects of the scene. This has caused a dilemma, as seen in the case of cellular phones, for example: as functionality for individuals becomes more and more robust, the bonds among family members and friends as well as the existential sense of place are weakening. How should this problem be solved by engineering? This question also means working out techniques for broadening communicability—that is, the possibility of making connections. In the inclusive dance mentioned above, dancing skills are not crucial. What is important, rather, is that we ask ourselves how we currently are and realize something at the scene of expression in order to connect to various people with different backgrounds and values. Generating We from I requires awareness at the scene achieved through the body.

Interaction with the shadow media

As a means of technologically assisting in the creation of a scene of expression with awareness, we decided to utilize shadows of bodies. Specifically, we process the figures of human bodies captured by thermocameras to create shadow images. Transforming them in various ways, we produce shadows that differ in shape from the actual body shadows, which we call the shadow media. We project these shadows on the screen from below as if they were actual shadows of the bodies. Then, all the people seeing them naturally begin to move their own bodies, and interaction with the shadow media is underway. Whether they are aware of it or not, there emerges a gap between the sense people have of their actual bodies and the shadow media that is their shadows. Observing this, we discovered that the reflection of expression which is normally felt internally was externalized as expression through their own shadows, and this in turn raised a clear awareness of their bodies, enhancing body sensitivity and facilitating the creation of images. From another point of view, the shadow media can be regarded as predicate and verbal media that is different from existing encoded media, with the effects of the scene bringing out the nominative or the subject. This has also been confirmed by our recent studies of groups of kindergarten children showing that minds are mutually connected through the shadow media.

The slit-screen projection system

In February 2010, we received from the host of the Genoa Science Festival (*1) an invitation for our physical performances and experience-based presentation with the shadow media. This world-famous festival is based on the spirit of community building by sciences, and features the participation of everyone—ranging from children to the elderly, together with scientists and artists—in the creation of a single scene for the province as a whole. So, we advanced our studies one step further and tried to present the possibility of building a scene of co-creation through the integration of shadow media with the physical performances on the stage in the Bolsa Palace, the venue for the festival. Our plan was that the space of the shadow media generated from the physical expression of the performers involved audiences—many of whom were inhabitants of Genoa—to make the entire hall an inclusive place. For that purpose, it is important for audiences to join in on stage (the scene) to create expression together in the form that the existence of the audiences affects the stage (the scene) of the shadow media, consisting of the physical expression of the performers, as well as images of the stage (the scene), which are evoked among the audiences. To this end, we contrived the slit-screen projection system that enables performers and audiences to be located on either side of the shadow media space and to pass through to the other side (Figure 1). This system incorporates a broad range of ingenious features, including separate processing of shadows fallen on the floor through the slit and shadows reflected on the belt-like screen, and multiple projection from both the stage and the audiences. This mechanism enables the shadows of performers and individual items to be displayed not only on the screen and on the floor of the stage, but also on the auditorium, walls, and ceiling.

Figure 1: The new method for projecting shadow media images (concept)

Dual 2010—Ancestors plant trees, descendants rest under the trees

Using this system, eight performers including two in wheelchairs delivered a performance entitled Dual 2010: Shadow Awareness II in the motif of the phrase "Ancestors plant trees, descendants rest under the trees" five times over three days, from October 29th (Figure 2, see (*2) for its movie). Thanks to direction by Ms. Yoko Nishi, the excellence of the performers, and the efforts of Mr. Shiro Itai and all of the other laboratory staff, the performance attracted increasingly larger audiences over time, and at the last show they played to a completely packed house. In particular, we were delighted that all the staff could witness the involvement in the scene of expression created by the shadow media which yielded an exchange of the minds among the performers and the audiences along the storyline of the performances.

When it comes to thinking of expression, we tend to pay attention only to the visible outer world. I believe, however, that the essence of expression is instead found in encounters and connections through effects of the invisible inner world. It seems that the technology of the shadow media is calmly appealing to us for the importance of communicability, which has recently been vanishing from society.

Figure 2: Scenes and all the staff of Dual 2010: Shadow Awareness II


Yoshiyuki Miwa
Professor, Faculty of Science and Engineering, Waseda University

1971: Graduated from the Department of Mechanical Engineering, the School of Science and Engineering, Waseda University
1976: Received his doctoral degree from the Graduate School of Science and Engineering at Waseda University, and became an Assistant at Waseda University
After serving as a full-time lecturer and an Assistant Professor;
1981: Became a Professor in the Department of Modern Mechanical Engineering in the School of Creative Science and Engineering at Waseda University
Currently, a Professor in the Department of Modern Mechanical Engineering in the School of Creative Science and Engineering at Waseda University
Doctor of Engineering

(1) Yoshiyuki Miwa (contributor), "The Shadow System [Kage Sisutemu]" in Encouragement of Integral Studies ["Togogaku" e no Susume] , Koyo Shobo (2007)
(2) Yoshiyuki Miwa (editor and author), Embodiment, Communication, and Mind [Shintaisei, Komyunike-shon, Kokoro] , Kyoritsu Shuppan (2007)
(3) Yoshiyuki Miwa and Ken Yabuno, The Book of Wabot: The Evolution of Robots and the Future of Humans [Wabotto no Hon: Robotto no Shinka to Ningen no Mirai], Chuokoron-Shinsha (2002)
(4) Yoshiyuki Miwa (contributor), "Bio-Communication in Co-Creation [Kyoso ni Okeru Seimeiteki Komyunike-shon]," Ba and Co-Creation [Ba to Kyoso], NTT Publishing, (2000)
(5) Yoshiyuki Miwa (editor and author), Biomechanical Engineering [Seimei Kikai Kogaku], Shokabo (1992)