Image Beyond the Screen E-Book
139,99 €
Mehr erfahren.
- Herausgeber: John Wiley & Sons
- Kategorie: Geisteswissenschaft
- Sprache: Englisch
Videomapping with its use of digital images is an audiovisual format that has gained traction with the creative industries. It consists of projecting images onto diverse surfaces, according to their geometric characteristics. It is also synonymous with spatial augmented reality, projection mapping and spatial correspondence. Image Beyond the Screen lays the foundations for a field of interdisciplinary study, encompassing the audiovisual, humanities, and digital creation and technologies. It brings together contributions from researchers, and testimonials from some of the creators, technicians and organizers who now make up the many-faceted community of videomapping. Live entertainment, museum, urban or event planning, cultural heritage, marketing, industry and the medical field are just a few examples of the applications of this media.
Sie lesen das E-Book in den Legimi-Apps auf:
Seitenzahl: 402
Veröffentlichungsjahr: 2020
Ähnliche
Table of Contents
Cover
Foreword
Introduction
PART 1: History and Identity
1 The Origins of Projection Mapping
1.1. Introduction
1.2. Let’s moonwalk! A short crossing through time
1.3. Immersion in hallucinated worlds
1.4. Examples of visual devices
1.5. The agencies
1.6. A figure of transgression and juxtaposition with a beyond
1.7. The invention of an “empty box” as an image container
1.8. Modern inflexions: obsolescence of old visual devices and tacit challenges to the Albertian model
1.9. Parastatic scenography
1.10. From expedition to investigation
1.11. Conclusion
1.12. References
2 The “Spatialization” of the Gaze with the Projection Mapping Dispositive
2.1. Introduction
2.2. The release of the “cinematographic cocoon”
2.3. Changing the projection mapping dispositive
2.4. The spatialization of the gaze or the perception of the projection mapping spectator
2.5. “Attractions set-up” or real content?
2.6. References
3 Projection Mapping: A New Symbolic Form?
3.1. Introduction
3.2. A shifting tool
3.3. The surface
3.4. The projection
3.5. Conclusion
3.6. References
4 Points of View: Origins, History and Limits of Projection Mapping
4.1. The origins of a movement towards alternative forms according to Romain Tardy
4.2. A short history of projection mapping according to Dominique Moulon
4.3. Projection mapping and its limits according to Christiane Paul
PART 2: Texts and Techniques
5 Listening to Creators in Residence
5.1. Creators, a residence and a festival
5.2. Capturing the genesis of a work
5.3. REMIND: a method to capture the dynamics of the situated creative experience
5.4. Space, tool and solitude
5.5. New residence arrangements
5.6. Prospects for the future
5.7. Increased attention to the place of creators in digital arts
5.8. Acknowledgements
5.9. References
6 Projection Mapping and Automatic Calibration: Beyond a Technique
6.1. Introduction
6.2. Towards a new projection dynamic
6.3. Automatic calibration
6.4. Automatic geometric calibration
6.5. Projector calibration using one or more pre-calibrated cameras
6.6. Automatic calibration applied
6.7. Automatic calibration in France
6.8. Conclusion
6.9. References
7 Projection Mapping Gaming
7.1. Introduction
7.2. Specifying the scope of the projection mapping game
7.3. The indoor projection mapping game
7.4. The outdoor projection mapping game
7.5. Conclusion
7.6. References
8 Projection Mapping and Photogrammetry: Interest, Contribution, Current Limitations and Future Perspectives
8.1. Introduction
8.2. State of the art
8.3. Photogrammetry for projection mapping
8.4. Contribution: an automated imaging device for object photogrammetry
8.5. Current limitations and future prospects
8.6. References
9 Points of View: Sound, Projection and Interaction
9.1. Sound creation projection mapping, a real composition of sound
9.2. Projectionist: a profession according to Pascal Leroy
9.3. Interactive projection mapping by Anne-Laure George-Molland
9.4. References
PART 3: Production and Dissemination
10 The Factory of the Future, Augmented Reality and Projection Mapping
10.1. Introduction
10.2. The factory of the future
10.3. Augmented reality
10.4. Factory of the future and augmented reality
10.5. Augmented reality and projection mapping
10.6. Future plant and projection mapping
10.7. Conclusion
11 Heritage Mediation through Projection Mapping
11.1. Introduction
11.2. The symbolic value of heritage
11.3. Projection mapping as a means of cultural heritage mediation
11.4. Conclusion: monumentalize the monumental
11.5. References
12 Projection Mapping: A Mediation Tool for Heritage Resilience?
12.1. Introduction
12.2. Architecture, a heritage trace and an art to be preserved
12.3. The architectural heritage between preservation and mediation issues
12.4. Meeting between architectural heritage and projection mapping
12.5. Classification of architectural projection mapping
12.6. Meeting between architecture and projection mapping
12.7. Conclusion
12.8. References
13 Architectural Projection Mapping Contests: An Opportunity for Experimentation and Discovery
13.1. Introduction
13.2. Different projection mapping projection contexts
13.3. Interests and functioning of the contests
13.4. Analysis of the 2018 season
13.5. Conclusion
14 Points of View: Supporting and Highlighting Projection Mapping
14.1. Video Mapping European Center according to Antoine Manier
14.2. Lighting design and sustainable projection mapping installations according to Alain Grisval
List of Authors
Index
End User License Agreement
List of Tables
Chapter 13
Table 13.1. Calls for festival projects published on the Facebook Group List of ...
Table 13.2. Festival winners published on the Facebook group List of internation...
List of Illustrations
Chapter 1
Figure 1.1. Evolution of the queries of the keyword “video mapping” in French on...
Chapter 2
Figure 2.1. Laterna Magika, Cube, director Pavel Knolle, National Theater of Pra...
Figure 2.2. Creative KAOS Management, Celebremos Campeche projection mapping on ...
Chapter 5
Figure 5.1. The resident is equipped with an eye-tracker during a work session (...
Chapter 6
Figure 6.1. Launch of Modulo Kinetic Designer, at ISE (photo credit: Holymage, 2...
Chapter 7
Figure 7.1. Different types of displays associated with video games (source: Alv...
Figure 7.2. Game console: Projector Mega Video Game/Light Games (source: Playtim...
Figure 7.3. RBG Racers (source: Play-Foul 2014)
Chapter 8
Figure 8.1. The rotary table (source: Nicolas Lissarrague). For a color version ...
Figure 8.2. The control box (source: Nicolas Lissarrague)
Figure 8.3. The complete system during a shooting session (source: Nicolas Lissa...
Figure 8.4. Contact sheet of some of the photos taken during a shooting session ...
Figure 8.5. 3D model obtained by photogrammetric reconstruction (source: Nicolas...
Figure 8.6. Diffuse map (source: Nicolas Lissarrague)
Figure 8.7. Alpha map (source: Nicolas Lissarrague)
Figure 8.8. ZDepth map (source: Nicolas Lissarrague)
Figure 8.9. Projection of the Diffuse texture retouched (left: view from above o...
Chapter 10
Figure 10.1. The “factory of the future” approaches in the world. For a color ve...
Figure 10.2. The “factory of the future” approach in Europe
Figure 10.3. H. Heilig’s sensorama
Figure 10.4. Steve Mann and the Eye Tap (source: Steve Mann Photo: Richard Howar...
Figure 10.5. Milgram Continuum
Figure 10.6. Some key dates in augmented reality (source: computer graphics by M...
Figure 10.7. Three views of a video mapping assembly device for a motorcycle eng...
Figure 10.8. Airframe assembly video mapping
Chapter 11
Figure 11.1. Cité Mémoire, a Montreal project. Various projections illuminate th...
Figure 11.2. Parabolic Gaudí at Palau Güell, 2018 (source: Playmodes Studio)
Figure 11.3. Frescos of the apse of the church of Sant Climent de Taüll, in Cata...
Figure 11.4. Terracotta warriors rediscover their colors in the exhibition “The ...
Figure 11.5. Memory Collector highlights the structure of the first collector se...
Chapter 12
Figure 12.1. Museum of Scientific Arts, Singapore (source: https://archibat.com/...
Figure 12.2. Vuitton Foundation
Figure 12.3. Remains of Timgad (source: Araibia Amine)
7
Figure 12.4. Projection mapping (source: A. Pogoda)
Figure 12.5. Madmapper, projection mapping software
11
Figure 12.6. Celebration of lights in Lyon
12
Figure 12.7. The Lexus advertising campaign
Figure 12.8. Lyrical perspective in Lyon (source: 1024 Architecture)
Figure 12.9. Projection mapping on the astronomical clock in Prague (source: 102...
Figure 12.10. Mapping on the works of the painter Murillo in Seville
Figure 12.11. Projection mapping of Seville City Hall
Chapter 13
Figure 13.1. (a) Overview of the various contest projects at the Circle Light Fe...
Figure 13.2. Distribution of the number of festivals by continent in 2018
Figure 13.3. Nationality of the European winners in the 2018 contests
Guide
Cover
Table of Contents
Begin Reading
Pages
v
iii
iv
xiii
xiv
xv
xvi
xvii
xviii
xix
1
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
69
70
71
72
73
74
75
76
77
78
79
80
81
83
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
107
108
109
110
111
112
113
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
159
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
229
230
231
232
233
234
235
236
237
238
239
241
242
243
244
245
246
247
Series Editor
Sylvie Leleu-Merviel
Image Beyond the Screen
Projection Mapping
Edited by
Daniel Schmitt
Marine Thébault
Ludovic Burczykowski
First published 2020 in Great Britain and the United States by ISTE Ltd and John Wiley & Sons, Inc.
Apart from any fair dealing for the purposes of research or private study, or criticism or review, as permitted under the Copyright, Designs and Patents Act 1988, this publication may only be reproduced, stored or transmitted, in any form or by any means, with the prior permission in writing of the publishers, or in the case of reprographic reproduction in accordance with the terms and licenses issued by the CLA. Enquiries concerning reproduction outside these terms should be sent to the publishers at the undermentioned address:
ISTE Ltd
27-37 St George’s Road
London SW19 4EU
UK
www.iste.co.uk
John Wiley & Sons, Inc.
111 River Street
Hoboken, NJ 07030
USA
www.wiley.com
Cover image: Copyright of Cité des Électriciens, Bruay-La-Buissière (France) © Rencontres Audiovisuelles – Video Mapping Festival 2018.
© ISTE Ltd 2020
The rights of Daniel Schmitt, Marine Thébault and Ludovic Burczykowski to be identified as the authors of this work have been asserted by them in accordance with the Copyright, Designs and Patents Act 1988.
Library of Congress Control Number: 2019952956
British Library Cataloguing-in-Publication Data
A CIP record for this book is available from the British Library
ISBN 978-1-78630-504-6
Foreword
It is a well-worn idea that this century is one of an abundance of images. It is another to say that we live in an era of screens: for a long time, we had two in our lives – cinema and television. We then went from three, with the computer screen that opens up access to the immense resources of the Web and the Internet, to four with mobile tools – smartphones. Some are pursuing inflation by adding five (the intermediate terminal that is the tablet), six (the games console) and seven (the immersive helmet). Little by little, screens are saturating all the dimensions of our private but also public spaces, as we can see with the recent replacement of traditional advertising billboards by screens in our developed cities – this, moreover, without any consideration for the ecological impact that this mutation implies. The screen is everywhere.
This book opens a much less widespread space: the space of the image beyond the screen. From monumental projections on façades to more intimate projections on sets or objects. Is it worth stopping by? Isn’t this just a simple change of media? An additional avatar in the diversification of screens?
Juxtaposing chapters by researchers and testimonies by artists and practitioners experienced in this new form of projection, the texts gathered in this first book on projection mapping show in various ways that, on the contrary, a completely new form of expression, or even an art, is being exhibited.
On the one hand, several chapters remind us of this: Leon Battista Alberti, in De Pictura in 1435, presents the painting as an “open window”. Inside the frame, the “white canvas” is an immaculate void replaced by an external scene, similar to the opening of a window that allows the inhabitant to see the outside of his residence while remaining inside. The screen takes up this characteristic of painting: in a similar way, it is a window open onto a landscape and/or a scene to be inserted, totally independent from the place where it is located. The situation is quite different with projection mapping, which brings out the image on the set, and where the specific geometry of the projection medium reappears: it is essentially a question of “dealing with” it, and not of going through it to replace it with something else that has no connection and that ignores it.
On the other hand, the screen is also defined by immobility. It cuts a window of escape into the space and time of reality. In cinema or video, the reported scene that fits into the window is dynamic and has movement, but the window itself is motionless. Moreover, the inscription of the reported image takes no account of the environment, as shown by Rem Koolhaas’ caricatural aphorism about the context, which Ludovic Burczykowski recalls in Chapter 1. On the contrary, projection mapping develops a new mediation mechanism, that is “an action involving a transformation of the situation or the communicational mechanism, and not a simple interaction between elements already constituted, and even less a circulation from one element from one pole to another” (Davallon 2004, p. 43), as Alexandra Georgescu Paquin states in Chapter 11.
Thus, projection mapping is an emergence in the sense of Morin (Morin 1977; Juignet 2015): neither the medium, nor the projection, but an in-between. Or rather a third-party composition that could not survive without the medium or projection. A type of link, of junction from which the unexpected springs forth. This is what Martina Stella refers to as a subject-matter in Chapter 3. In this form of “narrative composition”, the reciprocal action of one on top of the other creates movement on a monumental medium. This is particularly the case in these ever-impressive sequences where the building explodes or collapses, or when a window appears that opens into a well-known facade that is known to be blind. In another style, this is also the case when the image animates a white marble statue, making it cry or laugh, changing its expression at will, following the viewer’s gaze or winking at him. And we start dreaming of marble statues that might start dancing!
Last but not least, the way in which monumental projection mapping is broadcast brings back to life a practice that is in retreat, that of strolling and street performance. During the second projection mapping festival in Lille in March 2019, more than 100,000 people walked for an entire evening between the spots spread throughout the city1. And it is no coincidence that this form of projection is so successful in this region of Hauts-de-France, which has always loved the big popular parades and street carnivals. The device imposes the social sharing of a collective viewing in the public space, far from the intimate, internalized and silent contemplation specific to cinema, or the superficial and distracted viewing that can be practiced alone or in groups in front of television or on mobile devices.
Finally, projection mapping embodies in a new way the idea of a Gesamtkunstwerk dear to Richard Wagner. Indeed, sound is part of the projection mapping work, just like the image. It is therefore a form that links the object and/or the architectural environment with the audio-visual media. But experiments have already given way to gesture in the construction, such as this work which translates into images the gestures of the conductor conducting the musical work performed live. Taste and touch may follow in the dynamics of innovation.
In the end, we can only agree with Antoine Manier when he states that projection mapping works are in their infancy and will be structured. A grammar of projection mapping will be gradually developed. To master it, it will be essential to simultaneously understand how projection mapping makes sense, and what the viewer develops from the perceptual requests addressed to him (Leleu-Merviel 2018). Gradually, we will be able to move towards modelling in support of the design of works, as we were able to do some 20 years ago for hypertexts and hypermedia (Durand et al. 1997). Other techniques will emerge, such as photogrammetry (Chapter 8), while other algorithms will refine the correspondence calculations between the 3D surface and the mapped image, and other fields of implementation will be strengthened, such as the factory of the future (Chapter 10). Other concepts will structure the reflection about the image beyond the screen, such as mediatecture2 (Chapter 11). Undoubtedly, this book is only a foundation which many others will come to complete before we have covered the subject.
Sylvie LELEU-MERVIEL
Director of the DeVisu laboratory
Polytechnic University of Hauts-de-France
Valenciennes
References
Alberti, L.B. (1435). De Pictura. Macula, Paris.
Davallon, J. (2004). La médiation: la communication en procès? MEI, 19, 37–59.
Durand, A., Huart, J., Leleu-Merviel, S. (1997). Vers un modèle de programme pour la conception de documents. Hypertextes et Hypermédias, 1(1), 79–101.
Juignet, P. (2015). Edgar Morin et la complexité. Philosophie, science et société [Online]. Available at: https://philosciences.com/philosophie-generale/complexite-systeme-organisation-emergence/17-edgar-morin-complexite.
Kronhagel, C. (2010). Mediatecture: the Design of Medially Augmented Spaces. Springer, Vienna.
Leleu-Merviel, S. (2018). Information Tracking. ISTE Ltd, London, and Wiley, New York.
Morin, E. (1977). La méthode 1. La nature de la nature. Le Seuil, Paris.
1
There was no certified count, but estimates, based on objective criteria (for example, the number of people entering indoor spaces with limited capacity, or number of connections to the mobile application), cite around 130,000 participants.
2
“Mediatecture is the orchestration and temporalization of space, the loading of meaning into space and the creation of a sphere of communication” (Kronhagel 2010, p. 3).
Introduction
It is generally agreed that projection mapping consists of applying light whose geometry corresponds to a more or less complex surface made of heterogeneous materials on which it is projected. Is it really that simple? The practice of projection mapping has developed since the 1990s, particularly in the artistic field, by supporting the dynamics of digital technologies. This practice has now become almost commonplace, while paradoxically, research in the social sciences and humanities pays little attention to it. Certainly, projection mapping could be understood as a simple hybridization of cinema, animation and scenography, but that would miss the essential questions that this practice stimulates.
First of all, the very concept of projection mapping needs to be clearly defined: is it a tool, a device, a technique, a medium, a discipline, a practice, a trend or a movement? Can an image do without a connection to any surface? Projection mapping is synonymous with spatial augmented reality, video mapping and spatial correspondence. While the words video, projection, reality, augmented, spatial and correspondence have found their French equivalents, the word mapping presents itself as an obstacle to translation. Few say that they do “video cartography” or “projected cartography”. The term “mapping” refers to dressing, coating, texturing, covering, transposing and tuning. A rigorous translation of projection mapping as a medium in our context could then be “projection of correspondences” or “video-projected correspondences”.
In this book, many people use the expression projection mapping as we have retained it. Creators, broadcasters, sponsors, service providers, producers, audiences, enthusiasts, technophiles, critics from various disciplinary backgrounds define it and recognize themselves in its lexical field.
The Video Mapping European Center project, supported by Rencontres Audiovisuelles and the DeVisu laboratory of the Polytechnic University of Hauts-de-France, offers and organizes training courses, artist retreats, screenings, a festival and a conference dedicated to projection mapping: Image Beyond the Screen International Conference (IBSIC). This project has informed this book, which aims to identify the different conceptions and practices of projection mapping in order to give it a status, identity, issues and perspectives. For us, it is a question of setting a milestone in favor of its theoretical existence as a medium and discipline in order to stimulate reflection, propose a new perspective on the scriptures, and enrich future practices. The research and points of view proposed in this book contribute to the construction of a shared theoretical object that was previously lacking. It is divided into three parts:
1) history and identity;
2) writing and techniques;
3) production and dissemination.
As far back as we can go in the history of light projection devices in space, it remains difficult to define precisely the genesis of projection mapping (Burczykowski). A practice that remains close to cinema and animation, but which is not totally in line with it. The screen, fixed, mobile, inert, living, pre-existing to the project or created specifically, is at the heart of the matter (Alvarez; Tardy). Video mapping is available in XXL format as well as in miniature, transportable and intimate format, perhaps promoting interactivity (George-Molland). It is far from being systematically narrative and offers transmedia and sustainable opportunities (Grisval). It often has audio to develop the contours of the world that the work seeks to create (Oury). The invention and subsequent democratization of the video projector and digital tools have paved the way for new forms of expression (Moulon). Artists have sometimes made the city their studio, their relationship to the workplace being essential in the creative process, which is far from being a long quiet river (Thébault and Schmitt). Other technological opportunities such as the automation of the geometric calibration of projected images (Kourkoulakou) and photogrammetry (Lissarrague) allow time saving during content creation by way of an accelerated evolution of methods and uses. A society can be defined through the evolution of its appliances and devices, but above all the appliances build the sensitive potential of a society (Stella).
Projection mapping is being developed in multiple ways. It serves the industry of the future by guiding complex and delicate assemblies of machined parts (Level). Based on a low-intrusive creative action, it transforms a heritage or updates a memory trace (Georgescu Paquin); it highlights content that enhances the medium (George-Molland). In architecture, projection mapping is becoming a learning device that promotes reflexivity and memorization (Boulekbache and Chibane). It offers game experiences (Alvarez) and although projection mapping writing is in its infancy, it will probably structure and develop (Manier). Experiments, whether aesthetic, technical or mediation, will be expanded. This audio-visual form is born with the evolution of needs, disrupting and broadening our perceptions (Labadz).
Image technologies influence the way we represent ourselves. They extend our body and allow us to visualize it (Paul). Projection mapping also offers a special experience; it is a source of amazement and wonder (Leroy). Most often, the illusion created by the projection does not require any equipment on board the spectators. The festival is now one of its main forms of expression, particularly in South America (Oury). For as much as and beyond the major events, electronic, technological and digital art is now spreading to institutions and galleries (Moulon). This is not without influence on their presentations and documentation of the collections and their conservation (Moulon; Paul).
We hope that these points of view will contribute to giving projection mapping a theoretical consistency, and thus accompanying and enriching the future of what could well become a new art, a new discipline, and a new field of research.
Introduction written by Daniel SCHMITT, Marine THÉBAULT and Ludovic BURCZYKOWSKI.
PART 1History and Identity
1The Origins of Projection Mapping
1.1. Introduction
Figure 1.1.Evolution of the queries of the keyword “video mapping” in French on Google from 2004 to 2019 (source: Google Trends, 2019). For a color version of the figures in this book see, www.iste.co.uk/schmitt/image.zip
When did projection mapping start? More than 10 years separate this text from the emergence of the key word “video mapping” on Google, but this criterion alone is not enough to capture its origin and requires a more distant archaeology. The materials that make up this chapter come from a collection of significant historical elements during a series of expeditions in the reflector that is the archived history of ideas and works of art or technology up to the 20th Century. Presented with a desire to go beyond the passive collection, these facts are ordered in such a way that one can understand how projection mapping can be understood as an instrument of vision comparable by similarities and differences to many other devices that preceded it, but also as an original way of seeing that has been maintained over time.
1.2. Let’s moonwalk! A short crossing through time
1.2.1. The emergence of the expressions “video mapping”, “projection mapping”, “spatial augmented reality” and “spatial correspondence” between the beginning of the 21st Century and the end of the 20th Century
By literally writing “video mapping” on the Google Trends trend analysis tool, we can see that queries for this keyword increase significantly from a point of origin located in 2008. This information highlights the recognition of the unity of a concept based on Web searches for it and would make it possible to clearly date the birth certificate of projection mapping. Google and the use of this word alone, however, do not reflect what precedes them: before 2008, this expression was not used or very rarely used, whether written as one or two words, with or without an accent, and with or without the word “projection” instead of “video”. This initial year would therefore be sufficient if it was not very clear that the technical device and its spatial writing logic were already well identified in the 1990s by technologists or artists. The often cited “spatial correspondence” was first mentioned by Michael Naimark in 1984 in Los Angeles, while “spatial augmented reality” was first mentioned in 1998 in North Carolina by Ramesh Raskar.
Between these two publications, Paul Milgram proposed in 1994 in Toronto a taxonomy of “mixed realities”, and the first patents identified for digital video projections that match volume surfaces were filed for Disney Inc. in 1991 and General Electric Co. in 1994. Between these two patents, artist Tony Oursler exhibited The Watching in 1992. Later, in 1999, John Underkoffler, the designer of Minority Report’s famous interface, invented I/O bulb and Luminous room when the facade of Amiens Cathedral was painted. In 2004, choreographer Klaus Obermaier used infrared to divert the real-time video from the silhouette of a performer in the foreground of the stage and projected a video on his body that differed from the background. Johnny Chung Lee published his doctoral thesis on automated projector calibration the same year. The following year in 2005, Olivier Bimber planned to project onto a classical-age painting.
1.2.2. From 17th Century magic lanterns to ancient camera obscura
If we stick to the technical device and spatial writing logic, many artists, painters, visual artists, illustrators or directors have also used, since the beginning of the 20th Century, the unconventional image projected in relation to the reference that is cinema. They played with the image carriers and environments of their works in a way that projection mapping still offers today. Before them, in the 19th Century, it was travelling projectionists like Maximillian Skladanowsky who projected natural disasters, sometimes improvising projection spaces for storms, fires or earthquakes. These images are reminiscent of buildings that collapse with digital pixels. Savoyard lanternists brave the mountain slopes accompanied by marmots, monkeys, drums, or barrel organs like the contemporary off-road projection mapping projectionists do outside rooms.
At the end of the 18th Century, phantasmagorical spectacles such as those performed by Giuseppe Balsamo called the Count of Cagliostro, Johann Georg Schröpfer, Paul Philipsthal or Étienne-Gaspard Robert called Robertson used illusory projections of supernatural beings. They would double the projections of thunderous noises or the smell of burnt feathers. Between the 15th and 16th centuries, a magic lantern was designed by Leonardo da Vinci, but they were not made by Christiaan Huygens until 1659. Then they were carried around in 1664 by Thomas Walgenstein, at the time when anamorphoses were theorized. The lantern is widely disclosed by Johann Christoph Sturm and the Jesuit Athanasius Kircher. It was Milliet Dechales, in 1674, who produced a series of projected images approaching animation and, in 1698, Johann Christoph Weigel developed an overlay of images in the same way as a compositing of layers. This was shortly before the lantern was mounted on wheels and several were used to compose the same image, which today would be called stacking or shifting the image. Johannes Zahn, a Jesuit, also shows animated projections: he projects worms in a jar, the movements of a weather vane or those of a clock. Van Musschenbroek continued Zahn’s work around 1730 by inventing animated plates that made it possible, for example, to turn the wings of a mill. As early as 1608, Cornelis Drebbel said he could change the appearance of his clothes and make giants appear.
In 1650, another Jesuit, Gabriel Magalhaens, reported descriptions of the use of oriental lanterns that he said were quite convincing. Chiang Khuei and Fang Chheng, in the 13th Century, made animated projections on smoke. Sun Kuang-hsien, in 930, or Shao Ong the magician, as early as 121 BC, did comparable tricks. In the past, they would turn flying dragons. In the 16th Century, in the West, Giambattista della Porta described in his Magie naturelle how to reveal volumetric light in chalk powder. He provides a remarkable drawing of it. Girolamo Cardano and Benvenuto Cellini projected images for shows. In 1420, Johannes de Fontana produced a drawing of what he called the lantern of fear in the Bellicorum instrumentorum liber. Jokes and mysteries are played in the Middle Ages with these thaumaturgical lanterns. Arnaud de Villeneuve, from 1290, would have used it. The camera obscura was known to Aristotle and Mozi in the 4th Century BC, and even to Apollonius of Tyana, the miracle worker, in the 1st Century.
Contemporary archaeologist Matt Gatton argues that the camera obscura may have been used as an archaic projector to animate the faces of statues during the Eleusian feasts in ancient Greece – no doubt long before the use of 16 mm projector film from 1969 onwards to project on the busts sculpted inside Disneyland’s Haunted Mansion and on their facades and other crystal balls. Contemporary authors Jean-Jacques Lefrère and Bertrand David have even put forward the hypothesis, with supporting arguments, that figurative shadows were projected using tallow candles as early as the Palaeolithic era.
1.2.3. The screen as a material considered as a void: projection mapping in negative from the 15th Century onwards
As far as we can go in the history of light or shadow projection devices, pinpointing a beginning seems complex. Would it be necessary to have a kind of visual antinomy of what projection mapping is in order to be able to record its beginning by way of a contrast? One can find an antinomy of this type in an asserted disregard for the medium receiving the image, since it is the interplay with the latter that gives meaning here to the words mapping, correspondence or spatial. One of the most important key points of such disregard is undoubtedly the explicit verbalization of Leon Battista Alberti written in 1435 in De Pictura. He presents the painting board as an “open window”. Mapping on the void that is the open interior of a window is a misdirection: no more composition according to the medium is possible as soon as it is expressed as non-existent.
If we agree to consider the artistic movement that began with the transition to the 20th Century of so-called modern art – or at least one of the most important parts of this movement – as a challenge to this Albertian representation as a visual screen form that has become hegemonic, the hypothesis is that projection mapping would only be one manifestation, among other types, of what happened as a result of this paradigm shift. Extension of the painting and access to new dimensions sought by painters, removal of the base in sculpture, total or synesthetic art, sublimation or enhancement of the material, background scenery becoming an actor in scenographies, etc., are all fine art studies whose origins can be traced to projection mapping.
1.2.4. How far back in history can we go?
Thus, the first image not to consider its medium as an “open window” is the one that took advantage of the formal specific features of its surface, considering the material nature or volume of the latter. We then find ourselves in the astonishing situation of having to bring together the origin of projection mapping with that of the images themselves at the current state of knowledge of the history of human creations: wall representations of the Paleolithic. Some of them undoubtedly play with the volumes of the cave walls, which is a form of mapping in the sense used in projection mapping: a set of correspondence between an image and the specific features of a heterogeneous medium. Some of these images could be animated with the moving flame of a fire, in a sense close to what can be understood as the feeling of movement caused by the video. In addition, some of them took place where acoustic games were played. Would a study on projection mapping help to understand this mystery? It would have been easy to conclude that projection mapping was anthropologically inevitable since it was so latent in humanity that it became confused with it. Everyone dresses, every culture transforms or disguises their bodies, makes them up or masks them. What is more than mapping if not disguising a surface by pretending to transform it by adding light? Exposing it, perhaps.
So, what alternatives occurred when the Albertian representation was set up? Was there any kind of projection mapping before it was designated? Would we have always done it? If so, the words video or projection mapping, spatial correspondence, spatial augmented reality, would use new terms to describe something old that does not have this name, and this would legitimize an archaeological expedition. But if we insist on going back in time, we notice that projection mapping becomes a concept whose strength, like any good concept, is to find many dynamic echoes in various contexts. The difficulty of such an undertaking, then, is to know where to stop the investigations knowing that we will always find if we want a Greek, a Chinese, an Indian, an African tale or a disappeared civilization which will have made some initial progress. Or, at the very least, we could build a demonstration of it! The risk of seeing the new in the old is equal to the ambition when it comes to making the new from the old. Let’s extrapolate, then: hardly earlier than the first shapes drawn in the caves, we would make the White hole, White fountain or Big Bang, the first “projectors” of light, cosmic! Are we living in an illusory Matrix projection? To pose this question, we must take a look at some media and philosophical doctrines.
1.3. Immersion in hallucinated worlds
It was shown in September 2017 that the resources of a conventional computer do not allow the known universe to be simulated. A few months earlier, an experiment was attempted with the Swiss supercomputer Piz Daint, which provides an image of the universe composed of 15 billion galaxies generated from two trillion particles in 80 hours of computing. This representation of a closed world is nevertheless in contrast to the infinitization of the universe that marks the 16th Century described by Alexandre Koyré, but the idea of the anthropocene today also brings us back into a relatively closed system. Sustainable development problems linked to climate change or access to natural resources thus tend to show us our contemporary artificial environment as an ephemeral illusion in view of the changes deemed necessary for the future. To get an idea of this, it is sufficient to look around you, identifying everything that has required oil, comparing predictive studies on its extraction and consumption with the materials that you imagine you could produce in its absence. This appreciation of a hallucinated world refers to an idea that is not new and crosses history along various multicultural traditions: living in a coma, a dream, as a living being, an egg or a seed that is virtually a tree, a computer simulation, being just a brain covered with electro-stimulators, being possessed, etc., are as many cases of the abyss of illusory worlds where parallel realities of a geometry that is not very Euclidean can overlap or intertwine with each other.
1.3.1. Some films on the theme of nested or fallacious realities in line with the first digital projection mapping installations
The novels Simulacron 3 by Daniel Francis Galouye and Simulacres by Philip K. Dick, both published in 1964, are said to have founded the sci-fi theme of the world viewed as a computer simulation. The first one became Le Monde sur le fil (1973) and then Passé virtuel (1999). The second author would give the essence of Total Recall’s schizophrenic scenario (1990). Although Adolfo Bioy Casares’ novel Invention of Morel published in 1940 was earlier than them, it will not be part of the opuses that adapt this theme on the big screen. In the 1990s, there were The Lawnmower Man (1992), Ghost in the Shell (1995), Jumanji (1995), Serial Experiments Lain (1998), Dark City (1998), Truman Show (1998), the Matrix trilogy (1999) based on the novel Neuromancien (1984), ExistenZ (1999). It was also in 1999 that digital projectors were installed in cinemas, with the digital release of Episode I of Star Wars as a benchmark. Others followed, including Avalon (2001) and Vanilla Sky (2001). The latter is a remake of Open Your Eyes (1997), a Hispanic film that is itself part of the 17th Century Spanish tradition of playing with the rationality of perceived realities. The most important representatives of this tradition are the novel Don Quixote (Cervantes 1605), the play Life is a Dream (Calderon 1635) and the painting of Ménines (Velasquez 1656).
1.3.2. Some philosophies of illusion
In the East, in the Vedic texts, the Brahmin underlies the cosmos as a presence in all things and presents itself as the only reality whose manifestation named Maia makes us think of a world that we accept as real. But it remains an illusion to be overcome in order to see the transcendent realities. In Buddhism, which is most probably inspired by it, the ego projects an illusory reality onto a set of primordial laws called dharma. It is noteworthy that some Japanese Buddhist temples equipped themselves with projection mapping systems no later than by 2015. Chouang Tzu, in a Taoist fable, wonders if he is a man who has dreamed that he is a butterfly or if, rather, he is not this butterfly dreaming that he is Chouang Tzu. On the western side, the ancient Greeks put forward similar ideas. For the pre-Socratic Heraclitus, around the 5th Century BC, “nature likes to hide itself”. The “kora” described in the Timaeus by Plato a century later is presented as a dream, both a print and a matrix of a genesis perceived also under a veil. Of course, his cave is even more famous, and Plato defended the Pythagorean ideal of an invisible world that could be described mathematically. Galileo summarized this in the 17th Century by saying “Nature is a book written in mathematical language”. Shortly after Galileo, it was John Locke and Robert Boyle who imagined that they could bring Adam and Eve’s conditions in Eden back to heaven, thanks to exhaustive properties. Adam and Eve saw things as they really are.
The theory of emission, according to which visual perception occurs through light rays emitted by the eyes, has been debated since the ancient Greeks. For Lucretia, in the De Natura rerum, written in the 1st Century BC, there is a wandering sham crowd and the vision of the mind coincides with that of the eyes. The fable of the painter Parrhesias winning against Zeuxis illustrates this idea well. In the 18th Century, Francis Bacon wanted to inhibit the tendencies of the mind that he named idols and that distorted reality (Novum Organum 1620). Descartes, some 20 years later (Première méditation 1641), argues that he has sometimes experienced that “meanings were misleading, and it is prudent never to rely entirely on those who once deceived us”. Newton then demonstrates in Optics (1704) that what we see is not physical reality. In the 17th and 18th centuries, Berkley, Hume and Kant were famous for their criticism of the idea that reality exists independently of its human representations, known as “realism”. Hume, for example, argues that it does not matter whether impressions are produced by the creative power of the mind: what is important is that “we can draw inferences from the coherence of our perceptions, whether they are true or false, whether they represent nature accurately or whether they are pure illusions of the senses”.
1.4. Examples of visual devices
Without having to dwell on other more contemporary thoughts, it is already remarkable that the philosophical traditions that characterize the illusory or incomplete state of affairs make it a subject that seems inexhaustible. If we see both too much and too little, if there are things behind things and we perceive only part of them, if we imagine artificial additions, etc., it would be appropriate to inhibit or subject to analysis what potentially deprives us of a proper appreciation. Jean Cocteau thus repeated Tchouang Tseu in his 1950 Orpheus: “Some say that we are just his dream”, before adding “his bad dream”. The idea of a Promethean transgression or hubris presents itself as a result, since the challenge of finding some ways to differentiate true and false is symmetrically launched. Should we show ourselves our own ignorance of a fake, hidden or perceived incomplete nature to which we add our own falsehoods? In this sense, Umberto Eco advanced the idea that a process in man leads him to seek iconic processes that increasingly deceive his senses. The following few processes raise this question: would projection mapping answer the need to make reality habitable? With Aristotle, art in some cases completes what nature does not have the power to accomplish. According to Kant, only Art that is evaluated without criteria of utility makes it possible to exercise a free will, and thus to be free. “Augmented reality” would be the fruit of those ideas that have obviously crossed time and space: both an expression of perceptual limits and an apparatus of transgressive vision that tries to approach a beyond an imperfect illusion because it is incomplete and overdetermined.
1.4.1. Two visual instruments: anamorphoses and X-rays
The 17th Century Dutch man, living a golden age of painting at a time when we are perfecting the lenses of telescopes, the automation of calculation, the theory of anamorphoses and the diffusion of the magic lantern, does not hesitate to recall the dichotomy of the limits of our perceptions on which a set of devices emerge that aim to overcome them while being conditioned by them. The errors of our vision became a subject of representation for the Dutch of that time and it is in this prism that the theories of anamorphoses which were produced then can be understood. After about 1630, the complexity of the perspective play of the new optics, catoptric and perspective, devalued the human vision now perceived as a simple way of seeing the world among so many others. Jacob Leupold’s “machine for anamorphosis” (1713) is an apparatus that shows this devaluation and perhaps even aims to prevent it. A probable high point came in 1895 when Wilhelm Röntgen accidentally invented X-rays. The range of colors whose painting had hitherto celebrated the variety is in fact only a window into the rays that escape our senses. This further mocked the tradition of representation that photography too had already dulled considerably. The mocked painter “can’t see further than the tip of his brush”.
1.4.2. Immersive panoramas
Cinema is this original and radical form of visual immersion that the themes of the nested realities mentioned above translate as a self-reference from its own operating mode. In favor of a certain captivity in the cinema, the cinema does not allow a priori any floating attention or bodily mobility. This “immersion” is supported by renowned videographers: Bill Viola when he argues that to see a video image “you have to get wet” or Nam June Paik for whom it is “weightlessness”. In both cases, we would no longer have our feet on the ground with the video image. For the theorist of computer graphics, Philippe Quéau, the reason for the enthusiasm for virtual images is also “immersion in the image”.
The giant panoramas of the 18th Century precede this radical aspect of an exclusive image. They artificially stage and provoke the sensation of 360° immersion. Around 1779, the engineer of the Royal Corps of Bridges and Causes, Louis Le Masson, had a “great and new” idea “to show Rome as a whole”. The device designed is a large circular painting where the viewer’s eye finds the most complete pictorial illusion. It was patented in June 1787 by the Irish painter Robert Barker, who named it La Nature à Coup d’Œil in French. It is remarkable that the mausoleum of the philosopher Hume, mentioned above as going against the doctrines of “realism”, is painted in this first panorama. The year before, the idea of Panopticon, Jeremy Bentham’s “inspection house” was written, which conceived a circular prison project whose characteristics gave prisoners “the feeling of an invisible omnipresence”. Jules Damoizeau (1890) is credited with the first 360° panoramic shot. In another field, Claude Monet began the Water Lilies project in 1918. He painted a whole that formed a surface of about 200 m2 in eight compositions of the same height suspended in a circle.
1.4.3. Augmented reality and low-tech virtual reality
These panoramas place their observers in a portion of an imported or imaginary landscape. They are large in size and contrast with smaller portable objects such as Martin Engelbrecht’s “perspective boxes” from 1730 or the trompe-l’oeil painter Samuel van Hoogstraten from 1656. In a way, they anticipate today’s virtual reality (VR) headsets, as well as Charles Wheatstone’s two “stereoscopes” produced in 1838. Brunelleschi already proposed with his “experience” in 1415 to look, through a hole drilled inside a painting, at the reflection of this painting which made it merge with its unframed frame. In the 18th Century, Claud’s Glass turned its back on what was observed in a tinted mirror to give it the calibrated shape of the chromatic canon of landscape representation of the time. These mirrors
