The Occupation of the Natural by the UnNatural: About the Operation of the Superimposition in Augmented Reality and Trompe-l’œil
by Manuel van der Veen

This brief research report concentrates on the operation of visual superimposition, which today attains an unexpected topicality through the technology of augmented reality (AR). My doctoral research project relates the art-historical procedure of trompe-l’œil to AR.1 Both techniques want to embed images naturally into the real environment, and subtly make the natural unnatural.2 It is important to note at the outset, that in many exhibitions and reproductive representations of trompe-l’œil, the pictures are ripped out of their constitutive context, making this effect hard to see. In this text, I will focus on three examples in which objects from the real surroundings are superimposed by an interior.
I begin with a trompe-l’œil of Cornelis Norbertus Gjisbrecht, An Open Cabinet of Curiosities with a Hercules Group from 1670 (fig. 1). Its frame is camouflaged with a wooden edge, which physically attaches the trompe-l’œil into the real wooden wall. Then there is a small door, slightly opened, which integrates the space in front of the picture, but also refers to the space behind. The door creates a three-dimensional space. Therefore, the surface of the picture plane, as indicated by the wooden edges, is only the core level within a total of three spatial layers. With the door slightly open the trompe-l’œil invites viewers from the physical space in front of the picture to see what is visible in the depicted space behind the wall. While the frame is camouflaged, the inside of the cabinet is an ordinary illusion. Here, the trompe-l’œil takes advantage of its connection to the real environment to centrifugally amplify the center until it immerses into reality. Since it is not obvious where the picture begins or ends, the reality outside is also centripetally transformed into a pictorial being. Perhaps the boundaries arise with the hinges as thresholds, which mediate between the picture world and the real environment, but this remains uncertain. As theorist Bernard Siegert notes, “It is this oscillating between the transparency of the imaginary pictorial space and the opacity of the material carrier, and more importantly, it is the re-entry of the latter into the former, that keeps generating the trompe-l’œil.”3 The picture object concurrently appears as a ripped reality; as a flat surface (the wood, the glass, and the lead bars); as a real opaque object in the room (the open door, the hinges, the folded note); and finally as a picture superimposed upon the real environment to cave a fictitious interior into the solid wall.

My research pursues the theory that these strategies of trompe-l’œil are reactivated in contemporary art, and simultaneously investigates their relationship to the phenomenon of AR. The artist Cayetano Ferrer continues the tradition of trompe-l’œil by focusing on the entanglement of object and environment. In the series Western Imports, he affixes high-resolution photographs of an environment onto cardboard boxes placed in the space (fig. 2). The photographs have been processed in such a way that the surface of the package intermittently disappears but the fonts and logos printed on them are still visible. Subsequently the logos seem to float in space, while also marking the surface of the box. Hence, the object itself seems to be semi-transparent. As if seen through water, the materiality of the package itself is liquefied. As such, we observe an object that can assert itself as an object but is caught in the moment of its own disappearance. As in trompe-l’œil, the surface is maintained on the one hand and denied on the other.

A popular strategy within AR is to superimpose real objects with their own inside—visualizing the actual but also invisible interior. This is used in various applications, including the next example which is drawn from industry (fig. 3). The internal structures of the machine are superimposed upon its surface in order to make its functionality transparent and more comprehensible.4 A dynamic image of the inside is therefore etched into the machine. If one touched this representation, they would clash with reality. The semi-transparent quality of a digital screen (which allows both touching and looking through it) is transferred to the now semi-transparent object. Consequently, reality is mediated but not dissolved as such. AR makes use of different evidence production processes, like the fixed position of the animation above the machine, calculated light incidence, or interactive communication, to imply presence but it does not guarantee the reality of the image.
The techniques employed in each of these case studies make visible a layer of reality, which could not be seen without them. However, this superimposed multiplicity of events, all now located at one place and time, reduces them to a singular one in relation to the present.5 These superimpositions are in a way so excessive and immersive that they occupy the natural with the unnatural.6 Images cave themselves into reality, liquefy its consistency, or etch themselves into the material. Therefore, one of the most important challenges we encounter with AR is to constantly point out its very narrow boundaries. Since our field of vision is ripped apart, natural and unnatural are interwoven.
____________________
Manuel van der Veen
Manuel van der Veen is a PhD candidate in art science at the State Academy of Fine Arts Karlsruhe, Germany. He studied arts and philosophy to work at the intersections of practice and theory. His research focuses on the confrontation of traditional procedures, like trompe-l’œil and sculptural relief, with the more recent one of augmented reality.
____________________
Footnotes
1. The working title of my PhD project is “Augmented Reality. Trompe-l’œil and Relief as Technique and Theory.”
2. This is also the case with the image below, which is all the more important because the trompe-l’œil does not want to show up as a work of art. Instead its goal is to show up as a thing beside other things.
3. Bernhard Siegert, Cultural techniques: grids, filters, doors, and other articulations of the real (Fordham: Fordham University Press, 2015), 191.
4. The visible inner structure is also used for maintenance work. This allows one to refer to invisible parts and additionally enrich the field of view with references, instructions, and annotations.
5. In her text about AR applications, which register historical data and images into the real field of vision, Edith Blaschitz also emphasizes the reduction produced by this superimposition; Edith Blaschitz, “Mediale Zeugenschaft und Authentizität. Zeitgeschichtliche Vermittlungsarbeit im augmentierten Alltagsraum: Augmented Realities—Augmented Spaces. Digitale Texturen sozialer und kultureller Räume,” Hamburger Journal für Kulturanthropologie (HJK) no. 5 (2017): 3–13.
6. A term used by Louis Marin as excessive mimesis; Louis Marin, “Representation and Simulacrum,” in On Representation, trans. Catherine Porter (Stanford, CA: Stanford University Press, 2001), 316.