TELEVISUAL SCREEN SPACE, COLLAGE, AND THE REMEDIATION OF MODERNISM
lecture

In our 1999 book, Jay Bolter and I defined "remediation"as the logic with which media (particularly but not exclusively digital media) refashion prior media forms. In response to the question of what is new about digital media, we proposed the answer that these media are new precisely because of the ways in which they refashion older media. Specifically, we examined the ways in which computer graphics, video games, virtual reality, and the World Wide Web define themselves by borrowing from, paying homage to, critiquing, and refashioning their predecessors, principally television, film, photography, and painting, but also print. Video and computer games remediate film by styling themselves as "interactive movies," incorporating standard Hollywood cinematic techniques. Virtual reality remediates film as well as perspective painting. Digital photography remediates the analog photograph. The World Wide Web absorbs and refashions almost every previous visual and textual medium, including television, film, radio, and print. Furthermore, older media can remediate newer ones within the same media economy. Today, the traditional Hollywood cinema is attempting to maintain its influential cultural status by employing computer graphics in otherwise conventional films, by creating films entirely with computer animation, or (in more experimental films like David Cronenberg's Existenz or the German film Run, Lola, Run) by replacing the logic of linear narrative with more recursive, game-like logics.
Television, too, is making such extensive use of new media that TV screens often look like pages of the World Wide Web. Remediation seems to be a fundamental characteristic not only for contemporary media, but for all visual media at least since the Renaissance with its invention of linear-perspective painting-as evident in recent interest among art historians in the role that optical devices (including the camera lucida, camera obscura, and photographic projection) played in the history of realistic painting. Each medium seems to follow this pattern of borrowing and refashioning other media, and rivalry as well as homage seems always to be at work. Our account of remediation, with its emphasis on hybridity and intermediality, intends to intervene in the more enthusiastic discourse of the uniqueness of digital media. Among the most bombastic voices to be heard amidst the discursive din of digital exceptionalism, Janet Murray argues aggressively in Hamlet on the Holodeck that "the globally networked, multimedia desktop computer" is "an entirely new medium of representation."Comparing the birth of this new medium to the birth of film, Murray argues that new digital media must move "away from the formats of older media and toward new conventions in order to satisfy the desires aroused by the digital environment."Thus "it is important," she concludes, "to identify the essential properties of digital environments, that is, the qualities comparable to the variability of the lens, the movability of the camera, and the editability of film, that will determine the distinctive power and form of a mature electronic narrative art." Murray's rudimentary technical account of the art of film reveals her as among the more theoretically and historically naive of digital enthusiasts.
Although there may be a prima facie appeal to this line of thought, in the euphoria of digital desire Murray greatly oversimplifies both the diversity of new digital media and the complexity of their relation t o older media forms. In calling for the discovery of the uniqueness of digital media, writers like Murray or Jerome Holtzman, who urges us to "transcend the old to discover completely new worlds of expression" (15), unselfconsciously adopt a modernist rhetoric in which digital media cannot be truly significant until they make a radical break with the past. While I would agree that digital media do from earlier media forms in interesting ways, it is important to articulate these differences in terms of the cultural, historical, and technological specificity of particular media forms.
A more historically nuanced understanding of the relation between modernism and the newness of new media can be found in the work of Lev Manovich, who argues in "Avant-Garde as Software," for example, that digital technologies have not led "to the invention of radical new forms," but have rather transformed "avant-garde techniques" like "Constructivist design, New Typography, avant-garde cinematography and film editing, photomontage, etc." "into the conventions of modern human-computer interface." In Manovich's persuasive account, "What was a radical aesthetic vision in the 1920s" has become "a standard computer technology by the 1990s." In making this claim about the modernist antecedents of new digital media, however, Manovich does not completely dismiss the idea of a new media avant-garde. Rather he redefines this new avant-garde as a form of remediation: "The new media avant-garde is about new ways of accessing and manipulating information. Its techniques are hypermedia, databases, search engines, data mining, image processing, visualization, simulation. . . . The new avant-garde is no longer concerned with seeing or representing the world in new ways but rather with accessing and using in new ways previously accumulated media. In this respect," he concludes, "new media is post-media or meta-media, as it uses old media as its primary material." In this talk I want to pursue further this relation between new digital media and the development of modernist art. More specifically I am interested in looking at the way in which some of the formal features of new media can be seen to remediate the formal evolution of modernist visual aesthetics, particularly the significance of collage as articulated by Clement Greenberg.
In this influential and highly contested account of modernist painting, collage played a crucial role in challenging painting's two-dimensionality by first calling attention to the fact that "flatness, two-dimensionality, was the only condition painting shared with no other art." Because of this unique condition, Greenberg argues, "Modernist painting oriented itself to flatness as it did to nothing else." Where "realistic, illusionist art had dissembled the medium, using art to conceal art," modernist art in its later stages "abandoned in principle. . . the representation of the kind of space that recognizable, three-dimensional objects can inhabit." For Greenberg it was cubism that "brought about the destruction of the illusionist means and effects that had characterized Western painting since the fifteenth century. The fictive depths of the picture were drained, and its action was brought forward and identified with the immediate, physical surface of the canvas, board, or paper." Greenberg further credits "the Cubists' need for renewed contact with reality" with bringing about the practice of collage as developed by Braque and Picasso in 1912 (4:61).
Because cubism had succeeded so emphatically in flattening the pictorial space, the distinction between the plane of the painting and the depicted space of the picture needed simultaneously to be made more evident and elided. The method which both Braque and Picasso employed to renew their contact with reality was collage, the pasting of paper and other materials onto the surface of the canvas.
I invoke Greenberg here in part to distinguish my argument from the claim that "digital mosaic" or "digital collage" is in some sense the essence of new media.
For in this claim (and in countless uninspired examples to be found on the World Wide Web), collage is understood primarily in its horizontal aspects in terms of the juxtaposition of heterogeneous fragments within the plane of the computer screen. But this understanding of digital collage, I would argue, elides or represses the role of collage in Greenberg's account of modernist painting in calling attention to the materiality of painting. Even where digital artists have become interested in representing the three-dimensionality of information-space or cyber-space, this space is too often conceptualized as a virtual or immaterial simulacrum of space receding behind the computer screen, or as an information non-space like Simon Biggs's remediation of the Dewey Decimal System in Babel, a web-site which presents this library information system as a navigable, three-dimensional space that functions as a kind of web browser. As interesting as some of these information spaces are, they still do not come to terms with the idea that the space of painting (or the space of the computer screen) is pushing forward into the space of the viewer (or user) rather than receding into the pictorial space behind the canvas (or the screen).
If we look at the evolution of what has been called "screen space," beginning with film and moving through television and computer monitors, we can see a number of interesting affinities with Greenberg's account of the development of modernist painting, most notably the idea that it is the layering of one medium on top of another which offers the most intriguing remediation of collage for our understanding of new digital media. For as film scholars hardly need to be told, flatness is a condition of that medium as well as of painting. Indeed in The World Viewed Stanley Cavell has sketched out a number of important affiliations between his own account of the uniqueness of film as a medium and Greenberg's account of modernist painting. But as Cavell has also noted, another condition of film is precisely its ability to represent, like realistic, illusionistic painting, a "world viewed," a world with the kind of visual space that recognizable, three-dimensional objects can inhabit. Incorporating both the ontology and the pictorial assumptions of perspectival representation built into the medium of photography, traditional narrative cinema operates according to pictorial assumptions very similar to the representational painting of the Western tradition from the fifteenth until the end of the nineteenth century. Although Manovich has intriguingly argued that the incorporation of computer graphics and digital animation into contemporary film reveals that mechanically reproduced, realistic cinema is not the foundational condition of the medium of film, but a historical interruption in its trajectory as a medium of handiwork and animation, film is still, on this account, a fundamentally two-dimensional medium. Like painting and film, television, too, shares this condition of flatness and two-dimensionality; and like realistic painting and film, television traditionally employs its two-dimensionality to represent, or as Cavell persuasively explains, to monitor the three-dimensional world.
But the screen space of television or film differs from painting or photography in that, like the computer screen, it is "dynamic": "It can display an image changing over time" (Manovich, "Archaeology of Computer Screen"). And television and video, like the computer screen, have another characteristic lacking in film. As Manovich notes, these screens are not only dynamic, but they can change in real time. That is, they can function as live media, not simply as media that can present pre-recorded material.
There is an even more interesting characteristic, however, that distinguishes the computer screen from the screens of film, television, and video--the interactivity afforded by the graphical user interface (GUI) or windowed desktop. The human-computer interface community generally agrees that the desktop is meant to offer a more direct and "natural" interface than the earlier command-line interface. By representing three-dimensional objects (folders, files, documents, a trash can), the desktop GUI is said to allow the user to manipulate directly the data she is working with. Nonetheless, the desktop interface is not meant to represent three-dimensional, pictorial space, but a more flattened space akin to that of cubism and collage, which calls attention to the flatness of he screen itself by allowing the user to manipulate directly the image displayed on the screen.
Motivated by the same need for contact with reality that Greenberg associates with the development of modernist art, interface designers call attention to the flatness of the monitor by foregrounding or acknowledging the distinction between the depicted flatness of the interface (in frames, the heading bar, aliases, etc.) and the literal flatness of the monitor.
The dynamic, interactive nature of the desktop (documents and applications can move at a mouse-click from being "on top" to being "underneath" or "in the background") emphasizes not only the flatness, but also in some sense the materiality of the pixellated screen, by foregrounding the various layers or objects on the desktop. The appeal of interactivity is evident at what is arguably the mythical originary moment of both networked computing and the direct manipulation interface, Douglas Engelbart's demonstration of the prototype of the graphical user interface, described as follows on Stanford University's Science and Technology in the Making web site: "On December 9, 1968, Engelbart and the group of 17 researchers working with him in the Augmentation Research Center at Stanford Research Institute in Menlo Park, C[alifornia], presented a 90-minute live public demonstration of the online system, NLS, they had been working on since 1962. The public presentation was a session in the of the Fall Joint Computer Conference held at the Convention Center in San Francisco, and it was attended by about 1,000 computer professionals. This was the public debut of the computer mouse. But the mouse was only one of many innovations demonstrated that day, including hypertext, object addressing and dynamic file linking, as well as shared-screen collaboration involving two persons at different sites communicating over a network with audio and video interface." As the video of Engelbart's demo makes clear, one of the most remarkable features of this new system is the way in which textual representations on the screen can be directly and (almost) immediately manipulated by the user as if they were real objects in the world, rather than symbolic representations. It is telling, I would argue, that Engelbart uses the example of a shopping list to demonstrate the system's ability to reorganize text on the screen; in moving around words like "lettuce" and "banana," it is as if Engelbart is moving around not words but the things themselves: no words but in things. Indeed, the sense that this new interface seems to have created new objects in the world is even further suggested when Engelbart playfully claims to have created the "skinless banana" by simply typing the words on the screen and adding them to his shopping list. In suggesting some ways in which the evolution of screen space can be seen to recapitulate or remediate the history of Western painting, I do not mean to posit a direct identity between these two histories. For as I suggested earlier, the process of remediation is not one of technological teleology. Not only do newer media remediate older ones, but established media remediate new ones as well. Thus in recent years televisual screen space has begun to orient itself to flatness in ways that bear interesting affinities to the employment of collage in modernist painting.
As the space of the television screen (with its multiple windows, overlaid texts and graphics, and moving data streams) has come increasingly to resemble the space of the computer screen --both as desktop and as web browser--the two-dimensionality of the televisual medium has become increasingly not an attribute to be concealed or erased in the presentation of three-dimensional, televised space, but rather a condition that the medium seeks to foreground or acknowledge as paradigmatic of at least some of television's modes of visual address, but rather a condition that the medium seeks to foreground and acknowledge. A small but telling example of this acknowledgment of the medium's two-dimensionality is evident in what might seem an almost insignificant feature of contemporary televisual space--the network logos that are almost ubiquitous in the lower right-hand corner of today's television screens. The way in which these logos (which have also been seen as a form of "branding") are overlaid on the monitored space of a football game, a situation comedy, a cartoon, a newscast, or a televised movie suggests not primarily collage, but more closely the artist's signature conventionally found in the lower corner of most paintings. Like these signatures, which exist on the plane of the canvas rather than in the depicted three-dimensional space of the picture, these network logos remind viewers of the contrast between the television screen as a window onto our three-dimensional world and the television screen as a two-dimensional surface whose image is owned or we might say "authorized" by the network on which it is being shown. Whether in the case of these logos, or in the countless other textual and graphic images overlaid on the television screen, as the television viewer oscillates between looking at the screen and looking through it, televisual space is flattened, making the televisual image (as well as the TV itself) function more as an object within our world, and less like a monitor of (or window on) this or another three-dimensional world. In other words, by thinking of the evolution of televisual screen space in relation to modernist collage, we can see how television acknowledges both its role as monitor of our three-dimensional world and its own existence as a three-dimensional object with a two-dimensional screen within the space of the viewer.
One consequence of this flattening of televisual space and the concomitant acknowledgment of the materiality of the televisual image is the intensification of the emotional valence or affect which viewers attach to television. This is evident in the relatively recent advent of "reality TV," which itself remediates the talk and game show genres. But an even more widespread and well established instance of the way in which television becomes emotionally charged can be found in the variety of news shows, where one of the formal features of television coverage now is the extreme close-up. Due partly to changes in technology that make such shots possible, these intense close-ups work to flatten the space depicted on the television screen. The background tends to blur and the screen is filled with a worried face or an excited one, an angry one or a jubilant one. The depth of the space that is depicted is foreshortened as the close-up face is brought right up front to the screen. Indeed in some respects it becomes the screen. While this effect is made possible by the technological development of high-powered zoom television cameras, the need or desire for these images marks a change in the logic of televisual screen space. This change is dramatized by media coverage of the September 11 attack on the World Trade Center and the subsequent War against Terrorism both in Western television networks like CNN and in its non-Western imitators like Al-Jazeera.
The juxtaposition of theses two CNN images demonstrate the double logic of televisual space which aims both to multiply mediation in the now familiar collage-like look of CNN News and to erase the signs of mediation in presenting the immediacy of the extreme close-up of the Twin Towers in flame. [Images from Al-Jazeera] In these two images from Al-Jazeera, we also see this double logic at work. In one image we have a guest on an Al-Jazeera talk show framed in classic CNN style by a text and logo on the right-hand side of the screen, with a crawling text below reading "American planes conduct raids on Kandahar." In the other image, from the title segment for the Al-Jazeera talk show "The First of the Century's Wars," we see this double logic in the split-screen montage of close-ups of a worried George W. Bush juxtaposed with the Twin Towers in flame, overlaid by the title of the show and the Al-Jazeera logo in the bottom-right hand corner of the screen. Prior to the widespread adoption of computer technology both by the public and by the television industry, television, modeling itself more closely on the visual style of traditional Hollywood cinema, had traditionally sought to erase its own mediation in the depiction of an already there world; as these images from CNN and Al-Jazeera reveal, television today uses close-ups to absorb or appropriate an emotion that is either already there or evoked by the visual immediacy of close-up, and identifies this emotion with itself, indeed with its screen. Insofar as television does work to elide its own mediation, it does so to reveal itself not as a window to another world but as an object within this one. So that in these instances of televisual close-up, what television does is not like illusionistic art, which provides an invisible two-dimensional mediation of a three-dimensional world. Rather television functions more like modernist art in that it acknowledges and embraces both its flatness and its materiality. Furthermore television invests its flatness with the immediacy of an emotional charge so that viewers identify with their TV, relate to it, as if it were a person, or as if, like a person it was an audiovisual embodiment or mediation of human emotion. Just as modernist painters invested the flatness of the canvas, its slightly more than 2-dimensionality, with visual interest, making it in many cases the focus of their paintings, so these images from CNN and Al-Jazeera present the screen as a hypermediated space in which the viewer can focus alternately on the multiplicity of images on the screen or on the emotion with which the close-up image is charged, rather than on the three-dimensional space in which that image is situated. In so doing such moments are not so much about the depiction of a news event within space and time, as they are about the depiction of an emotional moment, crystallized in an instant on the TV screen.
This intensification of television as a physical medium, the flattening of its monitored or televised space, increasingly transforms the screen from a transparent, illusionistic medium into a real object in our three-dimensional space. In so doing, televisual screen space participates in the double logic of remediation that marks new media at the present time: the simultaneous proliferation of mediation evident in television's windowed, collage-like style, coupled with the erasure of mediation as televisual screen space is revealed, through techniques like the close-up, as an object like any other within our three-dimensional world. The increasing acknowledgment of the materiality of televisual screen space is suggestive of another aspect of the development of modernist art, the resurgence of sculpture as a medium for abstract art, which Greenberg traces directly to the formal problems raised by cubism and collage. In similar fashion, and motivated by a similar desire for contact with reality, the screen space of new media is no longer being confined to the computer monitor, or to the television, but has moved increasingly out into the world--as cell phones, two-way pagers, or PDAs. While it is true that all of these media have screens of a sort, these screens are subordinated to the objects or information appliances themselves; they are not in any traditional sense windows onto another world. This development is moving even further into three-dimensionality, with the increased interest in ubiquitous computing, augmented reality, "smart" houses, intelligent traffic systems, and other cases in which digital media are being implanted into physical objects or social environments. New media are becoming increasingly sculptural, functioning not as screens to be viewed, but as objects like any others within our world. As suggestive as this comparison to sculpture may be, however, it also serves to point out the limitations of Greenberg's analysis of modernism. For not unlike other received accounts of modernity, Greenberg's account of modernist painting sees it as seeking to purify or distill art or the aesthetic as an essence or set of formal qualities to be distinguished from the more mundane realities of politics, economics, nature, or society. But as Bruno Latour has argued, modernity has at least two projects, working simultaneously to try to purify or distill the world into its distinct, autonomous realms while at the same time proliferating quasi-objects or hybrids which partake of multiple, heterogeneous qualities or characteristics and thus cannot neatly be designated as aesthetic or political or natural or social. Greenberg's classic account of modernism insists on separating art from the real, but fails to recognize how artworks like the collages produced by Picasso or Braque function as hybrids or quasi-objects. What makes collage or new media quasi-objects or hybrids is their status as both real objects and as mediations. Their assertion of their own objectness or materiality produces what Yves-Alain Bois describes, in disagreeing with Greenberg's account of Picasso's cubist collages, as a "mixing of real space and the space of art," a hybridity which Bois finds "at the heart of cubism, of the objecthood that it wishes to confer on the work of art and that Greenberg's reading tends to minimize if not efface" (91). Indeed it is precisely this hybridity that I have meant to make manifest by looking at the role of collage in new media's remediation of modernism. New media, like older media before them, are not part of a virtual or parallel or immaterial world. But rather, as evidenced through their use of collage and their increasing presentation as, on, and among different objects within the world, new media assert themselves as real objects within our world, thus breaking down or complicating the modernist attempt to separate the mediated from the real.
this invention has helped to redraw the boundaries between science, art, and technology by redefining media not as representations of the kind of space that three-dimensional objects can inhabit but as new quasi-objects or hybrids within the space of our three-dimensional world.
Televised sporting events, where one of the formal features of television coverage now is the extreme close-up. Due partly to changes in technology that make such shots possible, these intense close-ups work to flatten the space depicted on the television screen. The background tends to blur and the screen is filled with a worried face or an excited one, an angry one or a jubilant one. The depth of the space that is depicted is foreshortened as the close-up face is brought right up front to the screen. Indeed in some respects it becomes the screen. While this effect is made possible by high-powered zoom television cameras, the need or desire for these images marks a change in what it means to broadcast sports. According to Alan Schwarz [NY Times, Sunday, October 15, 2000, Week in Review, page 4--Ideas & Trends: A Shot Seen 'Round the World], the accident of having a camera focused on Boston Red Sox catcher Carlton Fisk in Game Six of the Major League Baseball World Series in 1975, when he waved his extra-inning home run fair, prompted television executives and producers to realize that TV could show the emotions of sports, not just follow the ball--indeed that this is what TV does best. Bob Costas, a good sportscaster but a poor media theorist, gets the significance of this feature of televised sports completely backwards, seeing the goal of television to consist in its erasing its own mediation: "To me, television is at its best when it's simply eavesdropping on events that would have been essentially the same had there been no television.... These days, every athlete who crosses the finish line in the Olympics, when they celebrate it's genuine but they are all aware they are players in a movie. They are putting on a performance for the cameras." Costas is off the mark here: it is film, not television, that has traditionally sought to erase its own mediation in the depiction of an already there world; television in this case rather absorbs or appropriates the genuine emotion of an athlete, an emotion that is already there, and identifies it with itself, indeed with the screen.