Syndicate: A THEORY OF CULTURAL INTERFACES 3/3

[manovich@ucsd.edu (Lev Manovich)]

III. Human-Computer Interface

The development of human-computer interfaces, until recently, had
little to do with cultural applications.  Following some of the main
applications from the 1940's until the early 1980's, when the current
generation of GUI (Graphic User Interface) was developed and reached
the mass market together with the rise of a PC (personal computer), we
can list the most significant: real-time control of weapons and weapon
systems; scientific simulation; computer-aided design; finally, office
work with a secretary as a prototypical computer user, filing documents
in a folder, emptying a trash can, creating and editing documents
("word processing"). Today, as the computer is starting to host very
different applications for access and manipulation of cultural data and
cultural experiences, their interfaces still rely on old metaphors and
action grammars. Thus, cultural interfaces predictably use elements of
a general-purpose HCI such as scrollable windows containing text and
other data types, hierarchical menus, dialogue boxes, and command-
line input. For instance, a typical "art collection" CD-ROM may try to
recreate "the museum experience" by presenting a navigatible 3-D
rendering of a museum space, while still resorting to hierarchical
menus to allow the user to switch between different museum
collections. Even in the case of The Invisible Shape of Things Past
which uses a unique interface solution of "filmobjects" which is not
directly traceable to either old cultural forms or general-purpose HCI,
the designers are still relying on HCI convention in one case -- the use
of a pull-down menu to switch between different maps of Berlin.
        In general, cultural interfaces of the 1990's try to walk an uneasy
path between the richness of control provided in general-purpose HCI
and an "immersive" experience of traditional cultural objects such as
books and movies. Modern general-purpose HCI, be it MAC OS,
Windows or Unix, allow their users to perform complex and detailed
actions on the digital data: get information about an object, copy it,
move it to another location, change the way data is displayed, etc. In
contrast, a conventional book or a film positions the user inside the
imaginary universe whose structure is fixed by the author. Cultural
interfaces attempt to mediate between these two fundamentally
different and ultimately non-compatible approaches.
        As an example, consider how cultural interfaces conceptualize
the computer screen. If a general-purpose HCI clearly identifies to the
user that certain objects can be acted on while others cannot (icons of
files but not the desktop itself), cultural interfaces typically hide the
hyperlinks within a continuous representational field. (This technique
was already so widely accepted by the 1990's that the designers of HTML
offered it early on to their users by implementing the "imagemap"
feature). The field can be a two-dimensional collage of different images,
a mixture of representational elements and abstract textures, or a single
image of a space such as a city street or a landscape. By trial and error,
clicking all over the field, the user discovers that some parts of this
field are links. This concept of a screen combines two distinct pictorial
conventions: the older Western tradition of pictorial illusionism in
which a screen functions as a window into a virtual space, something
for the viewer to look into but not to act upon; and the more recent
convention of graphical human-computer interfaces which, by
dividing the computer screen into a set of controls with clearly
delineated functions, essentially treats it as a virtual instrument panel.
As a result, the computer screen becomes a battlefield for a number of
incompatible definitions: depth and surface, opaqueness and
transparency, image as an illusionary space and image as an
instrument for action. [28]
        Here is another example of how cultural interfaces try to find a
middle ground between the conventions of general-purpose HCI and
the conventions of traditional cultural forms. Again we encounter
tension and struggle -- in this case, between standardization and
originality. One of the main principles of modern HCI is consistency
principle. It dictates that menus, icons, dialogue boxes and other
interface elements should be the same in different applications. The
user knows that every application will contain a "file" menu, or that if
he/she encounters an icon which looks like a magnifying glass it can be
used to zoom on documents. In contrast, modern culture (including its
"post-modern" stage) stresses originality: every cultural object is
supposed to be different from the rest, and if it is quoting other objects,
these quotes have to be contextualized. Cultural interfaces try to
accommodate both the demand for consistency and the demand for
originality. Most of them contain the same set of interface elements
with standard semantics, such as "home," "forward" and "backward"
icons. But because every Web site and CD-ROM is striving to have its
own distinct design, these elements are always designed differently
from one product to the next. For instance, many games such as War
Craft II (Blizzard Entertainment, 1996) and Dungeon Keeper give their
icons a "historical" look consistent with the mood of an imaginary
universe portrayed in the game.
        The language of cultural interfaces is a hybrid. It is a strange,
often awkward mix between the conventions of traditional artistic
forms and the conventions of HCI -- between an immersive
environment and a set of controls; between standardization and
originality. Cultural interfaces try to balance the concept of a surface in
painting, photography, cinema, and the printed page as something to
be looked at, glanced at, read, but always from some distance, without
interfering with it, with the concept of the surface in a computer
interface as a virtual control panel, similar to the control panel on a
car, plane or any other complex machine. [29] Finally, on yet another
level, the traditions of the printed worde and of cinema also compete
between themselves. One pulls the computer screen towards being
dense and flat information surface, while another wants it to become a
window into a virtual space.
        To see that this hybrid language of the cultural interfaces of the
1990s represents only one historical possibility, consider a very different
scenario. Potentially, cultural interfaces could completely rely on
already existing metaphors and action grammars of a standard HCI, or,
at least, rely on them much more than they actually do. They don't
have to "dress up" HCI with custom icons and buttons, or hide links
within images, or organize the information as a series of pages or a 3-D
environment. For instance, texts can be presented simply as files inside
a directory, rather than as a set of pages connected by custom-designed
icons. This strategy of using standard HCI to present cultural objects is
encountered quite rarely. In fact, I am aware of only one project which
uses it quite successfully: a CD-ROM by Gerald Van Der Kaap entitled
BlindRom V.0.9. (Netherlands, 1993). The CD-ROM includes a
standard-looking folder named "Blind Letter." Inside the folder there
are a large number of text files. You don't have to learn yet another
cultural interface, search for hyperlinks hidden in images or navigate
through a 3-D environment. Reading these files required simply
opening them in standard Macintosh SimpleText, one by one. The
effect of this simple technique is remarkable. Rather than distracting
the user from experiencing the work, the computer interface becomes
part and parcel of the work. Opening these files, I felt that I was in the
presence of a new literary form for a new medium, perhaps the real
medium of a computer -- its interface.
        As the examples analyzed here illustrate, cultural interfaces try
to create their own language rather than simply using general-purpose
HCI. In doing so, these interfaces try to negotiate between metaphors
and ways of controlling a computer developed in HCI, and the
conventions of more traditional cultural forms. Indeed, neither
extreme is ultimately satisfactory by itself. It is one thing to use a
computer to control a weapon or to analyze statistical data, and it is
another to use it to represent cultural memories, values and
experiences. The interfaces developed for a computer in its functions of
a calculator, control mechanism or a communication device are not
necessarily suitable for a computer playing the role of a cultural
machine. Conversely, if we simply mimic the existing conventions of
older cultural forms such as the printed word and cinema, we will not
take advantage of all the new capacities offered by a computer: its
flexibility in displaying and manipulating data, interactive control by
the user, and the ability to run simulations, etc.
        Today the language of cultural interfaces is in its early stage, as
was the language of cinema a hundred years ago. We don't know what
the final result will be, or even if it will ever completely stabilize. Both
the printed word and cinema eventually achieved stable forms which
underwent little changes for long periods of time, in part because of the
material investments in their means of production and distribution.
Given that computer language is implemented in software, potentially
it can keep on changing forever. But there is one thing we can be sure
of. We are witnessing the emergence of a new cultural code, something
which will be at least as significant as the printed word and cinema
before it. We must try to understand its logic while we are in the midst
of its natal stage.


NOTES

1. For an analysis of the parallels between the language of the nineteenth
century moving image presentations and the language of computer
multimedia during the first half of the 1990's, see my "What is Digital
Cinema?", in The Digital Dialectics, edited by Peter Lunenfeld (Cambridge,
Mass.: The MIT Press, 1988).
2. Thomas S. Kuhn, The Structure of Scientific Revolutions (2nd. ed. Chicago:
University of Chicago Press, 1970).
3. Brad. A. Myers, "A Brief History of Human Computer Interaction
Technology," technical report CMU-CS-96-163 and Human Computer
Interaction Institute Technical Report CMU-HCII-96-103 (Pittsburgh,
Pennsylvania: Carnegie Mellon University, Human-Computer Interaction
Institute, 1996).
4. http://www.xanadu.net/the.project, accessed December 1, 1997.
5. XML which is supposed to replace HTML on the World Wide Web will
enable any user to create his/her customized markup language. Thus, the
next stage in digital media culture will involve authoring not simply new
documents but new languages. For more information on XML, see
http://www.ucc.ie/xml., accessed December 1, 1997.
6. http://www.hotwired.com/rgb/antirom/index2.html, accessed December 1,
1997.
7. See, for instance, Mark Pesce, "Ontos, Eros, Noos, Logos," keynote address
for International Symposium on Electronic Arts 1995,
http://www.xs4all.nl/~mpesce/iseakey.html, accessed December 1, 1997.
8. Roman Jakobson, "Deux aspects du langage et deux types d'aphasie", in
Temps Modernes, no. 188 (January 1962).
9. XLM promises to diversify types of links available to include bi-directional
links, multi-way links and links to a span of text rather than a simple point.
See http://www.ucc.ie/xml.
10. This may imply that new digital rhetoric may have less to do with
arranging information in a particular order and more to do simply with
selecting what is included and what is not included in the total corpus being
presented.
1. See
http://www.aw.sgi.com/pages/home/pages/products/pages/poweranimator
_film_sgi/index.html, accessed December 1, 1997.
12. Jacques Aumont et al., Aesthetics of Film (Austin: Texas University Press,
1992), 13.
13. By VR interface I mean the common forms of a head-mounted or head-
coupled directed display employed in VR systems. For a popular review of
such displays written when the popularity of VR was at its peak, see Steve
Aukstakalnis and David Blatner, Silicon Mirage: The Art and Science of
Virtual Reality (Berkeley: CA: Peachpit Press, 1992), pp. 80-98. For a more
technical treatment, see Dean Kocian and Lee Task, "Visually Coupled
Systems Hardware and the Human Interface" in Virtual Environments and
Advanced Interface Design, edited by Woodrow Barfield and Thomas Furness
III (New York and Oxford: Oxford University Press, 1995), 175-257.
14. See Kocian and Task for details on field of view of various VR displays.
Although it varies widely between different systems, the typical size of the
field of view in commercial head-mounted displays (HMD) available in the
first part of the 1990's was 30-50o.
15. The following examples refer to a particular VRML browser - WebSpace
Navigator 1.1 from Silicon Graphics, Inc. Other browsers have similar
features. http://webspace.sgi.com/WebSpace/Help/1.1/index.html, accessed
December 1, 1997.
16. See John Hartman and Josie Wernecke, The VRML 2.0 Handbook:
Building  Moving Worlds on the Web (Reading, Mass.: Addison-Wesley
Publishing Company, 1996), 363.
17. For a more detailed analysis of this narrative structure, see my article,
"The Aesthetics of Virtual Worlds: Report from Los Angeles," in CTHEORY
(www.ctheory.com).
18. Examples of an earlier trend are Return to Zork (Activision, 1993) and The
7th Guest (Trilobyte/Virgin Games, 1993). Examples of the later trend are
Soulblade (Namco, 1997) and Tomb Raider (Eidos, 1996).
19. Critical literature on computer games, and in particular on their language,
remains very slim. Useful facts on history of computer games, description of
different genres and the interviews with the designers can be found in Chris
McGowan and Jim McCullaugh, Entertainment in the Cyber Zone (New
York: Random House, 1995). Another useful source is J.C. Herz, Joystick
Nation: How Videogames Ate Our Quarters, Won Our Hearts, and Rewired
Our Minds (Boston: Little, Brown and Company, 1997).
20. Dungeon Keeper, MS-DOS/Windows 95 CD-ROM (Bullfrog Productions,
1997).
21. For a more detailed discussion of the history of computer imaging as
gradual automation, see my articles "Mapping Space: Perspective, Radar and
Computer Graphics," in SIGGRAPH '93 Visual Proceedings, edited by
Thomas Linehan, 143-147 (New York: ACM, 1993); and "Automation of Sight
from Photography to Computer Vision," in Electronic Culture: Technology
and Visual Representation, edited by Timothy Druckery and Michael Sand
(New York: Aperture, 1996).
22. Moses Ma's presentation, panel on "Putting a Human Face on Cyberspace:
Designing Avatars and the Virtual Worlds They Live In," SIGGRAPH '97,
August 7, 1997.
23. Overlapping windows were first proposed by Alan Kay in 1969.
24. The examples of Citizen Kane and Ivan the Terrible are from Aumont et
al., Aesthetics of Film, 41.
25. On the ideal of engineering efficiency in relation to the avant-garde and
digital media, see my article "The Engineering of Vision and the Aesthetics of
Computer Art," Computer Graphics 28, no. 4 (November 1984): 259-263.
26. Although this essay was written as late as 1936, Walter Benjamin still sees
film technology as an alien presence which disrupts the familiar patterns of
human perception and destroys an object's aura. Benjamin compares a
cameraman to a surgeon who "penetrates deeply into its [reality] web" (237);
his camera zooming in order to "pry an object from its shell" (225). Walter
Benjamin, "The Work of Art in the Age of Mechanical Reproduction," in
Illuminations, ed. Hannah Arendt (New York: Schochen Books, 1969).
27. See http://www.artcom.de/projects/invisible_shape/welcome.en,
accessed December 1, 1997.
28. The computer screen also functions both as a window into an illusionary
space and as a flat surface carrying text labels and graphical icons. We can
relate this  to a similar understanding of a pictorial surface in the Dutch
art of
the seventeenth century, as analyzed by Svetlana Alpers in her The Art of
Describing. In the chapter entitled "Mapping Impulse" she discusses how a
Dutch painting of this period functioned as a combined map / picture,
combining different kids of information and knowledge of the world. See
Svetlana Alpers, The Art of Describing: Dutch Art in the Seventeenth
Century (Chicago: University of Chicago Press, 1983).
29. This historical connection is illustrated by popular flight simulator games
where the computer screen is used to simulate the control panel of a plane,
i.e. the very type of object from which computer interfaces have developed.
The conceptual origin of modern GUI in a traditional instrument panel can
be seen even more clearly in the first graphical computer interfaces of the
late
1960's and early 1970's which used tiled windows. The first tiled window
interface was demonstrated by Douglas Engelbart in 1968.