Notes on the conjunction of percept and affect in VR
Hardly profound at this stage, but something I put together for the new media class I am doing this semester over here, titled Scopic Regimes of Virtuality, with Edward Shanken.
Introduction
Virtual Reality environments enable experiential opportunities of immersion where perceptions of what ‘is’ or seems to be, in the realm of the virtual world or environment, and the affects of the player/user, become thrown together, circuited, and interactively related. It is this apparent equivalence of percept and affect, identified in Andrew Murphie’s theorization of VR, which forms the embodied conceptualism of much artistic VR experimentation (2002, 10). While perception in VR environments is given attention in the study of VR, drawing on art history, optics, philosophy and cognitive psychology, often the affectivity of VR experience is under-discussed, weakly theorized, or overdetermined. Descriptions of VR as “affective” does little in the way of explanation or theory beyond that single adjective – “affect” alone presumably just connoting proof of user engagement, or a successful interpenetration of the VR world registering in and upon the spectator as internal shift or ‘move’ment. I wanted to take this post to consider this notion of the affectivity of VR environments, via the theoretical legacy of affect as a concept, alongside the dialogue of abstracted, affective VR art works with broader socio-political contexts of production and reception. In the spirit of the question mark that could easily be argued to exist at the end of our course title “Scopic Regimes of Virtuality” this post considers this non- or extra- scopic experience of affect in VR, as it is set up by Andrew Murphie via Deleuze. I proposed also a reading of Eve Kosofsky Sedgwick’s work on Silvan Tomkins to enable us to think speculatively backwards and forwards through the confusing mix of essentialist, structuralist, and poststructuralist thinking about affective experiences of VR that continues to appear in thinking and writing about it. Rather than introducing Tomkins via Sedgwick as a preferred theorist of affect or cognition however, I want to instead consider the ways in which his work at least expands our critical and political appreciation of the concept of affect as it is used in descriptions of immersive media. I will conclude with some remarks about the (i.e. this here) contemporary engagement with models of cognition, perception, and affect to do new media theory.
VR environments and what they do
Early popular and academic writing on VR often locked theories of VR down into discussions of current and future speculative versions of its hardware. For an important hard-ware driven definition formative to media studies theorizations of VR, see for example Steuer’s article ‘Defining Virtual Reality: Dimensions determining telepresence’ (1992). More recently, Andrew Murphie has drawn on Deleuze’s concepts of the actual and the virtual to more fully flesh out a philosophical engagement with virtual reality far beyond a technological emphasis (2002). Via Deleuze, Murphie suggests firstly that VR experiences offer up opportunities to “modulate” the virtual aspects of our being in the world (and/or have them modulated), which more often operate “below the general threshold of perception” (2002, 10). Murphie argues that VR crosses: “information thresholds” – by way of the computer chip; “body/machine thresholds” – through the use of goggles and other technical engagements with the interface; and, representation /reality thresholds (2002, 10). Fundamentally it is the “threshold of perception” itself in Murphy’s theorization of VR, that is “subjected to a massive broadening of it’s own limits (thereby) disrupting the solid notions of social existence” (2002, 10). Murphie also understands VR and VR experiences as granting access to the virtual as an expression of a reality, rather than as a representation of it – a system or setup offering one world among many, that is more conceptually about “operationalism” in the abstract, than any sense or control of the real, implied by representationalism (2002, 9). It is at this point, considering abstracted operationalism, that an appreciation of affect in the VR percept-affect conjunction increasingly tends to matter.
Affect theory in Deleuze
Affect in Deleuze is a kind of change piercing social logics and enabling transitions and transformations within bodies connected to the world. In Cinema 2, and in his Bergsonism, Deleuze gives us less a vocabulary than a kind of flight path, if you like, for thinking about agency and change at the material level of the body that occurs in parallel to the social. This theoretical work is compellingly resistant to purely social constructivist models of the body and embodied experience, and points to the something else that goes on, outside of language, in the concept of change and the body’s material performance. This rich unpredictability of the body and its change is accentuated, manipulated and productively engaged by VR structurations of experience and perception. Perhaps paradoxically, just as VR destabilizes the thresholds of perception, and conjoins percept to affect, we find here an implicit query about the innateness and constructedness of perception and affect at the core of a critical VR practice. Indeed, early utopian and dystopian considerations of VR, as well as authorings of actual VR space that attempt particular kinds of structuration, might be considered to be productively intertwined with this necessarily awkward critical and historical situation of VR. Foregrounding non-linguistic affect, produced in abstracted perceptual-affective environments, VR glances furtively, and strangely ‘back’ or sideways to biological and structuralist understandings of human, culture, and artistic engagement. So how does the idea of human difference appear at the abstracted VR interface? What kinds of claims for affect are made by VR spaces to what ends, and how might this inform our critical and aesthetic comprehension of critical VR inquiry?
Sedgwick on Tomkins’ affects
I wanted to consider in this context postructuralist gender and literary critic Eve Sedgwick’s specific contemporary engagement with 50s era psychologist Silvan Tomkins. In ‘Shame in the Cybernetic Fold…’, Sedgwick re-reads Tompkins’ outlining of the workings of the human affect system as a complex, systematic totality in his encyclopaedic text Affect, Imagery, Consciousness (Sedgwick 2003). In Tomkins, the affects are highly specific and unmodulated physiological reactions “present from birth”: surprise-startle, distress-anguish, anger-rage, enjoyment-joy, interest-excitement, fear-terror, shame-humiliation (protective), “dismell”, and disgust (Tomkins 1962-92, xv-xx). Tomkins theorizes affect separately from the biological drives, raised to prominence by psychoanalytic explanations of behaviour by Freud – these are breathing, ingestion, excretion, sleepiness, and sexuality in Tomkins’ own conception of a drives system (Tomkins 1962-92, xiv). While Freud’s individualistic drives encourage specific needs to be met at specific sites of the body, Tomkins’ affects, layered over the drives, are a kind of alerting mechanism that work much more complexly, through convoluted systemic processes, between the qualia of experience and the drives, and give a circuitry of affect to and between bodies. It is the affects, and not the drives, that lead variously to feeling (the conscious registeristing of affect), and to emotion or ‘scripts’ (the assemblage of feeling with memories of previous experiences of affect), and action (Tomkins 1962-92, xxv). Affects amplify and mobilize the drives into action and not the other way around, and can be interpersonally circuited.
The contemporary import of Tomkins’ supposedly suspect structuralist enterprise lies for Sedgwick (and perhaps our own considerations of scopic and non- scopic constructions of virtual media) not so much in the ‘correctness’ of his systemic categorization. Sedgwick’s reading instead hones in on three relevant conceptual movements of relevance to our understanding of affect in, and human difference at, the interface of VR spaces: 1) the critical release or “freedom” of the affect system from innate, monadic, or oedipal drives, but also from the Foucauldian repressive hypothesis (a theoretical premise of many early visions of VR space as we will see); 2) the unpredictable and non-‘natural’ relation of affects to objects – in Tomkins, “any affect can have any object”, or can be its own reward; and 3) a physiologically social and circuited comprehension of the subject. Tomkins’ rich and speculative “scientism” (Sedgwick 2003, 98) asks us to be in turn more critical of new media work, objects, and experiences offering up specific kinds of singular-aesthetic experiences of affect.
Tomkins on VR
Tomkins would treat VR engagements as dynamic and unpredictable negotiations of user scripts (a user’s physiological affect system considered interactively and inseperably from it’s unique historical experience and memory as a body-mind) interpenetrating unpredictably with authored environment scripts. The script set up by the VR system is completed by the user in a way that cannot bear straightforward hierarchical relations of author-over-immersant scriptedness (‘immersant’ is borrowed here from VR artist/author of Osmose, Char Davies). This is already a commonplace after a decade or more of VR work. But while Deleuze valorizes affect itself in the singular as bodily change in duration – a sense of the unpredictable liberty and liveness of the body in the social field – Tomkins, working beyond Deleuze on affect, employs a much more richer concept of the body’s intertextuality. To the “affected” subject he ascribes an elaborate array of motivations and scripts; physiological complexity; interactive, multiplied and modal concepts of relationality; but also, importantly, idiosyncratic (i.e. subjective) possibilities of individuation. Tomkins’ affect theories put more matter and circuitry into the difference and mattering of bodies at the interface, even prior to the geo-historical situatedness of immersant beings (Hemmings, 550).
Reading VRs affective promises: a small gesture towards case studies
In Murphie’s conceptualization of VR, VR seems much more about the modulation of difference in regard to perception, rather than affect per se. VR sets up a productive “differential” which describes or leads to an active fold of perception (Murphie 2002, 14). Here, “difference” is defined as the negotiation of perception, and actualizations of the virtuality of VR are actually individual solutions to “problems” set up for the immersant and their perceptual regime (2002, 14). But that fold is also worked up, or against, by pre-emptive affective promises of VR works too, by their necessary interpolative gestures prior to immersion, their marketing. Perhaps the promises of each VR environment’s authorial scripts are useful for media and art historical inquiry into VR. We looked in Week 2 at Char Davies’ Osmose. This was a great example to query VR affects: a work promising the immersant a “primary” affective experience of Nature, off the Cartesian grid. In Davies’ words, Osmose as an artwork “seeks to heal the rational Cartesian mind/body subject/object split which has shaped so many of our cultural values, especially towards nature”. The user after donning a head-mounted display and motion-tracking vest very literally progresses from a three-dimensional Cartesian Grid (called an “orientation space”) which gives way to a clearing in a forest after the user’s first breaths. They can negotiate world-spaces based on metaphorical aspects of nature: a Clearing, Forest, Tree, Leaf, Cloud, Pond, Subterranean Earth, and Abyss, and also shift into the work’s code (substratum), and a ‘superstratum’ of text, where quotes from the artist and texts on technology, the body and nature are interwoven. The discourse generated around and within Osmose is one of ambiguous floating, embodied freedom, and a transcendence of western Cartesian negotiations of spatiality and duality. Osmose’s healing might result from abstracted non-realistic spaces but was also developed and funded in a specific context and was very much a product of it’s historical time and place of development. It makes a lot of Hegelian sense, for example, that its affectivity was worked up in the context of a massive IT production house of 850+ employees in the United States in the 1990s, the first decade of standardized and mainstreamed screen-based productivity. It is not difficult nor irrelevant scholarship to comprehend the scripted affectivity of early VR spaces in this way, thus complicating the notion of VR’s percept-affect “abstractions”.
Conclusion: The dilemma of under and over-theorised affect
Andrew Murphie has more recently queried the easy conjunction of new media objects and experiences with variegated cognitive models. Murphy argues it is often difficult to determine, even, “whether science, philosophy, or cultural work come first” when it comes to theorizing cognition and engagement, in new media, and elsewhere (2005, 1). Further, he argues that: “cognitive science and philosophy are no longer just science and philosophy, but are playing a role in a complex, competitive ‘market’ involving cognitive models, a market that forms a growing, and increasingly important, submarket within the global markets of contemporary capital” (2005, 1). It is this tension between science, art/philosophy and capital, where ideological constructions of desiring subjectivity and their resistance takes place directly at an abstracted human-machine interface, the difficult theoretical lack of firstness in theorizing that interface epitomizes the dilemma of undertheorised affect. Given that mediated ‘affect’ and affective experiences can be indistinguishably narrated and celebrated as a phenomenological and social good across corporate, aesthetic and experimental engagements with new media forms, then in the realm of VR, where lines between the virtual and the actual are problematised and interconnected, the purportedness of ‘affectivity’ to signify the richness of any particular VR regime whatsoever hardly complete’s our appreciation of what a work does or actualizes in the cultural field of its conception. Rather than proposing a cognitive model as an alternative, I invoked Sedgwick on Silvan Tomkins’ work, to be considered alongside Deleuze, just to better glimpse what might be at stake in theorising VR affectivity experience outside of scopic regimes of perception. Tomkins and Deleuze, perhaps considered each of them before and ‘after’ the ‘linguistic turn’, and equally concerned with the materiality, drives, and circuitry of the body that goes on in parallel to, or beneath signification (language), can be used to think through the problem of the dialogue of perception with social logics and transitional non-core subjectivity in a way that is of interest to theorists of VR. My preliminary thoughts in this post are that theory about VR could benefit from more rigorous, but no less speculative, attention to non-scopic affect, as it is registered differently in art works and across the art work’s context of specific socio-political realities. Incidentally, I think Anna Munster’s writings on new media art are mindful of this and I look forward to upcoming posts on her work on blog here.
Bibliography
Hemmings, Clare. (2005). ‘Invoking Affect: Cultural Studies and the Ontological Turn’. Cultural Studies Vol. 19. No 5, pp. 548 – 567
Sedgwick, Eve Kosofsky and Frank, Adam. (2003) ‘Shame in the Cybernetic Fold: Reading Silvan Tomkins’. in Touching Feeling: Affect, Pedagogy, Performativity. Duke University Press: Durham, pp. 92-121.
Murphie, Andrew. (2002) ‘Putting the Virtual Back into VR’ in Massumi, Brian (ed.) A Shock to Thought: expression after Deleuze and Guattari. London: Routledge, pp. 188-214.
Murphie, Andrew. (2005) ‘The Mutation of “Cognition” and the Fracturing of Modernity: cognitive technics, extended mind and cultural crisis’, Scan 2, 2 (September) [Special issue on memory, media, and embodied cognition]
Steuer, Jonathan. (1992) ‘Defining Virtual Reality: Dimensions Determining Telepresence’ Journal of Communication, 42, Autumn, pp. 73-93. –Posted by Rachel O’Reilly for Stream 2.
