Editorial

Exploring affect in interaction design, interaction-based art and digital art.

Issue Edited by Jonas Fritsch and Thomas Markussen

   Issue:ePubIssue:pdf   Editorial:pdf

The notion of affect does take many forms, and you’re right to begin by emphasizing that. To get anywhere with the concept, you have to retain the manyness of its forms. It’s not something that can be reduced to one thing. Mainly, because it’s not a thing. It’s an event, or a dimension of every event. What interests me in the concept is that if you approach it respecting its variety, you are presented with a field of questioning, a problematic field, where the customary divisions that questions about subjectivity, becoming, or the political are usually couched in do not apply.
(Massumi, Of Microperception and Micropolitics, 2009, p. 1)

The aim of this special issue of the Fibreculture Journal is to address some of the contemporary challenges involved in working with affect across disciplines and practices that centre on the use of interactive- or digital technologies. The issue has a special focus on interaction design, interaction-based art and digital art. The pivotal question, as we see it, might be framed roughly like this: How do we explore the “field of questioning” that arises when we approach the affective in relation to interaction design, interaction-based art and digital art? What is the use of disciplinary goals when the affective has been proven most valuable in trans-disciplinary theory? Where do we go from here, that is, how can we continue working with the notion of affect, develop it in new theoretical, analytical and practical domains? What key concepts would emerge from this continued trajectory and how would they feed back onto the theoretical propositions? How would they resonate within and with-out existing disciplines, creating novel links and assemblages?

With this special issue we present ongoing practices across disciplines that all engage with this challenge of working with affect—both analytically and artistically, but always creatively. We are especially interested in the way in which changing concepts of affect are taken up and modulated within interaction design, interaction-based art and digital art. For example, some concepts of affect coming into these areas go beyond the “personal” interaction with the technology, indeed beyond (or run beside) many of the assumptions of interaction design, including those grounded in phenomenology. They understand affect as an impersonal as much as, or even sometimes as opposed to, an intimate dimension of relational capacity. As proposed in the work of Deleuze and Guattari, and more recently, in very different ways, in the work of Brian Massumi, Patricia T. Clough, Nigel Thrift, and others (see below), affect comprises intensities and speeds, in which the living and nonliving, human and nonhuman, differentially affect and are affected by each. Such new understandings of affect have consequences for notions of interaction or interactivity, and meet other concepts of affect and interaction in ways that challenge basic assumptions about interactive media and digital technology in material, processual and experiential terms.

It is important to underline that this issue of the Fibreculture Journal is not concerned with the ‘affective turn’ per se. Rather, assuming the importance of considering affect across a number of disciplines, we are particularly concerned with affect as it is worked with in interaction design, interaction-based art and digital art. As Marguerite La Caze and Henry Martyn Lloyd clearly demonstrate in their introduction to the Parrhesia issue on ‘Philosophy and the Affective Turn’, studies of affect have a long history within philosophy (La Caze & Lloyd, 2011). In their introduction, the ‘affective turn’ is used to describe a specific phenomenon in cultural studies/critical theory in the 90s marking an increased cross-disciplinary research interest in pre-cognitive bodily forces, notably in how these forces are involved in the construction of human subjectivity, identity and our engagement with other people and technology.

However, after years of intense study we have now reached a point where the analysis of the affective has proliferated and spread into a number of disciplines in an attempt to enrich the understanding of the pre-individual forces that function on the level of the formation of experience—from the micro-perceptual to the macro-political (Gregg & Seigworth, 2010). Lisa Blackman and Couze Venn have edited a special issue of Body & Society in which they attempt to sketch out the kinds of trans-disciplinary collaboration and engagement enabled by the concept of affect as these have emerged across the humanities and the natural, social and human sciences (Blackman & Venn, 2010). Indeed, Brian Massumi has described affect as a “world-glue” (2000: 187), bringing together different levels of experience and working across traditional dichotomies. As such, it seems that affect also has a further role to play as a kind of “disciplinary-glue”, making disparate practices resonate through the conceptual development and practical exploration of affect—and derived concepts, analyses and experimentation. Rather than seeking a unified understanding of what constitutes affect or the affective, it will be necessary to develop rigorous approaches across disciplines under an affective heading, thus bringing forth the multiplicity of these affective explorations, ensuring an enriching dialogue in-between disciplines, and reaching out of an academic context as well.

In an afterword to the above-mentioned special issue on affect published by Body and Society, Patricia T. Clough offers interesting ideas about the future of affect studies but leaves the question of technology relatively unaddressed (Clough, 2010). Turning towards the field of Human-Computer Interaction (HCI), however, a range of technology-oriented experiments have been carried out in the name of Affective Computing (e.g. Picard, 1997) or Emotional Design (Norman, 2004). These approaches have been criticized within HCI for reducing the complexity of the affective in an attempt to make it formalizable and structurable in computational and informational terms (Sengers et al., 2002). Recently, this informational approach to understanding affect has been countered with what has been termed an interactional approach (Boehner et al. 2005; Höök et al., 2008). Here, an alternative model of emotion as interaction is introduced, allowing an investigation into how interactive systems are experienced as culturally mediated and socially constructed. The relation between the affective and emotional remains relatively unexplained, however. All this leaves us with a possible space of resonance for many of the findings arising from the affect theoretical work done in and around cultural and critical theory.

Patricia T. Clough’s introduction to ‘The Affective Turn’ from 2007 is explicitly concerned with how the ‘affective turn is necessary to theorizing the social’ (Clough, 2007). Nigel Thrift identifies five different schools of affective thinking in ‘Turbulent Passions’ (Thrift, 2007). Interestingly, coming out of psycho-geography and non-representational theory, these schools end up mixing together new theoretical assemblages. Brian Massumi offers another affective trajectory. In Massumi’s work, the philosophical focus moves from Spinoza’s basic notion of affect as the ability to affect and be affected, through the writings of Gilles Deleuze, to other conceptual allies, Gilbert Simondon and Alfred N. Whitehead, at the same time making references to work in developmental psychology carried out by Daniel Stern, as well as building heavily on William James’ notion of radical empiricism. For Massumi the notion of the affective has been central for re-conceptualizing the emergence of subjectivity, which is not a pre-given entity. One aspect of this is the way in which interactive media and technologies may open up new territories for engaging pre-cognitive sensations and feelings in bodily experience, in what are sometimes referred to as ‘technologies of emergent experience’ (Markussen, 2005: 2). This re-conceptualization has not only been valuable for understanding the aesthetics of interaction as it is continuously explored in interaction-based art, digital art, design and architecture (see e.g. Massumi, 1998 & 2007). It has also become clear that we need to include the political and ethical in the notion of the aesthetic, which in Guattari’s terms leads us to consider the aesthetico-political. Bodies always find themselves affected by fields of forces—forces of ideology, techniques and practice—that attune these bodies to certain regions of action or potentialities for action (Massumi, 2008: 6).

With the advent of new media and related technologies, artists and interaction designers are offered rich opportunities for exploring the many intersections between affect, sensation and action. At the smallest scale we find imaging technologies that allow artists such as Olafur Eliasson or Bill Viola to explore microscopically affective layers of sensation, of which we may not usually be consciously aware. Turning towards the area known as tactical media, one could also find examples such as surveillance technology used subversively in public space, either to enhance the affective social attunement between bodies—as in projects by Rafael Lozano-Hemmer, Markus Kison or Ben Rubin—or as an instrument for micro-political acts of resistance that disrupt existing systems of control and power in order to liberate the body and construct counter-publics—as seen most vividly in iSee by The Institute for Applied Autonomy or Roderico Dominquez’s Transborder Immigration Tool.

It seems we are at an important point in the exploration of the affective today, one at which we are moving from arguing that it is important or even necessary to consider affect, to actually working with how affect theory changes different kinds of practices—and not least how these practical explorations feed back into and change the theoretical assumptions. This is why we are interested in how concepts and meetings of concepts feed into the practices that we find in interaction design, interaction-based art and digital art. How do you design affectively, for instance? How can we use the insights from and around current explorations of affect in a continuously mobilizing and dynamic way, creating new relational events across disciplines and practices, feeding into new ways of thinking, doing and acting? If the concept of change is so integral to the understanding of affect, how might we actually start “living” by it—academically, or in the manner of practice-based research, research-through-design or research-creation? What kinds of politics does the concept of affect offer? If, as Brian Massumi states, it is possible to talk about the affective as bringing about an expanded empirical field in various disciplines, how might we continue an exploratory politics of radical change pursued by other than philosophical means? And how do such questions come into interaction design, or the more general meeting of technology and the social?

Affect has been coupled with the notions of interaction and the virtual in an attempt to increase understanding of how technology engages and re-distributes human bodies in relation to processes, time and change. The need for addressing the question of what affect, as a new foundational concept, offers to the understanding of interaction design, interaction-based art and digital art seems clearer today than ever. From a variety of intersecting backgrounds, the contributions to this issue address this question in experimental, practical, and conceptually new ways.

We begin with Adam Nash’s article, ‘Affect and the Medium of Digital Data’, in which he argues that the notion of affect is critically important for understanding how digital data lends itself as a medium and material for creating virtual environments. Too often, Nash’s argument goes, the term ‘virtual’ is taken in the sense of virtual reality, a dematerialised realm of digital data, which is thought of as being ontologically distinct from material reality. While this idea of the virtual certainly was influential in the 1990s in terms of how, for instance, the internet was conceived of as a distant cyberspace, Nash is sceptical about it becoming ‘a signifier for any interaction that is facilitated on a digital network and induces affect in the material world, or vice versa’. Drawing attention to his own artwork in Second Life, on the internet and elsewhere, Nash gives various examples of the way that digital data can be modulated so as to enable non-human and human bodies to engage affectively with each other, beyond the digital-material divide. This is perhaps experienced most evidently in Trace Aureity, an interactive audiovisual sculpture in Second Life where avatars gradually gain a greater degree of autonomy, thereby transcending ‘the linear mapping between human user and the user’s humanoid avatar’. Hence, for Nash, there is a need for an integrative ontology, one that views virtual environments and more generally ‘virtual art’ as ‘a continuum of force and materiality which can be modulated and re-modulated by the artist so new cycles between digital networks and material reality can emerge, between non-human and human bodies.’ And the notion of affect is a core concept in this ontology.

The idea that the notion of affect is key for understanding how digital technology gives artists access to work with hitherto unexplored forms of interaction is also central in the second essay, ‘Affect and Care in Intimate Transactions ’, by Lone Bertelsen. Bertelsen shifts the focus of attention from ontology to ethics insofar as she discusses how ethics may be rooted in the way that bodies mutually affect and are affected by one another. In her article, she examines Intimate Transactions, an immersive interactive installation where participants situated at two distant locations can experience intimate transaction. Each participant uses a physical interface called a ‘Bodyshelf’. By gently moving their bodies on the Bodyshelf they instigate intimate transactions, which influence an evolving world of non-human creatures. As these creatures meet in the screen-world, they can ‘move together as one semi-merged avatar’. At the same time, vibrations onto the lower back and lower abdomen of the participant’s bodies are activated. For Bertelsen, this can be seen as an instance of ‘co-affective collaboration’, which focuses on ‘trans-subjective collaboration and a logic of affects’. This leads to a form of experience which cannot be adequately accounted for with the notion of interactivity. This is because, while the notion of interactivity presupposes the individual and the subjective as pre-existing categories, co-affective collaboration takes place at a pre-individual and trans-subjective level. In fact, the ultimate, but implicit conclusion of Bertelsen’s argument seems to be that, in the vocabulary of media art theory, interactivity must be supplemented with trans-activity as a new foundational concept. For years critical thinking on digital art has put its trust into the explanatory strength of interactivity. Bertelsen suggests that a re-thinking is necessary and that affect is a promising starting point for future work.

Susan Kozel’s ‘AffeXity: Performing Affect with Augmented Reality’ situates us immediately in the middle of a transdisciplinary inquiry into affect in cities and a-fixity as an urban condition. In the article, Kozel unfolds the affective explorations carried out in an ongoing interaction design project, AffeXity, experimenting with artistic practices from dance improvisation, video shooting, digital image editing to sound composition, combined with the daily practices of moving through a city and using mobile devices. Kozel writes at the intersection between conceptual coherence and artistic direction in an attempt to bring to life the way that working with affect simultaneously modifies both theory and practice, in a writing style that brings to the surface the affective explorations involved. Thinking with and through affect theory, digital media and social choreographies, Kozel develops a notion of performance triangulated across bodily movement, emergence and shimmering. Starting from the basic assumption that designing affectively and designing for affect are two different things, Kozel proposes a range of affective sensibilities. Her work straddles practical and theoretical activities because ‘it is used in the process of generating the movement and media at the same time, as it is a way of engaging with theories of affect—it is a method for generating artistic and theoretical content.’

In the fourth article Mark Gawne emphasises the lack of awareness inside compositionist analyses of the way in which affective technologies are used to organize labour in the post-Fordist condition. Theorists of affective labor such as Hardt and Negri have been successful in demonstrating that in ‘the passage to post-Fordism, the labour of producing affects, communication, knowledge, the creation and maintenance of relationships and the cultivation of attention emerge as key economic terrains’. Gawne argues however that while these theorists have identified the need to consider the problem of immaterial production, they have less to offer in critiquing how technology is used and misused ‘to subordinate user affect to the imperatives of capitalist valorization’. For over a decade, Human-Computer Interaction (HCI) has focused on developing ‘technologies that aim to sense, recognize and modulate user affect’. However, through his critical and enlightening examination of recent developments in affective HCI, Gawne identifies a blind spot insofar as HCI research seem reluctant to consider the impact that these technologies have on the bodies involved. Gawne’s contribution consists in integrating a discussion of affective HCI with the perspective of compositionist analysis. In so doing, he remedies the inherent limitations of both fields. The heuristic value of this endeavor becomes evident in Gawne’s analysis of smile-scan, a technology developed by Japanese company OMRON for the purpose of measuring the face expressions of workers within workplaces.

The four articles are followed by two conversations. The first of these is ‘Multimedia Mixing and Real-time Collaboration: Interview with Sher Doruff about the development and use of KeyWorx, the Translocal and Polyrhythmic Diagrams.’ This conversation recounts a largely undocumented chapter in the history of the media art and technology nexus. While books on media art flourish that contain accounts of the ways that artists have exploited existing technologies coming out of research and innovation in industry for artistic purposes, the conversation with Sher Doruff is remarkable. It reveals the way in which artistic experiments themselves have launched new networking technologies which in many ways anticipate more recent developments in social networks. In a personal conversation with Andrew Murphie, Doruff takes the reader behind the performance scene in the 80s and 90s, where, as an artist, working in New York and Amsterdam, she collaborated with various performers, dancers, musicians, and programmers in developing Keystroke. Keystroke , or Keyworx as it was later re-named, is a virtual studio environment, which in 97 gave artists and performers the opportunity to engage in translocal, real-time collaborative performances. That is in 97! Even though the notion of affect may not appear to be placed at centre stage, the affective is felt to be present as a form of co-existence among the artists and performers, one for which the Keystroke technology is designed. In translocal performances an intense ‘synchronous interaction’ emerged between the performing bodies. Performers always need to find a rhythm together. But translocal performance, where bodies are separated, is different, or as Doruff explains: ‘You don’t have perceivable body language between you. So you have to find other ways to find that kind of synchronisation. It’s incredibly intense and affective’. This affective synchronisation is not only being explored in many of Doruff’s own art projects, which are presented throughout the conversation; it is also closely aligned to the co-affective experiences discussed in the earlier essays by Nash and Bertelsen.

In ‘Entertaining the environment’, a conversation between Andrew Goodman and Erin Manning, the affective is conceptualized in terms of ‘relation’, which is counterposed to ‘interactivity’. ‘Interactivity’ or ‘interactive art work’ are two concepts used by Goodman and Manning to describe artwork, where the ‘art event’ is drowned out by the ‘technology event’. Or, as Goodman says, works of art ‘that seem invested in a demonstration of technology’s capabilities (and/or the artist’s technological skills)’. Relational art, on the other hand, is more interested in what art can do and not just what technology can do. It activates experiences that do not place the viewer at the centre of the experience, but invites her instead to participate in creating events, and this subverts the hierarchy of subject and object. This disruptive aesthetic effect or ‘tweaking of experience’ is not a result of technology use, but can be achieved by simple means and techniques such as those introduced by the conceptual art movement in the 1960s and 19070s. By tracing the roots of relational art back to conceptual art, Goodman and Manning point towards affective possibilities ‘at the fringes of technology’.

Biographical Notes

Jonas Fritsch has a PhD in Interaction Design and is currently Assistant Professor at Aarhus University, Denmark. Here he works on a multitudinous thinking-together of interaction design and affect theory in conjunction with practical design experiments carried out at CAVI.dk (Centre for Advanced Visualization and Interaction) and at the Participatory IT Centre (pit.au.dk) in the Department of Aesthetics and Communication. Jonas holds an MA in Information Studies with a supplementary degree in Aesthetics and Communication from la Nouvelle Sorbonne, Paris, is a member of the SenseLab, Concordia University and participates in the editorial collective of Inflexions , a journal for research-creation.

Thomas Markussen holds a MA in Comparative Literature and Semiotics and a PhD in Interaction Design focusing on embodied and emotional experiences. He is currently Associate professor at the Department of Communication Design at Kolding School of Design, Denmark. Markussen’s research interests lie within design activism and urban interventions, embodiment in interaction design and new media art, and methods of practice-based research in interaction design.

References

• Blackman, Lisa and Venn, Couze. ‘Affect’, Body & Society 16.1(2010): 7.
• Boehner, Kirsten, DePaula, Rogerio, Dourish, Paul, and Sengers, Phoebe. ’Affect: from information to interaction’, Proceedings of the 4th decennial conference on Critical computing: between sense and sensibility (New York: ACM Press, 2005): 59–68.
• Clough, Patricia T. ’Introduction’, in Patrica T. Clough and Jean Halley (eds.) The Affective Turn: Theorizing the Social (Durham: Duke University Press, 2007), 1–33.
• Clough, Patricia T. (2010). ’Afterword: The Future of Affect Studies’, Body & Society 16.1: 222–230.
• Gregg, Melissa and Seigworth, Gregory J. (eds.) The Affect Theory Reader (Durham: Duke University Press, 2010).
• Höök, Kristina, Ståhl, Anna, Sundström, Petra, and Laaksolahti, Jarmo. ‘Interactional Empowerment’, *CHI 2008: Proceeding of the twenty-sixth Annual SIGCHI Conference on Human Factors in Computing Systems (New York: ACM Press, 2008), * 647–656.
• La Caze, Marguerite, and Lloyd, Henry Martin. ‘Editors’ introduction: Philosophy and affective turn’, *Parrhesia *13 (2011): 1–13.
• Markussen, Thomas and Birch, Thomas, ‘Transforming Digital Architecture from Virtual to Neuro: an Interview with Brian Massumi’, Intelligent Agent 5.2 (2005), https://www.intelligentagent.com/archive/Vol5_No2_massumi_markussen+birch.htm.
• Massumi, Brian. ‘Sensing the virtual, building the insensible’, Architectural Design (Profile no. 133), 68.5–6 (1998): 16–24.
• Massumi, Brian. ‘Too-Blue: Colour-Patch for an Expanded Empiricism’, Cultural Studies 14.2 (2000), 177–226.
• Massumi, Brian. ‘The thinking-feeling of what happens’ in Joke Brouwer & Arjen Mulder (eds.) Interact or Die! (Rotterdam: NAi/UitgeversPublishers, 2007), 70–91.
• Massumi, Brian and McKim, Joel. ‘Of Microperception and Micropolitics: An Interview with Brian Massumi, 15 August 2008’, Inflexions: A Journal for Research-Creation 3* *(2009), https://www.senselab.ca/inflexions/volume_3/node_i3/massumi_en_inflexions_vol03.html.
• Norman, Donald A. Emotional design: Why we love (or hate) everyday things (New York: Basic Civitas Books, 2004).
• Picard, Rosalind W. Affective computing (Cambridge, MA: MIT Press, 1997).
• Sengers, Phoebe, Boehner, Kirsten, Mateas, Michael and Gay, Geri, 2008, ‘The Disenchantment of Affect’, Personal and Ubiquitous Computing 12.5 (2008): 347–358.
• Thrift, Nigel J. Non-representational theory: space, politics, affect (London: Routledge, 2007).