Multimodality and Intermediality: Humanist Research in a Digital World (MIDWorld)

The interdisciplinary graduate school MIDWorld is a collaboration between the Centre for Research in the Humanities and the research team in Multimodal Communication at Örebro University and the Centre for Intermedial and Multimodal Studies at Linnaeus University.

Digitalization has revolutionized our ways of learning, working, and socializing, and offers highly functional possibilities to create and share knowledge across communities. Individuals have direct access to more knowledge than ever before, yet society struggles with fast-spreading disinformation and isolating filter bubbles. A fundamental aspect of digitalization is the convergence of systems of information and communication into multimodal media spaces which combine, for instance, language, image, color, music, and motion, and form intermedial relationships in the way they are constantly redesigned and redistributed across platforms and contexts. This mix of modes and media calls for new methods and approaches. The interdisciplinary graduate school MIDWorld provides emerging researchers in the Humanities with theories and methods for critically analyzing the mix of modes and media in our digital world. Multimodality and intermediality research offer complementary frameworks to understand communication in a digitized society and for analyzing meaning-making as part of social, political, historical, aesthetic and educational practices.

A total of nine doctoral students are enrolled in MIDWorld, of which five are enrolled at Örebro University and four at Linnaeus University.

In addition to the research conducted by the doctoral students of the research school, the research groups on Multimodal Communication and the Centre for Intermedial and Multimodal Studies co-organize seminars, workshops, conferences, and research retreats.

Contact person at Örebro University: Anders Björkvall (anders.bjorkvall@oru.se)

Contact persons at Linnæus University: Beate Schirrmacher (beate.schirrmacher@lnu.se) and Niklas Salmose (niklas.salmose@lnu.se)

PhD students of MIDWorld

Yagmur Atlar, English Literature, Linnæus University

(De)polarised Borders and Digitalised Environments: Intermedial Enactment on Nature and Human Ecology in Contemporary Theatre 

This project focuses mainly on digitization of stage in contemporary theatre, foregrounding how the projection of ecocriticism is transformed, appearing in a more digitalised fashion. contemporary theatre’s effectiveness in tackling current subject matters, particularly nature and human ecologies, interconnects ecocriticism, digitalisation and media studies. Regarding the connectivity of contemporary theatre and humanist research, the intermedial and multimodal nature of contemporary theatre also catalyses the prominence of the audience’s perception rather than focusing on the playwright’s. The centralisation of the audience as the most significant component of theatre highlights the importance of autonomous meaning-making in increasing awareness towards global and current ecological issues. Therefore, the project aims to utilise the word depolarisation to refer to the blurred distance between the stage and auditorium as in the contemporary theatre with the impact of digitalisation, the distance between the actors/the playwright and the audience is minimised and are coalesced. This unification renders the audience more active participants, increasing their responsibility towards the ecological issues demonstrated on stage. As a result, such responsibility makes the audience not only the witnesses but also the components of the staged nature and human ecology. The second area to be analysed within the scope of depolarisation is the relationship between the human and non-human, focusing on the diminished distance between human beings and other species referring mainly to animals. The third aspect of depolarisation focuses on the notion of reality and the blurring of the distance between virtual reality and the real. Targeting to analyse the challenges stemming from the digitalisation of the world and stage, the depolarisation of the virtual and real brings out complexities regarding how it problematises the reliability of perception. Thus, the aim of this research is to analyse digitalisation in contemporary theatre and how digitalisation of stage enhances the multimodal and intermedial aspects within the context of foregrounding the phenomenon of depolarisation of nature and human ecology.

Anna Brynhildsen, comparative literature, Örebro University

Romantic Misanthropy – Images of America in the Media Ecologies of Father John Misty and Lana Del Rey

In my PhD project I am exploring how the image of America is made in the media ecologies of two contemporary American writers of song lyrics/artists: Elizabeth Grant, more known by the name Lana Del Rey, and Joshua Tillman, more known as Father John Misty. The two musicians often move in and out of each other’s media ecologies, textually as well as visually and audially. Another significant common denominator is that both assumed a persona, after years of making music under other names, with other expressions. I found it interesting how they seemed to change expressions while changing names and personas – which strengthens the notion of them doing a sort of performative literature. I’m examining them as media ecologies, since I need a theoretical framework free enough to include different medial and modal elements: verbal, visual, audial, but also intertextuality, cultural stereotypes, cultural history, media history. And finally, processes and activities – since my object of study is dynamic and perpetually changing, as opposed to a static one.

The main question is two-fold – the first part aims to answer the question of “What?” and the other one the question of “How?” What meaning and themes are produced in the media ecologies, and how are they made due to the affordances of different media types, and the combination of them?

When it comes to themes and the meaning being made, an important aspect is that The US is a central theme in both artists’ song lyrics and aesthetics – an aesthetic that often moves between opposing poles considering the depiction of time and space. The poles, to name a few, are made up from the nostalgic and romantic opposed to the realistic and the contemporary. America is thus partially presented romantically – both by text, images, and music – but just as often in a dark and cynical way. I want to examine and describe aspects about how intermedial literature, and the themes and contents of my study of choice – such as the hyper medialized image of America – are made in the 21st century.

Emma Flodqvist, English Literature, Linnæus University

Jack Huang, comparative literature, Örebro University

Life Writing in Games: Exploring Self-Experience and Life Narratives in Computer Games

In current times of digital fragmentation, there is a pressing need to understand how we craft memorable experiences and go about our lives in virtual/physical hybridity. This project’s overarching aim is to acquire this understanding from one site of investigation, i.e., immersion, specifically, the symbiotic relationship between players and player characters that engenders the performance/articulation of autobiographical/alternate identities in computer games (or what may be described as life writing in games).

To tackle this objective, Life writing in games will engage in and contribute to the scientific intersection of media studies with narratology, and with life writing studies. For the former, the project will expand upon existing narratological studies on subjective representations of scripted characters, or how the perspectives of these characters are constructed, with studies on player subjectivity to holistically account for what one may call perspectives in computer games.

For the latter, Life writing in games will align with and add knowledge to Sidonie Smith and Julie Watson’s automediality framework, which laid bare the rendition of subjectivity across various forms/modes of digital media. In contrast to past examinations of how we inscribe our lives in the digital, e.g., how digital blog entries are used for self-representation, I seek to grapple with how we inscribe our digital lives.

All in all, I seek to identify through what combinations of medium specific affordances we perform identities and cumulate lived experiences in gameworlds and the strategies/semiotic resources that may assist us in acquiring better understanding of our lives in computer games. I will do so by following players of computer games, as they craft autobiographical artifacts/write life stories about their experiences in virtual gameworlds. In this way, I hope to initiate discussions on our experiences in digital worlds in a time when digital technology becomes growingly integral to our being.

Anna Ishchenko, English Literature, Linnæus University

This doctoral project aims to examine the ways video games have the potential to aesthetically transmediate the ecological crisis as narrative and poetic entities. In particular, the project will focus on, first, how the literary and the poetic acquire new ontological status in the medium of the digital and, second, how this brings about new affordances to mediate the relationship between human and non-human worlds in an emotionally engaging way. While postclassical narratology will be utilized to conceptualize and analyze video games as eco-narratives, the ones that present affectual potential to facilitate emotional engagements with ecocritical questions and concerns, intermediality will be employed to bring to the fore the aspect of medium-specificity.

Elin Stenberg, Swedish Language, Örebro University

Presence at a distance

This PhD project is a study of how presence is realised and designed, multimodally and semiotically, in online learning environments in Swedish higher education. The study adopts a social semiotic approach (Kress 2010; van Leeuwen 2005) which is concerned with how modes and semiotic resources (such as speech, writing, gestures, images, audio etc.) contribute to communication and meaning-making in various social contexts (Bezemer & Kress 2016). A social semiotic approach makes it possible to analyse how various resources are co-deployed in online learning environments and how these resources contribute to a notion of presence, when physical presence is ruled-out. A presumption is that a realisation of presence is central in the processes of meaning-making and learning in online learning contexts. Questions raised in the study concern which semiotic resources are exploited in online learning environments and how these contribute to a realisation of presence. Another question addresses the affordances of the resources, i.e. their potentials and limitations for meaning-making, and how these are perceived by instructors and participants. The choice of resource is central to the shaping of knowledge (Jewitt 2013), and to explore instructors’ choices and design the study takes on a multimodal and design-oriented approach to learning (Björklund-Boistrup & Selander 2022). This brings an understanding of learning in different contexts and how resources are drawn upon in situated meaning-making.  The methods employed in the study are social semiotic multimodal and ethnographic consisting of long-term observations of classes in higher education. The observations are carried out through digital participation where communication between participants is observed throughout the course of their education. The classes observed are recorded and analysed through a multimodal lens. Semi-structured interviews are conducted to explore instructors’ design choices and participants’ perceptions of presence. These ethnographic elements are combined with semiotic analyses of online learning environments, artefacts, and resources used in the learning contexts. The combination of methods allows for a holistic understanding of how presence is realised in online teaching and learning.

Felicia Stenberg, English Literature, Linnæus University

Make It Make Sense: Internet Memes and Environmental Crisis In Speculative Fiction

There is a widespread view in today’s mainstream media: that the Earth is dying, and that we, the people, are killing it. Scientists, of course, know that while, yes, the sun will eventually burn out or fade away, killing the Earth 8 minutes later, the more accurate facts of the matter are these: we are not killing the Earth, we are only killing ourselves. These ever-increasing dire trajectories in the news permeate social media, where amongst the denial, there is an abundance of macabre acceptance, anger, satire, irony, hope, and humor, often taking the shape of Internet Memes (IMs). Internet memes are communicative tools that simultaneously evolve and devolve, eternally becoming, and are never static in meaning, form, function, or content. IMs have now become such an intricate part of online communication that they have begun to surface outside of the confines of the Internet. In particular, IMs have transmediated into literature. Whether viral text posts, or amalgamations of image and text, these inherently malleable intermedial social media phenomena are frequently used as macabre short-hand reactions to the increasingly dire state of the ecosystem. In my project, I will explore how IMs and other viral internet phenomena contribute to meaning-making with their adaptability in fiction, to explore what such transmediations mean for how speculative fiction narrates environmental breakdown, and what this can mean for the Earth of today and tomorrow. How does this disturbance of (formerly) clearly demarcated media utilize the in-between of transmediation to challenge normative understanding of ecology and the hierarchies of knowledge production and distribution that maintain them?

Piia Suomalainen, Swedish Language, Örebro University

Images of Sweden in a digitalized society ­– a multimodal discourse analysis of Sweden and Swedishness on TikTok.

The image of Sweden is undergoing change: Swedish society has become more heterogeneous demographically, while the Swedish self-image as a good and equal nation (Hübinette & Lundström 2014) is challenged by various contemporary crises and conflicts. At the same time as the image of Sweden changes, social media and various digital applications have become a central part of people's everyday lives and have played a significant role in shaping public opinion (Facht 2023: 56), with TikTok being the platform that has experienced the greatest increase in users in Sweden in 2023 (Internet Foundation 2023: 237–238). There is a lack of research on images of Sweden in a digitalized society, and there is a need for deeper understanding of how digitalized images of Sweden are shaped through the interaction between multimodal representations on social media and the algorithms that govern content flow within them.

The purpose of this dissertation is therefore to map out how images of Sweden and Swedishness are formed through the affordances of the globally popular social media platform TikTok, and to investigate how user agency (Jones et al. 2015: 10) influences this process.

The dissertation draws on both linguistic and media and communication studies, and combines thinking tools from social semiotics (van Leeuwen 2005) critical discourse studies (van Leeuwen 2008) and autoethnographic netnography (Zulli & Zulli 2022)

Wenting Zhao, rhetoric, Örebro University

Health communications on social media: the case of reproductive health in China

The proposed project will use Multimodal Critical Discourse Analysis, to analyze and evaluate social media content about women’s reproductive health in China. Aligned with the national initiative, "Healthy China," the project aims to enhance awareness of healthy lifestyles and elevate health literacy across the population. The Healthy China project has adopted a social media focus due to a growing awareness that people now mostly find health related information from social media which is often of poor quality. Against this context, the government aims at producing material which is accessible to all levels of education across the population.

This PhD research, situated within the broader context of social media, medical rhetoric, and public understanding of health, explores social media information on women's reproductive health within government initiatives and content generated by influential figures. There is a wider research gap in understanding how people now gain, share and discuss health matters across social media, particularly in the Chinese context. The ultimate aim of this project is to produce findings which contribute to understanding in particular to this local context, evaluating more recent government led social media content and how users engage with this, providing insights which can be used to guide and shape interventions by health care professionals and health authorities. This has particular relevance in the Chinese context in relation to the unfavourable gender and sexual norms that impose structural barriers to women in society.