Multimodal discourse analysis is a method that takes into account multiple modes of communication and how they interact with one another. 

Multimodal Discourse Analysis

Multimodal discourse analysis is an approach that looks at multiple modes of communication such as text, color, and images. It is a method of discursive analysis that looks at not just how individual modes communicate, but how they interact with one another to create semiotic meaning.

Kress (2010) states further that, “Using three modes in one sign - writing and image and colour as well - has real benefits. Each mode does a specific thing: image shows what takes too long to read, and writing names what would be difficult to show. Color is used to highlight specific aspects of the overall message” (p. 1). Therefore looking at multiple modes at once elicits a more nuanced and complex analysis, especially when looking at online environments. 

Illustration

Here is the beginning of a news item on US television channel ABC, on a mass shooting in Odessa, Texas.

We see

(a) how the words of the anchor are coordinated with images: relevant images pop up simultaneously while words are pronounced for which the images can offer illustrations.

(b) The words make the images understandable: they are "glosses" for the images, some of which do not convey much information in themselves. This photo, for instance, does not "speak for itself". Images become understandable because they are accompanied by words, and the words become more vivid by coordinating them with images.

(c) the words and images are, in turn, supported by visual text overlays. We thus get three "modalities" simultaneously interacting: the spoken words, the visual text, and the images. 

(d) all three modalities together are put in a "syntax": an orderly sequence which, jointly, conveys one message. Some of the modes used - think of the photos - would not suffice to achieve the same effect. It's the coordinated and ordered interplay of the different modalities that produces the message.

 

References

Kress, G. (2010). Multimodality: A social semiotic approach to contemporary communication. London: Routledge.