Multimodality: Foundations, Research and Analysis – A Problem-Oriented Introduction (2017)
By John Bateman, Janina Wildfeuer & Tuomo Hiippala
Berlin, De Gruyter Mouton, 2017, 415 pp. including index
ISBN 978-3-11-047942-3
https://www.degruyter.com/view/product/473272
Overview. This textbook provides the first foundational introduction to the practice of analysing multimodality, covering the full breadth of media and situations in which multimodality needs to be a concern. Readers learn via use cases how to approach any multimodal situation and to derive their own specifically tailored sets of methods for conducting and evaluating analyses. Extensive references and critical discussion of existing approaches from many disciplines and in each of the multimodal domains addressed are provided. The authors adopt a problem-oriented perspective throughout, showing how an appropriate foundation for understanding multimodality as a phenomenon can be used to derive strong methodological guidance for analysis as well as supporting the adoption and combination of appropriate theoretical tools. Theoretical positions found in the literature are consequently always related back to the purposes of analysis rather than being promoted as valuable in their own right. By these means the book establishes the necessary theoretical foundations to engage productively with today’s increasingly complex combinations of multimodal artefacts and performances of all kinds.
Frontmatter | Pages I-IV
Contents | Pages V-VIII
How to use this book | Pages 1-4
Part I: Working your way into ‘multimodality’
1 Introduction: the challenge of multimodality | Pages 5-21
2 Recognising multimodality: origins and inspirations | Pages 22-73
3 Where is multimodality? Communicative situations and their media | Pages 74-111
4 What is multimodality? Semiotic modes and a new ‘textuality’ | Pages 112-136
Part II: Methods and analysis
5 The scope and diversity of empirical research methods for multimodality | Pages 137-168
6 Are your results saying anything? Some basics | Pages 169-210
7 Multimodal navigator: how to plan your multimodal research | Pages 211-232
Part III: Use cases
Use case area 1: temporal, unscripted
8 Gesture and face-to-face interaction | Pages 239-248
Use case area 2: temporal, scripted
9 Performances and the performing arts | Pages 251-260
Use case area 3: spatial, static
10 Layout space | Pages 263-278
11 Diagrams and infographics | Pages 279-294
12 Comics and graphic novels | Pages 295-324
Use case area 4: spatial, dynamic
13 Film and the moving (audio-)visual image | Pages 327-339
14 Audiovisual presentations | Pages 340-344
Use case area 5: spatiotemporal, interactive: ‘media that bite back’
15 Webpages and dynamic visualisations | Pages 347-354
16 Social media | Pages 355-365
17 Computer and video games | Pages 366-378
18 Final words: ready, steady, analyse! | Pages 379-382
Bibliography | Pages 383-410
Index | Pages 411-416