Grimoire for the Digital Humanities: The Forbidden Atlas
1.1 Introduction
The magic contained herein is about digital mapping. More than that it is about mapping over time. More than that it is about mapping culture over time. Related to Moretti’s concept of distant reading, we are interested in the concept of ‘deep mapping’ (Moretti, 1998; Moretti, 2007; Bodenhamer et al, 2010, 2015), where many forms of information are layered over places, forming new associations and insight. This all presented as a Grimoire.
A Grimoire has often been thought of as a textbook of magic spells. For example, H. P. Lovecraft frequently wrote of a fictional terrible and forbidden grimoire called The Book of the Dead. We like this concept, so we are going to borrow it.
But why not a plain old methods book or a textbook or a book of recipes detailing how to solve digital mapping research problems? Or a carpentry or some other metaphor? A reference book perhaps? Or any of the other kinds of instructional books for that matter?
The problem with cookbooks or recipes on relatively technical topics is that they tend to stop working fairly soon after they are published. A cookbook is generally a list of step-by-step instructions that tell you how to make a specific thing with specialised software. But software gets updated, new versions come out, websites disappear and the instructions in the book stop working. Thus the experience of picking up an old Digital Humanities methods book or unmaintained website is usually the same as picking up an old book of spells: you follow all the instructions exactly, but the magic just doesn’t happen for reasons unknown. The original magic has faded.
Our Grimoire approach (we hope) works within this obsolescence problem. It is written with an eye towards a time when the magic contained herein ceases to function, yet it still contains recipes or instructions to follow. The instructions aim to provide a deeper lesson that will echo through the ages as newer versions of software are released and infrastructure falls into disrepair. But on the other hand, our Grimoire does not abstract the lessons so far away from step-by-step instructions as to be entirely theoretical.
In a practical sense, all this is acheived by accompanying specialised recipes/spells (often involving programming or obscure software) with their equivalent (often somewhat cumbersome) recipes/spells that use only spreadsheets or good old fashioned elbow grease to achieve the same ends. For example, one spell on how to use code to pull mapping data from an online database into a file is accompanied by another spell on how to use spreadsheet formulas to pull the same data from the same database into eagerly waiting spreadsheet cells. Both instructions will stop working one day as the respective software is updated, superseded, or grinds to a halt. In the space and contrasts between methods a timeless message will (we hope) call out through the ages like some terrible monster of vaguely humanoid outline, but with a face of unknowingly powerful tentacles and a mass of bat wings thrown in for good rubbery measure.
Secondly, this Grimoire considers the unintended consequences and ethical conundrums that often come with new digital research methods. The power that comes with magic can have a corrupting influence or magic can be wild. The ethical and societal implications of the use of magic should always be considered. For example, we show how to use free products from the tech giants such as Google, Twitter, Microsoft and others. But those tech giants only offer free services so they can watch what we do. The price for their services is our privacy or, for example, the privacy of those we interview for oral histories. The tech giant’s magic is cursed.
Thirdly, this Grimoire is all about creating magical objects. Conjuring. ‘Making stuff’ is often seen as the core goal of the Digital Humanities as a discipline and this Grimoire follows in that tradition, while still appreciating that the Digital Humanities is a broad church of the old gods and the new.
Finally, in answering why this publication is framed as a Grimoire we must acknowledge that seven copies exist that are bound with unicorn skin and kept in the basement of The University of Melbourne library.
1.2 Background
This is a publication in and of the Melbourne History Workshop, which can be found within the School of Historical and Philosophical Studies at The University of Melbourne. With few exceptions we cover Melbourne History, but what we cover will be of interest and applicable to urban historians everywhere. The Grimoire shows how to apply digital mapping techniques to many existing datasets produced by the Melbourne History Workshop, including The Encyclopedia of Melbourne entries (eMelbourne.net.au) and many other online collections. The Grimoire draws on our experience (May et al., 2018) in collaborating on digital urban history projects and making many mistakes. The Grimoire has also been created in collaboration with the Melbourne Data Analytics Platform (MDAP) at the University of Melbourne. MDAP is an interdisciplinary team of researchers who share a capacity to work across domains and a passion for collaborating with other researchers to facilitate data-intensive research.
This Grimoire comes out of the Time-Layered Cultural Maps project (www.TLCmap.org) which is funded through an Australian Research Council Linkage Infrastructure, Equipment and Facilities grant (PROJECT ID: LE190100019). TLCMap is an open online facility to create maps and timelines and combine new Australian cultural and historical datasets with existing ones (Arthur et al., 2020). The project centres on developing infrastructure around several technologies, including the Gazetteer of Historical Australian Placenames (GHAP) and Recogito (recogito.pelagios.org). This Grimoire is a gentle introduction to many of the concepts needed for the more advanced parts of TLCMap.
By showing how each research problem can be solved in multiple ways (spreadsheets, code, expensive tools, by hand and so on) each chapter demonstrates how the problem would be trickier or impossible to solve without user friendly infrastructure. In general, the datasets of Melbourne History Workshop provide the projects to ground the Grimoire in reality rather than in the realm of fantasy.
1.3 Structure of this Grimoire
Chapter 2 (Basic Magic Tricks) is a gentle introduction to some of the tools and concepts needed in later chapters to create maps. It is a chapter of magic ingredients.
The chapter includes a gentle introduction to structured data which takes the riveting form of a simplified history of Software Engineers saving files and then being hit by buses because of an ancient curse. An understanding of structured data is needed in order to be able to create the kinds of files (KML data files) that mapping software often uses to render great visualisations. This section is followed up by a gentle reminder about spreadsheet concepts, which can be skipped by those of you that would rather drink to forget all about spreadsheets.
Chapter 3 (The Ley Lines and Contours of Melbourne) visualises historical contour maps of Melbourne from 1853 and 1880 which exemplify suspected human induced or deliberate changes to the ground levels of the central business district. As a first step, inaccurate historical map images are stretched and shrunk to be consistent with contemporary maps because historical maps are not always perfectly to scale or perfectly accurate. Two alternatives to creating a dataset of elevations from the images are presented. The first approach outlines a relatively labour-intensive method using spreadsheets. The second approach uses the Google Earth application to create a KML data file filled with the elevations data. The files from both approaches can be readily opened/imported in mapping software and examined to reveal where the levelling likely occurred. Finally, some notes on the accuracy of the data generated and a discussion on the publication of datasets is presented. The chapter represents a very simple layering of time (the T and L of Time-Layered Cultural Mapping). The C(ultural) aspects come to the fore in later chapters.
Chapter 4 (A Gentle Word on Code) introduces a quick and easy way to run Python code without having to install software and go to all the effort of maintaining a server. It is a followup to the magic ingredients introduced in chapter 2 and continues to introduce the tools and concepts needed in later chapters to create maps.
Chapter 5 (My Marvellous Melbourne Podcast Locator Spell) uses computational text processing techniques on the Melbourne History Workshop’s My Marvellous Melbourne podcast transcripts. The techniques auto-magically extract the places mentioned in interview segments in order to present them on a map. This is a chance for Melbourne History Workshop podcasters to reflect on the temporal and spatial coverage of episodes to date and also to create an engaging visualisation to generate more interest in the podcasts. The chapter takes a brief detour from mapping to consider different auto-magical methods of creating transcripts from audio recordings.
The next three sections present alternative methods for creating maps from the transcript of a single podcast. The first approach uses Voyant Tools to very quickly and easily create a map from the transcripts. The next section is a point of contrast as it creates a map using a spreadsheet in a very long and tedious process, but in the tedium there is much more control over the final product. The last method uses Python code to create the map from the transcripts. As the previous methods were only applied to a single podcast, the next section attempts to semi-automate each of the proceeding methods to work on all the podcasts past, present and podcasts in the swirling mists of the uncertain future.
Chapter 6 (The Sentiment of Good News, Bad News and Darko News) applies Sentiment Analysis to datasets of historical local newspapers. This enables a sense of the sentiment associated with different locations over time. Two approaches to Sentiment Analysis are presented. First, a spreadsheet approach which arduously builds a sentiment analysis from scratch, but in doing so provides a deep understanding of the issues involved in this kind of analysis. Second, a Python code approach is presented which is slightly less uncool than the spreadsheet. Both are dictionary-based approaches to Sentiment Analysis. This chapter goes deep into the source code used and asks bigger questions about the algorithms in our lives. The chapter finishes by showing methods for creating KML data files of Sentiment Analysis across different regions that can then be viewed in most mapping software.
Chapter 8 (Under the Evil APEye of Trove) is a gentle introduction to Application Programming Interfaces (APIs), showing how an online collection of photographs can be found through the Australian National Library’s Trove portal and downloaded using code and/or a spreadsheet for offline analysis. In this way the chapter takes the Programming out of Application Programming Interfaces. As a point of contrast, the Programming is well and truly put back in using Python. The chapter then uses an online database of Australian historical place names to improve the location data in the Luly collection and generate a map. This section concludes with a discussion about how programmers think about APIs in general in order to consolidate some general principles for using APIs in research and in project collaboration with Software Engineers.
Chapter 8 (Encyclopedia of Melbourne Locator Spell) aims to improve the eMelbourne encyclopedic entries (eMelbourne.net.au) by adding location data derived from the text of the entries. The chapter splits this problem into smaller more solvable problems, such as obtaining the data and processing the data. This facilitates a discussion about how to effectively collaborate on complicated Digital Humanities projects. Spreadsheet and coding methods are presented as well as using alternative online place name authorities.
The concluding remarks of the chapter considers the relative successes and failures of the Grimoire approach this book has taken and points the reader to the next steps they may wish to take.
1.4 Expected Knowledge and Audience
The ideal audience for this Grimoire is a masters or PhD researcher just starting out. You are probably wondering if you should go down the path of learning a lot of technical skills as part of your research journey, or whether you should instead gain a working knowledge of some topics in order to collaborate well with your colleagues from Computer Science and Software Engineering. You have the spirit of adventure. You are willing to embark on a thrilling journey. You are willing to sacrifice a goat or two.
We expect that you are coming to this Grimoire from a history or historically sympathetic social sciences research background. If you are coming from a highly technical background, this book is not for you, although it could make you better at the process of requirements gathering for Humanities, Arts and Social Science researchers.
We expect that you have some experience with, or have done a course on spreadsheets at some point and promptly forgotten everything about spreadsheets. But you do have a vague recollection of what’s possible and what’s not possible and just need a bit of a reminder from time to time. We’ll give you those reminders as needed and also introduce the really advanced stuff.
We also expect you to have tried some kind of programming workshop for researchers or you took programming in high school. You thought that you understood most of what was going on, but have definitely forgotten all of it now. Again, we’ll re-introduce you to the concepts as needed. We are more interested in giving you the skills you need to collaborate with coders than in becoming an expert coder yourself. But if you do want to use this Grimoire as part of a path towards becoming a programmer, we will provide you with all the resources you need.
We expect you have no geographical information systems knowledge and no specialised knowledge of maps and mapping beyond what was taught in high school. For example,...
... PREVIEW ONLY ...
Next: 2. Basic Magic Tricks