Grimoire for the Digital Humanities:
The Forbidden Atlas

6. The Sentiment of Good News, Bad News and Darko News

Authors: Dr Mitchell Harrop, Kim Doyle

This chapter applies Sentiment Analysis to datasets of historical local newspapers. Sentiment Analysis is about trying to determine the sentiment (positive, negative or otherwise) of written text. A Sentiment Analysis of historical newspapers enables a sense of the sentiment associated with different locations over time. Two approaches to Sentiment Analysis are presented. First, a spreadsheet approach which arduously builds a Sentiment Analysis process from scratch. Second, a Python coding approach which is slightly less uncool. Both are dictionary based approaches to Sentiment Analysis. This chapter goes deep into the code used and asks bigger questions about the algorithms in our lives and the opaque blackbox nature of most software.

Okay, so we’ve got to tell you about one of our favourite bits of the Halloween movie Donnie Darko (McKittrick et al., 2001). Donnie, a teenager who time travels (that bit is not actually important) somehow manages to make great critiques of Sentiment Analysis while not actually talking about computational Sentiment Analysis.

So Donnie is in school and Mrs. Farmer has just finished playing a video to the class. She walks to the blackboard and draws a line. On one end she writes “Love” and on the other end she writes “Fear”. She explains how the “Lifeline” is an energy spectrum. Ms Farmer, played by Beth Grant, who, by the way, plays a very similar pageant operator character in Little Miss Sunshine (Turtletaub et al., 2006), hands out Character Dilemma cards. She tells the students to read them aloud and place an ‘X’ on the Lifeline in the appropriate place. And that’s what most Sentiment Analysis techniques sort of do. They take naturally spoken language (the Character Dilemma cards) and reduce them into score/s between good and bad (the ‘X’ on the lifeline).

6.1 Sentiment Analysis in a Spreadsheet

This section conducts a simple kind of Sentiment Analysis in a spreadsheet. It is arduous to do this from scratch, but it should get you thinking about the process and underlying assumptions and principles.

First up, the data to analyse. These are semi-fictional examples of historical Melbourne newspapers in a spreadsheet that include headlines, the text of the articles, the date of the articles and a location of the paper:

  A B C D
1 Headline Article Date Paper Location
2 Good dog goes bad MELBOURNE, Tuesday. A mysterious beast has been terrorising the homestead at Point Cooke... 1859 Melbourne
3 Bad Ghost. Melbourne, Saturday. The residents of a township near Ballarat have lately been troubled by the operations of a thief guised as a ghost. ... 1903 Ballarat
4 Melbourne's "Ghost" MELBOURNE. Jan. 13. The tramways board is planning a gaily-lit 'ghost' tram which will travel in Melbourne's streets day and night during the Royal visit... 1954 Adelaide
... ... ... ... ...

Let’s focus for now on doing a Sentiment Analysis on just the headlines. As such, the other columns in all the examples below have been removed for brevity. We’re going to start our simple Sentiment Analysis by looking for the words 'Good' and 'Bad' in the headlines. Whether or not the words are found will form the basis of the sentiment score for each headline. Some additional columns keep track of...... PREVIEW ONLY ...

Next: 7. Under the Evil APEye of Trove