Founder of Orodata Science, Blaise Aboh writes on AI and Journalism, data visualization and other forms of augmented reporting with a call on newsrooms to come up with a long term strategy for use of the new tools.
On August 21, 2019, I received a rare invitation from Michael Bloomberg to be a featured speaker and guest at Bloomberg’s Africa Business Media Innovators 2019 forum in Dakar, Senegal, on 10-12 November. This event was co-hosted by Bloomberg Media CEO Justin Smith and Editor-in-Chief Emeritus Matthew Winkler and brought together distinguished media leaders from Africa and around the world, along with business, government, and civil society leaders who play an important role in shaping the media industry in Africa.
The event which was the fifth instalment of the annual forum focused on the theme Business Strategies for African Media and examined a broad range of issues affecting African media companies and their role in supporting growth and prosperity across the continent. As the founder of Orodata Science and a leader in the tech and media sectors, I presented a tool-kit session on AI and Journalism, data visualization and other forms of augmented reporting; since my work sat at the intersection of the themes the convening was exploring.
As requested by many who valued my unique perspective and the values it contained, I am sharing more about the subject matter, and the message I hoped to pass to distinguished participants.
. . .
Augmented reporting
Augmented reporting is the future of reporting, this is the use of frontier technologies like Machine Learning and Data Visualization for news content generation, analysis and reporting. Another name for this sort of reporting is ‘automated journalism’. In 2014 I thought of presenting key election data and information to citizens for better decision making by using data visualization. What started out as a learning project became what I would do for the coming 6 years. I understood one thing at the inception, every issue had a solution, and to get to it, you had to look at the data behind it and ask the right questions.
I always asked questions, then I would wake up very early by 4 am, to read a breaking news story and ask myself; what questions was this story not answering? What were the gaps in the story? Where is the data? What questions would readers ask? Where are the answers? After that, I would proceed to find the answers to all these questions and represent them using data visualization techniques. By morning the visualizations fill the information gaps left by the publications, and of course, these news platforms end up republishing the visualizations.
In July 2018 I began researching how algorithms, automation, and AI could enhance how people accessed numbers and insights from a very large collection of data. Months after I built Fintel Envoy, a bot that provides accurate approaches for financial information extraction and scientific data analysis. I used the Nigeria Inter-Bank Settlement System (NIBSS) data sent to me by a journalist who covered the financial sector. Fintel was built using Google’s Tensor flow and Botmock for initial conversational flow design. It embodied faster knowledge discovery, all a user who seeks insight on this data needed to do was pick up an android phone and say; “Talk to Fintel Envoy”.
Since the core of the bot is powered by Google Assistant, the bot comes up and says; “Hi, I am Fin, and I can give you insights on the Volume and Value of Payment Transactions from Q1 2017 to Q1 2018” and then it reels out all the options. The NIBBS data had so much in it, so we focused on payment channels and banking sector credit data. I didn’t realize that a year after I would use the same techniques to build Robot V – Voteron, a bot that increased ease of access to election information by analyzing and simplifying large election datasets while delivering insights inform of snippets and very short stories to mobile devices and social media.
The future of news
All the while I worked on the bots, I was training journalists across newsrooms on data visualization and investigative reporting, and on very special occasions I would casually introduce them to the bots and pass the phone around letting them ask it questions. I told them that this was where the future of news was headed. A few times I took a poll to understand their thoughts. For many, it was fear and uncertainty. The thought of how to empower these journalists even more lingered and would have me create what I called ‘The Future of Work Classes’ many months later.
Currently, the use of algorithms to automatically produce news or content from structured data is riling up the journalism industry. During the 2019 elections, using a natural language generation tool powered by Articulate Text Language called Arria, my organization Orodata generated hundreds of stories from structured election data. In December 2019 the BBC wrote that it had generated numerous stories overnight for the UK constituency elections using machine journalism. Associated Press today automates the production of its quarterly corporate earnings reports, Yahoo Sports, Forbes, The New York Times and ProPublica have also begun automating their reporting, especially on stories with numbers.
Telling stories quicker and simplifying complex issues
I always say the endpoint of every story is either insight, decision making or problem-solving. In a time of information overload, people want to achieve these things quicker and the only way forward is by information simplification. Using data visualization for news, one makes data or numbers engaging, easy to consume, and readers are quicker to insight. The visual summary of information or data increases cognition, makes it easier to identify trends and patterns. It is so much simpler and faster to get insights from visualized data than from scrolling thousands of rows on a spreadsheet or reading hundreds of numbers in an article. It’s the way the human brain is encoded to work.
On automation and machine learning, once an algorithm is developed for this purpose, and the needed data well-structured and connected to it, it can create thousands of news stories from it quickly, cheaply, and potentially with fewer errors than any human journalist. To dissuade journalists’ fears that automated journalism or content production will eventually eliminate newsroom jobs, this is where the human and machine alliance comes to play. These algorithms do not completely annihilate human input, rather it empowers humans to even create more content, faster while having time for other things. For an almost flawless or without wax touch, even an automated story needs a small human touch to give it the ‘almost human’ feel.
The news media and the fear of the future
The news media in Africa are already awake to the concept of data visualization. We have a few newsrooms adding a bit of visualization in their stories and talking about it today than 5years ago. On the automation and machine learning part, not so much. In an in-depth global survey by POLIS, the only African country that could give unprecedented insight into how AI and Automation are perceived by those people leading their development or application inside news organizations was Botswana. During the survey, I was contacted by the Journalism AI program manager Mattia Perretti to assist in reaching out to my networks in Africa to find newsrooms actively using any form of AI or Automation, but I could not find any.
I have had extensive discussions on these new technologies with some news editors, and news media owners in the past, and at the inception, some of them do not understand these technologies or how it will immediately solve their major problems on engagement and revenue. A few bites on the idea of data visualization because they are seeing some of their competitors starting to do it, but not automation, that is too far out for consideration. And I understand. Globally, the news media is struggling to keep the lights on, revenue is not as it used to be and journalists are overworked and unsatisfied with their pay. Making a huge investment in technology will not be an easy decision.
Preparing your workforce for a Data and AI-driven future
Historically, technological progress has not created technological unemployment, rather new technologies have eliminated some jobs and substituted for some tasks formerly performed by humans, but they have also enabled the creation of many new jobs roles. When a newsroom decides to embrace new technologies, new tasks are created with new job roles and these new roles will need to be filled. The upskilling of existing staff to fill these roles will be inevitable. For a data-driven future, the biggest challenge is the lack of data experts. The second biggest challenge as news organizations struggle to break even is the cost. To produce stellar data-driven journalism is an expensive and time-consuming venture.
The technology cost includes new kinds of software to handle the data, software to visualize and share the data, resources to gain access to data and a commitment from data providers to keep resources up to date. There are other costs in terms of human resources such as time dedicated by staff on data-intensive projects and the hiring of technical staffs to complete the process. Not many news organizations will be open to expending these resources, however, a way out could be to collaborate with technology companies who are core on these technologies and work something out.
On automation and artificial intelligence, according to the Journalism AI report, the biggest challenges to adopting AI in newsrooms were financial resources (27 percent) followed by knowledge or skills (24 percent). Cultural resistance was also at (24 percent) which includes the fear of losing jobs, the fear of changing work habits, and a general hostility to new technology. Lack of knowledge about AI came at (19 percent) across the news organization along with a lack of strategic managerial insight (17 percent) were also key issues. They also described AI as often expensive to build and manage.
Should newsrooms do it?
With augmented reporting, newsrooms can do so much with the data out there either by finding stories in data or finding data in stories. 6 years ago the New York Times introduced The UpShot, a data visualization and graphics department. Since this shift, other organizations like The Washington Post and The Guardian have increased their focus on data and data visualization. With the latest Natural Language Generation (NLG) technologies, news organizations can produce more content automatically from data and achieve massive distributions. They will have better-customized content, increased user engagement which will throw open the door of better subscription options, hence more revenue.
The Associated Press uses and NLG is known as Wordsmith to transform raw earnings data into thousands of publishable stories, covering hundreds more quarterly earnings stories than previous manual efforts. First, AP could only produce 300 such stories per quarter, now it does 4,400 quarterly recaps and almost 15X increase over manual efforts. Yahoo! Sports uses the same NLG to produce over 70 million reports and match recaps—each one unique—that helps engage, monetize, and delight its massive user base. It has also increased the time spent on page for website visitors. The best part is that these stories retain the same quality and accuracy that readers expect from any of AP or Yahoo’s human-written articles.
Other examples of AI used by news organization include; The Press Association’s ‘RADAR’, The Guardian’s ReportersMate, Bloomberg’s Cyborg, The Times of London’s JAMES. The exclusive stories generated by augmented reporting can help convince subscribers to pay for the new product. Like Michael Bloomberg, said at the National Press Club presidential inauguration in January 2015, “The public will demand and pay for quality journalism”. So should newsrooms do it? Yes, I think they should lay out a long term strategy, it’s a race, and many of their competitors have started running 5-10 years ago.