A Measurement Study of Selected News Websites
During the 2016 U.S. Presidential campaign, the term “fake news” became a significant part of public vernacular and discourse. Fake news, it was posited, was used to spread disinformation and diminish the credibility of established news organizations in order to sow confusion about presidential candidates. In the months that followed the election, Donald Trump used the term to directly attack news outlets that published content that he disagreed with. In a heated press conference at Trump Tower, he directly told CNN’s Jim Acosta “You are fake news.” As a result of its sudden appearance in mainstream American politics, fake news as become one of the most significant topics among not just media researchers, but among the general public as well. It seems that virtually everyone is aware of the issue and accepts that fake news carries with it significant consequences, but it can be difficult to determine what specific effects it can have, and just how widespread they can be.
One of the reasons that fake news is such a significant problem is the fact that relatively small organizations and groups of people are able to easily produce partisan–and often misleading–news headlines and articles and spread them quickly via social media. This makes it possible for fake news to easily compete with large and established news organizations. Even though their content is not as well-produced or as high quality, the it is able to spread rapidly and reach a wide audience. In this study, I analyze three selected websites and examine their form, and style of writing–including both article content and headlines. I then compare those characteristics with the Facebook engagement metrics of the websites’ content to determine if there is a correlation between website content and overall online spread of said content. Though my study is limited in scope, the data suggest that even small partisan blog-style websites are able to compete with large established news organizations by tailoring their content to be shared rapidly and widely. It appears that the form in which content is presented is much more significant than the content itself. Perhaps it is time to move beyond the adage of many media producers that “content is king,” and instead confront the realities of the rapidly shifting media landscape.
Over the course of one month, I observed the content published on three websites: Politico, Eagle Rising, and Addicting Info. These three sites represent a opposite ends of the political spectrum, as well as one neutral baseline website for comparison purposes. Eagle Rising is a conservative-leaning website founded by Gary DeMar and Brandon Vallorani. Its self-purported mission is to “share breaking news … from a Christain perspective.” Addicting Info represents the opposite political ideology, and nearly all of its content has a distinctly liberal slant to it. It describes itself as a “resource to discredit all the lies and propaganda that the right-wing spreads.” Finally, Politico is an international news and information network that specifically focuses on politics and policy. Its stated mission is “to help sustain and vastly expand nonpartisan political and policy journalism.” Compared to Eagle Rising and Addicting Info, Politico is much more well-established and is generally seen as a credible news source.
I visited each website on five separate occasions: February 4, February 7, February 12, February 18, and March 2. Each time, I recorded metadata about the five articles that appeared at the top of each website’s homepage. These metadata included the article’s timestamp, authors, headline, URL, and total Facebook engagement. I also took general notes on the form of each article’s content, such as the tone of the writing, or the inclusion of external media and quotes from other websites.
To gather Facebook engagement statistics, I used the publicly available Facebook Graph API. This service enables users to request specific information about given Facebook content, including users, pages, and individual posts. It is possible to use an external URL as an input for a Graph API request, and receive data about all the times that URL was used on the Facebook platform. The data I was collecting was publicly available, so I was able to use my personal Facebook account’s API key as the authentication token for the following Graph API request:
I chose to record Facebook’s engagement data to serve as a measurement of the extent to which any given article had spread throughout online communities. Facebook engagement is the sum of interactions with that specific URL on the Facebook platform; this includes comments, shares, and all reactions (Like, Angry, Wow, Sad, Haha, Love). The number is not an entirely accurate representation of the number of times the article was read, or the dissemination of the URL via platforms other than Facebook. Gathering that information would involve using public APIs and analytics tools for many other platforms, and full viewership data would likely only be available to the website owners via tools such as Google Analytics. Those types of measurement are perhaps useful for understanding the full scope of a given article’s spread but is beyond the small scope of this study.
For each website, I recorded metadata and Facebook engagement data for 25 articles. Generally, the range of data that I collected was very large, and with both high and low outliers. This makes the mean an inaccurate measure of central tendency. For this reason, I have chosen to use the median for the purpose of comparison and discussion.
One important caveat about the collected Facebook engagement data is that is was gathered at the time of each observation. For each observation session I collected data for whatever each website’s top five articles were at that exact moment. This means that some data were collected only a few hours after the article was published, while others were gathered a full day after publication. Furthermore, Facebook engagement data for previously recorded articles were not updated during subsequent observations. Therefore these results and discussion are applicable to the short-term spread of news articles via Facebook, but should not be interpreted as an indication of the long-term spread of content.
Politico’s website has a large array of content types, from short informational articles, to much-longer thinkpieces. They have also had more “general interest” pieces that are less about specific news events, and instead focus more on broader issues — such as a federal minimum wage. All articles are still based in either politics or public policy issues.
Politico has a significantly large staff — with few repeated authors in the articles I pulled. Additionally, many of them contain photographs from their own staff, or licensed from other providers (and likely not just lifted from online). There are also often graphics and data visualizations that are also likely produced in-house. Most articles contain quotes that were obtained from actual Politico reporters on the Hill, and not just copied from other news sources and republished online.
The writing style is similar to that of actual print newspapers, and follows a lot of the “journalistic traditions,” such as the inverted pyramid model, specific style guides, and use of external quotes/reporting. Much of the articles are informative, and analytical — but not necessarily with a given agenda one way or the other.
The median Facebook engagement for Politico articles was 853, ranging from 17 to 7233. However, there is no specific correlation in type of content for articles that received the highest levels of engagement. “White House aide Rob Porter resigns after allegations from ex-wives” received 7233 engagements, possibly due to the scandalous nature of the article, and recency of the event. However, the significantly less scandalous candidate profile “Is Conor Lamb the Next Big Democratic Upset?” received 7075 engagements, a similar level.
Politico’s Facebook engagement outperforms that of Eagle Rising, but is overshadowed by the consistently high engagement levels for articles published on Addicting Info.
Eagle Rising’s website has a distinct blog-like feel to it, and appears to be built using a premade Wordpress theme. The writing style mirrors this aesthetic; Eagle Rising articles typically have a personal and casual tone as well as include personal opinions. The website appears to have a very small number of writers. During my observation, I only noted three unique authors: Joe Scudder, Dale Summitt, and Keely Sharp.
The content of Eagle Rising’s articles is limited, and most published content is quite short. Generally the website’s articles contain only a few paragraphs of text, and fit within the space of a single screen. Furthermore, much of its content is taken wholesale from other websites and news agencies, such as Fox News. Generally, the headlines contain politically divisive language, but are not necessarily aggressive, or meant to serve as clickbait. Some examples include “NeverTrump Republicans Mock the Memo,” “Pelosi Tortures House with 7-hour Speech,” and “CEO of Goldman Sachs Vindicates Trump on Economy.”
Most Eagle Rising posts receive low amounts of Facebook engagements, and several articles received zero interactions whatsoever. However, this may only be an indication of the small size of the Eagle Rising audience. The Eagle Rising Facebook page has 688,000 likes, while Politico and Addicting Info have 1.7 million and 1.4 million, respectively. However, this means that articles with really high levels of engagement stand out. The article “Pelosi Tortures House with 7-hour Speech” received 175 Facebook engagements, and “Former Mexican President Blames ‘Racist Trump’ for School Shooting” received 1210. However, these articles contain little substantive content themselves, which possibly indicates that Eagle Rising’s content is intended to be consumed in small snippets in the Facebook News Feed, rather than on the full website.
Nonetheless, for articles with little substantial content, it is noteworthy that the URLs are still being engaged with so highly compared to other Eagle Rising articles. It suggests that the headline and presentation of the content is more significant than the content itself. Eagle Rising’s median Facebook engagement is lower than that of Politico, but some Eagle Rising articles outperformed certain Politico articles.
Much like Eagle Rising, many portions of the Addicting Info website are written in an informal style, more like a personal blog rather than a traditional news source. In my observations, I only recorded four unique authors of Addicting Info’s content: John Prager, Shannon Barber, Conover Kent, and Tim Abel. Additionally, much of the website’s content is sourced from other websites. “Trump Staffers Desperately Want To Quit, But No One Else Will Hire Them” quotes Buzzfeed extensively, for instance.
Despite the small staff and low volume of content within its articles, Addicting Info handily outperformed both Eagle Rising and Politico in terms of Facebook Engagement, with a median of 2145. One of its articles, “JUST IN: Trump Campaign Chair In Kentucky Pleads Guilty To Sex Trafficking Of Minors” received 21,798 engagements, the highest amount throughout my entire study. However, removing this article from the dataset drops the range of engagement down to just 181, suggesting that Addicting Info’s audience is stable and enagages with its content on a regular basis.
Perhaps one reason that this website is so successful in terms of Facebook engagement is that many of its headlines seem specifically designed to grab attention and encourage other users to share and interact with the content on social media. This includes titles in all-caps, claiming that items are “breaking news,” and clickbait titles that tease the user with the promise of more content, such as “The Police Were Called Over Man’s Anti-Trump T-Shirt; His Response Is Perfect.”
The Facebook engagement for these three websites is a fairly small dataset, but nonetheless can provide some interesting insights to the current media landscape, which has been vastly shifting in the last several years. Eagle Rising and Addicting Info are significantly smaller than Politico, but thanks to social media are able to somewhat compete with its content, and draw their own share of online interactions. My study shows suggests that the current Facebook platform does not necessarily incentivize good content, but instead incentivizes content styles and methods that generate engagement.
Through attention-grabbing titles and divisive language, websites such as Eagle Rising and Addicting Info create headlines that draw in readers, despite the fact that the actual content of the articles is limited, and often simply just quoted extensively from other websites and news sources. Whether or not an individual actually reads the article is irrelevant; so long as people continue to engage with the content–commenting, liking, or sharing–it will continue to spread throughout online communities. It is by these methods that fake news, misinformation, and disinformation spread so rapidly. Even an established news organization such as Politico is unable to increase its online engagement simply by having higher quality, and more lengthy content; in the context of social media, it too is playing by the same rules as Eagle Rising and Addicting Info. The media landscape is still adapting to the proliferation of online communication technologies, and media researchers continue to feel out this new media environment and figure out just what the rules to this new game really are. However, the data from this study carry one significant implication–the possibility that content truly is no longer king.