Wednesday, 9 September 2009

An interesting week for misleading statistics

A few technology news headlines have attracted my attention this week that seem to be somewhat less than honest. Some of the ones that stood out included the 10% surge in BBC iPlayer traffic following its inclusion on the new PS3 Slim which in turn has been reported as bringing about a massive sales spike in Sony PS3 units following the slimmer redesign. The other noteworthy report involved the claim that 7 million of the 61 million UK residents is engaged in illegal file-sharing activity. Let's take a little look at those claims...

Sony's sales and why they might not be so impressive

There have been several reports that Sony is having a very successful PS3 Slim rollout. Popular games blog, Kotaku, noted that the redesigned PS3 sold more units in Japan in the first three days following the relaunch (150,252) than it did back in November 2006 when it was originally released. It sold 88,000 units in two days but there were issues with supply levels at launch, so not everyone who wanted one could get their hand on one.

The more interesting PS3 story came from VG247 courtesy of Chart-Track director, Dorian Bloch, who claimed that UK-based PS3 sales in financial week 36 were up over 999% on the week prior. This is a story which has snowballed and appeared in several other places. Great news for Sony. Or is it?

On the surface the 999% or 1000% sales boost seems impressive but this has to be seen in the harsh reality that very few people were likely to be buying the older PS3 model when the new slimmer model was about to be launched at a reduced price. It's not hard to see why a huge sales spike may occur when very few units were being sold in the week prior - but it makes for great headlines!

What is also unclear is exactly how many of these purchases were by new customers. Many existing PS3 owners have expressed an interest in trading their older machines in for the new model (the local GAME store where I live was accepting a trade-in deal of one 80gb PS3 plus two games and £60 cash for a new slimmer PS3).

This might not be so great for Sony after all. Sony are still producing the PS3 slim at a loss and require customers to purchase software in order to offset these losses. If they are selling lots of new units to existing customers who already own plenty software then they are unlikely to profit dramatically from these impressive sales figures. Also, Sony get no financial gain from the second hand hardware or software market. The more new customers the PS3 redesign attracts, the healthier Sony's bank balance.

It's understandable that Sony would want to view its new product launch as a success, given that the PS3 languishes in third place in the console hardware sales. It's also no surprise that Sony has announced a massive ad campaign to push its new hardware in the Christmas run in (with a campaign budget of £82 million!). Whether or not the PS3 proves to be the success that Sony needs it to be remains to be seen, especially since it experienced its first full year loss in fourteen years.

How many filesharers?

The other attention-grabbing (and more worrying) story that warrants a little exploration involves the claim that 7 million UK residents are file sharing criminals. This figure is one which has been around for some time now. The BBC ran a report back in May which cited the figure in a government-backed report. The report was issued by the Strategic Advisory Board for Intellectual Property (SABIP) and can be found on their site (.pdf here). Intellectual Property Minister David Lammy said the report put into context the impact illegal downloads had on copyright industries and the UK economy as a whole.

This figure of 7 million criminals is quite catching, seems precise and scientific and has helped perpetuate the notion of digital criminality amongst UK web users. It was also calculated that these file-sharers had access to £12 billion worth of free content. It has also helped keep file-sharing and its supposed criminality in the media spotlight in recent weeks - something Lord Mandelson has been using to great effect in his attempt to lobby for hard-hitting punitive measures against file-sharers.

Credit needs to be given to the BBC who took a second look at that figure of 7 million and decided to see if they could locate its origin, following an enquiry from the audience of the BBC Radio 4 show, "More or Less".

The figure may have been reported in a government document, but it's not a government figure. It transpired that the government commissioned a report from the CIBER research group at University College London, which contained the number. The CIBER report mentioned the figure four times. However, the figure actually came from yet another report from consultancy firm, Forrester. It doesn't end there. The Forrester report doesn't actually contain the 7 million figure despite the CIBER citation. The figure actually comes from a different piece of research called the Jupiter Industry Losses Project, which was an industry funded (BPI) attempt to gauge P2P use and the related losses produced by such internet use.

That's right. The official sounding figure cited by the government-backed report actually came from the British recording industry which has a vested interest in putting a figure on its losses at the hands of P2P users so that it can then lobby government to change policies and laws favourable to the industries interests. That explains why Lord Mandleson has been so anti-P2P in recent weeks...

It gets worse. The actual report was never published publicly and the industry declined to pass it over to the BBC but they managed to get in touch with Mark Mulligan, one of the report's authors, who revealed some interesting methodological assumptions. The report estimates that there are 6.7 million illegal file-sharers in the UK. It arrived at this figure when multiplying the total number of Internet users in the UK (estimated at 40 million by the report despite the UK government putting the figure at 33.9 million) against the percentage of the population engaged in file-sharing.

As for the estimate of the piracy percentage, that comes from a 2008 survey of 1,176 UK households. The survey actually found that 11.6 % of respondents admitted to using file-sharing software, but this figure was inflated to 16.3 % to account for "under-reporting". It's not quite clear how this figure was arrived at but it was based on the assumption that some respondents would lie about their P2P use, and Mulligan claims to it be 'based on evidence'.

The differences between these figures are staggering and produce vastly different results. If the lower figures are used instead it transpires that only 3.93 million UK residents are criminals, not 7 million.

And finally...

What does this all mean? Can we actually trust the figures used by goverenemtn and industry alike? As Nate Anderson (over at Ars Technica) puts it:
"The problem isn't that such calculations are done; they can serve as useful tools for industries and even for policymakers. But problems develop when the numbers are ripped from their original, provisional context by repetition and citation, eventually taking on the force of Fact. When such "facts" end up being used to make policy, the problems are compounded."
Statistics and facts are not always what they seem. Ben Goldacre, author of Bad Science and regular Guardian writer, puts it slightly differently:
"As far as I’m concerned, everything from this industry is false, until proven otherwise”
I'm inclined to agree, but there is a worrying point to these examples of misreported statistics and over-inflated figures. If they go unchecked, they become powerful rhetorical strategies that can gain momentum and if repeated enough times in the right places can take on the appearance of 'truth' from which it is only 'common sense' to act accordingly. I wonder how successful the new PS3 actually is and are shareholders seeing the benefit of increased sales? Does this equate to more inward investment in the technology at both the software and hardware levels, and does this bring benefits to the end user? Can we really believe the figures bandied around by the recording industry and what are the consequences if they are not contested? Who benefits from being disconnected from the internet on the back of wild exaggerations?

Monday, 7 September 2009

Crowdsourcing the sources of a citizen journalism essay

In the second half of last year, I ran a Level 2 module called Cybercultures in which students were introduced to the study of cyberculture and some of the current debates being prompted by developments in digital technology. One of the weekly sessions looked at the role of citizen journalism and its impact on traditional/professional journalism. The assessment question set for this session was as follows:
It could be argued that only those with professional training should be allowed to publish news. Critically examine the threat posed by citizen journalism.
I have received an essay from a nameless source which seemed very well composed, but it lacked sources. The essay was submitted via our electronic plagiarism detection, aka EPD, (TurnItIn) which flagged the essay as 96% original which was unusual in itself, as the service cross checks work against several electronic databases and scholarly sources. Usually, student work tends to hover around the 75% original point, taking into account the inclusion of primary and secondary sources as citations.

In the past, marking essays often involved typing phrases into a search engine to see if they produced any similar results but this was time consuming - the new process is much quicker but not always fool proof. Not every academic source is logged by the EPD. Also, work that has been created via a service offering custom written students essays are capable of sidestepping this service as these pieces of work are seldom indexed by search engines, hence are unlikely to appear when searching for key phrases.

A modest proposal

Given the above point I'd like to run a little experiment. All students who submit work for assessment are required to sign a form or agree that the work they submit adheres to the University's regulations regarding infringement. Consequently, any student who pays for an essay has already broken those regulations, yet it is very difficult to prove as those pieces of work are seldom freely available online and are unlikely to ever appear so unless someone published them to the web. A student who has knowingly plagiarised or infringed would be unlikely to publish their work as they risk being penalised for doing so. However, if that essay is published to the web anonymously then 2 things can occur:

  1. The essay will eventually be picked up and indexed by search engines so any subsequent submission of similar content will be flagged against it
  2. Any readers familiar with the content may be able to spot gaps in the EPD database and assist in a crowdsourcing experiment
How should an educator respond to a piece of work that seems suspicious but is unable to locate the sources? Ignoring the material is not an option as it brings the reliability and validity of the assessment process into question and those students who produce their own original work may feel short changed if nothing is done about suspected violations.

Below, you will find an argument that hasn't been picked up by our EPD services. Neither does a random search for key lines bring up anything. This may be a plagiarised essay. However, it may also be a unique piece of work. If you can spot sections which seem notable, please get in touch either in the comments section below or by emailing me at


When looking at the threat posed by participatory media to professional practice, it is first necessary to understand the seeds of the former's existence, and the stance of the latter. The concept of professional training poses immediately the idea of the amateur, self-educated free media, the effect of which is a much discussed topic among authors, of books and blogs. Citizen journalism's slingshot into the public awareness brought with it not only the benefits, but faults and flaws within the system. Ultimately, that which it seeks to change, modify or rectify, that is the rigidity of professional journalism, can offer things from that structure, that the open-sourced public bank of information can not. To establish the threat that citizen journalism poses to the commercial media, an assessment must be made of its impact on the existing standard, and an evaluation conducted as to the weighting of the negative or positive connotations of said impact. The flaws in professional journalism are what drove the rise of citizen journalism if it is to be a threat to the very foundations of the professional industry, then it should not only correct these flaws, but prevent an overriding, more useful source of information.

The history of citizen journalism and its ascension to such status is a product of a direct marketing supply and demand idea. The general public called for an alternative to existing journalistic style, and the Web 2.0 generation provided it. Aaron Barlow explains that the defining property of Web 2.0 was the personalisation of the internet, it moved from a communication and information device, to a social networking media that connected thousands of people from all corners of the globe. He explains that whilst professional journalists felt the public were becoming increasingly wilfully ignorant “the citizen journalists believe there is a desire in the public for information that allows careful consideration of the issues” (Barlow, p.141). With regards to the news, the members of this internet revolution; the blogosphere arose as more and more people hosted their thoughts, opinions and observations in the virtual world. Commentary news blogs came forward into the public scope of view, as Baase discusses in A Gift of Fire: Social, Legal and Ethical Issues for Computing and the Internet, online blogs were popular because they “present a personal view; they are often funny and creative; they provide varied, sometimes quirky perspectives on current events” (Baase, 2008, p.5) allowing the reader to form a connection with the writer. The absolute lack of mainstream filtering and the independent presentation of the opinion seems genuine, therefore evoking common emotions and feelings among its readership. This quality is what led blogs to becoming so popular among the people of the population.

Initially scorned by members of the mainstream media as being unreliable and less than objective, Baase explains how blogs were first viewed almost as gossip columns (although it was argued by some that this could push mainstream journalists to excel in their work) then eventually came to be accepted as complementary to existing news sources, that they “demonstrated their influence by digging up information before the mainstream media did and by pushing stories the mainstream media did not publish. Bloggers detect and report errors, bias and digitally falsified news photos in mainstream media.” (Baase, 2008, p.6)

He explains that this drive to be impartial, or to at least highlight flaws in professional impartiality, led to their popularisation and soon businesses and organisations realised the potential of blogs to inform and communicate with their consumers. The readership of some blogs is now in the hundreds of thousands, often peaking to millions when a particularly important new story breaks. Baase makes the interesting point that many of those bloggers at the forefront of their newly defined industry, are being hailed not only as near-celebrities, but as respected industry professionals, often given invitations to press conferences, fashion shows, art galleries and the various related media events usually populated by journalists (Baase, 2008, p.5). This new definition of status has taken bloggers out of the virtual world and cemented their status within the real community.

Blogging has become, not only a social and cultural phenomenon, but a legitimate opposer to the multi-conglomerate media centres such as the likes of the Rupert Murdoch group of publications. Stuart Allan looks at the seeds of blogging in more detail with regards to the negative public opinion of traditional journalism. He suggests that its popularity rose as it stood firmly as an alternative, indeed a remedy, to the faults of established journalism, perceived to be restrictive or misleading. Stuart Allan suggests that this comes from journalists’ apathy toward their audience, that there is “a certain degree of ambivalence about opinion polls” (Allan, 2005, p.100). Indeed, he goes so far as to suggest that journalists ignore public opinion and are guilty of staying too close to the factual aspect of reports and ignoring the wishes of the public. Whilst the professionals would argue that news should not be driven by the consumer, it is the consumer they are providing for and unfortunately, without answering the call, the industry would die.

Perhaps the best way to analyse the impact, and therefore any proposed threat, of reader interaction with news reports is to observe the journalistic participation of victims of global natural and terrorist disasters. For example, the Tsunami that struck the East on Christmas Eve 2004, Hurricane Katrina that struck New Orleans, America in 2005 and events such as the 9/11 bombing of the World Trade Centre in New York, America and the 07/07 London attack in England. In both sets of circumstances, evidence of the drive for normal citizens to record and broadcast the experience is apparent. Videos, still images, recorded interviews and statements provided news content to explain the situation to the watching public. Here, it can be observed that the news became not just a factual report, not even a witness statement, but an actual live recording of an event, first hand. In this, those citizens who chose to record and document their experiences placed themselves in the line of danger as would a highly trained war-time correspondents, overriding a biological drive to escape and survive, instead, they chose to record and broadcast.

The worry of professional journalists lies not in recording of events, as any footage or recollection of a situation contributes to the general public awareness of world-events, but in the analysis and discussion of the happenings. The concern of maintained objectivity arises as communities reporting on local events will ultimately by affected by the consequences and therefore, even unintentionally, biased toward the situation. Andrew Keen, in his book The Cult of the Amateur raises the idea that this loss of pure truth arises “because democratization, despite its lofty idealization, is undermining truth, souring civic discourse, and belittling expertise, experience, and talent.” (Keene, 2007, p.15)

Only those with professional training in remaining morally, ethically and (perhaps most importantly in the modern age) legally just, can seek to comment fairly and abstractly on a news issue. Keene likens the modern users of Web 2.0 to monkeys, typing away in “an infinite universe” (Keene, 2007, p.15), generating an endless stream of personal truths. Instead of there existing a factual, definite truth, each person takes and gives to the collective Web 2.0 knowledge, “shattering the world into a billion personalized truths, each seemingly equally valid and worthwhile”. (Keene, 2007, p.17) Truth has become an amalgamation of people's personal opinion, distributed onto a virtual storage system with no moderator, no censor, no fact-checker. The possible emergence of such a regulatory figure of organisation is discussed by Dan Gillmor, who looks at services like Technorati and Feedster, that browse and search web content and “may enable “consumers” of journalism to sort through the opinionated conversations and assemble something resembling reality, or maybe even truth” (Gillmor, 2006, p.42).

Hailing such tools as the pioneer of content screening, a kind of intelligent RSS, he is however, keen to point out the flaw in the system; that they are simply just tools. They cannot replace either journalism or user-generated content as they cannot assure the core values of journalism; “fairness, accuracy and thoroughness” (Gillmor, 2006, p.42), or span the full scope of the available public media.

With this in mind, it can be seen that citizen journalism, whilst an exciting new medium for the delivery of news information, also has its faults. The main problem it faces is that, unlike professional journalism, it has no code of conduct or set of underlying principles that it must abide by. Richard Keeble explains in Ethics for Journalists that professional journalism bases its ethics on a multitude of various codes of practice, each with the same underlying core values of truth and respect (Keeble, 2001, pp.14-15). Citizen, democratic journalism is not the result of formal training in the rights and wrongs of informing the general public of other people's business morally and legally. In such, it often becomes a melee of rumour and assumption; a free-for-all where any opinion has the same amount of credibility and value as any other. Martin & Hansen examine various critical approaches on this issue, drawing the conclusion that “it isn't that people can't get diverse information swiftly; it's that people can't decided what is significant, relevant and useful while dispensing with unwanted information” (Martin & Hansen, 1998, p.45). That, unfortunately, is the core issue; that without professional investigation, reported events are relayed as opinion and supposition. We expect the news to tell us the truth, but if our trust in journalists is waning, why should we turn to the public arena. What credibility does it posses that we hold it more reliable than the word of those trained to be objective, who seek to be accurately informed. Only professional practice can offer trained, impartial opinion on news, or any other issues. The conveyance of facts and figures, in a delicate situation, is best left to those who have studied the law and ethics of such problems.

Perhaps then, the answer is not to look on citizen journalism as a threat, but instead as a balanced opposer to professional practice; as independent film-makers are to Hollywood production companies. What must not be allowed is for the mass media to replace professional practice and become the sole source of information. Maybe the threats of citizen journalism should be considered not as threats, but additions. The balance of company-funded journalists with the constraints of public publishing, who may be tempted toward bias or corruption, with the free, community sourced media. The rigid limitations of the former cancelled, or harmonised by the latter. Equally, the professional deficit of training in law and ethics can be enforced by those journalists who have studied their craft. In all media; photography, writing, music, art, there are professionals and amateurs who interact within the same space of coverage, why can this not exist in journalism? This balance between those who are trained and those not, keeps the level of work produced as a whole interesting, with tight technical proficiency, balanced by freshness and work that breaks through boundaries, challenging the landscape of the craft. Citizen journalism challenges professional practice, forcing it to change, to acknowledge its own faults and be proud of its standing as a valuable source of information. The two offer to each other a method of self-analysis, a check and balance system for an assessment of how their work matches to their intent.

Since the rise of citizen journalism, spurred by the Web 2.0 generation, professional journalism has constantly faced an opposition. What citizen journalism offers to the general public is a chance for them to speak out on issues they feel need to be brought to public attention. It also allows the 'common man' to question the motives and sources of commercial reporters, either due to bias or inaccuracy. Not only this, but it offers a free, open exchange of information and opinions. However, with this comes the problem of unregulated media. Without an independently fact-checking source, the independent articles within the free media have no validation as to their accuracy, nor can they be excused of bias. As a whole, it offers every viewpoint, every position on an issue, but it the responsibility then lies with the reader to assess the piece objectively. Professional journalism removes this need as its practitioners are trained and expected to remain objective. Unfortunately, individuals may not uphold this, but the free media is not a solution, more a balanced weighing tool.

Allan, S. (2005) Journalism ; Critical issues. McGraw-Hill International.

Baase, S. (2008) A gift of fire: social, legal, and ethical issues for computing and the Internet. Prentice Hall.

Gillmor, D. (2006) We the media: grassroots journalism by the people, for the people. O'Reilly.

Keeble, R. (2001) Ethics for Journalists. Routledge.

Keen, A. (2007) The Cult of the Amateur: How Today's Internet is Killing our Culture. Doubleday/Currency:California.

Martin, S.E. & Hansen, K.A. (1998) Newspapers of record in a digital age: from hot type to hot link. Greenwood.