Reading the News: Critical Analysis of Graphic Data

Reading the News: Critical Analysis of Graphic Data

TIME SERIES ANALYSIS

In "Paying the Price for Power" (Oregonian, May 20, 2001), the author presents a better than average time series analysis of wholesale prices for electricity. Indeed, the graph does illustrate the potential impact of spikes in spot prices on consumers. However, there are some potential problems with the graphic display:

Are the values adjusted for inflation? If there are increases in electricity prices, can they be attributed fully to power prices alone or are they partially contingent on inflation? Did inflation remain steady from January 2000 to May 2001? If it did not, the analysis needs to adjust for inflation to provide the reader with a closer understanding in specific changes in electricity pricing.

2. While there are dramatic shifts in the price of electricity (and the article does

reference to a "cold snap"), are surges in electricity standard for the season or

out of the ordinary? In other words, are the values seasonally adjusted? While the power surge did indeed throw businesses into disarray, a comparison with standard November — January energy pricing might illuminate the example all the more. Or, it may show that price surges are not as abnormal as portrayed in the article. Jessica M. Utts, in Seeing through Statistics (Belmont: Wadsworth Publishing Company, 1996) provides a range of useful questions for reading time-series data. This is particularly important in "reading the media" because time-series data are common. She asks:

Are the time periods equally spaced?
Is the series adjusted for inflation?
Are the values seasonally adjusted?
Does the series cover enough of a time span to represent typical long-term behavior?
Is there an upward or a downward trend?
Are there other seasonal components that have not been removed?
Are there smooth cycles (Utts, 245)?

While "Paying the Price for Power" equally spaces time, covers enough time span to represent that there was indeed a spike in pricing, does display some trends in pricing, the author does not adjust for inflation, seasonally adjust values, or account for other seasonal components that may impact pricing. This means that the graph is not as descriptive as it potentially could be.

Here is another example: In USA Today (Monday, April 23, 2001 — Money Section B), the author of "Who Used to be a Millionaire states above a graph, "There’s been a steep decline in the number of millionaires in Silicon Valley." The graph does not mention any adjustment for inflation, but perhaps most importantly the graph does not cover enough of a time span to represent a typical long-term trend in income. In fact, there is — according to the data — no downward trend. Rather, 1999 represents an aberration — a higher than average year for millionaires. This may have to do with other components (e.g. value of primary residence; changes in tax laws, etc.) that have not been accounted for. If anything, the shift from 138,000 millionaires in 1998 to 159,000 in 1999 and back to 136,000 millionaires in 2000 shows that the standard number of millionaires is around 136-138,000 rather than 159,000.

"THE MISSING LINK": HOW MEDIA REPRESENTATIONS MAY HIDE THIRD VARIABLES

It is important to recognize that the realm of quantitative inquiry can also be biased. While a reliance on numbers, statistics, and graphs may illustrate a phenomenon, such a reliance may reify biases within reporting.

Does a New York Times account of increased divorce rates across the "Bible Belt" show a shift in religious orientation or are other factors at work? (Monday, May 21, 2001)

There are multiple dilemmas within this account. First, the categorization of "Bible Belt" runs the risk of being too vague. What is it and how is it different? Is there demonstrable evidence presented that the religious beliefs of the citizens of Wyoming are markedly divergent from those of Tennessee? If so, what are those differences? The divorce rate is in the same category for both, according to the data, yet are both states within the "Bible Belt"? What of Arizona or New Mexico? While the rate is higher for Oklahoma, Tennessee, Arkansas and Kentucky, there are a myriad of other states that also have a high rate of divorce. The graph states "Not only are the divorce rates high across the Bible Belt, but in the last ten years more people have chosen to live together without being married." Rather than pointing to a dramatic change in religious beliefs (which would need to be definitively demonstrated), this points to a lack of difference, i.e. that the category of "Bible Belt" is not a useful variable.

Jessica Utts provides an instructive observation regarding the possibility of what she calls "The Missing Link: A Third Variable" (Utts, 177). While a change in religious belief (i.e. a dissolution of beliefs in the "Bible Belt") may be an explanatory variable for changes in divorce rates, the category of religious beliefs may be masking a third variable. She writes, "another common mistake that can lead to an illegitimate correlation is combining two or more groups when they should actually be considered separately (perhaps religious and economic categories in this case)....When the groups are examined together (e.g. religious and economic status), the individual relationships may be masked (e.g. change in divorce rate results from change in religious affiliation). The Times graph may run the risk of concealing more than it reveals by not pursuing other possible third variables, such as economic status.

III. HOW ONE WORD MAY SKEW REPRESENTATION OF VALID RESEARCH

In USA Today’s "Snapshots" (April 25, 2001), the accompanying narrative states "Most businesses want tax-free shopping online, CFO’s say..." Note that the graph implicitly supports tax-free shopping by constructing it as a matter of "LIBERTY." There is a significant difference between tax-free shopping and "liberty." By presenting this "snapshot" of the data, the presentation is skewed. The data may well be "correct" (i.e. there may be no issues with most businesses wanting tax-free shopping online), but how are the data represented? Graphic representations of data can dramatically influence how that data is received and interpreted. In this case, the conclusion that the reader might draw is that most businesses are for liberty. Yet, the research conducted is not research on liberty (for that is a much fuzzier category!), but rather on taxes. One could be for liberty, but against tax-free shopping online. By reading the media critically, graphic representation of data is called into question rather than accepted wholesale.

IV. DISASTERS IN GRAPHIC REPRESENTATION OF DATA:

READING THE MEDIA CRITICALLY

In the Oregonian’s "Where the (Former) Californians Are," the objective is to track the "percent of people moving to Oregon," yet the blocks from California and Oregon are of the same color. The result is totally illogical or at least ineffective. The map confuses, rather than assists the points made in the sidebar narrative.

Jessica Utts notes five basic "difficulties and disasters in plots, graphs and pictures"

1. No labeling on one or more axes

2. Not starting at zero

3. Change(s) in labeling in one or more axes

4. Misleading units of measurement

5. Graphs based on poor information (Utts, 145).

In this case, the units of measurement are misleading and poorly constructed. Blocks representing "where the Californians are" (which aim to explain migration to Oregon) are also found in California in an illogical manner. Utts’ "checklist for statistical pictures" would have been instructive here and proves to be instructive for anyone reading quantitative reasoning in the media critically. She lists "ten questions you should ask when you look at a statistical picture."

1. Is the message of interest clearly standing out?

2. Is the purpose or title of the picture evident?

3. Is a source quoted for the data, either with the picture or in an accompanying article?

4. Did the information in the picture come from a reliable, believable source?

5. Is everything clearly labeled, leaving no ambiguity?

6. Do the axes start at zero or not?

7. Do the axes maintain a constant scale?

8. Are there any breaks in the numbers on the axes that may be easy to miss?

9. For financial data, have the numbers been adjusted for inflation?

10. Is there extraneous information cluttering the picture or misleading the eye? (Utts, 150-151).

In this case, the message of interest is not clearly standing out, while the purpose of the picture is evident by the title. A source (the IRS) is quoted for the data and in this context we can deem the IRS as reliable and believable. But, everything is not clearly labeled. Rather, the graph is highly ambiguous and problematic. Utts’ advice here points to a critical reading of quantitative reasoning within graphic representation in the media.

V. POTENTIAL PROBLEMS WITH RANKING POPULATIONS

Forbes.com ranked (as reported in the Oregonian Monday, May 21, 2001) the 40 most populated cities in its article "Where to Score As a Single: Best Places to Be Single." Here, Forbes determines the "best" places to be single by combining the number of "singles" (which according to Forbes are those people who have never been married, not those people who are not married), nightclubs, museums, sports teams, university population, theaters, job growth, and cost of living plus "our own light-weighted subjective buzz factor (i.e. "public perception")" and "voila: The Washington, D.C.-Baltimore metroplex came out on top."

As with the measuring of the "most livable city," Additionally, the creation of such "subjective" accounts of "public perception" is the very mechanism that often shapes public perception. Yet, the way in which the variables are combined (usually adding then averaging rankings) is statistically unsound. Moreover, other variables (e.g. STD rates, homicide rates, etc.) that might be significant in evaluating where it is nice to be single are left out altogether. In short, terms such as "being single" or "livability" are highly elastic and not very descriptive.