Snake-oils, supplements and the pinnacle of data presentation

Snake-oils, supplements and the presentation of data

A big part of the reason for our existence at The Science of Sport is the clear and creative presentation and discussion of scientific subjects and concepts.  The idea was to make sports science more accessible, whether you're an interested reader, a coach, a high school teacher, a student, a scientist in an unrelated field, or a sports scientist. 

Of course, we regularly deviate into debate and opinion, but it's the way in which concepts are presented that (we hope!) makes the subject more interesting.  I've often felt that if you can go to a cocktail party or a dinner and talk about a scientific/analytical subject related to sport thanks to one of our articles and its discussion, then we've done our job!

The presentation of data - a defining characteristic for the translation of science

Yet have you ever noticed how badly, in general, science present its data?  You see this all the time in University courses, where often, the most accomplished scientists make the worst lecturers.  Students pick this up immediately, when some lecturers cannot seem to filter their ideas, they don't appreciate the level of the audience they're speaking to, and they lack the visual 'discipline' to take data and translate it into clear and concise principles and concepts.  I suspect all of you are remembering one such academic right now... (and if I happened to lecture you, you'd better not be thinking about me!)

The key here is that designing the research study, gathering the data, and performing the stats (which is the academic focus, most times) represents only part of the "journey".  This is because data by itself is raw and has no significance beyond its existence.  It must be related to other data, it must be aggregated or collected, analyzed, visualized and designed.  They teach this as part of "systems thinking" in IT courses, and the following diagram, designed by a brilliant website called "Information is Beautiful" depicts this:

Translation

The ability to communicate becomes all the more valuable when a scientist goes OUTSIDE the academic world and must speak to the public or corporate world.  Within academia, there is a level of acceptance of how data is presented - look at a dozen scientific journals and you'll see much the same style of language and graphics in every one.  Tables and line graphs are common and the language is, for all but a few in the field, very difficult to follow.  That's because it works in the field.  But outside, whether it's to the public, or the corporate sector, the rules of the game change a little.

I learned this the hard way, incidentally, because in 2006, I deviated from my PhD and went to the UCT Faculty of Commerce to do a Post-graduate in Sports Management.  There, I did marketing and finance, and spent 2 years working in the sports sponsorship industry, where a big part of the job was to put together presentations and show market-related, financial and sponsorship data to companies.  Gone was the classic approach of line graphs, tables and standard deviations.  I learned more about data presentation in those 2 years than in the previous 8.

And I can't help feeling that science loses out in this area, because it is often left to the individual to learn how to manage and present information.  I was lucky, because in my supervisor Tim Noakes, I had someone who had a gift for the communication of science, and then I was fortunate enough to work in sports sponsorship and learn on that job.  Not that I'm very good at it (it's not my place to say!), but I was at least given direct, tangible advice on how to do it.  For the most part, it's neglected (I haven't seen it taught before) and so some succeed, but most fail.

And so in line with that, today is a post on the presentation of data, and specifically, the scientific evidence for popular health supplements.

Snake-oils and health supplements

Right, so here is the single best piece of data presentation that I have ever seen.  It comes, once again, from the "Information is Beautiful" website, run by David McCandless (when you're done here, play around on the "Visualizations" tab on the top of his home-page.  Amazing work).

So what you're looking at below is an image depicting the level of evidence and popularity of a range of health supplements.  The higher the balloon, the stronger the evidence for the supplement (but only for the conditions listed in or linked to the bubbles).

The larger the balloon, the more popular it has been, based on Google hits.

So there's a lot to be said for this graphic.  It's easy to follow - so obvious that I dare say anyone will understand it almost instantly.  It's also concise - no need to read a 32-page review of the literature in a scientific journal to grasp the key points.  For example, in one glance, you can see that Vitamin A lacks evidence, whereas Vitamin D has strong evidence.  As easy as that.  In fact, it's so easy to follow that I don't even need to comment on it...!

The graphic uses "relativity" (in the size and position of the bubbles) to get across those key points, and it uses colour to further emphasize strength of evidence.  It's a masterpiece of clear and accurate data presentation.

Strength of evidence - not just externally impressive!

And then most important of all, the evidence is just about as "stringent" as you'll find it - it comes from Cochrane reviews and PubMed analyses in which only randomized, double-blinded placebo studies were used.  And this is a vital point - you cannot compensate for weak data with spectacular design and visualization.  Or at least, you shouldn't. 

This happens too often, and one of the challenges faced by science is that marketers and designers often end up working on projects with zero scientific backing, but they win the battle for "the mind of the consumer" because they know how to present what is actually hollow and worthless "science" in a much more appealing way (think Power Balance bands and other hocus pocus products that become "scientifically proven").  The end result is that you have this "debate" in which the companies present their visually impressive material and the science argues the "nuts and bolts", and in the end, the consumer loses (usually because it's easier to believe the fancy graphic than the dry science). 

The interactive - taking presentation to an even higher level

But wait, there's even more to it.  Taking the above image, McCandless then turned it into an interactive graphic (click here to open in a new tab).  Here, you can:

• Hover over each balloon to see which conditions it is effective for
• Click on "Show me" on the right hand side of the image to see which supplements are produced for each of the listed conditions
• (Most impressively), click on the balloon and you'll be redirected to the page which carries the evidence for the image - the Pubmed and Cochrane review papers. 
  
  This is the strength of evidence I was talking about earlier - it is indispensable, because without that scientific evidence, this would just be wallpaper that does more damage than good.  And I dare say, this is part of the reason why many scientists will be skeptical of this kind of data.  Bizarrely, there seems to be a culture that "if it looks too good, then it's probably not accurate or credible". (This is the data equivalent of medicine - if it doesn't taste terrible, it probably doesn't work...!)

 Contrasting with the "typical" approach

A final point to make is that the "typical" approach to this kind of question (how effective are supplements?) would be to conduct a meta-analysis, and then publish the findings in a scientific journal, in 32-pages of black and white, scientific language and probably with multiple tables showing the level of evidence and the p-value.  

This is as accurate as anything you see above, and it contributes enormously to the value WITHIN that field, but for anyone outside of the health science-academic world, it has little significance.  The general public, as informed as they may wish to be, will not see that data - they will remain uninformed, not as a result of this knowledge not being available, but because it has not been translated and delivered to them in such an interactive, palatable (and stimulating) way.

I realize I may sound over-critical of science, and this is not my intention (I suspect some academics will have stopped reading at this heretic talk by now!)

Rather, I want to emphasize that science can be so much more effective, powerful even, if it meets with good design and presentation.  Whether that means partnering with a designer like David McCandless (probably quite costly, but I believe worth it in many cases), or simply learning the discipline of turning words, tables and line graphs into meaningful and elegantly presented information, I think it's indispensable!

And lastly, here is a slide that was labeled as the "worst Powerpoint slide ever".  It comes from the military, and shows part of the strategy in Afghanistan.  Just for contrast...

Ross