The scientific interpretation of Oscar Pistorius research

From the go-kart racer to the F1 driver: The  scientific interpretation of the Pistorius research

The possibility that Oscar Pistorius has an advantage from his carbon-fibre blades is a topic that I honestly wasn't going to cover on this site.  For one thing, I've covered it already, dating back to 2007 when the story first emerged.  And secondly, it inspires in people a reaction that buries the scientific question/debate in a tsunami of emotion fueled by social and political correctness.  Even in a recent scientific debate between two camps that formed with Pistorius' own team split turned into name-calling, and the ghost of John McEnroe was called upon in a "You cannot be serious barb".  And that was in a scientific journal... 

However, I've received about a dozen emails in the last few weeks (most of them surprisingly supportive) and a lot of people have asked about the scientific discussion around the blades.  There are a lot more readers now than before, and so the "old news" may not be that old to many.  And then I got to thinking - The Science of Sport began specifically to provide a more detailed analysis of sports news, with a special focus on how sports science might add to our interpretation of what we see.  As we say in our "Mission", we're about the how and the why of sports news.  And there can be no bigger current sports story involving sports science than the Pistorius one.

Added to this is that the media seem scared of this issue - or at least, that's the only reason I can think of for why some of the facts haven't emerged, or why they never challenge the claims made by those associated with Pistorius, and most recently Hugh Herr.  Every claim made is accepted as fact without actually looking at what is being said, and some of the counter-points border on ridiculous.  And so this series of posts is the result - they will summarize my views on the science, the scientific process and the misdirection that has characterized what is one of the most controversial illustrations of science applied to sport encountered since this site started (with the exception of Caster Semenya in 2009 and 2010)

And it will be controversial.  Those who have followed the site for a while will know that I believe there is an advantage, and that is a conclusion based on ALL THREE scientific papers published about Pistorius, as well as the years of hypothesis and discussion, and the known theory of sprinting.  But it would be a glaring oversight to ignore it any longer.  (It's a long, and technical piece, so I may break it up into parts)

Being right vs being correct - the concession, and agreeing to disagree

Let me start off by saying that even once all the science is considered, and even if you arrive at the conclusion that Pistorius has an advantage, your response to that may not necessarily be to stop him competing in able-bodied races.  The more I think about it, the more I can appreciate the views of those who may be saying that he should run regardless of the possible advantage.  They have a point - it's possible that his "rarity" and the fact that he is such an inspirational figure outweigh the potential advantage and future implications of allowing him to compete. 

I'm not one of those people - my interpretation, having followed this from before the hypothesis-generation stage to the research studies, is that he has an advantage.  And because I come at this from an athletics standpoint (as opposed to the human interest side of it), my opinion is that he shouldn't be allowed to run regardless of it.  However, I bring this up counter-point to emphasize that this controversial issue may not have a "right" solution.  It may be a matter of being "correct" (with regards to the science) but not "right" (with regards to his participation).  So while I disagree, I can respect and appreciate a balanced view that says "there may well be an advantage, but given the circumstances and bigger picture, allowing him to run is the right thing to do".  All I ask is the same respect and courtesy, free of emotional name-calling!

What I do take exception to, however, is the dishonest scientific process in this story, and the lies that are actually shown up by data - that's where my criticism is leveled.  This is therefore not only a post about Pistorius, it is about the scientific process behind him.  My criticism is directed primarily at the science that, I believe (and as I'll describe) was twisted and manipulated to create the desired finding.  It is not at Pistorius personally.

However, I do also want to use these posts to respond to some personal comments made by Pistorius at me - unfortunately, journalists don't allow for opportunities to respond to being called a "kart racer" and accused of hunting publicity.  These are minor issues, so I leave those for a footnote at the end of this post.

The bigger issue is some of the claims being made in this debate, which are outright lies, to the point of deliberate dishonesty.  The media of course don't challenge these statements - they accept every claim at face value and instead prefer to sensationalize and create "hero vs villain" sagas.  But the claims that the science proves that there is no advantage demand a response.  And since the media are not bothering to question them, someone else has to.

Right back to the beginning: The starting hypothesis

Before looking at the evidence, it's important to understand WHY specific things were being measured, and what the basis might be for saying that Pistorius has an advantage, because this will help explain the implications of what were later found.  As far back as 2007, it was possible, on a theoretical level only, to debate whether the blades would provide an advantage.

The key points suggesting advantage were:

1. The reduced mass of the carbon fibre limbs
   
   The ability to accelerate the limbs faster is the first theoretical advantage.  A study had previously found that in distance runners, athletes with smaller calves (by mass) were more economical.  And while economy isn't as valuable for a 400m sprinter, the reduced mass is a potential advantage because it allows lower forces and work to be done in accelerating a smaller mass.  Conceptualize this by imagining how your 400m performance would be affected if you raced with 1.5kg strapped to each foot, and then imagine taking that mass away - that is the mass advantage of the carbon-fibre Cheetahs.  This would be detected in a reduction in the work or cost of running at a given speed, and is potential difference number one.
   
   
2. Enhanced energy return from carbon fiber
   
   The company that manufacture the blades, Ossur, make the claim that the carbon fibre blades return around 80 to 90% of the energy they store under compression.  That is, land on carbon fibre that is specially designed and shaped to compress under the body, and more than 80% of the energy will be returned on recoil. 
   
   In contrast, human tendons are relatively poor at returning energy - estimates vary depending on the way it is measured, but energy return ranges between 30% and 70%.  Greater energy return would reduce the need for muscle work, further reducing the "cost" of running.  There's no question that 400m races are metabolically limited.  The details are debatable, but key is that every 400m runner is regulating the rate of metabolite accumulation or energy depletion.  They are limited/regulated by the consequences of metabolic energy use, and anything to lower this (as carbon fibre blades would, according to the theory) would be advantageous.
   
   There was a lot of confusion about this, with some people saying that carbon fiber was only passive energy return and thus a disadvantage.  The reality is that human muscle can return energy, but it requires muscle contraction twice - first to store energy AND then to release energy.  Therefore, there is a cost to energy storage AND return in human muscle and tendon.  The fact that carbon fiber releases energy stored under compression is thus an advantage - there is no cost. 
   
   Once again, this would be reflected in a reduced energy cost of running - same effort, faster speed, or same speeds with less work.  On top of this, there is the matter of fatigue, which causes the force-producing properties of muscle to decline, whereas carbon-fiber never fatigues (in an athletic sense).  The result may be less fatigue at the same speed or for the same effort, over time.

Given those two theoretical considerations, the hypothesis before testing was that the metabolic energy cost of running would be substantially lower compared to able-bodied controls, and that the mechanics of running would differ significantly.  The word "bounce" came up many times in the initial debates.  The question remained, however: Had anyone collected data to confirm these hypotheses?  The answer, until 2007, was no.

The summary version of the evidence

Then, over the course of about 6 months, that evidence was gathered.  It took a while for it to emerge, but when it did, it confirmed much of the above.  This is a summary of the findings, and if you don't have the appetite to go through the detail and you trust the summary, then this is where to stop!  Or read on for the detail of the first study...

1. Pistorius used significantly less oxygen than able-bodied sprinters.  25% less during sprinting (the IAAF Study) and 17% less during jogging (the Herr study), to be specific. Therefore, his metabolic cost of running was lower.  This was true at all speeds, from jogging to sprinting, and of course the carbon-fiber limbs are not designed for jogging in the first place, so one can question how valid a measurement would be during jogging.  It was still found that he used 17%, or three Standard Deviations, less energy when jogging than able-bodied sprinters.
   
   
2. When you add in elite distance runners, then Pistorius becomes "similar" to other runners.  Herr et al conveniently did this when they found that Pistorius uses 17% (or 3 SD) less oxygen than able-bodied sprinters.  Rather than actually testing runners themselves, they turned to the literature and found fifteen-year old research studies on elite distance runners, and sure enough, when they added other people's data to their sample, it helped bring the average down and he became statistically similar to distance runners.  Even then, he used 4% less oxygen than the elite distance runners, which is quite remarkable.
   
   
3. Pistorius had significantly lower vertical ground reaction forces and horizontal braking forces than able-bodied runners.  That means less braking force, but interestingly, the same propulsive forces.  This in turn means less work at the same speed than able-bodied runners.  There is however a disadvantage of lower peak vertical forces, compromising the acceleration from the start.
   
   
4. The energy return from the carbon fiber limbs was 92% compared to 59% for the able-bodied runners.  This, in part, explains the reduced physiological cost compared to sprinters.
   
   
5. Pistorius' rate of fatigue was similar to the able-bodied sprinters, using running trials to fatigue.  This is interesting, and I have my doubts about whether testing an untrained athlete who knows the hypothesis reveals anything of value.  It is also questionable as to how relevant it is to a self-paced 400m race, but this is the only one of five findings that doesn't suggest advantage.

Now, for more on the findings, as well why they were so hotly debated, read on.

First testing - the IAAF study in Germany: Bouncing locomotion with lower metabolic cost

The first round of testing was conducted on behalf of the IAAF by Pieter Bruggemann in Germany in October 2007.  Below is a brief explanation of the three key findings from that paper, the reasons it challenged by Pistorius' science team (led by Hugh Herr, and at that stage, Peter Weyand), as well as some responses from other biomechanists to those challenges.

1. Oxygen use 25% lower = lower metabolic cost of sprinting

First, they measured his oxygen uptake during a simulated 400m sprint.  It's really important to understand that measuring oxygen is done because it's a "barometer" of sorts for the cost of running.  It's not because oxygen is limiting to a sprinter or because a sprinter needs a high VO2max (as was alleged as part of the normal obfuscation of the issue), but rather because much like fuel use in a car, oxygen measurements give an indication of the energy requirements for running.

So if you recall the hypothesis and the theoretical background to the question, measuring oxygen is important because it allows you to establish whether the work requirement is in fact different for carbon-fibre compared to able-bodied limbs, and not whether the runner has a better cardiorespiratory system.  The hypothesis was that Pistorius would show a reduced metabolic cost of running.

Below is a graph of oxygen use during the 400m simulated trial:

So, Pistorius uses significantly less (25% once up to speed) oxygen than able-bodied sprinters during a 400m race.  Also of interest is that he uses the same volume of oxygen over the first 15 seconds, when the balance problem (which many have said would increase his metabolic cost) is theoretically greatest.  Once up to speed, balance is actually not as much of a factor as some suggest - if you want to test that claim, jump your bicycle and ride at 2 km/hour, and then at 20 km/hour, and you soon realise that movement assists balance.

The mechanism for this reduced oxygen, and hence energy cost would be two-fold, and goes back to those two points raised earlier - increased energy return and mass of the limb mean less work, and that means a lower metabolic cost, indicated by lower oxygen use.

However, there was a problem with this conclusion, and the research was not complete in this regard.  That is, when someone is sprinting, energy comes from oxygen-dependent and oxygen-independent sources.  Measuring oxygen uptake during a sprint means that both are contributing, but only one is being measured.  The graph above thus gives an incomplete picture, and the conclusion that the metabolic cost of running is lower was challenged at the CAS.  There is a counter-point, and that is that if he uses less oxygen and thus less energy from oxygen-dependent sources, then there's no reason to suggest that his oxygen-independent contribution would be higher.  But only part was measured, and this would go on to become perhaps THE key point that was challenged in the CAS hearing.

The mechanical energy - 41% vs 8% energy loss

The other Bruggemann finding is summarized by the quotes below.  All are drawn from the report after testing and from a great piece by Edward Ovadia on the issue.

"The energy loss in the blade during the stance in sprinting was measured at 8% and is significantly lower than in the human ankle joints of the controls (41%). This results in a mechanical advantage of more than 30% (it is reported as a 7-fold greater energy return elsewhere in the paper) when the leg is substituted with the prosthesis" - Bruggemann report

Bruggemann attributed most of the 41% energy loss to heat.  This was disputed by Hugh Herr, on the grounds that a) the energy loss in able-bodied runners cannot be that large, that it is not possible to dissipate that much heat in a 400m race, and b) that measuring energy loss in joints is extremely complex, and that there is a possibility that some of the energy in the human joint is not lost, but rather transferred across the joint.  This would bring the energy loss down from 41%.  Bruggemann however disagreed, saying that firstly, the frictional energy loss is as high as claimed, and secondly, the energy transfer across joints is very small.

What is significant is that other studies of energy in joints support the Bruggemann evidence, showing that energy loss ranges between 70% and 30%, depending on whether the person is walking, jogging or sprinting.  The 41% energy loss found by Bruggemann is thus not unrealistic - he may have overestimated, and at the time, a group of biomechanists did debate the model he'd used, but concluded that even with certain small imperfections, the general conclusion was correct - human tendon returns only around 60% to 70% of its energy, compared to 92% for the carbon fiber blades.

However, this discussion was another point of contention for the CAS hearing, who said that the uncertainty re the exact energy loss could not be confirmed. The paradoxical thing about this, however, is that if energy is transferred from the ankle to the knee, as Herr and Kram argued to the CAS, it is not actually an advantage. The problem, as Bruggemann explained, is that this would be "a disadvantage for the able-bodied athlete, because this energy will bend the knee in a phase when the knee is in extension. It's an argument against Pistorius" (Bruggemann, quoted by Edward Ovadia)

Ground forces - less energy loss and less work required

The third finding is that Pistorius had very different ground forces during running. His vertical forces were 20% lower and the horizontal braking forces were 50% lower than those measured in able-bodied controls. Interestingly, the horizontal braking force is reduced, but he doesn't lose anything in the propulsive component of the horizontal force. This is shown below:

To Bruggemann, the reduced vertical force was a distinct advantage, because it was measured at a constant speed, similar to that of able-bodied runners, and meant that Pistorius would do less work to run at the same speed, a finding that is supported by the lower oxygen cost shown earlier.

Herr however argued that this would be a disadvantage during the acceleration phase of running and that faster runners would need more force. There's no question that this reduced vertical force would be detrimental to the start, and goes some of the way towards explaining Pistorius' relatively slow starts.

However, the debate about peak forces and acceleration typically obscures the real significance of Bruggemann's finding - it's a false comparison because we should not be comparing runners at different speeds, but rather comparing Pistorius to other runners at 46 second 400m pace. The comparison to people running faster is irrelevant. As Bruggemann explains: "If we look at subjects running at different speeds, it’s logical to say that the higher the force, the higher the speed. But with all subjects running at a given speed, lower force is an advantage.”

These sentiments were echoed by Benno Nigg, one of the world's leading biomechanists (if not THE leading biomechanist of running):

"He needs less vertical force as well as less horizontal braking and propulsion force – which means he has to work less at the same speed than the control subjects. Pistorius lost less energy and had to produce less work during each [instance of] ground contact than the athletes in the control group.” (as quoted in Ovadia's article)

Conclusion from the first study


The ultimate conclusion reached by Bruggemann, as you all no doubt know, was as follows:

Sprinting with the artificial limbs (Cheetah) is – from a biomechanical perspective – a “bouncing” locomotion and is significantly different to sprinting of able-bodied athletes on hard surface. It is a different kind of locomotion at lower metabolic cost.

This was however challenged at the CAS for the reasons explained above - the metabolic cost was challenged because of the method of measurement, and the mechanical/kinetic data on the basis of alternative interpretations.

I think it's clear that when trying to model the joint loads, the forces, the energy turn, much is model-dependent.  The biomechanical model used by Bruggemann (called inverse linear dynamic model) was questioned by biomechanists, but they nonetheless agreed with his overall conclusions.  So too, as shown for a specific part, did Nigg. 

The importance of the physiological data 

The challenges made by Herr & Weyand were of course equally unprovable.  They offered no alternative to the 41% energy loss, only a question about frictional heat loss.  Of course, this was all that was required, because the CAS hearing only asked for doubt to be cast on Bruggemann's findings, and not proof of a lack of advantage (the context would have been significantly different then).  The result is that the biomechanics part of the debate reaches a stalemate, and finding "conclusive proof" for either position would prove impossible.  CAS of course required conclusive proof and perhaps it is not surprising that they ruled the way they did, on this question anyway.

However, it is for this reason that I would suggest that the physiological data, and not the biomechanical data, hold the more important information.  That's not to say the mechanics are unimportant - the case made by Bruggemann, both in his research and in his responses to the questions, is, I believe, compelling and correct.

But it's when you look at the metabolic factors that things become really insightful, because metabolic cost is a symptom of the mechanics, and so given that there are two ways to interpret Bruggemann's kinetic and energy data, the way to test the options is to use metabolic cost.

And so what was needed was a comparison between Pistorius and sprinters at sub-maximal speeds to ascertain whether that oxygen cost would be lower even then.  That is, repeat the experiment but at slower running speeds.  And this is where Herr enters the picture with a research study that can only be described as "manipulative".  It was, to be blunt, one of the most astonishingly selective research articles I have ever seen, to the point of being dishonest.

But that is the topic of tomorrow's post, when I'll consider Herr's evidence, and some of the claims made in the media about what he showed, and more importantly, what he chose not to show for the sake of the finding.  I realise there are many unanswered questions - but this is only the first part - the analysis of the Herr paper reveals much more, including why I believe the CAS process was so farcical.

More to come.

Ross

The fineprint

And finally, because the tabloid journalist did not allow a response, I must make the following statements, which I do at the end of the piece because they are a footnote, not necessary to the debate, but need to be said.

First, at no point, not even once, have I contacted a journalist to "push an agenda".  Ironically enough, the fact that I have conducted no "pertinent research" on Pistorius is in fact because Pistorius didn't want research.  Back in 2005, I offered to do the research, but he was not interested enough in the science.  Then in 2008, he requested help from SA sports scientists, but again, he would not work with us because of the likelihood that we would confirm the Bruggemann findings.  Ultimately, Pistorius went to Hugh Herr because Herr was going to find what Pistorius wanted.  In tomorrow's post, I'll explain how they did it. 

When you stop to think about it, any person who says "Oscar Pistorius has an advantage" is walking into unpopularity and guaranteed hostile reactions.  That is hardly the way to raise your profile.  The reality (and this is a direct message to the Pistorius camp) is that the journalists are contacting me with questions, and I am answering as honestly as I can, about the evidence and the CAS process.  The alternatives are to lie ("there is no advantage), or to say "the evidence is unclear", but it's not.  It clearly points in one direction.

Secondly, when Pistorius personalizes the issue by calling me a "kart racer", it reveals one of the key problems in this issue - this is not an issue of personalities and of WHO is saying what.  It's about the evidence, independent of the people.  I don't care that Hugh Herr got a write-up in Time, what concerns me is the distortion of the evidence (again, see tomorrow's post).  I also care to point out that Herr is a big recipient of funding from Ossur - the idea of "independent research" from him is utterly false.  There are vested interests everywhere on this one, and it clouds how the media are reporting it. 

That's all, an unnecessary distraction from the evidence.  I'll get back to it tomorrow!