Backdoor Breath Testing 3: Scientific Community Regarding Traceability and Accuracy
So, if you've taken the time to read our previous post regarding measurement uncertainty, you've already seen that the leading scientific bodies (the International Standards Organization, the National Institute of Standards and Technology, and the National Academy of Sciences) all demand that any scientific measurement be reported as a range of values, not a solitary number. Remember that the Minnesota BCA already does this whenever it tests a blood or a urine sample for alcohol -- it's just breath tests that are reported as a bald number, rather than a range of values that have a high likelihood of including the driver's true alcohol concentration.
Which brings us to the next topic(s): "Traceability" and "Accuracy." We've presented whole seminars on each of these topics, but for the purposes of this blog post, it's enough to know that all scientific measurements are required to be traceable in order for the final result to be deemed valid, accurate, and reliable. In this context, a traceable measurement means a measurement that can be compared to a known, national/international standard.
That comparison comes from one of many "gold standards" held by the National Institute of Standards and Technology. NIST has "the" ounce, "the" pound, "the" 32 degree benchmark, and so forth -- anything you can measure has a standard sitting in NIST's vault, that others can use to make their own "standards." This also includes "the" standard for 0.08 grams of alcohol per 210 liter of breath, the standard that matters for breath testing. Therefore, having a traceable breath alcohol measurement simply means that you can look at how the breath test machine was calibrated, how the test was performed, and how the final result was reported, and through a trivial amount of math and documentation "trace back" that final result from the police station all the way to the NIST vault in Washington D.C.
The BCA has repeatedly admitted under oath that Minnesota's breath tests are not traceable (trust us, we have the transcripts!)
On to "accuracy." Like most metrological terms, "accuracy" has a specific definition: a measurement is only "accurate" if you can mathematically say how likely it is that whatever you are measuring is fairly close to what your actual measurement result was. In laymen's terms, any measurement is "accurate" as long as you admit how inaccurate it actually is.
Confused? Don't be. Say I take the outside temperature, and my thermometer says it's 40 degrees. That measurement is scientifically inaccurate -- I can't draw any conclusions from it at all. I can't say that it's above freezing or below freezing. Now, say I take the outside temperature and my measurement is "40 degrees plus-or-minus 3 degrees with a 99.7% level of confidence." NOW I've told you just how inaccurate my measurement is, which in turn means that my measurement actually IS accurate. NOW I can tell you "yup, it's above freezing." Accuracy isn't about perfection: it's about admitting your imperfections, so that you can actually use the measurement you produced.
The BCA has repeatedly admitted under oath that Minnesota's breath tests are not accurate (trust us, we have the transcripts!). In fact, one scientists admitted that he can not even tell us what the "outer bound" is when it comes to how inaccurate our breath tests could be. Another one, when asked if he could say under oath that our breath tests are accurate to within 0.1 (that's HUGE) admitted that he could not even say that.
Again, we have to stress that this is basic science; nothing fancy, nothing novel or new, nothing mysterious or sneaky. The concepts of traceability and accuracy are fundamental principles that apply to all measurements, and have nothing to do with how good (or bad) your measurement system is. In fact, these principles assume that whatever device you are using to take measurements -- whether it is a tape measure, a thermometer, or a breath test machine -- is the absolute best device on the market. Even the best measurement devices are imperfect, which is why every measurement that matters needs to be both traceable and accurate . . . . but don't take our word for it!
International Vocabulary of Metrology: "The objective of measurement in the Uncertainty Approach is not to determine a true value as closely as possible. Rather, it is assumed that the information from measurement only permits assignment of an interval of reasonable values to the measurand, based on the assumption that no mistakes have been made in performing the measurement . . . . However, even the most refined measurement cannot reduce the interval to a single value."
Guide to Uncertainty of Measurement: |When reporting the result of a measurement of a physical quantity, it is obligatory that some quantitative indication of the quality of the result be given so those who use it can assess its reliability. Without such an indication, measurement results cannot be compared, either among themselves or with reference values given in a specification or standard.| (emphasis added).
International Standards Organization: "The laboratory shall at least attempt to identify all the components of uncertainty and make a reasonable estimation and ensure that the form of reporting of the result does not give the wrong impression of the uncertainty"
These last few blog posts should have made it crystal clear that Minnesota's method of conducting breath tests is so fundamentally unsound so as to be effectively useless. But anyone with any experience with the criminal defense system will tell you that holding scientists accountable for their measurements takes more than being right -- you can't just be scientifically right, you need to be legally right as well (and sometimes the two are very different).
Luckily, not only is the science on our side, but the law is too. Up next, we'll discuss the legal standards that apply to these untraceable, inaccurate breath alcohol measurements (and after that, we'll show you some of those transcripts!).
Next Post:
Part 4: Legal Standards From Our Implied Consent Law And Standard Criminal Jury Instructions