“The past has no existence except as it is recorded in the present…we would seem forced to say that no phenomenon is a phenomenon until it is an observed phenomenon. The universe does not ‘exist, out there’ independent of all acts of observation. Instead, it is in some strange sense a participatory universe.”
― Professor John Archibald Wheeler, physicist
Physicist John Wheeler once made an interesting observation about the famous double-slit experiment. Set up one way, it measures light as composed of individual particles or photons. Set up another way, it demonstrates that light is a wave. The results show that light exhibits both wave-like and particle-like properties depending on how it is observed or measured. Prof. Wheeler’s observation had less to do with the experimental technicalities and more to do with the scientist performing the experiment. The results of the experiment quite literally depended on the preference – or bias – of the researcher. Set it up one way and prove light is a wave; set it up another way and prove light is composed of discrete particles.
Red meat consumption is another observed phenomenon that elicits a duality of responses. Some proponents claim it was an important component of humankind’s evolutionary ascendancy and is still important for proper health; others condemn it as a harbinger of disability and disease. However, the issues of red meat extend beyond the boundary of health and enter into many other emotionally- and politically-charged environments. It is within this cauldron of emotion and belief that such a heady brew may unknowingly taint the perspective of the most well-meaning and upright investigator.
Over the course of the last several decades, there has been a concerted message from many leaders within the fields of healthcare and dietetics to move away from the consumption of red meat, a message they say is based on science that clearly demonstrates ill health effects. Such a motivation is perceived to be above emotions and politics and, as a driver, is amplified and exemplified in the mainstream media’s obsessive push for everything “plant-based.” But what if, as Prof. Wheeler observed, there is no observation that is truly free of observer bias? Can we somehow get to a less muddled response?
This week’s study spotlight highlights an analysis that examines that question in terms of decades worth of epidemiologic studies on red meat consumption and health. The study employs an analytic method called specification curve analysis, which employs several specific steps.
- Identifying Plausible Model Specifications: The researchers identify a range of plausible model specifications that could be reasonably used to analyze the data.
- Conducting Multiple Analyses: Each of the specified models is then estimated using the same dataset.
- Evaluating Results Across Specifications: The results are examined to see how they vary across the different specifications.
- Interpreting Robustness: The goal of specification curve analysis is to assess the robustness of results. Robust findings are those that are consistent across a wide range of plausible model specifications, suggesting a consistency of findings that is not dependent on specific modeling choices. Conversely, findings that are sensitive to particular specifications may be less reliable and may warrant further investigation or caution in interpretation.
- Transparency and Reporting: Transparency is crucial in specification curve analysis, and the results need to fully disclose the range of model specifications tested and report the results of each analysis.
- This analysis applied specification curve analyses to estimate the effect of unprocessed red meat consumption on all-cause mortality.
- 1,208 unique analyses were performed.
- There was significant variability in the results with hazard ratios (HRs), ranging from 0.51 to 1.75.
- The primary objective of the analysis was not to draw inferences about the health effects of red meat but to provide a proof-of-concept illustration of specification curve analysis applied to nutritional epidemiology.
The Take-Away:
- The findings suggest that the results from observational nutritional epidemiological studies may be highly contingent upon the analytic methods utilized.
- Nutritional epidemiology studies continue to play a critical role in shaping dietary recommendations and policies.
- The specification curve analysis yielded a median hazard ratio of 0.94 (interquartile range: 0.83-1.05), or a six percent reduction in early all-cause mortality associated with consumption of unprocessed red meat.
- Forty-eight specifications (3.97 percent) were statistically significant, 40 of which indicated unprocessed red meat to reduce all-cause mortality, and eight of which indicated red meat to increase mortality.
The Caveat:
In the performance of the current experimental gold standard, a placebo-controlled double-blinded randomized controlled trial (RCT), the protocol, including how the data will be collected and subsequently analyzed, is put into place before any information is ever collected. However, a significant portion, if not most, of the data that steers dietary recommendations is derived from epidemiologic analysis of observational studies. When engaging in the analysis of such data, there are “often hundreds of equally justifiable ways of analyzing the data, each of which may produce results that vary in direction, magnitude, and statistical significance. Empirical evidence shows that results from observational studies may be highly dependent on analytic choices. [The end result is that] some investigators may test many alternative analytic specifications and, intentionally or unintentionally, selectively report results for the specification that yields the most statistically significant or interesting results or results that support their preconceived hypotheses.”[1]
Despite the conventional wisdom that eschews red meat consumption (in this study analysis, that consumption was specifically unprocessed red meat) in favor of an aggressive transition to everything “plant-based,” humankind remains tethered to our ancestral omnivorous physiology. Indeed, a previous column here explored the apparent benefits of such a “balanced” dietary approach in terms of brain health, mental health, and preserved cognitive function.
In the current study, of the 1,208 unique analytic specifications examined, 435 (36 percent) suggested that the effect of consuming red meat on all-cause mortality was either detrimental (HR>1.0) or neutral (HR=1.0). However, 773 (64 percent), suggested that the inclusion of unprocessed red meat in the diet reduced the risk of early all-cause mortality by six percent. Of the 48 statistically significant specifications examined, 40 (more than 83 percent), reaffirmed the reduced risk of early all-cause mortality by consuming unprocessed red meat.
Although this paper is the first to apply specification curve analyses to nutritional epidemiology, the technique has been validated and applied in diverse fields, such as psychology and economics. While the primary purpose of the paper was to discuss, display, and validate the methodology of specification curve analyses so as to sow it into the field of nutritional epidemiology, this particular study certainly bears other fruit. And while this method does not eliminate observer subjectivity – and Prof. Wheeler might argue we never can – it does address analytic flexibility. Analytic flexibility refers to the many choices that researchers have when looking to analyze observational data. The variability of effect that results from this analytic flexibility is called the “vibration of effects.” By processing large amounts of data through the full spectrum of analytic flexibility, we can hope to dampen the vibration of effects and increase the robustness of what the data tells us—in other words, get a straight(er) answer.
At the end of the day, isn’t that what we are all looking for?
The researchers asked a simple, straightforward question: does eating unprocessed red meat kill you? The answer – at least for now –is “no.” It may even help you live a little longer, if not happier. As Prof. Wheeler intuited, the universe is participatory in nature. So, participate, and don’t just rely on conventional wisdom. Because, at least in this case, it is neither conventional nor wise.
[1] (Wang, 2024)
