Two recently published meta-analyses, one on meat and another on milk, conclude that levels of polyunsaturated fatty acids (PUFA) and omega-3 fatty acids are higher in the organic versions of both. This is being widely reported as evidence that organic meat and milk is healthier, but a closer look at these two studies shows a different picture.
The organic false dichotomy
I am going to start with what I consider to be the biggest problem with how both studies are being presented and reported. This relates to a deeper issue – the false dichotomy of organic vs conventional. In my opinion, we should use whatever farming techniques prove to be best (for the quality of the food, sustainability, cost-effectiveness, and environmental impact) regardless of whether or not they are considered organic.
While proponents of organic farming claim that for them food quality and sustainability are the real issue, they aren’t. Organic techniques are based on the appeal to nature, not an evidence-based approach to best practices.
The conclusions of the meta-analysis addressing milk are:
Redundancy analysis of data from a large cross-European milk quality survey indicates that the higher grazing/conserved forage intakes in organic systems were the main reason for milk composition differences.
And for meat:
Evidence from controlled experimental studies indicates that the high grazing/forage-based diets prescribed under organic farming standards may be the main reason for differences in FA profiles.
In other words, the differences found can be entirely due to the difference between grass feeding and grain feeding – that’s it. That is the real variable here, not organic vs conventional. Grass fed conventional meat and milk has the same profile as grass-fed organic. It is therefore deceptive, in my opinion, to frame this difference as organic vs conventional when it is clearly grass-fed vs grain-fed.
The differences are really insignificant
For milk, the study found that PUFA was 7% higher in organic (meaning grass-fed) while omega-3 PUFA were 56% higher. For meat these numbers were 23% and 47% respectively.
These sound like impressive numbers, and most reporting simplifies this to around 50% for both. Regular readers of SBM by now should recognize that relative percentages can seem impressive but be misleading.
Neither milk nor red meat are significant sources of omega-3 fatty acids, and therefore even a 50% relative increase in this small amount likely has few health implications.
For example, if you drank a half liter of whole milk every day, consuming grain-fed milk would provide 11% of the recommended daily intake of omega-3, while drinking grass-fed would provide 16%. If you drink 2% or 1% milk the effect is even smaller. Simply put, you are not going to reach your targets of omega-3 by consuming milk.
The numbers are even worse for meat. Red meat is high in saturated fat. The net effect of eating grass-fed beef on your fatty acid profile is going to be negative. You are better off substituting fish or poultry for red meat. Another way to look at this is – because of the low amounts of omega-3 in red meat, in order for the grass-fed vs grain-fed difference to be significant in your diet, you would have to be eating way too much red meat.
In summary, while the relative percentages of omega-3 in grass-fed vs grain-fed meat and milk sound impressive, in absolute terms the differences are insignificant.
Problems with the meta-analyses
To further dilute the implications of these two studies, we need to consider the challenges of doing a proper meta-analysis. Meta-analyses in general are no substitute for single large rigorous trials. They are a method for combining the data from many studies to gain more statistical power.
That is the advantage of a meta-analysis, the statistical power. However, there are also some drawbacks. They suffer from the GIGO problem – garbage in, garbage out. If you take a lot of small and less rigorous studies and combine them, you don’t get one large rigorous trial. You get a mess – but a mess that can seem statistically impressive.
A meta-analysis is also an opportunity to introduce a host of new biases. You have to decide which studies to include, how to weight them, and how to combine different outcomes and different techniques. There are many “researcher degrees of freedom” which potentially lead to biased outcomes.
In the milk meta-analysis the authors did not weight the studies, which means they gave equal weight to small uncontrolled studies as they did to large controlled experiments.
The meat meta-analysis was more problematic. They could not find enough studies of any one type of meat, so they combined them into one analysis. Further, 21 of the 63 studies they included were not even controlled. They report a high degree of heterogeneity among the studies they looked at, which means the results were all over the place and any conclusions are highly suspect.
Conclusion: Organic is still not any better
The three main points that are important to understanding the relevance of these two meta-analyses, and which have not been consistently reported, include:
- The differences observed are not between organic and conventional farming, but between grass-fed and grain-fed animals. It is misleading to focus on an arbitrary category (organic) rather than the actual variable having the observed effect.
- The absolute differences are insignificant in practical terms. For the amount of red meat and milk that most people consume (or should consume) the differences are small and the chance of any net health effect is slight.
- These two studies were meta-analyses, which have generic limitations, but these particular meta-analyses have serious problems. Their results are therefore not very reliable.