Vlog 17: Chiropractic Debates on YouTube
A Canadian study analyzed the comments section of 4 popular YouTube videos on chiropractic to see which arguments were used by the "pro" and "against" commenters and to assess the degree of hostility. What do you think were the findings?
The original paper can be accessed here: https://www.cogentoa.com/article/10.1080/2331205X.2016.1277450
Hey, this is Jonathan from The Body of Evidence.
As you probably know, I am fascinated by how the defenders of alternative medicine argue in support of their beliefs. As a science communicator, it's not sufficient for me to provide facts; I also need to understand how best to convince someone to look at these facts because, if they hold a belief that is threatened by them, then they will probably find ways to rationalize these facts away.
My knowledge of how these believers argue-and even how skeptics argue-is mostly anecdotal: it comes from reading, interacting with people, debating, getting into Twitter skirmishes. But now, I'm happy to report on a study that is a bit more objective than my perception.
The study is called "Commenting on chiropractic: A YouTube analysis". It was published in Cogent Medicine and comes from the Health Law Institute at the University of Alberta. The first author is Alessandro Marcon, a research associate, and he is joined by last author Timothy Caulfield, whom many of who will know as the author of The Cure for Everything! and Is Gwyneth Paltrow Wrong About Everything?, and the man who tweets on health matters every waking minute.
They were interested in mapping out the types of discussions that were happening in the comments section of popular YouTube videos on chiropractic, specifically to look at the types of arguments made in those comments by people for and against chiropractic, and to determine the levels of hostility in these discussions.
Had I been asked for predictions before reading the article, I probably would have said that these comments would display a lot of hostility, and that people against chiropractic would point to the lack of scientific evidence behind it, while people for chiropractic would share anecdotal evidence.
What do you think? Do you think constructive health-related discussions take place in these YouTube comments?
Before we get to the findings, a short bit on what the authors did. They searched YouTube using keywords "chiropractic", "spinal manipulation", and "back cracking". They filtered out videos with fewer than a million views and fewer than one thousand comments. They ended up with four popular videos, representing a total of 24 million views and 14 000 comments. The authors did a qualitative analysis of these comments, looking at what emerged from them, in order to create codes. They verified throughout that a given comment was coded the same way independently by two researchers to improve the objectivity of their analysis.
All right, findings. One of the main types of discussion that was seen in these comments was debate over the efficacy and legitimacy of chiropractic. When they dug deeper, they found 11 key elements of these debates. Here they are from most frequent to most rare:
Real questions (meaning the solicitation of information from other commenters, like "I read the report in that link three times and I still don't know what they are talking about. Can you please post a few more links?"); use of technical terms (like a long description of a medical case that would not be intelligible by a lay person); comparing chiropractors to medical doctors; debates over the concept of a chiropractic subluxation, which is the vertebral anomaly chiropractors claim they can see and fix; comments about the legitimacy of the video itself; questioning the expertise of the video maker or one of the commenters; comparing chiropractors to homeopaths and physical therapists; comparing them to osteopaths; to acupuncturists; and finally to dentists.
My favourite part of the study was the breakdown of the arguments made by those critiquing and those defending chiropractic in the comments section.
Let's start with the arguments and tactics made by people against chiropractic. If you are a skeptic, is this representative of your community?
In 12 of the 14 discussions on the legitimacy of chiropractic, commenters against chiropractic mentioned that this practice is not supported by science, evidence, or evidence-based science. This was the #1 argument made by people who were against chiropractic in those comments. In 10 of the 14 discussions analyzed, links were provided by these people to studies and articles that supported their arguments. In 9 of the 14 discussions, the word "BS" or "bullshit" was used to qualify chiropractic.
Other negative terms were "snake oil salesman", "witchcraft", "con/scam", "quack", and "hacks". The placebo response was also mentioned numerous times. The risks of chiropractic were also put forward by these commenters. They also mentioned that chiropractic doesn't cure anything, that you are stuck in an endless cycle of maintenance treatments, that chiros make false medical claims, and that there are good chiropractors out there and that chiropractic can be effective in some situations, but that numerous medical benefits should not be attributed to the treatment.
That was for the YouTube commenters who were against chiropractic. Now, let's look at the arguments and tactics used by the people who are for chiropractic. What are your guesses? What do you think were the most common ones?
At #1, being used in 11 out of the 14 discussions, it's personal anecdotes! Of course, though it's nice to see hard data on this. At #2, it's mentions of harmful pills and the pharmaceutical industry. At #3, ex aequo, it's the argument that some chiropractors are bad, but some are good, and that chiropractic is in fact science based.
An interesting finding is that links were used to support claims made by the defenders of chiropractic, but less frequently than the people against it.
Before I talk about hostility, I want to quickly tackle the main arguments that were made for chiropractic in those comments.
Chiropractic is natural. So is scorpion venom. The naturalistic fallacy is really not a valid argument.
Chiropractic gets to the root of the problem, whereas medicine treats the symptom. It's actually the other way around.
Chiropractic is safe. No, it isn't. There is a fair amount of debate happening over the harm and deaths possibly caused by rapid neck thrusts that can cause an important artery to tear.
Personal anecdotes. The lower form of evidence because too many variables are not being controlled for. I made a whole video about this.
Chiropractic is science based. Not really. It was made up of whole cloth and, although studies have been made of it, it is hard to qualify a field as science-based when its main pathology, the chiropractic subluxation, does not exist. Ask any radiologist.
Chiropractic has a long history. So does blood letting. Just because something was used for a long time, doesn't mean it works well.
There is a conspiracy by the medical community to smear chiropractic. Valid criticism is not the same as a conspiracy.
Medicine is under the control of pharmaceutical companies. Now, there is a good point to be made here, one that Ben Goldacre makes really well in Bad Pharma, but if planes were unreliable, the answer wouldn't be flying carpets.
Some chiropractors are good, some bad. If what they are practicing is not based on good evidence, you are wasting your time.
Keep an open mind. But not so open your brains fall out.
The last finding I wanted to mention was the authors' assessment of hostility in these YouTube discussions. They defined a hostile message as one inciting "anger or exasperation through the use of name-calling, character assassination, offensive language, profanity, and/or insulting language".
The surprise for me was that hostility, according to the authors, was low, present in less than 6% of total comments. Over two thirds of these came from the people for chiropractic, whereas less than one third came from the other camp. In 3% of total comments, the word "troll" was used in an accusatory manner, all iterations of which but one coming from... people for chiropractic.
So, not nearly as hostile as we might have thought; genuine debate over the legitimacy of the practice; an over-reliance on anecdotes on the part of pro-chiropractic commenters as well as distrust of the medical establishment; a primarily pro-chiropractic minority using hostility and name-calling, possibly representing an emotional response to the negative labelling of their belief system; and a surprising (to me) open-mindedness on the part of the against-chiropractic commenters, since 57% of the discussions by people against chiropractic contained a mention that chiropractic might be good for some things but not all, or that some chiropractors are better than others.
The article is open access and can be read by anyone. Link down below. What do you think? Are you surprised by the findings? Did your learn something? And, in light of these findings and everything else that we know about changing people's minds, do you think these comments are constructive? Do you think the links posted by skeptics make a difference? Do you think the personal anecdotes by the pro camp convince people? How do you think we should move the conversation forward? Let me know in the comments below and, as always, keep it civil. You don't want to become a statistic in a future study on YouTube hostility.