Tuesday, February 18, 2014

On miraculous healing

I've heard a lot recently about miraculous healings serving as evidence for God and Christianity. Obviously, I have major reservations about these claims, and I could go a lot of ways with what I'm about to say. I'll give some of the points a quick superficial treatment, and then I'll get to a little mathematics to reveal what looks like a paradoxical result that works against these weird occurrences as proof of miracles, Christianity, or God.

Odds and ends
  1. Every religion makes these claims, as do some supernaturalists who do not believe in God, so while they could conceivably be taken as evidence for the supernatural, it's less likely that they could be considered evidence for God and even less likely that they constitute evidence for any particular religion. The Christian has to be able to explain Islamic, Mormon, Hindu, Shamanic, and all sorts of other miraculous healing claims--seeing as those would not be in accordance with the Christian faith--to be able to claim that such claims constitute evidence for Christianity or even for God generally.
  2. If the (epistemic) probability of the supernatural is effectively zero or otherwise utterly negligible, then even if such events can be taken as evidence for the supernatural, God, or some religion, they don't get us very far at all. In fact, they get us effectively nowhere.
  3. As was pointed out to a believer in such miracles on Twitter, the problem with all "evidence" of this kind is that it is anecdotal, which means that we are completely ignorant of which details are left out of the anecdote and thus cannot use them to draw many, if any, conclusions.
  4. As pointed out on this blog by a commenter the other day, the Texas sharpshooter fallacy accounts for much of the observation of such "evidence." In short, we have lots of data of people who heal via whatever means, some of them had religious circumstances around them, and then the believer circles all of those examples together and "paints a bulls-eye" around them (it's a form of cherry picking).
That's all the addressing I'm doing of those points, at least for now. On to math.

Nuts and bolts

Really, this is a mathematical analysis on the fourth point above, the Texas sharpshooter fallacy. I will be using a very simple Bayesian-style analysis to show numerically just how fallacious the Texas sharpshooter fallacy can be.

Consider the following circumstances.
  1. Let's suppose, generously, that if a religiously influenced medical miracle occurs, we have a 100% chance of healing, which is to say that when religious circumstances effect a miracle, it has a 100% cure rate of whatever disease or problem is presented.
  2. Let's also suppose, in many cases generously to the believer again, that there's a baseline 5% natural recovery rate, be that spontaneous remission or any other modicum of healing that operates on purely natural causes. (Even many aggressive cancers have better survival rates than this.)
  3. Now, since miracles are rare occurrences, we will also assume, yet again quite generously to the believer, that religious interventions effect a miracle in only one out of every one thousand (0.1%) of cases in which they are called for. This may feel ungenerous, but if miracle healings were happening more frequently for this, a very easy case could be made that this discussion wouldn't even be happening. Miracles are rare. (Note: at 1/1000 chances, if one opportunity arose per day on most days, the average person should expect to run into one "miracle" just by chance roughly every two years--in the context of a church of hundreds of people each of whom know hundreds more, such events should occur routinely just by chance, but I digress.)
Now, suppose that given these generous assumptions, we witness what appears to be a genuine medical miracle. There was some medical problem; there was a sincere religious intervention on the patient; there was an apparently spontaneous healing; and the doctors were left flummoxed as to how it happened. To all appearances, a miracle occurred, but did it?

We can apply Bayes's theorem to get an estimate of how confident we can be in concluding a miracle occurred in these circumstances. The theorem looks like this (mostly just to show you that I'm not making things up).

                                  P(H|M)P(M)
P(Miracle|Healing) =  -------------------------------------------------------------------
                                    P(H|M)P(M) + P(H|no M)P(no M)

I hope it is clear that H is shorthand for "Healing" and M is for "Miracle," and "no M" means "no miracle occurred."

To further disambiguate, P means "probability," specifically the probability of the event symbolically described inside the parentheses immediately following the P. The vertical bar is read "given" and means that the event represented by the stuff before the bar is conditioned upon the occurrence of what comes after the bar. Specifically, P(H|M) is read "the probability of a healing given a miracle," and it means the probability that healing occurred if we are assuming that a miracle did, in fact, occur.

In words, what we're calculating is the probability--the confidence with which we can assert--that a miracle occurred given that we saw a healing. That's the P(Miracle|Healing) on the left side of the equals sign. The right-hand side of the equals sign evaluates this possibility via Bayes's theorem.

In the light of our assumptions, we know that P(Healing|Miracle), the probability that we have healing given that a miracle occurred, is 100%, or 1.00. The probability that a miracle occurred at all, P(M), is 0.1%, or 0.001. Also, P(Healing|no miracle), the probability that healing occurred without a miracle and just by matters of usual circumstance (to which the religious intervention is irrelevant), is 5%, or 0.05. Finally, by symmetry, the probability that no miracle occurred, P(no M), is 99.9%, or 0.999.

I will not drag us through the calculation but will simply report the result.

P(Miracle|Healing) = 0.01963

In plain language, what this number means is that given the assumption that miracles are always effective at healing, but that they are 1-in-1000 rare events, with spontaneous natural recovery happening at a rate of 5%, if we witness a healing that appears miraculous, there's only a 1.96% chance that we're right to believe it is actually a miracle. In other words, on these assumptions, if a churchgoer believes he has witnessed a miracle, even under these generous assumptions, we can conclude that there's more than a 98% chance that he is simply wrong about it.

To put this number in context, if we wanted to have a 50% confidence in concluding that miracles of this kind do occur, we would have to have at least 43 independent reports come in. For 95% confidence, we'd need 186 independent reports, and for 99.99% confidence, 572 of them. These seem like low bars to clear, but I have never heard of a church clearing them. Indeed, they only seem like low bars to clear because so many people are so susceptible to this 98% likely error (on these very generous assumptions).

Tightening the bolts

We've done some studies on medical miracles, and we've detected no statistically significant difference between religious interventions and a lack thereof. Therefore, it seems too generous to offer a 1/1000 chance that religiously induced medical miracles occur, if we haven't ruled them out completely.

I don't know how likely medical miracles might be, which is to say that I don't know what a fair estimate for them is. I think it's a good bet that one-in-a-thousand is too likely, given that the studies we've done haven't detected them, so let's tighten the bolts in light of this fact. For now, let's assume that medical miracles are one-in-a-million shots.

On the assumption that miracles only occur in one out of every one million cases, with the rest of our assumptions remaining as generous as before, the confidence that we can put in an apparent medical miracle--an unexplained healing that occurred in or following a religious intervention--is a staggeringly low 0.002%, one chance in fifty thousand.

Pause for context here. If miracles are completely effective, but one-in-a-million chance rare events, and we have a 5% natural recovery rate, and we witness an apparent miracle, we'd be wrong to conclude that a miracle occurred 49,999 times out of 50,000, 99.998% of the time.

In this case, we would need 34,657 independent cases, all of which have been admitted by a thorough medical analysis to be unexplainable by any known natural means, just to have a coin-toss's chance, 50%, of being right in saying that medical miracles happen. To be 99% sure, we'd need 149,785 confirmed cases. This kind of thing would stand out.

Tightening the context

Just reflect for a minute on our assumptions now that we've seen these numbers. One of them is that we were paying attention only to medical circumstances with a 5% survival/remission/healing rate by natural means. To be sure, there are many illnesses with such grim circumstances, but heart attacks have a 4.7% death rate, or a 95.3% survival rate (according to the University of Michigan). 

Obviously, the higher the chances of survival, remission, or healing, the higher the likelihood that believers would attribute false positives (for miraculous intervention) to their beliefs. Indeed, if miracles are only one-in-a-million events, surviving a heart attack would only be miraculous in one out of 953,000 cases. In reference to this actual data, we'd need more than 660,000 independent and confirmed (otherwise unexplained) cases for a 50% chance of miraculous intervention and more than 2.5 million for 95% certainty.

In this light, we have no good reasons whatsoever to accept from anyone any anecdotal claims that miraculous religious healings occurred. Even being ridiculously generous to the possibility of a miraculous healing claim results in more than a 98% chance that the claimant is wrong, and by being a bit more honest, almost a 99.9999% chance.

18 comments:

  1. Checking the numbers, and I think you made a mistaken when "tightening the bolts". I think you left P(no M) as .999 rather than updating it to .999999 (since P(no M) = 1 - P(M). That makes things even worse for the miracle claimant.

    Also, I'm not sure how to get the results from multiple miracle claims that you have. Could you be more explicit on the arithmetic, please?

    ReplyDelete
    Replies
    1. Thanks for checking that, but no, I just rounded off 0.000019999. Changing from 0.999999 to 0.999 changes it to 0.000020004 or something like that.

      The calculation for the other is straightforward. Say that the probability of a miracle given a healing is denoted p. Then 1-p is the probability that given a healing we don't see a miracle, and (1-p)^n is the probability that we do not see a miracle in any of n (i.i.d.) trials. So take (1-p)^n=0.5 and solve for n. To calculate for 95%, note that we want (1-p)^n=0.05 because we're really interested in the opposite probability: the chance that we see at least one such miracle among the n trials.

      Delete
  2. >> "The Christian has to be able to explain Islamic, Mormon, Hindu, Shamanic, and all sorts of other miraculous healing claims--seeing as those would not be in accordance with the Christian faith--to be able to claim that such claims constitute evidence for Christianity or even for God generally."

    I'm baffled by your phrase "seeing as those would not be in accordance with the Christian faith".

    These things are in accordance with the Christian faith. What I mean by that is they all can be accounted for under what Christianity teaches about angels, demons, sin, God, etc interacting in the world. It fits.

    These things aren't evidence for Christianity, but they certainly aren't evidence against it either.

    ReplyDelete
    Replies
    1. You may fail to recognize that those other faiths directly contradict Christianity. If they are true, then Christianity is false.

      Delete
    2. I'm talking about the healing itself and how it fits within the Christian faith. I can accept that healings by Muslims may have actually occurred.

      Delete
  3. >> "In other words, on these assumptions, if a churchgoer believes he has witnessed a miracle, even under these generous assumptions, we can conclude that there's more than a 98% chance that he is simply wrong about it."

    Suppose this churchgoer knows that natural biological cause/effect relationships have never been shown to produce the specific biological effect being reported - or perhaps it occurs as often as miracles do. How would this knowledge change your numbers, if at all?

    ReplyDelete
    Replies
    1. The reason I asked this question, James, is because your situation seems to be missing something very important. There is a good reason it appears to be a miracle, you said so yourself in the setup.

      >> "Now, suppose that given these generous assumptions, we witness what appears to be a genuine medical miracle. "

      If there is a reason then that information should be factored into the equation. The probability for a miracle, whatever it is in this particular situation, wouldn't be the same as without this information.

      If your point was to calculate a general probability without adding any of this information into the mix, then I don't see the significance of your point because your numbers wouldn't apply to any situation where there was a reason for thinking a miracle occurred.

      Am I wrong?

      Delete
    2. Yes, you are wrong. The reason it appears to be genuine is because people make this mistake easily because it's an emotionally charged event that is looked for and yet has a very low true positive rate (which people are desperately bad at understanding because they have poor understanding of three probabilities: P(A given B), P(B given A), and P(A and B). Those get conflated in their confusion and lead to a lot of important social consequences, one of the probably less important of which is the belief in miraculous healing. (Most religious people still go to the doctor in emergencies, so this is mostly an annoying novelty that supports other religious evils.)

      If you wonder what more serious issues are fueled by this failure to clearly understand basic probability theory, racism is one of the biggest ones. There are also issues regarding drug testing (but that's tied up in racism as well) and the identification of disease states.

      A great deal of racism is perpetuated because P(in prison given black) is far lower than P(black given in prison), and also because P(poor given black) is far lower than P(black given poor). These extend to other racial groups as well, and getting them backwards seems to be the primary modus operandi of racist conservatives throughout the ages.

      Delete
    3. James,
      You said "we witness what appears to be a genuine medical miracle". Why would it *appear* to be a genuine miracle to anyone - you, me, a doctor? Your response - as far as I can tell - says that you think this information is irrelevant to the probability question being posed.

      Delete
    4. This hypothetical example would appear to be a miracle: a doctor reports that a patient's severed finger grew back withing 1 week. The reason it appears to be a miracle is that fingers don't naturally grow back. I'd expect this information (fingers don't naturally grow back) would factor into your calculations somehow but I don't see where you've allowed for this variable.

      Delete
    5. We never see cases of severed limbs growing back so let's go ahead and ignore examples that aren't valid.

      It would appear like a medical miracle for two main reasons.
      1. People look for them, largely in hopeful desperation and because of their unfounded beliefs in magic.
      2. We don't know everything about how the body works, and so we're often surprised. For example, we do not understand cancer at a level to where we can explain the cause for many cases of spontaneous remission. The same is true for autoimmune disorders. There are just so many confounding variables that we often end up with doctors who aren't sure what happened to effect a cure.

      Delete
    6. James,

      (1) This is not a universal truth for all people. I'm interested, as I'm sure you are, in those reports from people who never thought to look for them.

      (2) I'm not asking you to calculate based on on what we don't know. I'm asking you to calculate based on what we do know.

      If we know that some biological effect occurs about as often as a miracle then this would change your calculations, right? I'm saying you didn't account for this in your analysis. You gave every situation the same probability, and that isn't accurate.

      >> "There are just so many confounding variables that we often end up with doctors who aren't sure what happened to effect a cure"

      This happens, sure. You seemed confident enough in your knowledge of medicine to assign probability numbers so you must have some insight.

      Delete
    7. Steve, please read more carefully in the future.
      I said miracles are 1-in-1000 events. Spontaneous remission is 1-in-20 (5%). Those were the hypothetical numbers offered.

      I also made a case that for many illnesses, spontaneous remission is actually more common than 1-in-20. Further, we don't even have good reasons to believe that miracles occur with a frequency of 1-in-1000. Indeed, I don't think they occur at all, so even 1-in-1,000,000 drastically overestimates the case. (That is, for this calculation, I was ridiculously generous to the Christian position already. That it doesn't work out for you isn't my problem. It's yours.)

      Delete
    8. Yes, I do have a problem and I realize it truly may be *my* problem and mine alone. My problem is about the application of this information. I don't understand how your hypothetical numbers would apply to a specific documented case so I'm unable to use your numbers in any meaningful way other than to conclude that, yes, miracles are unlikely events. But I knew that already.

      Let me take miracles of the table and look at something uncontroversial.

      I know that royal flushes are very unlikely hands compared to other more common hands. Does this statistic (frequency of occurrence) help me in any way know if the girl at my table actually had a royal flush dealt to her last night, even with the added benefit of knowing that 1000's of other people have also been dealt this hand? Not really. Other information can help me decide, but the statistics associated with royal flushes vs. other hands doesn't seem to help me very much here.

      I'll think about it some more though. Thanks, James.

      Delete
    9. What would constitute a false-positive royal flush? Specify that, and I will think about it, giving you the numbers in a similar analysis. Without that kind of information, though, all you're really doing is showing how you are confused between something you cannot distinguish (miraculous healing and unexpected spontaneous recovery) and something you can (a royal flush from another combination of five standard playing cards).

      Delete
    10. I am guessing misreading the hand and cheating would constitute incorrectly identification, I don' t know the frequency of such things, but I'd think it would have to be very infrequent.

      Delete
    11. I'd also like to say that everyone would make their decision without doing any calculations whatsoever so I'm not sure what value the numbers actually have.

      Delete
  4. The miracle question always cuts both ways and like belief it finally rests in the mind of the beholder.

    Many things happen in this world that we cannot find or know efficient cause for. There are hosts of documented accounts around of "miracles" around for which trained medical professionals and other reliable, mostly unbiased witnesses can attest there is no know efficient cause to effect the unexpected new condition. By virtue of a context (a group praying for the person, a religious healing service, etc.) some will claim that God is the efficient cause. Others will simply say "I don't know". By virtue of the nature of event (no currently common or reproducible efficient cause apparent) no scientific conclusion can be established because science is all about reproducibility. By definition (convenient though it may be) a miracle is not reproducible.

    Miracles of course have a lot of impact on observers because like miracles themselves, people and their beliefs are not very reproducible. The factors are too many. We can attempt predictions of people in groups but individuals can be highly difficult to predict. Miracles can "prove" nothing by definition while they can remain highly persuasive. Ironically, Jesus himself didn't value the type of faith that his own miracles provoked.

    ReplyDelete