Success is your...wait a second, how do you know it's a success?

My somewhat lengthy discussion with occult author Donald Michael Kraig over on his blog (read it in the comments section of this post) seems to have come to an end. Looking back over it, I see a number of curious strategies on Kraig’s part that deserve some analysis and examination here.

Much, much more after the cut.

First, a (very) brief summary: Kraig’s post concerned speculations about how magick “works,” and one of the points he makes in the post is that it doesn’t really matter *how* it works since the fact is – so he claims – magick *does* work. He likens it to a car: one doesn’t need to know exactly how a car works in order to use it, and so too, he argues, it’s not necessary to know *how* magick works since the only thing that matter is *that* it works.

In my first comment, I correctly point out that before one can speculate about *how* magick works, one needs to establish *that* magick works in the first place. Importantly, this isn’t a question of “you’ve got to prove it to the skeptics!” As I consistently noted, it’s an issue of the practitioner proving it *to himself* for the benefit of *himself.*

If someone’s going to go around believing that he’s doing rituals that cause results, then he needs good reason to think that the rituals really do cause those results (assuming that he doesn’t want to fool himself, that is). He needs to be able to demonstrate it to himself, and he therefore needs some method of distinguishing these rituals from not having done any ritual at all. He needs, in short, a means of distinguishing a ritual that works from a ritual that seems to work (but does not).

Let me pause here and give an example of what I mean, using the example that came up in our discussion. Say a magician desperately needs $1,000 by the end of the month, so he does a ritual for this money and – lo and behold – sometime before the end of the month, in some form or another, he manages to come up with the money. “Huzzah!” celebrates the magician. “My ritual worked!”

But did it? In any given month, there are probably lots and lots of people who desperately need to come up with $1,000. Of the people who need it, a certain number *will* somehow come up with the money they need, without doing any magick at all. Indeed, some of those people who manage to come up with the money – without any magick at all – will do so by coincidental means that may subjectively seem “miraculous.” How can a magician tell that his ritual *caused* the outcome? That is, how can he tell that he was not part of the group that was going to manage to come up with the money anyway? He clearly needs a method of distinguishing a ritual that works from a ritual that seems to work but does not.

Much of the conversation between DMK and myself centered around this question, or rather centered around me asking him this question and getting no response.

Having engaged with Kraig in a little back-and-forth in the past, I knew that he frequently employs a tactic known in some debate circles as a “spread”: he makes many points in a single response, so many – and so many that sound kind of plausible and thus require a bit of explaining to establish why they are spurious – that his opponent can easily get lost in a labyrinth of unrelated points and lose track of the main thread of the argument. Aware that Kraig would likely try this tactic again, I went into our conversation with a strategy of my own: to keep the focus on one specific point and to continue to press it until my question was answered or until it became clear that Kraig has no answer.

Thus, I continued to ask Kraig how he demonstrated – to himself, now, not necessarily to anyone else – that magick works, how he distinguishes between a ritual that causes results and a ritual that seems to cause results but does not.

In response to this question, Kraig took a very unusual approach. At first, he maintained that he did not wish to tell me his method, implying that he *does* indeed have such a method, just not one he feels like sharing.

I said to him:

I don’t think I’m being terribly unreasonable in asking how you think a person can distinguish (“to them”) between rituals that “work” and rituals that just *seem* to work. (Post #8)

Kraig responded:

no, I don’t think you’re “being terribly unreasonable.” (Post #9)

It doesn’t matter what my method to distinguish between causation and coincidence is. (Post #11)

To agree that it’s not unreasonable to be asking this question – and to say that “it doesn’t matter what [his] method is” – strongly implies that Kraig sees the value in distinguishing between the two, that Kraig has a method of distinguishing between the two, and that Kraig does not wish to share his method.

Yet later in the discussion – in a response posted not even a week after the above comments – Kraig denies that distinguishing between the two is important at all:

that’s an interesting claim, trying to “distinguish rituals that work from rituals that merely seem to work.” The outcome, in either case, is that you got the desired results. So what difference does it make? Suppose a doctor gives you an injection to cure you of a fatal disease. It turns out that the drug doesn’t work for you but your mind thought it would work and ended up curing the disease. Are you going to say, “What a minute, the drug wasn’t effective so even though I’m cured I’m still going to die?” I don’t think so. (Post #28)

This passage is, quite frankly, baffling. Kraig’s example features medicine that doesn’t work “for you.” In this way, by analogy, he implies that magick does work for him but not for me. This implication assumes that the claim “magick works” is true (thus begging the question), and it further suggests that he thinks my objection stems from a personal inability to get magick to work (which not only demonstrates that he doesn’t understand the actual source of my objection, but demonstrates that he’s committed to using the old believer script "you're not doing it right!")

A better comparison – one that matches the situation we’re talking about and that doesn’t beg the question – would be to a situation in which a doctor is selling a patient medicine that has never been demonstrated to have an effect that could be distinguished from not giving the patient any medicine.

I think that I – and I daresay just about anyone – absolutely would care if I am paying a doctor to give me medicine that no one has ever demonstrated does anything. And indeed, if the doctor were to start dismissing such concerns, saying suspicious things like, “Oh, it doesn’t matter whether the medicine is really doing anything, as long as you get better!” or “Hey, there’s no real way to ‘test’ whether it actually does anything” or “Gee, a couple of patients took it and felt better afterwards, isn’t that proof? I mean, their subjective experience is just as valid a paradigm as actually conducting a rigorous study of the effects” – I think that many consumers would become extremely cautious of trusting such a doctor.

Not to put too fine a point on the comparison, but it’s worth at least noting that Kraig profits from the claims that he makes in public – selling books and workshops and the like. As such, just as it is well within the rights of any member of the public to ask how the doctor knows that the medicine he is selling to patients has been demonstrated to actually do something, it is well within the rights of a member of the public to ask how an author knows that the claims that he is selling to others have been demonstrated to be true.

Just to be extra clear – since I’m bound to be misread by people who insist on looking at everything through the lens of their imaginative and emotional responses to reality – I’m not accusing Kraig of deliberate fraud or anything like that: he seems sincere in his beliefs, but he also seems – like many intelligent people who believe wacky things – to have compartmentalized his thinking, accepting a line of argument for the claims he *wants* to believe that he would not accept for other kinds of claim.

Getting back to the strategy he uses, Kraig had at first seemed to go along with what I’m saying – seemingly agreeing that it’s important to distinguish between causality and the regular old happenstance of life –but then suddenly reversed this implied position, and declared it unimportant to be able to distinguish between them.

This is not the only example in our discussion of Kraig appearing at first to agree with me, only to later seemingly switch positions entirely. Later in the discussion, we start to talk about statistics, a subject Kraig at least claims to “understand.”

He had just finished asking me if the following scenario would constitute evidence:

Let’s say there are ten people trained in magick and in real need of getting at least $1,000 by a specific date. If these people create and perform rituals and then achieve their goals—a more accurate version of what is likely to happen—where is the confirmation bias? (Post #16)

But in fact, as anyone who thinks about it for more than a few seconds can agree, having a small number of people who happen to be able to come up with the money tells us nothing about whether the ritual *caused* them to come up with the money.

Considered in isolation, apart from the wider context, those ten anecdotes tell us nothing: we would need to know, for example, how common it is for people who need the money to end up coming up with it one way or another. Then we would have to select a representative sample of magicians and see if they, after performing this ritual, came up with the money more often than the general population, to a statistically significant degree. And we would have to repeat the test, to demonstrate that it happens consistently.

Obviously, if the ritual *does* cause people to come up with the money (or at least be more likely to come up with the money), this would be demonstrated by such an investigation, but it would have to be demonstrated by measuring the magicians against a wider context. A couple of anecdotes, on their own, would demonstrate nothing.

When informed of these simple facts, Kraig ineptly attempts to accuse me of “moving the goalposts”:

Let’s say we double the sample size and include twenty well trained and experienced magicians and they’re all successful. Will a person supporting your model agree that magick works then? Of course not. Such a person will simply ask for a larger and larger sample until the result fit into a preconceived model of reality (Post #23)

It’s almost embarrassing how badly Kraig misses the point: he seems to think that I’m after larger numbers of anecdotes, but – as I proceed to point out to him – what I’m actually after is results that we can put in a context. As I write:

I can’t believe you fail to comprehend this simple point: it’s not about a number – it’s about a number in a context. You need to demonstrate that a representative sampling of magicians deviates from the general population to a statistically significant degree.

First we would need to study the group of people who need $1,000 every month and determine the percentage of them, on average, that manages to come up with the money. This tells us what we would expect to happen, based on random chance.

Then, we’d have to find a group of magicians who are representative of the wider group (and we’d have to talk to statisticians to be able to accurately calculate how many we need and what their economic backgrounds would have to be in order to make them representative, etc.).

Then we’d have to run lots of trials – not ten, which is absurdly low, but a number determined by statisticians that would yield results that would give us a sense of whether this group differs from the larger population by a statistically significant

margin. And then we’d have to repeat this experiment quite a few times.

If these rituals really do *cause* people to come up with the money (or at least be more likely to come up with the money), then measuring the results would reveal that the group that performs the ritual consistently comes up with the money vastly more often than we would expect based on chance – to a statistically significant degree that we could measure.

Naturally, if one were really going to do this, one would have to sit down with neutral people who study probability and statistics in order to generate the numbers and to control as much as is possible for variables.

Just looking at ten cases is absurdly myopic and tells us nothing *at all* about causality. (Post #25)

Kraig’s subsequent response to my point is beyond ridiculous: though I have been at pains to point out that I’m arguing that his claim is testable, his objection rests entirely on the fact that I haven’t told him exactly what numbers we would need and the *precise* number it would take to confirm the claim:

Further, you have not supplied even one number in your proposal. As I’ve written several times before, a proposal like this, without any specifics, is a typical approach of a pseudo-scientist because he or she can always change the numbers and say the sample isn’t big enough, doesn’t have enough controls, isn’t focused enough, or in some other way move the goalposts to prevent a challenge to the pseudo-scientist’s belief system.

As a result of my experience I’m satisfied that magick works. (Post #26)

A response like this is so flabbergasting because it totally misses the point: that I’m not personally a statistician capable of calculating the precise numbers is irrelevant to the point that Kraig’s claim is, in fact, testable. It obviously is a testable claim, and it remains so whether or not anyone ever does calculate the numbers needed or organizes and runs the tests. That the claim is testable – and that no one has yet tested it and thus has no basis for accepting the claim – is my entire point.

Kraig labors under the delusion that what I’m pointing out is some sort of device for maintaining a staunch rejection of the supernatural, but the opposite is true: believing claims that are testable but have not been tested and confirmed – as Kraig clearly admits that he does – is a device for maintaining a dogmatic belief in the supernatural, supported by nothing more than how it subjectively feels.

But the truly astounding part of this section of the conversation is that after spending many posts going back and forth with me on what would constitute sufficient evidence – whether ten anecdotes would be sufficient or twenty anecdotes or a statistical analysis – Kraig switches gears and denies that an evidence-based “paradigm” even applies in this case:

You say that “Looking at ten cases is absurdly myopic.” The entire science of modern psychoanalysis, as created by Freud, began with fewer cases than that. Often he would base things on just one example. [which, by the way, partially explains why many of Freud’s conclusions are rejected by modern students of psychology –Los]

[…]

What about a pure science: mathematics? None of your examples are necessary to prove that the square of the hypotenuse of a right triangle is equal to the sum of the squares of the other sides. So the challenge you’re presenting isn’t valid for many sciences. And yet both you and pseudo-scientists seem to demand that it must be done your way and only your way. (Post #26)

But your paradigm is NOT the only paradigm of reality. The Tarot explains reality. So does astrology. So does Newtonian physics. So does quantum physics. And yet, they are entirely different paradigms. They are all correct within their own structure.

Yes, the effects of magick are measurable. But the statistical measurements you’re asking about have nothing to do with the results or effects.  (Post #28)

It’s hard to know what to say to such unadulterated nonsense. In the first place, Kraig points out that there are some areas of study that don’t make testable claims. But the notion that there are some areas of study that don’t make testable claims is, of course, irrelevant to the point that Kraig is, in fact, making testable claims.

If he were just proposing some psychological theory consistent with discussions he’s had with patients, that would be one thing. If he were performing mathematical proofs that follow logically from axiomatic propositions, that would be one thing. But he’s not doing either of those things: he’s making a claim that doing an action can affect the physical world. We’re in the realm of testable claims here, for such a claim – if it’s true – would be measurable.

The kind of statistical analysis I’m proposing does have its limits: it only applies to claims about being able to affect the physical world in some way. But since that’s the kind of claim Kraig is talking about, it obviously applies here.

The question is what kind of “paradigm” we should use to evaluate a given claim. Kraig’s argument here, taken to its logical end, suggests that any old “paradigm” will do for any old claim, which is clearly asinine.

To use another example, take  the issue of “cancer clustsers” – a higher-than usual clustering of instances of cancer in a given area. Though many people interpret such clusters as evidence that there must be some environmental factor at work in those areas, the truth is that such interpretations are frequently examples of a kind of fallacy that resembles the fallacy Kraig is employing: the Texas Sharpshooter fallacy.

[Read an excellent article about this fallacy here]

The fallacy draws its name from the image of a Texan cowboy blasting holes in a barnyard door. The holes he shoots are completely random, but – if we later draw a bullseye around some of the holes that are close together – it will seem as if our cowboy is a real sharpshooter.

That’s what happens when someone just focuses on a handful of cases without considering the wider context: that ten holes are really close together doesn’t tell us anything about the shooting skills of the cowboy.  If we looked at the whole barn door, we might learn that he was just randomly blasting away and that he wasn’t aiming for a particular target at all.

Similarly, that a lot of cancer cases are close together tells us nothing about whether an environmental factor is the cause. From the article I linked to above:

If you look at a map of the United States with dots assigned to where cancer rates are highest, you will notice areas of clumping. It looks like you have a pretty good indication of where the groundwater must be poisoned, or high-voltage power lines are bombarding people with damaging energy fields, or where cell phone towers are frying people’s organs, or where nuclear bombs must have been tested.

A map like that is a lot like the side of the sharpshooter’s barn, and presuming there must be a cause for cancer clusters is the same as drawing bullseyes around them.

More often than not, cancer clusters have no scary environmental cause.

“A community that is afflicted with an unusual number of cancers quite naturally looks for a cause in the environment – in the ground, the water, the air. And the correlations are sometimes found: the cluster may arise after, say, contamination of the water supply by a possible carcinogen. The problem is that when scientists have tried to confirm such causes, they haven’t been able to. Raymond Richard Neutra, California’s chief environmental health investigator and an expert on cancer clusters, points out that among hundreds of exhaustive, published investigations of residential clusters in the United States, not one has convincingly identified an underlying environmental cause. Abroad, in only a handful of cases has a neighborhood cancer cluster been shown to arise from an environmental cause. And only one of these cases ended with the discovery of an unrecognized carcinogen.”

The Cancer Cluster Myth, The New Yorker, Feb. 1999

There are many agents at work. People who are related tend to live near each other. Old people tend to retire in the same areas. Eating, smoking and exercise habits tend to be similar region to region. And, after all, one in three people will develop cancer in their lifetime.

To accept something like residential cancer clusters are often just coincidence is deeply unsatisfying. The powerlessness, the feeling you are defenseless to the whims of chance, can be assuaged by singling out an antagonist. Sometimes you need a bad guy, and The Texas Sharpshooter Fallacy is one way you can create one.

And, just to complete the point, that there are a lot of anecdotes of magicians happening to achieve their goals after doing rituals – or that there are a lot of anecdotes of other kinds of religionists achieving their goals after saying various types of prayers – tells us nothing about whether these rituals/prayers *caused* the goals to come to pass. We would need to see the entire barn door to make that estimation.

So how would we evaluate it? Let’s say that we’re environmental investigators trying to discover whether a cancer cluster is, in fact, caused by something in the environment. What “paradigm” are we going to use to investigate this claim and get a picture of the cluster in relation to the wider context?

Will we:

Perform a statistical analysis, placing this area in a wider context?

Perform a tarot reading?

Find ten people who have lived in the area their whole lives and interview them to see if they have ever developed cancer?

Spin around until we get dizzy – to bypass our pesky “reason” – and consult our magical intuitions?

Ask the invisible race of  gremlins that knows all and speaks to people in their dreams?

Which of these “paradigms” do you think is best suited for evaluating a claim about something *causing* a result in the world? Are they all equally valid means of evaluating this claim?

Obviously, the answer is no: in fact, only one of them is capable of determining causality here. And equally obviously, the fact that there might be some disciplines that don’t make these kinds of causal claims in no way indicates that we can just chuck aside the best method of evaluating these kinds of claims.

But here again, notice that Kraig plays along with the notion that evidence-based inquiry is the way to go about evaluating claims and eventually – after being painted into a corner – declares that evidence-based inquiry is just one paradigm among many and that it shouldn’t be used to evaluate these claims after all.

As I noted to him:

I consider it a great victory for me that I’ve backed you into a corner that you can only get out of by appealing to laughable, paper-thin New Age platitudes (Post #31).

One last point here: Kraig’s favorite word – by a long shot – appears to be “respectfully.” Now, normally, I don’t mention little verbal quirks that people have because they're usually not relevant, but here, Kraig uses this word and its variations a lot, both in his e-mails and on his blog – no less than ten times in our conversation – and it’s relevant to a point I’m about to make.

[Heck, even other posters on his blog – like “G-Man,” who posted comment #5 in our discussion – use the word in the same way that Kraig does. I guess he learned it from Kraig and feels like he’s being a good student by dutifully repeating it?]

In the first place, it seems very odd to me to have to keep saying “respectfully.” Rhetorically, there’s no need to constantly affirm one’s tone: the tone of a piece should be obvious from the word choice (as the old adage runs, show…don’t tell). In other words, one shouldn’t have to keep saying, “I’m being respectful!” It should be clear that one’s comments are respectful.

But in the second place, “respectful” is hardly how I would characterize Kraig’s attitude throughout the discussion (and particularly toward the end): unable to answer any of my questions and forced into a corner in which he had to pull out one of the lamest cards in the murky-new-age-relativist toolbag, Kraig turned to the place that many occultists turn when I back them into a corner – baseless assumptions about me and distractionary speculation about my experience and motivation.

Here’s a taste, for those of you too bored with Kraig’s antics to make it to the end:

You can present whatever you want. But that will never resolve the psychological dichotomy within you. In Tantric tradition there are blockages to advancement known as “kleshas.” One of those is the false ego. That’s where what you think of yourself and the world has distinct differences with what is actually real. That’s where you are now, and until you work on that no amount of proof of anything you don’t have an ersatz-religious belief in will ever satisfy you. (Post #33)

You totally miss the point that this entire discussion is about a psychological dichotomy within you.

[…]

What I’m saying is that your interpretation of yourself, who you are, doesn’t correspond to the reality of who you actually are. This is proven by your split attitude of saying you mocking magickal rituals as “pretend” but you perform them daily.

Without self-examination we can all easily get lost in false beliefs about ourselves. Crowley described this by saying there were people who thought they had crossed the abyss but actually were flung back down to the lower levels of the Tree of Life and left wandering through those paths while thinking they were among the Supernals. Los, where are you? Crowley has beginning exercises that were filled with developing self-knowledge. Have you been through them?

[…]

you should consider some introspection and resolve your internal issues. (Post #35)

Comments like these, accusing me of having a “psychological dichotomy” and suggesting that I haven’t “been through” various exercises for developing self-knowledge testify not only to Kraig’s desire to talk about anything other than the subject at hand, but to a marked lack of respect for me.

After all, a respectful response is one that takes the time to address the subject under discussion. Just saying “I’m being respectful!” over and over doesn’t make a comment respectful, particularly when it’s filled with distractionary assumptions about me and what I may or may not have experienced.

For the record, I could care less what Kraig thinks of me, but the contrast between his insistence on being "respectful" and his disrespectful and irrelevant speculations is curious.

Incidentally, the false dichotomy Kraig draws is just weird: by his logic, either someone must believe that magick causes the kinds of “results” he talks about OR someone must not “believe in magick” at all and must also be some kind of dogmatic rationalist opposed to the practice of magick.

It’s as if Kraig isn’t aware that a person can practice magick for purely psychological results while also rejecting spurious supernatural theories and beliefs that some people insist on attaching to these practices.

It’s odd that he seems unaware of this, especially since arguably the most important twentieth century author who covers the subject of magick (Aleister Crowley) consistently describes rituals as having purely psychological effects. Regardless of what Crowley may or may not have believed – and regardless of how often he performed magick for “results” – those passages of his writing where he actually describes what magick *does* and how it works are almost always psychological in nature and are devoid of supernaturalism.

Conclusion

There’s little more to add to this exchange, and I feel I’ve probably gone on about it for too long, but I think it’s worth noting that one of the more well-known occult authors, when pressed on how he knows that his claims are true, not only refuses to answer but actually goes a step further and opposes trying to find out – and instead just trusts how it subjectively feels based on a small number of anecdotes.

I’ve said it before, and I’ll say it again: this is not a path to truth of any kind.