This is a very fair treatment of both sides, I think. The take home message is that we simply don't have enough data to comment on whether or not lithium is effective in preventing suicide.
Dr. Ghaemi has lectured to my residency class many times. He is clearly a very smart and well read individual, but "soldier mindset" perfectly describes his approach to disagreements. I have always been perplexed by his propensity to make excellent, valid criticisms of the opposing side, only to turn around and make statements like "actually p = 0.07 indicates high confidence of a real effect" which totally undermines his valid critiques.
I can believe he is very smart. Here, he has indeed undermined himself and surely would have benefitted from a statistics expert. Sometimes being a sole author is not a wise choice.
My understanding is that the threshold for statistical significance - p = 0.05 - is a convention. Whilst by convention, p values 0.06 and 0.99 would fall short of statistical significance, the difference between these values reflects a gradient of evidence, not a dichotomy of 'significant' and 'not significant.' This underscores the importance of interpreting p-values as part of a broader context, rather than as definitive proof.
Yes, I think it is true that a single statistical test does not constitute definitive evidence. But the whole point of statistical hypothesis testing is that you set your significance level a priori (by convention this is p=0.05 but is often more stringent) and if you find a higher p value then you cannot reject the null hypothesis.
Obviously this is not a hard and fast cut-off, but if you are writing an article in which you are critiquing methodology, you should probably make sure your own analysis is watertight.
I think of p = 0.05 as passing the laugh test. I like to imagine a table with four 5-shot revolvers with a single bullet among them and being asked to select one, hold it to my head and pull the trigger once, as a risk and 30 as confirmation, taking advantage of the Central Limit Theorem. Fortunately, I'm not quite that dedicated to motivated reasoning.
The analogy isn't quite right bc it really depends on the prior probability. A p of .05 only tells you that *if the null hypothesis is true*, you'd only expect a result at least as extreme 1 out of 20 times. But it tells you diddly squat about *how likely it is that the null hypothesis is true*. So if your null hypothesis is really really likely, even a p of .0001 isn't enough, and if the null is really unlikely, a p much larger than .05 shouldn't necessarily convince you to embrace the null.
Great discussion. I don't know Ghaemi but Moncrieff clearly has an agenda. All observations are theory-laden, as Karl Popper said. Our field is so susceptible to forking paths...maybe lithium helps with some suicide attempts and not others? Maybe the type of person who attempts suicide now is not the same as somebody attempting suicide in 1980? The culture has changed a lot.
She clearly does have an agenda, or more charitably, a viewpoint which is outside the consensus. I kinda of feel that’s ok as long as she’s willing to put her ideas to the test.
Better to have people like her inside the tent than outside.
My perspective is from the experience of having been treated with lithium once and also having some experience in statistics.
i did not tolerate the therapy well. The effect was an urge to urinate frustrated by BPE, disrupting sleep and detracting from the efficacy of CPAP therapy. My empiricist psychiatrist switched to an alternative pharmacological regime that has been mostly effective over the past 20 years up until his retirement when his successor discontinued one component that is now the geriatric equivalent of fentanyl. (Poor, poor pitiful me).
In addition, i'm BP2, rather than BP1. Treatment of BP2 presents difficulty because it is often misdiagnosed as major depressive disorder since no BP2 patient ever presents with "doc, you've got to help me. I feel GREAT!" As a result, I believe BP2 is often seen, in error, as having the same etiology as BP1 but to a lesser degree. Perhaps, there is non-clinical evidence, I don't know.
In any of the physical or non-medical life sciences, a primary frequentist analysis of a binomial outcome Y with n = 9 for 1 and n = unknown for 0 from a treatment X is likely to be greeted, at most, as problematic for the following reasons:
1. Sample size: With only 9 instances, the sample size is quite small. This limits the statistical power and precision of any analysis.
2. Expected frequencies: For a binomial analysis, we typically want at least 5 expected instances in each category. With only 9 total instances, this may not be met depending on the probability of success.
3. Confidence intervals: With such a small sample, confidence intervals will likely be very wide, reducing the practical utility of the results.
4. Type I and II errors: The small sample size increases the risk of both false positives and false negatives especially when Bayesian priors are missing.
5. Assumptions: Binomial tests assume independent trials and a fixed probability of success, which may or may not be met in this case.
Given these considerations, while it's possible to perform a binomial analysis on 9 instances, the results should be interpreted with extreme caution. They may provide some preliminary insights, but would generally not be considered robust enough for definitive conclusions.
I haven't read the literature cited due to lack of subscription access to academic journals so I may be wrong about this, as about so much else, so there's that.
Thank you for sharing your personal and statistical experience! I appreciate lithium isn’t the easiest medication to take and for some, the benefits will not outweigh the side effects. I’m not sure about the shared aetiology between bipolar 1 and 2 - perhaps the clinical finding that antidepressants don’t seem to be effective for either condition points to some shared origins.
If you did want to access the academic articles then there is always the option to email the author directly, or alternatively…Sci-hub! I think your concerns about their sample size is spot on though
Thanks. Imposition of writing the authors isn’t really justified by my level of interest or expertise. If you haven’t already, check out Frank Harrell’s https://discourse.datamethods.org for high level discussion of medical statistics by people who know what they are talking about. I wish I were tall enough to stand on his shoulders.
Well I am pleased to hear that, personally I find a lot of groupthink and fear of challenging orthodoxy among most of my colleagues.....sort of a "we don't go there" kind of vibe....I expect there would be a bit more openness outside the peer group perhaps...
It's sad that it's so unusual to find a psychiatrist with an open mind, who is prepared to put himself out there and has the research and stats skills to do so well and in a way that invites reflection....and so wonderful that we have you.
This is a very fair treatment of both sides, I think. The take home message is that we simply don't have enough data to comment on whether or not lithium is effective in preventing suicide.
Dr. Ghaemi has lectured to my residency class many times. He is clearly a very smart and well read individual, but "soldier mindset" perfectly describes his approach to disagreements. I have always been perplexed by his propensity to make excellent, valid criticisms of the opposing side, only to turn around and make statements like "actually p = 0.07 indicates high confidence of a real effect" which totally undermines his valid critiques.
I can believe he is very smart. Here, he has indeed undermined himself and surely would have benefitted from a statistics expert. Sometimes being a sole author is not a wise choice.
My understanding is that the threshold for statistical significance - p = 0.05 - is a convention. Whilst by convention, p values 0.06 and 0.99 would fall short of statistical significance, the difference between these values reflects a gradient of evidence, not a dichotomy of 'significant' and 'not significant.' This underscores the importance of interpreting p-values as part of a broader context, rather than as definitive proof.
Yes, I think it is true that a single statistical test does not constitute definitive evidence. But the whole point of statistical hypothesis testing is that you set your significance level a priori (by convention this is p=0.05 but is often more stringent) and if you find a higher p value then you cannot reject the null hypothesis.
Obviously this is not a hard and fast cut-off, but if you are writing an article in which you are critiquing methodology, you should probably make sure your own analysis is watertight.
I think of p = 0.05 as passing the laugh test. I like to imagine a table with four 5-shot revolvers with a single bullet among them and being asked to select one, hold it to my head and pull the trigger once, as a risk and 30 as confirmation, taking advantage of the Central Limit Theorem. Fortunately, I'm not quite that dedicated to motivated reasoning.
The analogy isn't quite right bc it really depends on the prior probability. A p of .05 only tells you that *if the null hypothesis is true*, you'd only expect a result at least as extreme 1 out of 20 times. But it tells you diddly squat about *how likely it is that the null hypothesis is true*. So if your null hypothesis is really really likely, even a p of .0001 isn't enough, and if the null is really unlikely, a p much larger than .05 shouldn't necessarily convince you to embrace the null.
But what if revolvers were subject to Bonfarroni correction?)
I guess in that case it depends on the balance of preference between precision and recall. Might be hard to recruit volunteers for a RCT. 😈
Great discussion. I don't know Ghaemi but Moncrieff clearly has an agenda. All observations are theory-laden, as Karl Popper said. Our field is so susceptible to forking paths...maybe lithium helps with some suicide attempts and not others? Maybe the type of person who attempts suicide now is not the same as somebody attempting suicide in 1980? The culture has changed a lot.
She clearly does have an agenda, or more charitably, a viewpoint which is outside the consensus. I kinda of feel that’s ok as long as she’s willing to put her ideas to the test.
Better to have people like her inside the tent than outside.
What a magnificent ending to the piece!
My perspective is from the experience of having been treated with lithium once and also having some experience in statistics.
i did not tolerate the therapy well. The effect was an urge to urinate frustrated by BPE, disrupting sleep and detracting from the efficacy of CPAP therapy. My empiricist psychiatrist switched to an alternative pharmacological regime that has been mostly effective over the past 20 years up until his retirement when his successor discontinued one component that is now the geriatric equivalent of fentanyl. (Poor, poor pitiful me).
In addition, i'm BP2, rather than BP1. Treatment of BP2 presents difficulty because it is often misdiagnosed as major depressive disorder since no BP2 patient ever presents with "doc, you've got to help me. I feel GREAT!" As a result, I believe BP2 is often seen, in error, as having the same etiology as BP1 but to a lesser degree. Perhaps, there is non-clinical evidence, I don't know.
In any of the physical or non-medical life sciences, a primary frequentist analysis of a binomial outcome Y with n = 9 for 1 and n = unknown for 0 from a treatment X is likely to be greeted, at most, as problematic for the following reasons:
1. Sample size: With only 9 instances, the sample size is quite small. This limits the statistical power and precision of any analysis.
2. Expected frequencies: For a binomial analysis, we typically want at least 5 expected instances in each category. With only 9 total instances, this may not be met depending on the probability of success.
3. Confidence intervals: With such a small sample, confidence intervals will likely be very wide, reducing the practical utility of the results.
4. Type I and II errors: The small sample size increases the risk of both false positives and false negatives especially when Bayesian priors are missing.
5. Assumptions: Binomial tests assume independent trials and a fixed probability of success, which may or may not be met in this case.
Given these considerations, while it's possible to perform a binomial analysis on 9 instances, the results should be interpreted with extreme caution. They may provide some preliminary insights, but would generally not be considered robust enough for definitive conclusions.
I haven't read the literature cited due to lack of subscription access to academic journals so I may be wrong about this, as about so much else, so there's that.
Thanks for the great guided tour
Thank you for sharing your personal and statistical experience! I appreciate lithium isn’t the easiest medication to take and for some, the benefits will not outweigh the side effects. I’m not sure about the shared aetiology between bipolar 1 and 2 - perhaps the clinical finding that antidepressants don’t seem to be effective for either condition points to some shared origins.
If you did want to access the academic articles then there is always the option to email the author directly, or alternatively…Sci-hub! I think your concerns about their sample size is spot on though
Thanks. Imposition of writing the authors isn’t really justified by my level of interest or expertise. If you haven’t already, check out Frank Harrell’s https://discourse.datamethods.org for high level discussion of medical statistics by people who know what they are talking about. I wish I were tall enough to stand on his shoulders.
Well I am pleased to hear that, personally I find a lot of groupthink and fear of challenging orthodoxy among most of my colleagues.....sort of a "we don't go there" kind of vibe....I expect there would be a bit more openness outside the peer group perhaps...
It's sad that it's so unusual to find a psychiatrist with an open mind, who is prepared to put himself out there and has the research and stats skills to do so well and in a way that invites reflection....and so wonderful that we have you.
Thank you - I would hope most practicing psychiatrists have an ability to tolerate uncertainty and opposing views at least!
Read and retold. Any opinion of your own?