Astonishingly high significance levels were claimed by MG for his groups of professional data, as published for example in a British psychology journal in 19751:
Those significance-levels were derived using a chi-squared test: eg, his group of Jupiter-military reached chi-square = 29, which gave that very high significance level.2
However, a primary axiom of probability theory is that no significance test should be performed on the set of data from which a prediction is made: but only on some further set of data, on which it is tested. The significance test can then tell, whether or not the initial effect has replicated. Michel never seemed to grasp this subtle point.
We've seen (page 14) that there were two main stages of his data-release: the first in 1955 (L'Influence des Astres) which was then followed in 1970 by the data-volumes. The first proposed the hypotheses to be tested. MG's quoted probability-values were derived from the 1970-72 data.
We therefore need to subtract the initial, 1955 data from the final set, to ascertain if anything has replicated, and judge its significance. Thus, for physicians3:
Saturn in Key Sectors for Physicans - Did it Replicate? | |||||
---|---|---|---|---|---|
Physicians | Total | KS Expect | Observed | % Excess | Chi-Square |
1955 | 576 | 96 | 130 | 35% | 12 |
1970 | 2552 | 425 | 486 | 14% | 9 |
2018 Difference | 1976 | 329 | 356 | 8% | 2 |
Chi-Squared = 2 so the physicians-and-Saturn hypothesis fails to attain statistical significance, it did not replicate.
Let's do the same from sports champions and Mars, this time comparing published vs 'correct' values (See Page 13).
Mars and Sports champions - did it replicate? | ||||||
---|---|---|---|---|---|---|
Total Sports | Key Sector Scores | Expected Score | % Excess | Chi-Square | Probability | |
1. MG 1955 | 570 | 68+68=136 | 97.8 | 39% | 32 | 1 in 70,0004 |
2. NK 2018 | 570 | 59+67+126 | 99.85 | 26% | 7 | 1 in 20 |
3. M&FG 1972 | 2088 | 240+212=452 | 365.4 | 24% | ||
4. CURA 2018 | 2088 | 227+208=435 | 365.4 | 19% | 13 | |
Difference 4-1 | 1518 | 299 | 256.6 | 13% | 4 |
A Chi-square value of 4 is not adequate. But, if we instead use the corrected score for his 1955 group of sports champions (Number 2 in the above Table, see Section 13) then the chi-square goes up to 7, which is significant. The Mars-and-sports-champion effect has replicated. It's around a 1 in 100 level of significance, which is not much, but it will do.
We take one more example, this time a deficit. In 1975 MG asserted:
Only 203 of 1,473 great painters were born with Mars in sectors one or four. The odds against this: 200 to one.6
His original book of 1955 had 905 eminent painters scoring Mars in deficit (127 in Key Sectors), then in 1970 the Gauquelins had collected 1472 with much the same deficit (207 Mars in Key Sectors). Subtracting one from the other as before gives us the same 20% deficit, but the chi-square significance is just 3.7 which is merely borderline: not adequate as a significant replication.
The significance levels claimed by Michel Gauquelin were greatly mistaken.