Rootclaim takes a deep, data-driven look at the issues that interest society. The platform integrates all available evidence, assesses it for credibility and uses probabilistic models to reach conclusions about the likelihood of competing hypotheses.
Rootclaim is grounded in two pillars:
Rootclaim outperforms human reasoning by correcting for the biases and flaws of human intuition. Its conclusions represent the best available understanding of the complexity and uncertainty in our world.
Watch out though—the crowd evaluates the hypothesis-to-evidence relationship opposite to what you might intuitively expect.
Instead of first looking at all the evidence, and then trying to figure out which hypothesis is the best match, start from the hypotheses: Supposing Hypothesis A were true, how well does each piece of evidence fit with that storyline? This reversal is the key to Rootclaim’s approach. It lets us break down impossibly complex issues into a series of answerable questions.
Second, all the inputs, from every side of the issue, are combined and sent through Rootclaim’s Bayesian probabilistic engine. Unlike other types of analyses, Rootclaim doesn’t cherry-pick the evidence. Since anyone can add new information or challenge the existing inputs, all the evidence, from every side of the issue, is balanced together using Rootclaim’s Bayesian inference algorithm, letting us reconstruct the complex issue at hand.
Finally, the engine outputs its conclusions, showing which hypothesis came out on top. The conclusions are a direct mathematical result of the inputs, so if the inputs are right, the outputs are indisputable.
Remember, the whole analysis is transparent and open to scrutiny. If the conclusions don't seem right to you, challenge the inputs and improve the results!
Each analysis begins with a question, and compares several possible hypotheses (answers). For example, let’s say we want to know the answer to the following question:
Once the question and hypotheses are established, all the information related to the question is collected and assessed for its reliability. There are two main types of information evaluated in the analysis:
Finally, the relationship between each hypothesis and each piece of specific evidence is evaluated: How likely is each piece of evidence, given each hypothesis? In this example, there would be four different hypothesis-to-evidence relationships to assess:
Each new piece of evidence added to the analysis must be assessed in the same way, asking how likely that evidence would be under each of the competing hypotheses. The numbers that represent each of these relationships must be supported by solid reasoning. Anyone in the crowd can challenge these inputs and propose a different number, backed by better reasoning.
Ready to start contributing to a story? Each part of the analysis is discussed in further detail below.
Each competing hypothesis is one possible answer to the main question. All of the hypotheses must be mutually exclusive (two can’t be true at the same time), and collectively exhaustive (there are no other reasonable answers to consider). As a group, the competing hypotheses capture all the main points of view surrounding the topic under debate.
In this example, the hypothesis choices are exhaustive because one of the hypotheses must be true, and they are mutually exclusive because if one hypothesis is true then the other(s) must be false.
Each hypothesis should meet these criteria:
In some cases, it is impossible for the list of hypotheses to truly be collectively exhaustive, since there may be an infinite number of possible answers to the question at hand. Even in these situations, the list of hypotheses is still assumed to be collectively exhaustive for the sake of mathematical validity.
The evidence includes all the relevant information known about the topic at hand, compiled and organized by category. Each piece of evidence is supported by a source, whose reliability is assessed independently.
It’s important to phrase the evidence in a way that conveys the essence of the claim. In other words, it should contain exactly the amount of information that is relevant to the analysis--no more, and no less.
Depending on what the relevance to the analysis is, the essence of this evidence might look like this:
In a different analysis, however, the essence might look like this:
As this example demonstrates, the essence of the evidence depends on the context of the analysis.
The starting point of each analysis consists of general statistics and past examples related to each hypothesis. This general information is used to calculate how plausible each hypothesis is without regard to specifics of the case at hand.
Why is this necessary? If a hypothesis starts out already unlikely (e.g., “Santa did it!”), one would need a lot more evidence to prove that it’s true. If a hypothesis is already reasonable, one would need less evidence to back it up.
The similar cases considered must be specific enough to reflect the essential differences between hypotheses, but general enough to create a large enough sample to infer meaningful statistics.
Each piece of evidence is supported by a source, which is treated as an additional node in the Bayesian network. The source’s reliability is measured as one minus the probability that the source would provide such a report, if the report weren’t in fact true.
For example, if a source’s reliability score is 95%, then that means there's a 5% probability that this source would report such evidence when that evidence is false. When the information was brought to the public through several sources, we evaluate the weakest link in the chain of communication.
Sometimes, the way the evidence is phrased has to account for dependencies among sources, evidence, and hypotheses, in order to allow for all the hypothesis-to-evidence relationships to be assessed on equal footing.
If, for example, the source of a piece of evidence is a potential suspect in the crime under investigation, then the reliability of that suspect as a source might change dramatically depending on whether we’re looking at the hypothesis in which the suspect is guilty versus the hypothesis in which he is innocent.
This discrepancy is resolved by changing the definition of the evidence to the report of the event, rather than the event itself.
Instead of this wording:
The evidence must be phrased like this:
Now the hypothesis-to-evidence relationship can account for the evidence directly, without any potential differences in the reliability of the source.
To assess the source reliability, we try to estimate the probability that the source will provide such a report if it weren’t true. The first step in this process is estimating the most likely reason that the source would make a false report.
While it is possible that some sources might knowingly lie, there are other reasons that a source might report something other than the truth – like flaws in the raw data, or innocent misinterpretation of the data.
The source reliability questionnaire opens with exactly this question:
Depending on the answer to that question, a reliability score is assigned to the source. This score applies only to this specific piece of evidence - it is not a general score of the source’s overall reliability related to other topics, since a source may be more reliable in certain subjects than in others.
The full source reliability questionnaire is reproduced here:
Probability source will tell the truth | Net benefit | |||||
Expected cost if exposed | Huge | Major | Moderate | Minor | Almost None | |
Huge | .9 | .95 | .99 | .999 | .9999 | |
Major | .7 | .9 | .95 | .99 | .999 | |
Moderate | .3 | .5 | .9 | .95 | .99 | |
Minor | .01 | .1 | .5 | .9 | .95 | |
In general, nothing is 100% certain. However, in some specific cases, the certainty is so high that for the purposes of simplicity and readability of the analysis, we set its likelihood at 100% :
At any time, a piece of evidence set as 100% certain may be contested and/or analyzed in further depth. A piece of evidence (or source) is only considered 100% unless and until it is challenged by the crowd.
Instead of asking how likely each hypothesis is given the evidence, the inputs to the probabilistic engine reverse the direction of the analysis: how well does each piece of evidence fit under each of the competing hypotheses?
For example, let’s say the competing hypotheses are:
And the evidence is:
All the aforementioned pieces of the puzzle: starting points of hypothesis, evidence and their relationships to hypotheses, and source reliabilities are used to construct a unique Bayesian Network. A Bayesian inference algorithm takes the input network and calculates the output (posterior) probabilities.
The formal expression of Bayes' theorem is:
In cases where we have multiple pieces of evidence we use the Bayesian Network structure. This structure allows us to account for many pieces of evidence and their relationships with the hypotheses, the potential dependencies among them, the reliability of their sources, etc.
Rootclaim uses a Bayesian Tree, which is a particular case of a Bayesian Network. The tree starts with its root node: the group of competing hypotheses (each hypothesis is a possible state of that node). Evidence is organized below the root node, such that each piece of evidence is connected below the root node, and above its source node. Additional intermediary levels may be added to the tree structure in order to make sure that there are no invalid dependencies throughout the structure.
Two pieces of evidence are dependent if a change in the likelihood of the first produces a change in the likelihood of the second. For example, take the following two pieces of evidence:
If Bob got more sleep, would this affect how well he would do on his math test? If the answer is yes, then there is a dependency between these two pieces of evidence.
Formally, Bayesian networks assume that all of the dependencies among nodes in the graph are explicitly represented by arcs (connections) between them. In other words, all nodes of the network should be independent given their parent (the node pointing to them). In order to make the analysis mathematically valid, therefore, we have to address this dependency by changing the structure of the Bayesian network:
In order to make the analysis mathematically valid, therefore, we have to address this dependency by changing the structure of the Bayesian network:
A sub-analysis pits all the competing possible causes for the dependency against one another. For example, there could be a couple of different reasons that could explain why Bob failing his math test and only sleeping for four hours are dependent pieces of evidence:
All pieces of evidence whose dependencies are resolved by the addition of the sub-analysis are positioned in the network as child nodes of the sub-analysis node. Evidence and sources are evaluated exactly the same way as in the main analysis, but instead of looking at the likelihood of the evidence under each hypothesis, we look at the likelihood of the evidence under each scenario.
At the level of the main analysis, each of the competing scenarios in the sub-analysis are evaluated exactly the same way that the hypothesis-to-evidence relationship evaluates other evidence. However, just like with the hypotheses, the scenarios in a sub-analysis must be mutually exclusive and collectively exhaustive, given the hypotheses above. This means that the likelihoods of the scenarios given each hypothesis must sum to 100%.
By structuring these competing possible causes as a sub-analysis, the dependency is resolved.
Storyline assumptions are another possible tool that can be used to resolve dependencies among multiple pieces of evidence. If there is a reasonable common cause that can explain all the dependent pieces of evidence, and there are no other reasonable common causes competing with it, then that common cause is integrated as an assumption directly into the storyline of the hypothesis itself.
How does this work?
Taking into consideration this third piece of evidence (where he explicitly states that he didn’t study), it seems very implausible that Bob studied for the math test.
This leaves only one reasonable common cause that would explain all three of these pieces of evidence. So instead of creating a sub-analysis, we can incorporate the most reasonable explanation into the storyline of the hypothesis itself:
Each storyline assumption added makes the hypothesis more specific, and therefore it also makes the hypothesis less likely. In order to determine how much less likely this new, more specific version of the hypothesis is, we evaluate the storyline assumption similar to the hypothesis-to-evidence relationship: supposing the hypothesis were true, what is the likelihood that this new storyline assumption would also be true?
The rest of the analysis should also be adjusted so that the new storyline assumption is considered part of the hypothesis. This can affect the evaluation of the hypothesis-to-evidence relationships, as well as the way evidence is phrased, and even how source reliability is assessed.
By adding the most likely storyline assumption, and adjusting the rest of the network based on the more specific version of the hypothesis, the dependency is resolved and the adjusted hypothesis is the most likely it can be. Its initial likelihood is lower (since it is more specific) but the conditional probability of the evidence given this hypothesis is higher.
The prior probability (starting point) of each hypothesis is multiplied by the likelihood assigned to each storyline assumption, given the hypothesis. Then, these results are normalized to yield the adjusted prior probability for the hypothesis.
For example, let’s say the starting point for an analysis gives Hypothesis 1 a prior probability of 90% and Hypothesis 2 a prior probability of 10%. Hypothesis 1 requires one storyline assumption, which was assessed as 40% likely given the original hypothesis. Hypothesis 2 requires no storyline assumptions.
The calculation looks like this:
Hypothesis 1 | Hypothesis 2 | |
Initial prior probability (starting point) | 90% | 10% |
Storyline assumptions | 40% | None |
Adjusted prior probability | 36% | 36% |
Normalized adjusted prior probability | 78.3% | 21.7% |
For the purposes of the Bayesian inference algorithm, the normalized adjusted prior probability (the last line in the table above) is treated as the prior probability in the Bayesian network.