Lacking, poor quality information significantly increases uncertainty around a decision. While it is not the only problem that a decision-maker has to face (noise, cognitive bias, etc.), it is one of the most centra. Hence, information-related issues were among the main points we had to tackle when building the first iteration of our app. Below are the four classical information-related problems (Schmitt, Klein, 1996) and what solutions we have decided to design to solve them.
Missing information: When the decision-maker does not have the information at hand when required. The information may be out-of-reach of the organization. Alternatively, it can be available for the organization but not within reach of the relevant decision-maker. The main solution for out-of-organization-reach rests on multiplying the channels of collection:
news monitoring
academic articles
reports
local network reporting
open-source and third-party databases
a community-driven system allows detection of useful content
Subsequently, we had to ensure that this increased volume of information would not lead to overload and create another issue: out-of-decision-maker-reach.
Complex information: The information is so complex that it is difficult to synthesize all facts and understand the outcomes of data. This can arise from the data format - when it is laid out in a manner that is hard to consume - or from its volume - when there is just too much information. Information overload is especially tricky: analysts are both drowning in data facts AND feeling that they are better equipped to answer a question. In order to overcome complexity and overload, we had to come up with:
a community-driven content assessment detection and sorting of information that sorts and filters timely information
a highly visual way to consume data
a visual representation of systems that allows seeing interconnections between the components of a problematic very clearly
an editing process nudging for clear, concise content
Unreliable information: When the information is perceived as untrustworthy. This is an instance when the source is considered dubious and of low credibility, or when the information itself is judged as difficult to believe, even though its source may be reputable. We had to layer multiple solutions exists with various degrees of sophistication to mitigate this risk:
a community-driven information assessment where each piece is commented on and voted upon by a community of analysts
a conflict resolution mechanism to solve disagreements among analysts. Both cross-checks and believability are taken into account. If an analyst has a tendency to be more right about a topic, s.he will carry more weight in the resolution of the disagreement
an in-app reviewing system that allows a human-centered editing process for information coming from the field.
Conflicting information: When the impact of a piece of information is unclear. Information can be ambiguous and interpreted in multiple ways. This is for instance the case when there are contradictory accounts of the same event. To solve this demanding puzzle of interpretation, we set up a forecasting tournament:
all analysts can anonymously propose their interpretation of information and its impact
forecasts are compared to reality
analysts with a better track record on a topic are given more weight in the system