Swedish Agency for Health Technology Assessment and Assessment of Social Services
Facing residual uncertainty
When evidence is scarce, decision-making in health and social care is a risky business. But some uncertainty is unavoidable – especially in areas where multiple factors interact and policy-makers disagree regarding values and goals.
Uncertainty is an unwanted state. Nevertheless, it must be addressed – it is integral to daily life in health care and social services. Evidence from relevant, well-conducted studies, such as those reviewed by SBU, provides important support by clarifying anticipated consequences. But they fall far short of providing all the answers.
The rationale for basing health care and social service decisions on research evidence is that lives will improve and resources will be used more effectively. The aim is to reduce the risk of harm and wasting resources due to false expectations among policy-makers regarding consequences of interventions.
Meanwhile, expectations relating to research may be inflated. The belief that simply more and better research could dispel all uncertainty is wishful thinking, according to researchers at the Science Advice for Policy by European Academies (SAPEA), a scientific advisory body to the EU Commission.
In its report Making sense of science [1], SAPEA emphasises that scientific knowledge should never be expected to provide perfect predictions, contribute absolute and universally applicable truths, or to adequately serve as the sole basis for decision-making. On the contrary, decision-makers are cautioned against such over-confidence – a warning that gained unanticipated relevance with the COVID-19 outbreak less than one year later. Decision-makers must take research findings into account, even when substantial uncertainty remains.
Researchers formulate hypotheses about reality and then subject them to systematic testing. Their assertions concern the nature of reality and how it functions, how various occurrences are related and – depending on subject – how the situation can, or in some cases should, be affected and changed. The endeavour to accurately describe reality is a common denominator.
However, this does not mean that all uncertainty is dispelled. According to the advisors at SAPEA, the scientific basis for decision-making will always be more or less uncertain – depending on the complexity of the issues, limitations in scientific knowledge and ambiguities concerning the ultimate goals of the decisions.
An issue becomes complex when different components in a system strongly interact so that whatever occurs in the moment determines the likelihood of various subsequent events. [2] For example, the dynamics may depend upon how interactions between components can either boost or impede one another, the presence of control mechanisms that can either be turned on or off, and effects that may manifest at different rates and in different ways among different individuals. [3]
Greater complexity entails greater uncertainty concerning the benefits of interventions. When making decisions about complex issues, according to SAPEA, the system must be considered as a whole, and knowledge from multiple disciplines must often be applied – in order to be more confident about the outcome. To achieve the desired results, a whole range of simultaneous interventions must be combined, while carefully monitoring the effects so that decisions can be continuously adjusted as needed.
Limitations in scientific knowledge pose another challenge for decision-makers, since researchers are unable to reliably assess the likelihood of various effects of an intervention. This situation may be caused by the absence of research, or the presence of findings that are ambiguous, inconsistent or contradictory due to random or systematic errors, bias. SBU’s work clarifies for decision-makers both what the research shows, with varying degrees of scientific certainty, and what it does not show.
Scientific uncertainty may be rooted in methodological flaws, such as failure to take sources of error into account when designing trials; technical problems, for example related to poor or improperly used instruments for measurement or analysis; or epistemic uncertainty due to insufficient knowledge on underlying fundamentals or ignorance of alternative scenarios. Limited knowledge may also be related to the roles and incentives of scientists, as well as to who has the mandate to formulate research and to interpret and question the results.
Contradictory points of view may ultimately be present, even when there is scientific certainty about the risks, costs and benefits of the interventions. Decision-makers and experts may differ in their interpretation of facts, as well as in their core values and outlooks on life. The same is true for the people who are affected by the decisions. Such different perspectives can be incompatible, though equally well-rooted in fact.
For example, although research findings may be unambiguous concerning the effects of measures against tobacco use and related costs, different decision-makers and different countries may have completely different opinions concerning the appropriate policy. Clearly, disagreements among experts regarding evidence-based policy are not necessarily due to scientific uncertainty.
Moreover, an individual may find it difficult to address his or her own conflicting goals and interests. A decision-maker to whom various conflicting goals are equally important will become uncertain when forced to make a choice, for example due to scarcity of resources. It will be difficult to determine which of all the critical goals take precedence and in each case, weigh what risks and costs become acceptable.
It is not uncommon for the three types of uncertainty to occur simultaneously, in regard to one and the same issue, for which reason reducing uncertainty to zero is rarely possible. Nevertheless, presenting patients, practitioners and policy-makers with the most comprehensive and reliable evidence base possible should result in better informed and more transparent decisions. This practice makes it easier to distinguish what is reasonably certain from remaining uncertainties in specific areas, and of various types and degrees.
This approach makes it easier to discuss alternative actions and to cope with the inevitable residual uncertainty. Ultimately, many may find it easier to accept and comply with the decisions that are taken. • RL
References
- Science Advice for Policy by European Academies. Making sense of science for policy under conditions of complexity and uncertainty. Berlin: SAPEA, 2019. https://doi.org/10.26356/MASOS
- Axelrod RM, et al. Harnessing complexity: organizational implications of a scientific frontier. New York: Basic Books, 2000.
- Chu D, et al. Theories of complexity: Common denominators of complex systems. Complexity, 2003;8:19-30. doi: 10.1002/ cplx.10059
UNCERTAINTY IN DECISION-MAKING
- Complex issue – Involves multifaceted problem in which the many components either strongly facilitate or impede one another in a manner that is difficult to comprehend or predict.
- Limited knowledge – Important information is missing, for example due to the absence of research, or research findings that are ambiguous, inconsistent or contradictory.
- Contradictory points of view – Available knowledge is interpreted and assessed differently. The varying perspectives are difficult to reconcile, and differing goals are in conflict with one another.
