skip to content

Timetable (FOSW02)

Bayesian networks and argumentation in evidence analysis

Monday 26th September 2016 to Thursday 29th September 2016

Monday 26th September 2016
09:30 to 10:15 Registration
10:05 to 10:15 Welcome from John Toland (INI Director) INI 1
10:15 to 11:00 Philip Dawid (University of Cambridge)
An Introduction to Bayesian Networks

I will outline some basic theory of Bayesian Networks, with forensic applications.  Topics will include qualitative and quantitative representation,  object-oriented networks, and (time permitting) causal diagrams.

11:00 to 11:30 Morning Coffee
11:30 to 12:15 Henry Prakken (Universiteit Utrecht); (University of Groningen)
On how expert witnesses can give useful Bayesian analyses of complex criminal cases

In this talk I will discuss how expert witnesses in criminal trials can give useful Bayesian analyses of complex criminal cases. I will discuss several questions that have to be answered to make Bayesian networks useful in this context and what kinds of expertise are required to answer these questions. The discussions will be partly based on my recent experiences as an expert witness in a murder trial and a serial arson case.

12:15 to 13:30 Lunch @ INI
13:30 to 14:15 Julia Mortera (Università degli Studi Roma Tre)
Paternity testing and other inference about relationships from DNA mixtures

DNA is now routinely used in criminal  and civil investigations.  DNA samples are of varying quality and therefore present challenging problems for their interpretation. We present a statistical model for the quantitative peak information obtained from an electropherogram (EPG) of a forensic DNA sample and illustrate its potential use for the analysis of civil and criminal cases. In contrast to most previously used methods, we directly model the peak height information and incorporate important artefacts associated with the production of the EPG. The model has a number of unknown parameters, that  can be estimated in the presence of multiple unknown contributors; the computations exploit a Bayesian network   representation of the model.

We illustrate real casework examples from a criminal case and a disputed paternity case, where in both cases part of the evidence was from a DNA mixture.   We present methods for inference about the relationships between contributors to a DNA mixture of unknown genotype and other individuals of known genotype: a basic example would be testing whether a contributor to a mixture is the father of a child of known genotype (or indeed the similar question with the roles of parent and child reversed). Following commonly accepted practice, the evidence for such a relationship is presented as the likelihood ratio for the specified relationship versus the alternative that there is no such relationship, so the father is taken to be a random member of the population. Our methods are based on the statistical model for DNA mixtures, in which a Bayesian network is used as a computational device for efficiently computing likelihoods; the present work builds on that approach, but makes more explicit use of the BN in the modelling.
Based on joint work with Peter Green.

14:15 to 15:00 Paolo Garbolino (IUAV University of Venice)
Bayes Nets for the evaluation of testimony

David Schum gave in his book The Evidential Foundations of Probabilistic Reasoning an analysis of testimony that can be formalized by a Bayes net whose nodes represent the hypothesis of interest, the basic attributes of the credibility of human witnesses (observational accuracy, objectivity and veracity) and the ancillary evidence bearing upon those attributes. It will be shown as, given some plausible hypotheses, the net can provide a new answer to a classical riddle in the literature about evidential probabilistic reasoning, the so-called “blue bus problem”.

15:00 to 15:30 Afternoon Tea
15:30 to 16:15 Giulio D'Agostini (Università degli Studi di Roma La Sapienza); (INFN, Roma La Sapienza)
Basic probabilistic issues in the Sciences and in Forensics (hopefully) clarified by a Toy Experiment modelled by a Bayesian Network
16:15 to 17:00 Peter Vergeer (Netherlands Forensic Institute)
A Bayesian network for glass evidence evaluation at activity level: a novel approach to model the background distribution


In burglary cases the comparison of glass particles found on a piece of clothing of a suspect and a broken reference glass pane is of importance. Often, suspects are known as multiple offenders and may have a large collection of glass on their clothing. Therefore, in order to evaluate the strength of evidence, current likelihood ratio formulas contain parameters such as the number of groups of glass found on a piece of clothing, and the size of the matching group [1]. In order to obtain probabilities for these parameters, glass particles found on clothing of suspects have been counted and grouped, see e.g. [2]. In general, the amount of glass particles found on a suspect is limited in these studies.
A database of glass from suspects in the Netherlands shows quite deviant results. Up to a few hundred of glass particles are often encountered and only samples may be analyzed. In order to evaluate the evidential strength of a sample of particles with a background model based on samples from casework requires a different background model. We propose to model the background distribution of the sample by a ‘Chinese restaurant process’ [3].  

[1] Forensic Interpretation of Glass Evidence, CRC Press. (2000). (accessed April 19, 2016).
[2] J.A. Lambert, M.J. Satterthwaite, P.H. Harrison, A survey of glass fragments recovered from clothing of persons suspected of involvement in crime, Sci. Justice. 35 (1995) 273–281. doi:10.1016/S1355-0306(95)72681-8.
[3] D.J. Aldous, Exchangeability and related topics, in: P.L. Hennequin (Ed.), Éc. DÉté Probab. St.-Flour XIII — 1983, Springer Berlin Heidelberg, 1985: pp. 1–198. (accessed September 2, 2016).


17:00 to 18:00 Welcome Wine Reception at INI
Tuesday 27th September 2016
09:30 to 10:15 Charles Berger (Ministerie van Veiligheid en Justitie)
On activity level propositions addressing the actor or activity

Bas Kokshoorn, Bart Blankers, Jacob de Zoete, Charles Berger

More often than not, the source of DNA traces found at a crime scene is not disputed, but the activity or timing of events that resulted in their transfer is. As a consequence, practitioners are increasingly asked to assign a value to DNA evidence given propositions about activities provided by prosecution and defense counsel. Given that the dispute concerns the nature of the activity that took place or the identity of the actor that carried out the activity, several factors will determine how to formulate the propositions. Determining factors are (1) whether defense claims the crime never took place, (2) whether defense claims someone other than the accused (either an unknown individual or a known person) performed the criminal activity, and (3) whether it is claimed and disputed that the suspect performed an alternative, legitimate activity or has a relation to the victim, the object, or the scene of crime that implies a legitimate interaction. Addressing such propositions using Bayesian networks, we demonstrate the effects of the various proposition sets on the evaluation of the evidence. 

10:15 to 11:00 Norman Fenton (Queen Mary, University of London)
Bayesian networks: challenges and opportunities in the law
11:00 to 11:30 Morning Coffee
11:30 to 12:15 Anjali Mazumder (Carnegie Mellon University); (University of Warwick)
Using Chain Event Graphs to Address Asymmetric Evidence in Legal Reasoning
Co-author: James Q. Smith (University of Warwick)

Bayesian networks (BNs), a class of probabilistic graphical models, have been useful in providing a graphical representation of a problem, calculating marginal and conditional probabilities of interest, and making inferences particularly addressing propositions about the source or an evidential-sample. To address propositions relating to activities, there is a need to account for different plausible explanations of a suspect/perpetrator’s actions and events as it relates to the evidence. We propose the use of another class of graphical models, chain event graphs (CEGs), exploiting event tree structures to depict the unfolding events as postulated by each side (defence and prosecution) and differing explanations/scenarios. Different explanations/scenarios can introduce different sets of relevant information affecting the dependence relationship between variables and symmetry of the structure. With the use of case examples involving transfer and persistence and different evidence types (but in which DNA provides a sub-source level of attribution), we further show how CEGs can assist in the careful pairing and development of propositions and analysis of the evidence by addressing uncertainty and the asymmetric unfolding of the events to better assist the courts.
12:15 to 13:30 Lunch @ INI
13:30 to 14:15 Barbaros Yet (Other)
A Framework to Present Bayesian Networks to Domain Experts and Potential Users
Knowledge and assumptions behind most Bayesian network models are often not clear to anyone other than their developers. This limits their use as decision support tools in clinical and legal domains where the outcomes of decisions can be critical. We propose a framework for representing knowledge supporting or conflicting with BN, and knowledge associated with factors that are relevant but excluded from the BN. The aim of this framework is to enable domain experts and potential users to browse, review, criticise and modify a BN model without having deep technical knowledge about BNs.

Co-authors: Zane Perkins (Queen Mary University of London), Nigel Tai (The Royal London Hospital), William Marsh (Queen Mary University of London)

14:15 to 15:00 Hana Chockler (King's College London)
Causality and Responsibility in Formal Verification and Beyond
In this talk, I will (briefly) introduce the theory of actual causality as defined by Halpern and Pearl. This theory turns out to be extremely useful in various areas of computer science (and also, perhaps surprisingly, psychology) due to a good match between the results it produces and our intuition. I will outline the evolution of the definitions of actual causality and intuitive reasons for the many parameters in the definition using examples from formal verification. I will also introduce the definitions of responsibility and blame, which quantify the definition of causality.

We will look in more detail at the applications of causality to formal verification, namely, explanation of counter-examples, refinement of coverage metrics, and symbolic trajectory evaluation. It is interesting to note that explanation of counter-examples using the definition of actual causality is implemented in an industrial tool and produces results that are usually consistent with the users’ intuition, hence it is a popular and widely used feature of the tool.

Finally, I will briefly discuss recent applications of causality to legal reasoning and to understanding of political phenomena, and will conclude with outlining promising future directions.

The talk is based on many papers written by many people, and is not limited to my own research. The talk is reasonably self-contained.
15:00 to 15:30 Afternoon Tea
15:30 to 16:15 Richard Overill (King's College London)
Using Bayesian Networks to Quantify Digital Forensic Evidence and Hypotheses

In what appears to be an increasingly litigious age, courts, legal officials and law enforcement officers in a number of adversarial legal jurisdictions are starting to look for quantitative indications of (i) the probative value (or weight) of individual items of digital evidence connected with a case; and (ii) the relative plausibility of competing hypotheses (or narratives) purporting to explain how the recovered items of digital evidence (traces) were created.

In this presentation, we review the contributions that Bayesian Networks are capable of making to the understanding, analysis and evaluation of crimes whose primary items of evidence are digital in nature, and show how as a consequence they may fulfill both of the two above desiderata. 

16:15 to 17:00 Eyal Zamir (Hebrew University of Jerusalem)
The Anti-Inference Bias and Circumstantial Evidence
My presentation will be based on two studies: Seeing is Believing: The Anti-Inference Bias, co-authored with Ilana Ritov and Doron Teichman (see here), and New Evidence about Circumstantial Evidence, co-authored with Elisha Harlev and Ilana Ritov (see here).

Judicial fact-finders are commonly instructed to determine the reliability and weight of any evidence, be it direct or circumstantial, without prejudice to the latter. Nonetheless, studies have shown that people are reluctant to impose liability based on circumstantial evidence alone, even when this evidence is more reliable than direct evidence. Proposed explanations for this reluctance have focused on factors such as the statistical nature of some circumstantial evidence, the tendency of fact-finders to assign low subjective probabilities to circumstantial evidence, and the fact that direct evidence can rule out with greater ease any competing factual theory regarding liability.

In the first article, we demonstrated experimentally that even when these factors are controlled for, the disinclination to impose liability based on non-direct evidence remains. For instance, people are much more willing to convict a driver of a speeding violation on the basis of a speed camera than on the basis of two cameras documenting the exact time a car passes by them — from which the driver’s speed in the pertinent section of the road is inferred. While these findings do not necessarily refute the previous theories, they indicate that they are incomplete. The new findings point to the existence of a deep-seated bias against basing liability on inferences — an anti-inference bias.  

The second article describe seven new experiments that explore the scope and resilience of the anti-inference bias. It shows that this bias is significantly reduced when legal decision-makers confer benefits, rather than impose liability. We thus point to a new legal implication of the psychological phenomenon of loss aversion. In contrast, we find no support for the hypothesis that the reluctance to impose legal liability on the basis of circumstantial evidence correlates with the severity of the legal sanctions. Finally, the article demonstrates the robustness of the anti-inference bias and its resilience to simple debiasing techniques.

Taken together, the studies show that the anti-inference bias reflects primarily normative intuitions, rather than merely epistemological ones, and that it reflects conscious intuitions, rather than wholly unconscious ones. The articles discuss the policy implications of the new findings for procedural and substantive legal norms, including the limited potential (and questionable desirability) of debiasing techniques, the role of legal presumptions, and the advantages of redefining offenses in a way that obviates the need for inferences.

17:00 to 18:00 Drinks Reception at INI
Wednesday 28th September 2016
09:30 to 10:15 Bart Verheij (University of Groningen)
Three Normative Frameworks for Evidential Reasoning and their Connections: Arguments, Scenarios and Probabilities
Artificial intelligence research on reasoning with criminal evidence in terms of arguments, hypothetical scenarios, and probabilities showed that Bayesian networks can be used for modeling arguments and structured hypotheses. Also well-known issues with Bayesian networks were encountered: More numbers are needed than are available, and there is a risk of misinterpretation of the graph underlying the Bayesian network, for instance as a causal model. The formalism presented models arguments and scenarios in terms of models that have a probabilistic interpretation, but do not represent a full distribution over the variables.
10:15 to 11:00 Floris Bex (Universiteit Utrecht)
From Natural Language to Bayesian Networks (and back again)
Decision makers and analysts often use informal, linguistic concepts when they talk about a case: they tell the story that explains the evidence, or argue against a particular interpretation of the evidence. On the other hand, mathematicians and logicians present formal frameworks to precisely capture and support reasoning about evidence. 

In this talk, I will show how different Artificial Intelligence techniques can be used to close the gap between these two extremes - messy, informal natural language and specific, well-defined formalisms such as Bayesian Networks.  

11:00 to 11:30 Morning Coffee
11:30 to 12:15 Martin Neil (Queen Mary, University of London)
Modelling competing Legal Arguments using Bayesian Model Comparison and Averaging

Bayesian models of legal arguments generally aim to produce a single integrated model, combining each of the legal arguments under consideration. This combined approach implicitly assumes that variables and their relationships can be represented without any contradiction or misalignment and in a way that makes sense with respect to the competing argument narratives. In contrast to this integrated approach, Non-Bayesian approaches to legal argumentation have tended to be narrative based and have focused on comparisons between competing stories and explanations. This paper describes a novel approach to compare and ‘average’ Bayesian models of legal arguments that have been built independently and with no attempt to make them consistent in terms of variables, causal assumptions or parameterization. The approach is consistent with subjectivist Bayesian philosophy. Practically, competing models of legal arguments are assessed by the extent to which the credibility of the sources of evidence are confirmed or disconfirmed in court. Those models that are more heavily disconfirmed are assigned lower weights, as model confidence measures, in the Bayesian model comparison and averaging approach adopted. In this way plurality of arguments are allowed yet a single judgement based on all arguments is possible and rational.

Authors: Prof. Martin Neil (Queen Mary, University of London), Prof. Norman Fenton (Queen Mary, University of London), Prof David Lagnado (UCL), and Prof Richard Gill
(Leiden University)

12:15 to 13:30 Lunch @ INI
13:30 to 14:15 Jacob de Zoete (Universiteit van Amsterdam)
Generating Bayesian networks in Forensic Science An example from crime linkage
Co-author: Marjan Sjerps (Netherlands Forensic Institute)

The likelihood ratio framework for evaluating evidence is becoming more common in forensic practice. As a result, the interest in Bayesian networks as a tool to analyse cases and performing computations has increased. However, constructing a Bayesian network from scratch for every situation that one encounters is too costly. Therefore, several researchers have proposed Bayesian networks that correspond with frequent problems [1,2]. These `building blocks' allow the user to only concentrate on the conditional probabilities that fit their particular situation. This results in a more efficient workflow: the effort to construct the Bayesian network is taken away. Furthermore, it is no longer necessary that the user is experienced in constructing Bayesian networks. However, when the problem does not follow the `exact' assumptions of the building block, the Bayesian network can only serve as a starting point when constructing a model that does. In some situations, it is clear how one should model a certain problem, regardless of the case specific details. For example, a Bayesian network for a source level hypotheses pair where the evidence consists of a DNA profile has the same structure for any number of loci. Each locus can be added as a node together with it's corresponding drop-out/drop-in probabilities. For these type of problems, one can take away the effort of constructing the network. This facilitates the practical application of Bayesian networks for forensic casework. We will show an example of `generating Bayesian networks' for a problem from crime linkage. In [3] a structure for modeling crime linkage with Bayesian networks is introduced. This structure is implemented in R which allows the user to insert the parameters corresponding to their situation (e.g. the number of crimes/number of different types of evidence). Subsequently, this network can be used to obtain posterior probabilities or likelihood ratios. We will show how this is useful in casework.
14:15 to 15:00 Marjan Sjerps (Universiteit van Amsterdam)
DNA myth busting with Bayesian networks

co-authors J de Koeijer, J de Zoete and B Kokshoorn 

The Netherlands Forensic Institute is currently exploring the use of Bayesian networks in their forensic casework. We identified a number of different ways that networks can be used, e.g., as a probability calculator, as an exploratory tool for complex problems and as a reasoning tool. In this presentation we focus on the latter and discuss a recent case in which Bayesian networks were used in this way. The case concerned a series of six different robberies in which several DNA matches with a suspect were found on “movable objects” in each case. We were asked to assess the evidential value of the combined DNA evidence. Bayesian networks proved to be very valuable to assist in our reasoning and in busting a few important DNA myths that are common in the legal domain. We chose not to report the networks themselves but only the reasoning, and explain why. 
15:00 to 15:30 Afternoon Tea
15:30 to 16:15 Maria Cuellar (Carnegie Mellon University)
Shaken Baby Syndrome on Trial: Causal Problems and Sources of Bias
Over 1,100 individuals are in prison today on charges related to the diagnosis of Shaken Baby Syndrome (SBS). In recent years this diagnosis has come under scrutiny, and more than 20 convictions made on the basis of SBS have been overturned. The overturned convictions have fueled a controversy about alleged cases of SBS. In this talk, I will review the arguments made by the prosecution and defense in cases related to SBS and point out two problems: much of the evidence used has contextual bias, and the expert witnesses and attorneys ask the wrong causal questions. To resolve the problem of asking the wrong causal questions, I suggest that a Causes of Effects framework be used in formulating the causal questions and answers given by attorneys and expert witnesses. To resolve the problem of bias, I suggest that only the task-relevant information be provided to the individual who determines the diagnosis. I also suggest that in order for this to be possible, there must be a change in the definition of SBS so it does not include the manner in which the injuries were caused. I close with recommendations to researchers in statistics and the law about how to use scientific results in court.
16:15 to 17:00 Bart Verheij (University of Groningen); Henry Prakken (Universiteit Utrecht); (University of Groningen)
Report on a pre-workshop on analysing a murder case
Floris Bex, Anne Ruth Mackor, Henry Prakken, Christian Dahlman, Martin Neil, Bart Verheij

We will discuss the results of a pre-workshop held at the INI on September 23-24. During this workshop a Dutch murder case was analyzed from several theoretical perspectives, including Bayesian, argumentative and narrative approaches. 
19:30 to 22:00 Formal Dinner at Christ's College
Thursday 29th September 2016
09:30 to 10:15 Amanda Luby (Carnegie Mellon University)
A Graphical Model Approach to Eyewitness Identification Data
Although eyewitness identification is generally regarded as relatively inaccurate among cognitive psychologists and other experts, testimony from eyewitnesses continues to be prolific in the court system today. There is great interest among psychologists and the criminal justice system to reform eyewitness identification procedures to make the outcomes as accurate as possible. There has been a recent push to adopt Receiver Operating Characteristic (ROC) curve methodology to analyze lineup procedures, but has not been universally accepted in the field. This work addresses some of the shortcomings of the ROC approach and proposes an analytical approach based on log-linear models as an alternative method to evaluate lineup procedures. Unlike approaches that emphasize correct and incorrect identifications and rejections, our log-linear model approach can distinguish among all possible outcomes and allows for a more complete understanding of the variables at work during a lineup task. Due to the high-dimensional nature of the resulting model, representing the results through a dependence graph leads to a deeper understanding of conditional dependencies and causal relationships between variables involved. We believe that graphical models have been under-utilized in the field, and demonstrate their utility for not only broader statistical insights, but as an intuitive way to communicate complex relationships between variables to practitioners. We find that log-linear models can incorporate more information than previous approaches, and provide flexibility needed for data of this nature
10:15 to 11:00 Ulrike Hahn (University of London)
Bayesian Argumentation in the Real World
The talk provides a brief introduction to Bayesian Argumentation, including work on fallacies of argumentation (both inside and outside the law) and then focusses on the scope for Bayesian Argumentation and the normative considerations it provides in actual real world contexts, such as legal decisions. 
11:00 to 11:30 Morning Coffee
11:30 to 12:15 David Lagnado (University College London)
Causal networks in evidential reasoning
How do people reason in the face of complex and contradictory evidence? Focusing on investigative and legal contexts, we present an idiom-based approach to evidential reasoning, in which people combine and reuse causal schemas to capture large bodies of interrelated evidence and hypotheses. We examine both the normative and descriptive status of this framework, illustrating with real legal cases and empirical studies. We also argue that it is qualitative causal reasoning, rather than fully Bayesian computation, that lies at the heart of human evidential reasoning. 
12:15 to 13:30 Lunch @ INI
13:30 to 14:15 Christian Dahlman (Lund University)
Prior Probability and the Presumption of Innocence

My talk will address a problem of fundamental importance for the Bayesian approach to evidence assessment in criminal cases. How shall a court, operating under the presumption of innocence, determine the prior probability that the defendant is guilty, before the evidence has been presented? I will examine some ways to approach this problem, and review different solutions. The considerations that determine the prior probability can be epistemic or normative. If they are purely epistemic, the fact that the defendant has been selected for prosecution must be considered as evidence for guilt, and this violates the presumption of innocence (Dawid 1993, 12). The prior probability must therefore be determined completely or partly on normative grounds. It has been suggested by Dennis Lindley and others that the prior probability shall be determined as 1/N, where N is the number of people who could have committed the act that the defendant is accused of (Lindley 1977, 218; Dawid 1993, 11; Bender & Nack 1995, 236), but there are several objections to this solution. As Leonard Jaffee has pointed out, the prior probability will not be equal in all criminal trials, as N will vary from case to case (Jaffee 1988, 978). This is problematic since the doctrine of fair trial requires that defendants are treated equally. Furthermore, the court will not have sufficient knowledge about all possible scenarios to determine N with the robustness required by the standard of proof (Dahlman, Wahlberg & Sarwar 2015, 19). My suggestion for the problem is that the prior probability should be determined completely on normative grounds, by assigning a standardized number to N, for example N = 100. If the number of people who could have committed the crime is always presumed to be 100, the probability that the defendant is guilty before the evidence has been presented will be 1% in all trials. According to this solution, the prior probability is an institutional fact (Searle 1995, 104).

14:15 to 15:00 Mehera San Roque (University of New South Wales)
Admissibility and evaluation of identification evidence: 'experts', bystanders and shapeshifters
15:00 to 15:30 Afternoon Tea
15:30 to 16:15 Simon De Smet (University of Cambridge)
16:15 to 17:00 William Thompson (University of California, Irvine)
Using Bayesian Networks to Analyze What Experts Need to Know (and When they Know Too Much)
What is the proper evidentiary basis for an expert opinion? When do procedures designed to reduce bias (e.g., blinding, sequential unmasking) hide too much from the expert; when do they hide too little? When will an expert’s opinion be enhanced, and when might it be degraded or biased, by the expert’s consideration of contextual information? Questions like this are important in a variety of domains in which decision makers rely on expert analysis or opinion. This presentation will discuss the use of conditional probabilities and Bayesian networks to analyze these questions, providing examples from forensic science, security analysis, and clinical medicine. It will include discussion of the recommendations of the U.S. National Commission on Forensic Science on determining the “task-relevance” of information needed for forensic science assessments. 
University of Cambridge Research Councils UK
    Clay Mathematics Institute London Mathematical Society NM Rothschild and Sons