# Workshop Programme

## for period 21 - 25 July 2008

### Advanced topics in Design of Experiments

21 - 25 July 2008

Timetable

Monday 21 July | ||||

08:30-11:00 | Registration | |||

11:00-11:30 | Coffee | |||

12:30-13:30 | Lunch at Wolfson Court | |||

14:00-15:30 | Schwabe, R (Otto von Guericke University Magdeburg) |
|||

B2 Optimal design for linear and non-linear models | Sem 2 | |||

This tutorial is meant to provide an introductory course to basic principles in the theory of optimal design of experiments. We start with a general linear model setup, which includes regression and analysis of variance as special cases as well as more sophisticated functional relationships. The objective of optimal design is to determine experimental settings which optimize the quality of statistical analyses to be performed after data collection. For measuring the quality the information matrix plays a crucial role. In order to make the information comparable obtained from competing designs, various design criteria are introduced. The design problem is embedded in a convex optimization setting ("approximate theory"), which allows for characterizations of optimal designs by so-called equivalence theorems. This approach also provides numerical algorithms to generate nearly optimal designs and bounds for measuring the efficiency of a given design. In this context common principles like invariance ("symmetry") and majorization can be applied to reduce the complexity of the optimization problem. In non-linear models usually the quality of the statistical analyses can only be measured by asymptotic criteria. To this end the information matrix is either obtained from a linearized model or, if available, as the exact Fisher information matrix, which is proportional to the inverse of the asymptotic covariance matrix of the maximum likelihood estimator. If appropriate, estimation equations (e.g. quasi-likelihood) may be used instead. The coincidence of these approaches is established for the class of generalized linear models including logistic and Poisson regression. Due to the non-linearity optimal designs may depend on the true values of the parameters, i.e. on the true shape of the response. To avoid this complication weighted and minimax criteria are proposed. While so far the theory was developed for models, in which only fixed effects are present, we introduce some simple random effects, as they typically occur in statistical applications in biosciences, where individuals (humans or animals) are involved and multiple measurements are obtained from these individuals. This means, in general, that each individual has its own mean response curve, which randomly deviates from a mean population response curve. Starting from models with random intercept we switch over to general random coefficients and derive optimality concepts for population parameters ("averaged" over the individuals), for prediction of individual response curves and for the analysis of the variance components. We end up with some critical remarks on frequently used optimality criteria. |
||||

15:30-16:00 | Tea | |||

16:00-17:30 | Brien, CJ (University of South Australia) |
|||

B3 Multi-tiered experiments | Sem 2 | |||

Experiments that involve multiple randomizations [Brien and Bailey (2006) Multiple randomizations (with discussion). Journal of the Royal Statistical Society, Series B, 68, 571-609.] are termed multitiered. Two-phase, and multiphase, experiments are an important group of multitiered experiments that are widely applicable, including to agriculture, food processing, industrial, medical and microarray experiments. Also, some grazing, plant, superimposed, human interaction and multistage reprocessing experiments are multitiered. At the outset the difference between multitiered designs and standard textbook designs, almost all of which are two-tiered, will be discussed. An overview will be provided of the use of the different types of multiple randomizations in designs for multitiered experiments and of the evaluation of the properties of such designs, principally the confounding entailed. The concepts and methods required to do this will be introduced in the simpler and more familiar context of standard textbook designs. Techniques for analysing the results of multitiered experiments will be suggested. A wide range of examples of multitiered experiments will be presented. |

Tuesday 22 July | ||||

09:00-11:00 | Gilmour, SG (Queen Mary, University of London) |
|||

B1 Multi-stratum experiments | Sem 2 | |||

This course will cover the design and analysis of split-plot, and more general multi-stratum, experiments. Applications with factors which are hard to change will motivate the general ideas. As well as standard multi-stratum structures, we will describe recent research in regular fractional factorial structures, robust product design, response surface designs and experiments with mixtures. Outline 1. Standard multi-stratum structures: randomization, blocking and strata; applications with factors which are hard to change, restrictions on randomization; model-based approaches; choice of design; combination of information. 2. Regular fractional factorial structures: fraction in top stratum; fraction in bottom stratum; graphical analyses for unreplicated designs; robust product experimentation; joint modelling of mean and dispersion; fractions in more than one stratum. 3. Irregular fractional factorial structures: multi-stratum response surface designs; experiments with mixtures; mixed models and REML/GLS analysis; other approaches to data analysis; randomized-not-reset factors. |
||||

11:00-11:30 | Coffee | |||

11:30-12:30 | Wynn, H (London School of Economics) |
|||

A2 Computer Experiments | Sem 2 | |||

This is 2 x 2 hours sessions run by Henry Wynn and Peter Challenor. Session 1 (H. Wynn). Introduction to Computer Experiments. Gaussian process emulators, basic theory. Bayes versus BLUE estimation. Effects plots. Types of covariance function. Low discrepancy sequences. Maximum entropy sampling. Uses: modelling, optimisation, sensitivity analysis. |
||||

12:30-13:30 | Lunch at Wolfson Court | |||

14:00-15:00 | Wynn, H (London School of Economics) |
|||

A2 Computer Experiments | Sem 2 | |||

This is 2 x 2 hours sessions run by Henry Wynn and Peter Challenor. Session 1 (H. Wynn). Introduction to Computer Experiments. Gaussian process emulators, basic theory. Bayes versus BLUE estimation. Effects plots. Types of covariance function. Low discrepancy sequences. Maximum entropy sampling. Uses: modelling, optimisation, sensitivity analysis. |
||||

15:00-15:30 | Tea | |||

15:30-17:30 | Challenor, P (University of Southampton) |
|||

A2 Computer Experiments | Sem 2 | |||

This is 2 x 2 hours sessions run by Henry Wynn and Peter Challenor. Session 2 (P. Challenor). Climate model simulators. Computer experiments in climate modelling. Use of Gaussian process emulators. Special problems in experimental design: sequential design, estimation of scale parameters. Design of ensemble experiments. |
||||

17:30-18:30 | Wine and Cheese Reception |

Wednesday 23 July | ||||

09:00-11:00 | Schwabe, R (Otto von Guericke University Magdeburg) |
|||

B2 Optimal design for linear and non-linear models | Sem 2 | |||

This tutorial is meant to provide an introductory course to basic principles in the theory of optimal design of experiments. We start with a general linear model setup, which includes regression and analysis of variance as special cases as well as more sophisticated functional relationships. The objective of optimal design is to determine experimental settings which optimize the quality of statistical analyses to be performed after data collection. For measuring the quality the information matrix plays a crucial role. In order to make the information comparable obtained from competing designs, various design criteria are introduced. The design problem is embedded in a convex optimization setting ("approximate theory"), which allows for characterizations of optimal designs by so-called equivalence theorems. This approach also provides numerical algorithms to generate nearly optimal designs and bounds for measuring the efficiency of a given design. In this context common principles like invariance ("symmetry") and majorization can be applied to reduce the complexity of the optimization problem. In non-linear models usually the quality of the statistical analyses can only be measured by asymptotic criteria. To this end the information matrix is either obtained from a linearized model or, if available, as the exact Fisher information matrix, which is proportional to the inverse of the asymptotic covariance matrix of the maximum likelihood estimator. If appropriate, estimation equations (e.g. quasi-likelihood) may be used instead. The coincidence of these approaches is established for the class of generalized linear models including logistic and Poisson regression. Due to the non-linearity optimal designs may depend on the true values of the parameters, i.e. on the true shape of the response. To avoid this complication weighted and minimax criteria are proposed. While so far the theory was developed for models, in which only fixed effects are present, we introduce some simple random effects, as they typically occur in statistical applications in biosciences, where individuals (humans or animals) are involved and multiple measurements are obtained from these individuals. This means, in general, that each individual has its own mean response curve, which randomly deviates from a mean population response curve. Starting from models with random intercept we switch over to general random coefficients and derive optimality concepts for population parameters ("averaged" over the individuals), for prediction of individual response curves and for the analysis of the variance components. We end up with some critical remarks on frequently used optimality criteria. |
||||

11:00-11:30 | Coffee | |||

11:30-12:30 | Speed, TP (University of California at Berkeley) |
|||

A1 Experiments in genomics and proteomics | Sem 2 | |||

This will be four one-hour sessions by Terry Speed, James Brenton, Julian Griffin and Andy Lynch. Session 1 (TP Speed) Introduction and elementary overview of experiments in genomics and proteomics. For most experiments in this area, we can recognize three main phases that are relevant to experimental design. I: choice and preparation of experimental material, choice of platform technology; II: assignment of experimental reagents to components of the technology; III: actual conduct of the experiment, including times, places and conditions of experiment, and protocols, reagents, operators and equipment used. In this session we will mainly focus on examples where the experiments were poorly designed and led to unsatisfactory outcomes. This will show the need for good design. |
||||

12:30-13:30 | Lunch at Wolfson Court | |||

14:00-15:00 | Brenton, JD (University of Cambridge) |
|||

A1 Experiments in genomics and proteomics | Sem 2 | |||

This will be four one-hour sessions by Terry Speed, James Brenton, Julian Griffin and Andy Lynch. Session 2 (JD Brenton) This session will describe the science and technology of experiments in genomics from the viewpoint of a principal investigator. |
||||

15:00-15:30 | Tea | |||

15:30-16:30 | Griffin, JL (University of Cambridge) |
|||

A1 Experiments in genomics and proteomics | Sem 2 | |||

This will be four one-hour sessions by Terry Speed, James Brenton, Julian Griffin and Andy Lynch. Session 3 (JL Griffin) This session will describe the science and technology of experiments in proteomics and metabolomics from the viewpoint of a principal investigator. |
||||

16:30-17:30 | Lynch, AG (University of Cambridge) |
|||

A1 Experiments in genomics and proteomics | Sem 2 | |||

This will be four one-hour sessions by Terry Speed, James Brenton, Julian Griffin and Andy Lynch. Session 4 (AG Lynch) In this final introductory session, we will turn to specifics, here of the Illumina BeadArray platform, and set the scene for discussions later on in the workshop. After a brief description of the platform, key issues such as platform choice, the role of replication, randomisation and local control, validation, optimality and robustness will be discussed. |

Thursday 24 July | ||||

09:00-11:00 | Brien, CJ (University of South Australia) |
|||

B3 Multi-tiered experiments | Sem 2 | |||

Experiments that involve multiple randomizations [Brien and Bailey (2006) Multiple randomizations (with discussion). Journal of the Royal Statistical Society, Series B, 68, 571-609.] are termed multitiered. Two-phase, and multiphase, experiments are an important group of multitiered experiments that are widely applicable, including to agriculture, food processing, industrial, medical and microarray experiments. Also, some grazing, plant, superimposed, human interaction and multistage reprocessing experiments are multitiered. At the outset the difference between multitiered designs and standard textbook designs, almost all of which are two-tiered, will be discussed. An overview will be provided of the use of the different types of multiple randomizations in designs for multitiered experiments and of the evaluation of the properties of such designs, principally the confounding entailed. The concepts and methods required to do this will be introduced in the simpler and more familiar context of standard textbook designs. Techniques for analysing the results of multitiered experiments will be suggested. A wide range of examples of multitiered experiments will be presented. |
||||

11:00-11:30 | Coffee | |||

11:30-12:30 | Senn, S (University of Glasgow) |
|||

A3 Clinical Trials | Sem 2 | |||

Stephen Senn will discuss lost opportunities for the application of design theory in drug development giving examples where theoreticians have not aid sufficient attention to design realities but also cases where practitioners have missed applying useful theory. Michael Krams will cover the 'hot topic' of adaptive design using various case studies, illustrating these from many perspectives, including those of investigators, researchers, patients and data monitoring committees. The objective of this session will be to illustrate areas in which design theory can make a real contribution to drug development provided that genuine collaboration with practitioners is sought. |
||||

12:30-13:30 | Lunch at Wolfson Court | |||

14:00-14:30 | Senn, S (University of Glasgow) |
|||

A3 Clinical Trials | Sem 2 | |||

Stephen Senn will discuss lost opportunities for the application of design theory in drug development giving examples where theoreticians have not aid sufficient attention to design realities but also cases where practitioners have missed applying useful theory. Michael Krams will cover the 'hot topic' of adaptive design using various case studies, illustrating these from many perspectives, including those of investigators, researchers, patients and data monitoring committees. The objective of this session will be to illustrate areas in which design theory can make a real contribution to drug development provided that genuine collaboration with practitioners is sought. |
||||

14:30-15:00 | Krams, M (Wyeth Pharmaceuticals) |
|||

A3 Clinical Trials | Sem 2 | |||

Stephen Senn will discuss lost opportunities for the application of design theory in drug development giving examples where theoreticians have not aid sufficient attention to design realities but also cases where practitioners have missed applying useful theory. Michael Krams will cover the 'hot topic' of adaptive design using various case studies, illustrating these from many perspectives, including those of investigators, researchers, patients and data monitoring committees. The objective of this session will be to illustrate areas in which design theory can make a real contribution to drug development provided that genuine collaboration with practitioners is sought. |
||||

15:00-15:30 | Tea | |||

15:30-16:30 | Krams, M (Wyeth Pharmaceuticals) |
|||

A3 Clinical Trials | Sem 2 | |||

16:30-17:30 | Krams, M; Senn, S (Wyeth Pharmaceuticals / Univ of Glasgow) |
|||

A3 Clinical Trials | Sem 2 |

Friday 25 July | ||||

09:00-11:00 | Gilmour, SG (Queen Mary, University of London) |
|||

B1 Multi-stratum experiments | Sem 2 | |||

This course will cover the design and analysis of split-plot, and more general multi-stratum, experiments. Applications with factors which are hard to change will motivate the general ideas. As well as standard multi-stratum structures, we will describe recent research in regular fractional factorial structures, robust product design, response surface designs and experiments with mixtures. Outline 1. Standard multi-stratum structures: randomization, blocking and strata; applications with factors which are hard to change, restrictions on randomization; model-based approaches; choice of design; combination of information. 2. Regular fractional factorial structures: fraction in top stratum; fraction in bottom stratum; graphical analyses for unreplicated designs; robust product experimentation; joint modelling of mean and dispersion; fractions in more than one stratum. 3. Irregular fractional factorial structures: multi-stratum response surface designs; experiments with mixtures; mixed models and REML/GLS analysis; other approaches to data analysis; randomized-not-reset factors. |
||||

11:00-11:30 | Coffee | |||

11:30-12:30 | Schwabe, R (Otto von Guericke University Magdeburg) |
|||

B2 Optimal design for linear and non-linear models | Sem 2 | |||

This tutorial is meant to provide an introductory course to basic principles in the theory of optimal design of experiments. We start with a general linear model setup, which includes regression and analysis of variance as special cases as well as more sophisticated functional relationships. The objective of optimal design is to determine experimental settings which optimize the quality of statistical analyses to be performed after data collection. For measuring the quality the information matrix plays a crucial role. In order to make the information comparable obtained from competing designs, various design criteria are introduced. The design problem is embedded in a convex optimization setting ("approximate theory"), which allows for characterizations of optimal designs by so-called equivalence theorems. This approach also provides numerical algorithms to generate nearly optimal designs and bounds for measuring the efficiency of a given design. In this context common principles like invariance ("symmetry") and majorization can be applied to reduce the complexity of the optimization problem. In non-linear models usually the quality of the statistical analyses can only be measured by asymptotic criteria. To this end the information matrix is either obtained from a linearized model or, if available, as the exact Fisher information matrix, which is proportional to the inverse of the asymptotic covariance matrix of the maximum likelihood estimator. If appropriate, estimation equations (e.g. quasi-likelihood) may be used instead. The coincidence of these approaches is established for the class of generalized linear models including logistic and Poisson regression. Due to the non-linearity optimal designs may depend on the true values of the parameters, i.e. on the true shape of the response. To avoid this complication weighted and minimax criteria are proposed. While so far the theory was developed for models, in which only fixed effects are present, we introduce some simple random effects, as they typically occur in statistical applications in biosciences, where individuals (humans or animals) are involved and multiple measurements are obtained from these individuals. This means, in general, that each individual has its own mean response curve, which randomly deviates from a mean population response curve. Starting from models with random intercept we switch over to general random coefficients and derive optimality concepts for population parameters ("averaged" over the individuals), for prediction of individual response curves and for the analysis of the variance components. We end up with some critical remarks on frequently used optimality criteria. |
||||

12:30-13:30 | Lunch at Wolfson Court | |||

14:00-15:00 | Brien, CJ (University of South Australia) |
|||

B3 Multi-tiered experiments | Sem 2 | |||

Experiments that involve multiple randomizations [Brien and Bailey (2006) Multiple randomizations (with discussion). Journal of the Royal Statistical Society, Series B, 68, 571-609.] are termed multitiered. Two-phase, and multiphase, experiments are an important group of multitiered experiments that are widely applicable, including to agriculture, food processing, industrial, medical and microarray experiments. Also, some grazing, plant, superimposed, human interaction and multistage reprocessing experiments are multitiered. At the outset the difference between multitiered designs and standard textbook designs, almost all of which are two-tiered, will be discussed. An overview will be provided of the use of the different types of multiple randomizations in designs for multitiered experiments and of the evaluation of the properties of such designs, principally the confounding entailed. The concepts and methods required to do this will be introduced in the simpler and more familiar context of standard textbook designs. Techniques for analysing the results of multitiered experiments will be suggested. A wide range of examples of multitiered experiments will be presented. |
||||

15:00-15:30 | Tea | |||

15:30-17:30 | Gilmour, SG (Queen Mary, University of London) |
|||

B1 Multi-stratum experiments | Sem 2 | |||

This course will cover the design and analysis of split-plot, and more general multi-stratum, experiments. Applications with factors which are hard to change will motivate the general ideas. As well as standard multi-stratum structures, we will describe recent research in regular fractional factorial structures, robust product design, response surface designs and experiments with mixtures. Outline 1. Standard multi-stratum structures: randomization, blocking and strata; applications with factors which are hard to change, restrictions on randomization; model-based approaches; choice of design; combination of information. 2. Regular fractional factorial structures: fraction in top stratum; fraction in bottom stratum; graphical analyses for unreplicated designs; robust product experimentation; joint modelling of mean and dispersion; fractions in more than one stratum. 3. Irregular fractional factorial structures: multi-stratum response surface designs; experiments with mixtures; mixed models and REML/GLS analysis; other approaches to data analysis; randomized-not-reset factors. |