[c]

Professor Stewart Robinson, Dr Katy Hoad, Professor Ruth Davies

Figure 1: Overview of the Automated Analysis Procedure

 

Appropriate use of a simulation model requires accurate measures of model performance.  This in turn, requires decisions concerning three key areas: warm-up, run-length and number of replications.  Simulation software, however, gives little or no guidance in making these decisions.  A three year project, "Automatic Simulation Output Analysis", sponsored by the EPSRC (Grant ref: EP/D033640/1) and SIMUL8 and undertaken by Warwick Business School, is investigating the development of a methodology for automatically advising a simulation user on these three decisions.  This web site outlines the approach being taken, discusses progress to-date and provides links for research outputs.

---

Obtaining Accurate Simulation Results

Even with the best constructed simulation model if it is not used properly it is likely that the results will be inaccurate.  If the results from a simulation of a system are not accurate they will not provide a good prediction of what would happen in the real life system even if the model itself is valid.  The results could be biased and misleading, with the consequence that incorrect conclusions could be drawn.

There are many decisions that need to be made about how a simulation model is to be run.  These include:

• Do I need to use a warm-up period; set initial conditions; or perhaps use a combination of both; or do I in fact want my simulation to start from empty ?
• If I do use a warm-up period how long should this period be ?
• Should I perform one long simulation run, or a number of independent runs (i.e. runs with different random streams, also called replications) ?
• How long should this one simulation run be ?
• How many replications is enough ?

In particular, the decisions on run-length, number of replications to perform in a trial and warm-up period require specific skills in statistics.   Because of the variety and complexity of statistical methods used to make informed decisions on these matters, at the moment, most simulation software provide little or no guidance to users on making these important decisions.  Because the setting of inappropriate run-lengths, run (replication) numbers and warm-up periods can lead to inaccurate and misleading results, it is considered important to deliver tools to simulation users that allow them to make informed decisions regarding these matters, whatever their statistical background.

It is with this in mind that Warwick Business School and SIMUL8 have collaborated on a project to create an automated tool that can analyse preliminary simulation output and hence provide recommendations on warm-up, run-length and number of replications. 

---

Refs:

• “Automated Analysis Of Simulation Output Data”, Stewart Robinson (Warwick Business School, University of Warwick, Coventry, CV4 7AL, UK), Proceedings of the 2005 Winter Simulation Conference.
• “A Statistical Process Control Approach for Estimating the Warm-up Period”, Stewart Robinson (Warwick Business School, University of Warwick, Coventry, CV4 7AL, UK), Proceedings of the 2002 Winter Simulation Conference.
• “Simulation: The Practice of Model Development and Use”, Stewart Robinson (Warwick Business School, University of Warwick, Coventry, CV4 7AL, UK), 2004, Wiley, Chichester, UK.
• “The Problem of the Initial Transient: Techniques for Estimating the Warm-Up Period for Discrete-Event Simulation Models”, Stewart Robinson (Warwick Business School, University of Warwick, Coventry, CV4 7AL, UK) and Agis Ioannou (Process Solutions Centre, BT Exact, Orion Building, Adastral Park, Martlesham Heath, Ipswich, IP5 3RE, UK).  (Not yet published)
• Full Project proposal

 

 

 

 

 

 

Project Personnel

Principal Web Site editor & Research Fellow: Dr Katy Hoad

Principal Investigator: Professor Stewart Robinson

e-mail: stewart robinson@warwick.ac.uk

---

Co-Investigator: Professor Ruth Davies

---

Project Partner: SIMUL8 Corporation

---