|
Welcome to this month's newsletter! In this issue you'll find:
Testing Simulations by Helen Cook
One of the most important stages of simulation development is the
testing stage. This stage ensures that it meets the specifications
set by the client and also that it provides the client with reliable
and accurate results. No matter how much effort was put into planning
and building the simulation, it is always necessary to carry out
some testing. This is particularly true for large or complex simulations.
Testing can be broken down into 2 activities: Validation and Verification:
These are, in principle, distinct but can often be confused.
Validation -
The validation stage of testing involves ensuring that you have
solved the correct question. You need to ensure that the simulation
you have built behaves in the same way as the real-life system you
intended to simulate.
Verification -
Verification involves making sure that the simulation you have built
is correct - i.e. that there are no mistakes in your logic (this
is particularly the case for simulations models that involve Visual
Logic programming).
The following are some tips to help you verify and validate your
simulations:
Verification
- Change one variable at a time.
This will ensure that each time you make a change to your simulation,
you know which variable was responsible for a change in output.
For example, if I want to look at the impact of number of resources
and number of workstations, I would firstly examine one of these
variables and then the other, so that I could tell if number of
resources had more or less of an impact on output than number
of workstations.
- Run trials.
If you have randomness in your simulation, running trials rather
than single runs will give you more accurate results, and also
give you confidence intervals which may help you judge the accuracy
of your results.
- Monitor your simulation.
Use the watch window facility to view the state of any part of
your simulation at any time.
- Keep a record.
Try to keep a detailed record of the tests you have carried out,
what the outputs were and other details such as length of run,
number of runs in trial, so that your results can be replicated
and your tests repeated. The compare function in results can be
used to record results from each run, and these can be copied
into Excel for analysis. The screenshot below shows how the compare
function can be used.
- If in doubt: Simplify!
If you are unsure as to the accuracy of your simulation, try simplifying
it by, for example, putting simple figures in, or taking out resources,
so that your results are easier to understand. Complexity can
then be gradually built back.
- What to test?
Tests should be designed according to the types of things the
simulation is intended to do. For example, if your simulation
involves labels, you should carry out some tests to make sure
the labeling is working properly.
- Helping Hand.
Get someone else to test your simulation for usability and logic
- sometimes you can get too close to your simulation to see what's
going wrong!!
Validation
- Involve your client.
If it is possible for you to involve your client in the development
process, you can ensure the output of your simulation is what
he/she is expecting and allow them to participate in the decision
making process.
- Work it out by hand.
If you are not sure what the output from your simulation should
be, try building a spreadsheet model to work out, for example,
throughput, mathematically so that you can compare this figure
with your simulation's output. Simulation will be more accurate,
but the spreadsheet may be able to give you the right order of
magnitude and it will help you think through the logic.
More Reading
If you would like to learn more about validating and verifying
simulations, you can find back issues of the newsletter referring
to this subject on the SIMUL8
website. Also related to testing is debugging, which also appears
as the subject of a past newsletter.
You can also refer to Law and Kelton's book called "Simulation
Modeling and Analysis", Third Edition, McGraw Hill, and Stewart
Robinson's book "Simulation" (2004, John Wiley & Sons)
which discusses development in general and has a section on simplicity.
SIMUL8 Hints and Tips
Copying and Pasting Visual Logic
You can use the window's shortcut keys Ctrl C and Ctrl V to quickly
copy and paste Visual Logic code. If you select the top line in
a nested section, for example the first line of an IF statement,
everything nested below will be copied as well.
Go To Simulation Time
You can quickly fast forward your simulation to an exact time using
the "Go To Simulation Time" feature. You'll find this
under the Clock menu. This feature is really useful when you're
debugging a simulation.
SIMUL8 Master Classes
Master Classes give you
a chance to get access to our most experienced consultants. You
learn lots of non-documented hints and tips and get a chance to
make suggestions on how we can improve SIMUL8. Master Classes cover
different technical support topics each month and include a Question
and Answer session.
Master Classes are free
and are conducted over the internet via WEBEX Conferencing software.
The next scheduled Master Class is on 3rd September 10am (EST) and
will be presented by Mark Elder our CEO who has over 20 years experience
in simulation modeling. The topic will be 'Debugging Simulations'.
See our support website
for more details
and to sign up.
If you have any questions generated from, or comments about this
newsletter then email Newsletter@SIMUL8.com
|
|
