Nine participants, representing eight different Swedish fire consultancy firms participated in a Round Robin study where two different cases were simulated with the Fire Dynamics Simulator. The first case included a large open warehouse where the activation of a sprinkler system was to be studied. In the second case time to critical conditions in a theatre was to be calculated. The participants were given clear instructions on the building layout and heat release rate for the two cases. Still, the results demonstrate a significant variation in time to sprinkler system activation (range of 110 seconds) and available safe escape time (range of 60 seconds), between the participants. It is important to emphasise that some degree of variation is unavoidable, as engineers can model things differently without the modelling solution necessarily being incorrect. Even though it is hard to isolate and specific cause of the variation, some of the variation seen in this study is related to modelling choices that are questionable and consequently problematic for the reliability of the fire safety design. © 2020 The Authors.