Aim: This paper aims to report on two modifications made to improve the measurement precision of the Ms. Olsen test. Specifically, three items were added to the Ms. Olsen test and an extended scoring was applied to some items. Design: The competence assessment had a quantitative cross-sectional design comprising of test results from 111 Registered Nurses in four municipalities in South-Eastern Norway. Methods: The Rasch model was applied to evaluate the measurement properties in four versions: the Ms. Olsen test with 19 items and the same with three added items, and the Ms. Olsen test with 19 items with an expanded scoring for the original and expanded version, respectively. Results: The person separation indexes were improved from 0.50 to 0.62. Other measurement properties were not alternating between the four versions; all had shortcomings in terms of targeting (person measure means 3.02–3.87) and unidimensionality (% t-test >5% 9.01%–13.51%). Conclusion: The clinical relevance and relatively short time spent on testing makes the Ms. Olsen test a reasonable choice and a step in the right direction for assessing competence as a means of targeting continuous professional development of nurses throughout their career. Nevertheless, depending on what kind of decisions are to be made, reliability might still be too low and further development is suggested. Public Contribution: The initiative for the competence assessment came from nursing leaders in the four municipalities involved. The municipalities were represented in all stages of the research process through co-author LS, that is, design, data collection, data analysis, writing of the manuscript as well as dissemination of the results to the four municipalities. Implications for Practice: The Ms. Olsen test exemplifies a move towards objective assessments in nursing. Precise and reliable measurements are essential to support the ongoing professional development of nurses.