The stability of rock masses is influenced by the occurrence of rock joints. Therefore, the shear strength of rock joints must be considered in dimensioning of underground constructions. One way to predict the shear strength is through usage of failure criteria, which are validated from results of direct shear tests under controlled laboratory conditions. Consequently, the quality of the results from the tests are crucial to the accuracy with which the criteria will be able to predict the shear strength. Since rock joints are unique by nature usage of replicas (man-made copies of rock joints) is of importance in parameter studies. The overall objective of this work is to facilitate the development of improved criteria for predictions of the shear strength of rock joints. To support this objective, two sources of uncertainty have been investigated, namely the geometry of replicas and the influence of the normal stiffness of test systems. Two quality assurance parameters for evaluation of geometrical differences between replicas and rock joints based on scanning data have been derived. The first parameter describes the morphological deviations. The second parameter describes the deviations in orientation with respect to the shear plane. The effective normal stiffness approach, which compensates for the influence of the normal stiffness of the test system in direct shear testing, has been developed, validated, and applied. With help of the quality assurance parameters it is demonstrated that it is possible to reproduce replicas within narrow tolerances. Application of the effective normal stiffness approach basically eliminates the normal load error. In all, the results support generation of improved quality of test data and consequently, the development of shear strength criteria with improved accuracy will also be facilitated.