Psychometrics have long relied on rule-of-thumb critical values for goodness of fit metrics. With powerful personal computers it is both feasible and desirable to use simulation methods to determine appropriate cutoff values. This paper evaluates the use of an R package for Rasch psychometrics that has implemented functions to simplify the process of determining simulation-based cutoff values. Through six simulation studies, comparisons are made between information-weighted conditional item fit (“infit”) and item-restscore correlations using Goodman and Kruskal’s γ. Results indicate the limitations of small samples (n < 500) in correctly detecting item misfit, especially when a larger proportion of items are misfit and/or when misfit items are off-target. Infit with simulation-based cutoffs outperforms item-restscore with sample sizes below 500. Both methods result in problematic rates of false positives with large samples (n >= 1000). Large datasets should be analyzed using nonparametric bootstrap of subsamples with item-restscore to reduce the risk of type-1 errors. Finally, the importance of an iterative analysis process is emphasized, since a situation where several items show underfit will cause other items to show overfit. Underfit items should be removed one at a time, and a re-analysis conducted for each step to avoid erroneously eliminating items.

Depression is one of the most common psychiatric conditions. Given its high prevalence and disease burden, accurate diagnostic procedures and valid instruments are warranted to identify those in need of treatment. The Patient Health Questionnaire-9 (PHQ-9) is one of the most widely used self-report measures of depression, and its validity and reliability has been evaluated in several languages. However, the Swedish translation has yet not been subject to psychometric evaluation, and no previous psychometric evaluation of the instrument have used both Rasch analysis and classic test theory. The aim of this study was to investigate validity and reliability of the PHQ-9 in a Swedish sample of individuals with self-reported current or past mental health problems using Rasch analysis and confirmatory factor analysis. A sample of 4958 participants was recruited from an online survey covering different aspects of covid-19 and mental health targeted towards individuals with mental health problems. Results showed that a one-factor solution fit the data after removing one item and that some items had a different response pattern for older respondents. Furthermore, after removing item 2 and merging the two middle response categories for item 9, the Swedish translation of PHQ-9 showed adequate psychometric properties. The findings suggests that the Swedish translation of PHQ-9 may need to be adapted in order to make adequate interpretations of an individual score and to differentiate between populations. Further tests of its validity and reliability in other Swedish samples are needed to fully understand the properties of the Swedish version of PHQ-9. 

Background and Aim: To address knowledge gaps in safe patient handling and mobility methods (SPHMM) among rehabilitation professionals in Sweden, a national knowledge center, HMC, provides training on SPHMM. The study’s aim was to report on outcomes at 3 months after training in terms of self-perceived competence in SPHMM and self-perceived utility of acquired competence in SPHMM. Methods: Training participants (occupational therapists and physiotherapists) completed a study-specific questionnaire, with 1065 doing so at 3 weeks before the training and 389 at 3 months after training. Linear mixed models were used for analyses. Results: Self-perceived competence improved significantly at 3 months after training (p < 0.001). Occupational therapists, specifically, and rehabilitation professionals generally with less clinical experience reported greater improvement compared to physiotherapists and those with longer experience (p < 0.001). Rehabilitation professionals overall with shorter clinical experience also reported higher self-perceived utility of the acquired SPHMM competence (p < 0.001), but occupational therapists and physiotherapists did not differ significantly (p = 0.369). Conclusion: Among rehabilitation professionals, greater self-perceived competence and self-perceived utility of the acquired competence in SPHMM is possible after HMC training. Improvements were greater for those with shorter clinical experience. By enhancing competence among rehabilitation professionals, HMC training has the potential to advance working person-centered and efficient transfer practices in interdisciplinary care settings.

Background

Stress reflects physical and psychological reactions to imposing demands and is often measured using self-reports. A widely-used instrument is the Perceived Stress Scale (PSS), intended to capture more general aspects of stress. A Swedish translation of the PSS is available but has not previously been examined using modern test theory approaches. The aim of the current study is to apply Rasch analysis to further the understanding of the PSS’ measurement properties, and, in turn, improve its utility in different settings.

Methods

Data from 793 university students was used to investigate the dimensionality of different version of the PSS (14, 10, and 4 items) as well as potential response patterns among the participants.

Results

The current study demonstrates that the PSS-14 has two separate factors, divided between negatively worded items (perceived stress) and positively worded items (perceived [lack of] control), although with only the negative subscale exhibiting good reliability. Response patterns were analyzed using Differential Item Functioning, which did not find an influence of gender on any of the items, but for age regarding the positive subscale (items 6 and 9). The PSS-10 also demonstrated adequate reliability for the negative subscale, but the PSS-4 was not deemed suitable as a unidimensional scale.

Conclusions

Based on the results, none of the versions of the PSS should be used by sum-scoring all of the items. Only the negative items from the PSS-14 or PSS-10 can be used as unidimensional scales to measure general aspects of stress. As for different response patterns, gender may nevertheless be important to consider, as prior research has found differences on several items. Meanwhile, content validity is discussed, questioning the relevance of anger and being upset when measuring more general aspects of stress. Finally, a table to convert the PSS-7 (i.e., negative items) ordinal sum scores to interval level scores is provided.

Why?

Society's need for knowledge about experiences, feelings, behaviours and abilities is increasing. It is important to know how people and society feel or how a product or service is perceived, and to be able to compare how these change over time and between different areas, for example when making decisions about priorities, efforts and effects. In order for measurements of experiences, feelings, behaviours and abilities to be useful and provide reliable data, quality-assured measurement is required for socalled category based measurements1. It is therefore necessary that basic measurement principles are applied, which is often not the case today in this field. For the physical quantities (length, mass, time, etc.) there has long been an internationally established measurement quality infrastructure. In Sweden, it is mainly RISE as the National Metrological Institute which, through its national measurement sites, is responsible for ensuring access to quality-assured and traceable measurements for society. However, the corresponding measurement quality infrastructure is so far missing for category based measurements.

What?

RISE has been working for several years to develop methodology and working methods for measurement quality assurance of experiences, feelings, behaviours and abilities. Through dialogue over several years with actors at various levels, including in health care and academia, it has emerged that there is a great need for support for this development. RISE has therefore produced a first draft of a possible sustainable organization for measurement quality assurance of category based measurements. A national centre for quality assurance of category based measurements would be tasked with coordinating and ensuring reliable and comparable measurements of experiences, feelings, behaviours and abilities. Such infrastructural support must make methodologies available to enable quality assurance for category based measurements. The organization must meet society's need for support by offering services based on internationally accepted metrological principles and scientific methods, as well as adopting a neutral and independent role.

How?

We have humility about the fact that it is a long process to establish an organization for category based measurements and that it needs to happen through a dynamic and iterative development in co-creation with the actors involved. We look forward to continuing to develop ideas and revise the proposal in dialogue in the coming years. It is also important to start a dialogue together with relevant actors about how the implementation of the organization can take place in the future.

Varför?

Samhällets behov av kunskap om upplevelser, känslor, beteenden och förmågor ökar. Det är viktigt att veta hur personer och samhället mår eller hur en produkt eller tjänst upplevs samt att kunna jämföra hur detta förändras över tid och mellan olika områden, för att exempelvis fatta beslut om prioriteringar, insatser och effekter. För att mätningar av upplevelser, känslor, beteenden och förmågor ska vara användbara och ge tillförlitliga underlag krävs kvalitetssäkrad mätteknik för s.k. kategoribaserade mätningar. Det är alltså nödvändigt att grundläggande mättekniska principer tillämpas, vilket ofta inte är fallet i dag inom detta område. För de fysikaliska storheterna (längd, massa, tid etc.) finns sedan länge en internationellt etablerad mätkvalitetsinfrastruktur. I Sverige är det huvudsakligen RISE som Nationellt Metrologiskt Institut som genom sina riksmätplatser ansvarar för att säkerställa tillgång till kvalitetssäkrade och spårbara mätningar för samhället. Motsvarande mätkvalitetsinfrastruktur saknas dock för kategoribaserade mätningar.

Vad?

RISE arbetar sedan flera år för att utveckla metoder och arbetssätt för mätkvalitetssäkring av upplevelser, känslor, beteenden och förmågor. Genom dialoger med aktörer på olika nivåer, bland annat inom hälso- och sjukvården och akademin, som pågått under flera års tid har det framkommit att det finns ett stort behov av stöd för denna utveckling. RISE har därför tagit fram ett första utkast för en möjlig hållbar organisering för mätkvalitetssäkring av kategoribaserade mätningar. Ett nationellt centrum för mätkvalitetssäkring av kategoribaserade mätningar skulle ha i uppdrag att samordna och säkerställa tillförlitliga och jämförbara mätningar av upplevelser, känslor, beteenden och förmågor. Ett sådant infrastrukturellt stöd ska tillgängliggöra metodiker för att möjliggöra mätkvalitetssäkring för kategoribaserade mätningar. Organiseringen ska möta samhällets behov av stöd genom att erbjuda tjänster baserade på internationellt vedertagna mättekniska principer och vetenskapliga metoder samt ha en neutral och oberoende roll.

Hur?

Det finns en stor ödmjukhet kring att det är en lång process att etablera en organisering för kategoribaserade mätningar och att det behöver ske genom en dynamisk och iterativ utveckling i samskapande med berörda aktörer. Vi ser fram emot att fortsätta utveckla tankar och revidera förslaget i dialog under kommande år. Det är också viktigt att påbörja en dialog tillsammans med relevanta aktörer om hur implementeringen av organiseringen kan ske i framtiden.