Continuous responses: sliders and bounded scales

Module: Beyond the standard models
Tranche: extension (beyond a first course)
Before this: Models for polytomous responses · Factor analysis II: confirmatory models, omega, and IRT as factor analysis
Builds toward: none
Picks up threads: One standard error of measurement for everyone (from Classical test theory and reliability); Ordinal factor analysis is the graded response model (from Factor analysis II: confirmatory models, omega, and IRT as factor analysis); Choose the estimator from the data: categories, shape, missingness (from Factor analysis II: confirmatory models, omega, and IRT as factor analysis); Merging categories can’t add information (from Models for polytomous responses)
IRW tables: estcrm_epia, karlsson_2023_climate_risk
From EDUC 252: n/a
Status: draft
Authorship: Written largely by Claude (Anthropic), from Ben Domingue’s EDUC 252 materials and under Ben’s direction. Ben reviews each lesson before it is marked done.
Comments: open an issue on GitHub about this lesson.

What this is for

Some items offer no categories: the respondent marks a point on a line, or drags a slider from 0 to 100. Factor analysis II imagined a continuous latent response behind every ordered category; a line seems to hand it to us directly. So do we just fit a factor model? That rests on assumptions about how respondents use the line, and this lesson asks what they are and how the data can check them. But a line has ends, and respondents use them: on one climate-risk slider, almost a quarter of the ratings sit at 100. We build the linear model and Samejima’s continuous response model, look at models that treat the ends as part of the response, and ask what cutting a line into categories costs. On five impulsivity items marked on a line, the graded model fitted to the cut line reports more information than the continuous models, which it can’t have if they hold. Checking that claim takes up most of the real-data section.

Goals

By the end of this lesson you should be able to:

  • write the linear model for continuous item responses, and say why its information, \(\lambda^2/\psi\), is the same at every \(\theta\);
  • fit Samejima’s continuous response model as a factor model for logits, and convert its estimates to EstCRM’s;
  • say what the ends of a bounded scale do to these models, and what a squeeze constant, a beta model and an ordered beta model each assume about them;
  • compute what cutting a line into \(K\) categories costs, and where;
  • check a model’s claim about information against the data, with a split into halves that doesn’t depend on the model.

Core ideas

The latent response, observed

Recall from Factor analysis II: an ordered response can be read as a latent response \(y^* = \lambda\eta + \varepsilon\) cut at thresholds, and the ordinal factor model that results is the graded response model.

On a line there is nothing to cut. A respondent’s mark \(x_{ij}\), in millimetres or slider points, can be modelled directly: \[x_{ij} = \mu_i + \lambda_i\theta_j + \varepsilon_{ij}, \qquad \varepsilon_{ij} \sim N(0, \psi_i),\] with \(\theta_j\) standard normal. This is the one-factor model of Factor analysis I, with \(\psi_i\) the unique variance, read as an item response model for continuous responses (Mellenbergh, 1994; Bejar, 1977, applies such a model to personality items).

How much does one response say about \(\theta\)? Given \(\theta\), the response is normal with mean \(\mu_i + \lambda_i\theta\) and variance \(\psi_i\), so its log likelihood is \[\ell_i(\theta) = -\frac{(x_{ij} - \mu_i - \lambda_i\theta)^2}{2\psi_i} + \text{const},\] a parabola in \(\theta\). Information is the expected curvature of the log likelihood, \(-E[\ell_i''(\theta)]\). Differentiating twice gives \(\ell_i''(\theta) = -\lambda_i^2/\psi_i\), so the item’s information is \(\lambda_i^2/\psi_i\). Neither \(\theta\) nor the response appears in it, for two reasons. The mean of the response is a straight line in \(\theta\), so a step in \(\theta\) moves it by \(\lambda_i\) everywhere. And the noise around that mean, \(\psi_i\), is the same at every \(\theta\). For a 0/1 item neither holds: the curve is steepest near \(b\) and flat far from it, and the variance \(P(1-P)\) changes with \(\theta\). That is why its information has a peak.

Recall from Information, precision, and short forms: the conditional SEM is \(1/\sqrt{I(\theta)}\), and it replaced the single standard error of measurement of Classical test theory and reliability with one that varies with \(\theta\).

Under the linear model the single SEM comes back: \(1/\sqrt{\sum_i \lambda_i^2/\psi_i}\) for everyone. With standardized loadings, \(\psi_i = 1 - \lambda_i^2\), and a test information of \(I\) gives a reliability of \(I/(1+I)\).

The ends, and Samejima’s model

A line has no ends, but the responses we collect on one usually do: the mark falls on a 112 mm line printed on the page, or a slider stops at 0 and 100. The linear model doesn’t know about these bounds: its mean keeps rising past the top of the scale, and its error variance is the same at the ends as in the middle. Samejima (1973) extended her graded model to a continuous response model. In the parameterization of Wang and Zeng (1998) it is easy to fit: divide by the scale’s maximum \(M\), take logits, \[z_{ij} = \ln\frac{x_{ij}}{M - x_{ij}},\] and fit the linear model to \(z_{ij}\). The logit stretches the ends out to \(\pm\infty\), so the responses stay bounded, and are squeezed near the ends. The EstCRM package (Zopluoglu, 2012) fits it by EM; factanal() on the logits gives the same estimates, as the real-data section shows.

Try it: the linear model against Samejima’s model

An item follows Samejima’s model. The shaded band holds 95% of responses at each \(\theta\), and the solid blue line is the median. The orange lines are the linear model that best matches the item across respondents (the regression of the response on \(\theta\)), with its own 95% band dashed.

At the defaults the item sits low on the scale for most respondents (\(b = -1\)). The linear band spills over the top for 30% of them, while Samejima’s responses bunch below 1: their SD falls from 0.139 at \(\theta = b\) to 0.069 two units above it. Move \(b\) to 0 and the linear band still leaves the scale for 18%, in both tails.

The widget’s last number is the odd one. Samejima’s responses change shape along \(\theta\), yet the information is flat. That follows from a general fact: Fisher information is unchanged by a one-to-one transformation of the data, because the densities of \(x\) and of \(z\) differ only by a Jacobian that doesn’t involve \(\theta\) (problem 1). The IRW’s bounded continuous responses vignette makes the same point. So Samejima’s model keeps the single SEM.

If you’ve done Response times: the lognormal model for response times is the linear model for log time. Samejima’s model is the same construction, with the logit in place of the log.

Bounded models: beta IRT and its relatives

Another route builds the bounds into the distribution. Beta IRT (Noel & Dauvier, 2007) takes the response, rescaled to (0, 1), to be beta-distributed given \(\theta\), with a mean that rises with \(\theta\) and a precision \(\phi\) that sets the spread. Müller (1987) built a Rasch model for continuous ratings, and Ferrando (2001) a nonlinear factor model for them. Under these models information varies with \(\theta\). The IRW vignette fits them to 11 slider tables: on held-out log probability the bounded models beat the linear model on every table, but on the error of a predicted response (RMSE) they didn’t. They pay off when a question needs the shape of the response, such as the chance of a rating at the ceiling.

Try it: three densities at one θ

One item, three models: Samejima’s (logit slope 1.5, residual SD 1), a beta with the same median and precision \(\phi = 10\), and the same beta with shares \(\omega_0\) and \(\omega_1\) of responses exactly at 0 and 1. The shares are set at \(\theta = 0\) and move with \(\theta\) on the logit scale, with slope 1.5.

At \(\theta = 1.5\) the Samejima and beta curves have the same median, 0.90, but different tails: Samejima’s puts 1.0% of responses above 0.99, the beta 9.9%. Neither puts any response exactly at 1. The inflated model does, 16.2% of them, and 24.5% above 0.99 in all. Set both shares to 0 and the inflated curve becomes the beta.

Responses at the ends

A response of exactly 0 or \(M\) has an infinite logit. Samejima’s model needs the ends pulled in first, \(z = \ln\{(x + \nu)/(M - x + \nu)\}\) with a small squeeze constant \(\nu\). The beta needs the same (Smithson & Verkuilen, 2006, squeeze by an amount that shrinks with the sample size). When few responses sit at the ends, the choice hardly matters. When many do, it matters a good deal: Molenaar, Cúri and Bazán (2022) found estimates affected with as few as 5% of responses on a bound.

Try it: the squeeze constant

Five hundred respondents answer two items on a 0–100 slider; each logit is \(1.5\theta\) plus normal noise, and responses are recorded as whole numbers, so everything above 99.5 becomes 100. Move the items up the scale and pick \(\nu\). The histogram shows item 1’s logits.

At the default, 22% of responses are at 100, the tall bar on the right. With \(\nu = 0.5\) it sits just past the other logits; with \(\nu = 0.005\) it stands far out, and the fitted information falls from 4.01 to 2.54, against 4.50 with no ceiling. Move the items down to 0 and the three agree within 0.25. The squeeze constant decides where to put responses the model can’t place.

Two families give the ends their own probabilities instead. Zero-one inflated models add a share at each end to a continuous density (Ospina & Ferrari, 2010), and Molenaar, Cúri and Bazán (2022) build such shares into four IRT models for bounded responses. They allow responses at the ends to arise differently from the rest, for example as a choice of the end itself. The ordered beta (Kubinec, 2023) uses cutpoints, as ordered logit does, to fit one linear model to 0, to 1 and to the beta-distributed responses in between. It treats a respondent at 100 as one who would rate higher still. Both can be fitted with glmmTMB (Brooks et al., 2017).

Cutting the line

Recall from Models for polytomous responses: merging two categories never adds information (the Cauchy–Schwarz step in its second Go deeper callout), and collapsing a four-category item costs most away from the boundary that is kept.

A line is the limit of that argument. If the linear model holds, a line cut into \(K\) categories is an ordinal item with its thresholds at the cuts, and it can’t carry more than the line’s \(\lambda^2/\psi\). MacCallum, Zhang, Preacher and Rucker (2002) make the parallel case about dichotomizing a continuous variable. How much does a cut item carry?

Try it: cutting a line into K categories

The latent response has loading λ and is observed on a line spanning ±3 SD. Cut it into K categories of equal width, or holding equal shares of respondents.

Five categories holding equal shares keep 92% of the line’s information at \(\theta = 0\) and 67% at \(\theta = -2\); cuts of equal width spread the loss more evenly (84% and 82%). Two categories keep 64% and 27%. No setting reaches the dashed line.

Assumptions: one item; the response (or its logit) is \(y = \mu + \lambda\theta + \varepsilon\) with \(\varepsilon \sim N(0, \psi)\); item parameters known.

Cutting gives a graded item. Cut \(y\) at thresholds \(t_1 < \dots < t_{K-1}\). The chance of reaching category \(k\) or above is \(\Pr(y > t_k \mid \theta) = \Phi\big((\mu + \lambda\theta - t_k)/\sqrt\psi\big)\): a normal-ogive graded item with slope \(\lambda/\sqrt\psi\) and boundaries \((t_k - \mu)/\lambda\).

The limit. Let the thresholds fill the line, spaced \(h\) apart, with \(h \to 0\). Category \(k\)’s probability becomes \(h\,f(t_k \mid \theta)\), with \(f\) the normal density of \(y\) given \(\theta\), and the polytomous information \(\sum_k (P_k')^2/P_k\) becomes \[\int \frac{\big(\partial f(y \mid \theta)/\partial\theta\big)^2}{f(y \mid \theta)}\,dy = \frac{\lambda^2}{\psi},\] the Fisher information about a normal mean \(\mu + \lambda\theta\) with variance \(\psi\). With the logit of the response as \(y\), this is Samejima’s model.

Cutting loses information. Any cut version is a merger of the fine categories. By the Cauchy–Schwarz step in Models for polytomous responses, each merger keeps or lowers the information, so a cut item has at most \(\lambda^2/\psi\) at every \(\theta\), with equality only in the limit.

What the proof needs. Both steps assume the model holds. If the response isn’t normal given \(\theta\), then \(\lambda^2/\psi\) is what a normal model reports, not necessarily the information in the data, and a graded model fitted to the cut line can report more. The real-data section meets exactly this.

Do sliders behave like items with more categories?

That depends on how respondents use them. A respondent who picks only multiples of 10 is answering an 11-category item. In a web experiment by Couper, Tourangeau, Conrad and Singer (2006), visual analogue scales gave the same response distributions as radio buttons, with more missing data and longer completion times. In one by Funke and Reips (2012), respondents adjusted their ratings more often on the line than on five-point scales, with no difference in means, nonresponse or time. Matejka, Glueck, Grossman and Fitzmaurice (2016) found that a slider’s ticks and labels shift where respondents put it. Preston and Colman (2000) found reliability and validity rising with the number of categories up to about seven. Heaping can be counted: an evenly spread 0–100 response lands on a multiple of 10 about 11% of the time, and the vignette found a table where over half the responses sat on the scale’s quarter points. A paper line scored with a ruler gives respondents no numbers to round to.

Simulate

The code simulates 500 respondents answering six slider items from Samejima’s model, fits the linear model, Samejima’s model (\(\nu = 0.5\)) and the graded model on K bins, and compares each with the true test information. The first run takes 20–30 seconds while R and mirt load.

With the seed as written, the true test information is 5.2; the linear model gives 4.7 and Samejima’s 5.6. The graded model on five bins gives 5.6 at \(\theta = 0\) and 1.6 at \(\theta = -2\). That 5.6 is above the truth, which a cut line can’t carry: sampling error, and perhaps the graded model’s logistic link where these data are normal. Keep it in mind. Set ends <- 0.1 to push responses to the ends: information falls to 3.3 (linear) and 2.5 (Samejima). Heaping, round_to <- 10, costs little (4.7 for both).

Download this code to run it in your own R session.

With real data

Impulsivity on a line

Our main example is estcrm_epia: 1,033 undergraduates each marked five items of the impulsivity subscale of the Spanish Eysenck Personality Inventory (EPI-A) on a 112 mm line running from “almost never” to “almost always”, scored in millimetres from 1 to 111 (Ferrando, 2002; distributed as EPIA in EstCRM, Zopluoglu, 2012). Its job here: a line that respondents use away from its ends, where the continuous models should do well. Higher means more impulsive, and all five items are keyed that way. Download the code to run every chunk.

Code 
library(mirt)
set.seed(252)
# Each IRW table has a landing page (itemresponsewarehouse.org/tables/<name>/) with a
# CSV download. This link is pinned to one version of the data.
epia_url <- "https://redivis.com/api/v1/tables/datapages.item_response_warehouse:v64_0.estcrm_epia/rows?format=csv"
df <- read.csv(epia_url)
X <- as.matrix(unstack(df[order(df$id), ], resp ~ item))   # 1,033 x 5, rows = respondents
colnames(X) <- paste0("item", 1:5)
M <- 112          # the line is 112 mm; marks are scored 1-111 mm
c(respondents = nrow(X), min = min(X), max = max(X))
respondents         min         max 
       1033           1         111
Code 
c(at_1 = mean(X == 1), at_111 = mean(X == 111))           # marks at the ends
      at_1     at_111 
0.02594385 0.01781220
Code 
# Heaping: evenly spread marks would put 1 in 5 on a multiple of 5, 1 in 10 on 10.
c(mult5 = mean(X %% 5 == 0), mult10 = mean(X %% 10 == 0))
    mult5    mult10 
0.2205227 0.1163601
Code 
round(cor(X), 2)
      item1 item2 item3 item4 item5
item1  1.00  0.20  0.19  0.23  0.18
item2  0.20  1.00  0.15  0.25  0.25
item3  0.19  0.15  1.00  0.16  0.13
item4  0.23  0.25  0.16  1.00  0.31
item5  0.18  0.25  0.13  0.31  1.00
Code 
kurt <- function(v) mean((v - mean(v))^4) / mean((v - mean(v))^2)^2   # normal: 3
round(apply(X, 2, kurt), 1)
item1 item2 item3 item4 item5 
  2.0   3.1   1.6   2.0   1.9

Only 2.6% of marks are at 1 and 1.8% at 111, and there is little heaping: 22% fall on multiples of 5 and 12% on multiples of 10, against 20% and 10% for marks spread evenly. The items correlate only 0.13 to 0.31, and every item is flatter than a normal distribution (kurtosis 1.6 to 3.1, against 3).

Recall from Factor analysis II: ML treats the responses as continuous and multivariate normal, and the simulations there found it did well when they roughly are.

These marks are continuous, seldom at the ends and not far from symmetric: ML, on the raw marks and on their logits.

Code 
# The linear model: a one-factor model fitted by ML to the raw marks.
# Samejima's model: the same model fitted to the logits z = log(x / (M - x)).
Z <- log(X / (M - X))
f_lin <- factanal(X, 1, scores = "regression")
f_crm <- factanal(Z, 1, scores = "regression")
l_lin <- f_lin$loadings[, 1]; l_crm <- f_crm$loadings[, 1]
round(rbind(linear = l_lin, samejima = l_crm), 2)          # standardized loadings
         item1 item2 item3 item4 item5
linear    0.41  0.47  0.31  0.57  0.51
samejima  0.37  0.48  0.31  0.56  0.53
Code 
# Item information about theta (variance 1) is lambda^2 / psi, psi = 1 - lambda^2;
# test information is the sum, the same at every theta.
info <- function(l) sum(l^2 / (1 - l^2))
I_lin <- info(l_lin); I_crm <- info(l_crm)
c(info_linear = I_lin, info_samejima = I_crm)
  info_linear info_samejima 
     1.426047      1.403862
Code 
c(reliability_linear = I_lin / (1 + I_lin), reliability_samejima = I_crm / (1 + I_crm))
  reliability_linear reliability_samejima 
           0.5878069            0.5840028
Code 
cor(f_lin$scores[, 1], f_crm$scores[, 1])                   # the two sets of scores
[1] 0.9649437

The two models agree. Loadings run from 0.31 to 0.57 under either, test information is 1.4 under both (reliability 0.59 and 0.58), and the scores correlate 0.96. With five weakly related items the SEM is \(1/\sqrt{1.43} \approx 0.84\), for every respondent.

Code 
# Check: EstCRM fits Samejima's model by EM (Wang & Zeng, 1998; Zopluoglu, 2012).
# Its a is the logit slope over the residual SD, which is l / sqrt(1 - l^2) in
# standardized terms; its b is -(mean logit) / (logit slope).
if (requireNamespace("EstCRM", quietly = TRUE)) {
  crm <- EstCRM::EstCRMitem(as.data.frame(X), max.item = rep(M, 5), min.item = rep(0, 5),
                            max.EMCycle = 500, converge = 0.001)
  slope <- l_crm * apply(Z, 2, sd)
  print(round(cbind(EstCRM_a = crm$param[, "a"], factanal_a = l_crm / sqrt(1 - l_crm^2),
                    EstCRM_b = crm$param[, "b"], factanal_b = -colMeans(Z) / slope), 2))
}
      EstCRM_a factanal_a EstCRM_b factanal_b
item1     0.39       0.39    -0.62      -0.62
item2     0.55       0.55     1.30       1.30
item3     0.32       0.32     0.23       0.23
item4     0.67       0.67     0.39       0.39
item5     0.63       0.63     0.34       0.34

EstCRM gives the same item parameters as factanal() on the logits, to two decimals: its \(a\) is the logit slope over the residual SD, and its \(b\) is minus the mean logit over the logit slope. (Its \(\alpha\) is the logit slope itself; the course keeps \(\alpha\) for other things.)

Now cut the lines into bins.

Code 
# Cut each line into K equal-width bins and fit the graded response model.
cut_line <- function(X, K, M) apply(X, 2, function(v)
  cut(v, breaks = seq(0, M, length.out = K + 1), labels = FALSE, include.lowest = TRUE) - 1)
bins <- t(sapply(c(2, 3, 5, 7, 21), function(K) {
  g <- mirt(as.data.frame(cut_line(X, K, M)), 1, itemtype = "graded", verbose = FALSE)
  c(K = K, marginal_rel = marginal_rxx(g),
    info = setNames(testinfo(g, Theta = matrix(c(-2, 0, 2))), c("at_-2", "at_0", "at_2")))
}))
round(bins, 2)
      K marginal_rel info.at_-2 info.at_0 info.at_2
[1,]  2         0.48       0.36      1.14      0.76
[2,]  3         0.56       0.60      1.47      1.10
[3,]  5         0.62       1.10      1.75      1.56
[4,]  7         0.64       1.39      1.83      1.75
[5,] 21         0.66       1.91      1.97      1.96

More bins mean more information, as the widget says: at \(\theta = -2\) it rises from 0.36 with two bins to 1.91 with 21. Five bins keep 89% of the 21-bin information at \(\theta = 0\) (1.75 against 1.97) and 58% at \(\theta = -2\) (1.10 against 1.91). But from five bins on, every graded fit reports more information at \(\theta = 0\) than the continuous models’ 1.4.

Does the graded model inflate information on its own? Simulate from the fitted linear model, cut the same way and refit:

Code 
# Does the graded model inflate information by itself? Simulate responses from the
# fitted linear model (true test information I_lin), put them on a line, cut into
# 21 bins as above, and fit the graded model.
set.seed(252)
th <- rnorm(nrow(X))
S <- sapply(l_lin, function(l) l * th + rnorm(nrow(X), 0, sqrt(1 - l^2)))
S <- apply(S, 2, function(v) (v - min(v)) / (max(v) - min(v)) * M)
g_sim <- mirt(as.data.frame(cut_line(S, 21, M)), 1, itemtype = "graded", verbose = FALSE)
c(true_info = I_lin, graded_info_at_0 = testinfo(g_sim, Theta = matrix(0)))
       true_info graded_info_at_0 
        1.426047         1.353969

On data that follow the linear model, the graded model reports 1.35, a little below the true 1.43. So the gap comes from something in the EPI lines. Which model is right about them? A check that leans on neither: split the five items into a pair and a triple (10 ways) and correlate the halves’ sums. More information means halves that agree more, and each fitted model says how much they should.

Code 
# A check that doesn't trust either model: split the five items into two halves (a
# pair and a triple; 10 ways) and correlate the halves' sums. More information in
# the items means halves that agree more. Compare the observed agreement with what
# each fitted model implies.
splits <- combn(5, 2, simplify = FALSE)
halves <- function(Y) mean(sapply(splits, function(A) {
  B <- setdiff(1:5, A); cor(rowSums(scale(Y[, A])), rowSums(scale(Y[, B])))
}))
R_lin <- tcrossprod(l_lin); diag(R_lin) <- 1           # correlations the linear model implies
implied_lin <- mean(sapply(splits, function(A) {
  B <- setdiff(1:5, A); sum(R_lin[A, B]) / sqrt(sum(R_lin[A, A]) * sum(R_lin[B, B]))
}))
set.seed(252)
g21 <- mirt(as.data.frame(cut_line(X, 21, M)), 1, itemtype = "graded", verbose = FALSE)
implied_grm <- halves(simdata(model = g21, N = 20000))  # a large sample from the fitted GRM
round(c(observed_lines = halves(X), observed_21_bins = halves(cut_line(X, 21, M)),
        linear_implies = implied_lin, graded_21_implies = implied_grm), 2)
   observed_lines  observed_21_bins    linear_implies graded_21_implies 
             0.39              0.39              0.38              0.44

The halves agree at 0.39, whether we sum the lines or the 21 bins. The linear model implies 0.38; the graded model fitted to the 21 bins implies 0.44. The extra information the graded model reports doesn’t show up where we can see information without a model. Why the graded model fits steeper slopes to these lines, whether through the items’ flat distributions or something else, I don’t know (problem 6).

Climate risk on a slider

For contrast, karlsson_2023_climate_risk: 1,001 US adults rated the risk of climate change on six 0–100 visual analogue scales before and after a mental-imagery manipulation (Karlsson, Asutay & Västfjäll, 2023; data at https://osf.io/k6vp9/). Its job: a slider where responses pile up at the top. Two items each ask about the risk to oneself, to other people and to non-human populations (item 4 reads “How big of a risk is climate change for you?”, from the deposit’s variable label as quoted in the IRW’s processing script). Higher means more perceived risk. We use wave 1, before the manipulation.

Code 
# Wave 1 only: the ratings before the imagery manipulation (wave 2 is after it).
karl_url <- "https://redivis.com/api/v1/tables/datapages.item_response_warehouse_6:v3_8.karlsson_2023_climate_risk/rows?format=csv"
dk <- read.csv(karl_url)
dk1 <- subset(dk, wave == 1)
W <- as.matrix(unstack(dk1[order(dk1$id), ], resp ~ item))  # 1,001 x 6, 0-100
W <- W[, sort(colnames(W))]
c(respondents = nrow(W), complete = sum(complete.cases(W)))
respondents    complete 
       1001        1001
Code 
round(rbind(at_0 = colMeans(W == 0), at_100 = colMeans(W == 100)), 2)
       Risk_DV_1 Risk_DV_2 Risk_DV_3 Risk_DV_4 Risk_DV_5 Risk_DV_6
at_0        0.04      0.03      0.03      0.04      0.03      0.03
at_100      0.15      0.20      0.31      0.17      0.20      0.32
Code 
c(at_0 = mean(W == 0), at_100 = mean(W == 100), all_six_at_100 = mean(rowSums(W == 100) == 6))
          at_0         at_100 all_six_at_100 
    0.03230103     0.22660673     0.11788212
Code 
c(mult5 = mean(W %% 5 == 0), mult10 = mean(W %% 10 == 0))
    mult5    mult10 
0.5860806 0.4390609
Code 
round(cor(W), 2)
          Risk_DV_1 Risk_DV_2 Risk_DV_3 Risk_DV_4 Risk_DV_5 Risk_DV_6
Risk_DV_1      1.00      0.83      0.75      0.93      0.81      0.74
Risk_DV_2      0.83      1.00      0.87      0.83      0.93      0.87
Risk_DV_3      0.75      0.87      1.00      0.77      0.85      0.96
Risk_DV_4      0.93      0.83      0.77      1.00      0.85      0.77
Risk_DV_5      0.81      0.93      0.85      0.85      1.00      0.87
Risk_DV_6      0.74      0.87      0.96      0.77      0.87      1.00

Here 23% of ratings are at 100 (15% to 32% by item) and 3.2% at 0, and 12% of respondents gave 100 on all six. Ratings heap: 59% fall on multiples of 5 and 44% on multiples of 10, against 21% and 11% for values spread evenly. The items pair up, correlating 0.93 to 0.96 within three pairs (items 1 and 4, 2 and 5, 3 and 6), which fits two items per target. That is local dependence, and the information figures below, which treat the six items as independent given \(\theta\), overstate the precision for that reason.

Code 
# Samejima's model needs finite logits, so the ends are pulled in by a squeeze
# constant nu (in scale points) before taking logits. The choice moves the fit.
info_W <- function(Y) info(factanal(Y, 1)$loadings[, 1])
sq <- sapply(c(0.5, 0.05, 0.005), function(nu) info_W(log((W + nu) / (100 - W + nu))))
round(c(linear = info_W(W), samejima = setNames(sq, paste0("nu=", c(0.5, 0.05, 0.005)))), 1)
           linear   samejima.nu=0.5  samejima.nu=0.05 samejima.nu=0.005 
             37.9              35.7              31.3              28.2

The squeeze constant now moves Samejima’s fit: test information 36, 31 and 28 as \(\nu\) goes from 0.5 to 0.005 slider points, against 38 for the linear model.

Code 
# Beta and ordered beta regressions with item fixed effects and a random intercept
# per respondent (a Rasch-like model for a bounded response), fitted with glmmTMB
# (Brooks et al., 2017). The beta needs the ends squeezed in (Smithson & Verkuilen,
# 2006); the ordered beta (Kubinec, 2023) gives 0 and 1 their own probabilities.
if (requireNamespace("glmmTMB", quietly = TRUE)) {
  long <- data.frame(id = factor(rep(seq_len(nrow(W)), 6)), item = factor(rep(colnames(W), each = nrow(W))),
                     y = as.vector(W) / 100)
  n <- nrow(long)
  long$y_sq <- (long$y * (n - 1) + 0.5) / n
  m_beta <- glmmTMB::glmmTMB(y_sq ~ 0 + item + (1 | id), family = glmmTMB::beta_family(), data = long)
  m_ob <- glmmTMB::glmmTMB(y ~ 0 + item + (1 | id), family = glmmTMB::ordbeta(), data = long)
  set.seed(252)
  sim_beta <- unlist(simulate(m_beta, nsim = 20)); sim_ob <- unlist(simulate(m_ob, nsim = 20))
  print(round(rbind(observed = c(at_0 = mean(long$y == 0), at_1 = mean(long$y == 1)),
                    beta = c(mean(sim_beta < 0.005), mean(sim_beta > 0.995)),
                    ordered_beta = c(mean(sim_ob == 0), mean(sim_ob == 1))), 3))
}
              at_0  at_1
observed     0.032 0.227
beta         0.010 0.161
ordered_beta 0.029 0.247

A beta model fitted to squeezed responses puts 16% of simulated responses above 0.995 and 1.0% below 0.005, against 23% and 3.2% observed. The ordered beta gives the ends their own probabilities and comes close to both: 25% at 1 and 2.9% at 0.

Code 
binsK <- t(sapply(c(2, 3, 5, 7, 21), function(K) {
  g <- mirt(as.data.frame(cut_line(W, K, 100)), 1, itemtype = "graded", verbose = FALSE)
  c(K = K, marginal_rel = marginal_rxx(g),
    info = setNames(testinfo(g, Theta = matrix(c(-2, 0, 2))), c("at_-2", "at_0", "at_2")))
}))
round(binsK, 2)
      K marginal_rel info.at_-2 info.at_0 info.at_2
[1,]  2         0.45       0.25      0.96      0.00
[2,]  3         0.67       1.58     14.48      0.00
[3,]  5         0.84       5.56     44.74      0.03
[4,]  7         0.88      15.67     44.78      0.07
[5,] 21         0.93      59.54     67.95      0.41

How much do the ratings measure above the ceiling? Cut into 21 bins, the graded model has 68 units of information at \(\theta = 0\) but 0.41 at \(\theta = 2\), and under 0.1 there with seven bins or fewer. Marginal reliability is 0.84 with five bins and 0.93 with 21.

What does the choice among these models turn on? Two assumptions: that the response is normal given \(\theta\) (on the raw or the logit scale), and that responses at the ends are ordinary responses that happen to be extreme. On the EPI lines, marks sit away from the ends, the two continuous models agree, and the linear model’s claim about the halves matches the data. In the case that responses sit away from the ends and the goal is a score, the linear factor model on the raw responses may be sufficient for some purposes. In the case that a fifth or more of the responses sit at an end, as on the climate slider, a model that gives the ends their own probability, such as the ordered beta, is the one whose predictions at the ends can be checked against the data. And in the case that a slider is to be cut, five categories holding equal shares may be sufficient for some purposes: under the linear model they keep about nine tenths of the information in the middle and two thirds two SDs out. Where these assumptions are in doubt, the vignette’s comparison on held-out responses can be run on any of these models.

Problems

  1. Information on two scales. (a) Show that the linear model’s item information is \(\lambda^2/\psi\) at every \(\theta\). (b) Show that Fisher information about \(\theta\) is unchanged by a one-to-one, differentiable transformation of the response that doesn’t depend on \(\theta\). (c) Show that EstCRM’s \(a\) equals \(\lambda/\sqrt{1 - \lambda^2}\), with \(\lambda\) the standardized loading of the logits.
  2. Where does the scale end? estcrm_selfeff has 307 pre-service teachers’ marks on ten teaching self-efficacy items, made on an 11 cm line (Kan, 2009; EstCRM::SelfEff). Many marks fall between 9.5 and 10.5 cm, and a few above. Fit Samejima’s model with the scale maximum at 11 and at 10.5 (squeezing the marks above 10.5). What moves, and why? Which maximum would you defend?
  3. Replacing a five-point item. A team plans to replace a five-point item with a 0–100 slider “to get more information”. What would you look for in pilot data (heaping, the ends, the information by \(K\) from this lesson’s code) before agreeing, and what result would make the switch worth it?
  4. Designing a slider. Design a slider item to limit heaping and piling at the ends. Decide whether it shows numbers, where the handle starts and how the ends are labelled, drawing on Couper et al. (2006) and Matejka et al. (2016). How would you test your design against a five-point version?
  5. Does imagery move the ends? Use wave 2 of karlsson_2023_climate_risk and the three arms in cov_condition. Fit the ordered beta with the arm as a predictor. Does the manipulation move the share at 100, the middle of the scale, or both? What does the local dependence within pairs do to your standard errors?
  6. Challenge (open): the graded model’s extra information. On the EPI lines the graded model fitted to bins reports about 40% more information than the linear model, yet the halves agree only as much as the linear model implies. Why does the graded model report more? Test one explanation: error flatter than normal, a variance that shrinks near the ends, a beta response, or the logistic link. I don’t know which one holds.

Stuck, or curious about something this lesson doesn’t cover? Ask Claude about this lesson opens a new conversation that already knows what the lesson is about; add your question at the end. Claude shows a caution notice on any prompt that arrives through a link; that’s expected.

Going further

  • The IRW vignette Bounded continuous responses. The linear, beta, Samejima and Müller models compared on held-out responses across 11 slider tables, with information curves for each; this lesson builds on it rather than repeating it.
  • Mellenbergh, G. J. (1994). A unidimensional latent trait model for continuous item responses. Multivariate Behavioral Research, 29(3), 223–236. https://doi.org/10.1207/s15327906mbr2903_2. The linear model as an item response model.
  • Mellenbergh, G. J. (2016). Models for continuous responses. In W. J. van der Linden (Ed.), Handbook of item response theory: Vol. 1. Models. Chapman and Hall/CRC. https://doi.org/10.1201/9781315374512. A handbook chapter on the models here (chapter pages not verified).
  • Samejima, F. (1973). Homogeneous case of the continuous response model. Psychometrika, 38(2), 203–219. https://doi.org/10.1007/BF02291114. The continuous response model.
  • Wang, T., & Zeng, L. (1998). Item parameter estimation for a continuous response model using an EM algorithm. Applied Psychological Measurement, 22(4), 333–344. https://doi.org/10.1177/014662169802200402. The parameterization and the EM algorithm that EstCRM uses.
  • Zopluoglu, C. (2012). EstCRM: An R package for Samejima’s continuous IRT model. Applied Psychological Measurement, 36(2), 149–150. https://doi.org/10.1177/0146621612436599. The package, and the distributor of both the EPI and self-efficacy data.
  • Ferrando, P. J. (2002). Theoretical and empirical comparisons between two models for continuous item responses. Multivariate Behavioral Research, 37(4), 521–542. https://doi.org/10.1207/S15327906MBR3704_05. The source of the EPI lines.
  • Ferrando, P. J. (2001). A nonlinear congeneric model for continuous item responses. British Journal of Mathematical and Statistical Psychology, 54(2), 293–313. https://doi.org/10.1348/000711001159573. A factor model that bends at the bounds.
  • Bejar, I. I. (1977). An application of the continuous response level model to personality measurement. Applied Psychological Measurement, 1(4), 509–521. https://doi.org/10.1177/014662167700100407. An early application.
  • Noel, Y., & Dauvier, B. (2007). A beta item response model for continuous bounded responses. Applied Psychological Measurement, 31(1), 47–73. https://doi.org/10.1177/0146621605287691. Beta IRT.
  • Müller, H. (1987). A Rasch model for continuous ratings. Psychometrika, 52(2), 165–181. https://doi.org/10.1007/BF02294232. A Rasch model for the line.
  • Noel, Y. (2014). A beta unfolding model for continuous bounded responses. Psychometrika, 79(4), 647–674. https://doi.org/10.1007/s11336-013-9361-1. If you’ve done Unfolding models: an ideal-point model for bounded responses.
  • Smithson, M., & Verkuilen, J. (2006). A better lemon squeezer? Maximum-likelihood regression with beta-distributed dependent variables. Psychological Methods, 11(1), 54–71. https://doi.org/10.1037/1082-989X.11.1.54. Beta regression, and the squeeze used in the code.
  • Ospina, R., & Ferrari, S. L. P. (2010). Inflated beta distributions. Statistical Papers, 51(1), 111–126. https://doi.org/10.1007/s00362-008-0125-4. Beta distributions with point masses at the ends.
  • Molenaar, D., Cúri, M., & Bazán, J. L. (2022). Zero and one inflated item response theory models for bounded continuous data. Journal of Educational and Behavioral Statistics, 47(6), 693–735. https://doi.org/10.3102/10769986221108455. Responses on the bounds, in IRT.
  • Kubinec, R. (2023). Ordered beta regression: A parsimonious, well-fitting model for continuous data with lower and upper bounds. Political Analysis, 31(4), 519–536. https://doi.org/10.1017/pan.2022.20. The ordered beta.
  • Brooks, M. E., Kristensen, K., van Benthem, K. J., Magnusson, A., Berg, C. W., Nielsen, A., Skaug, H. J., Mächler, M., & Bolker, B. M. (2017). glmmTMB balances speed and flexibility among packages for zero-inflated generalized linear mixed modeling. The R Journal, 9(2), 378–400. https://doi.org/10.32614/RJ-2017-066. The package for the beta fits.
  • MacCallum, R. C., Zhang, S., Preacher, K. J., & Rucker, D. D. (2002). On the practice of dichotomization of quantitative variables. Psychological Methods, 7(1), 19–40. https://doi.org/10.1037/1082-989X.7.1.19. What cutting a continuous variable costs.
  • Lubke, G. H., & Muthén, B. O. (2004). Applying multigroup confirmatory factor models for continuous outcomes to Likert scale data complicates meaningful group comparisons. Structural Equation Modeling, 11(4), 514–534. https://doi.org/10.1207/S15328007SEM1104_2. The reverse move: treating categories as continuous.
  • Rhemtulla, M., Brosseau-Liard, P. É., & Savalei, V. (2012). When can categorical variables be treated as continuous? Psychological Methods, 17(3), 354–373. https://doi.org/10.1037/a0029315. Recalled from Factor analysis II.
  • Couper, M. P., Tourangeau, R., Conrad, F. G., & Singer, E. (2006). Evaluating the effectiveness of visual analog scales: A web experiment. Social Science Computer Review, 24(2), 227–245. https://doi.org/10.1177/0894439305281503. Sliders against radio buttons and text boxes.
  • Funke, F., & Reips, U.-D. (2012). Why semantic differentials in web-based research should be made from visual analogue scales and not from 5-point scales. Field Methods, 24(3), 310–327. https://doi.org/10.1177/1525822X12444061. Lines against five-point scales.
  • Preston, C. C., & Colman, A. M. (2000). Optimal number of response categories in rating scales: Reliability, validity, discriminating power, and respondent preferences. Acta Psychologica, 104(1), 1–15. https://doi.org/10.1016/S0001-6918(99)00050-5. From 2 to 11 categories, and a 101-point scale.
  • Matejka, J., Glueck, M., Grossman, T., & Fitzmaurice, G. (2016). The effect of visual appearance on the performance of continuous sliders and visual analogue scales. Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, 5421–5432. https://doi.org/10.1145/2858036.2858063. How a slider’s look moves responses.
  • Kloft, M., Hartmann, R., Voss, A., & Heck, D. W. (2023). The Dirichlet dual response model: An item response model for continuous bounded interval responses. Psychometrika, 88(3), 888–916. https://doi.org/10.1007/s11336-023-09924-7. When the response is an interval rather than a point.
  • Finch, W. H. (2023). The impact and detection of uniform differential item functioning for continuous item response models. Educational and Psychological Measurement, 83(5), 929–952. https://doi.org/10.1177/00131644221111993. DIF for continuous items, which this lesson leaves out.

Data sources

Data from the Item Response Warehouse (each table name links to its IRW page); references from IRW biblio.

For instructors

  • Source: new material, added with the lessons Ben approved on 09-28 (#153). It starts from the IRW vignette Bounded continuous responses, cites its held-out comparison rather than repeating it, and takes up what it leaves aside: Samejima’s model as a factor model, the ends, and cutting the line.
  • Timing: about 50 minutes for the core ideas and widgets; the real-data section is a 30-minute lab. The ordered beta fit takes about 30 seconds locally; every other chunk takes seconds.
  • Software: the page runs factanal() and mirt in the browser. EstCRM and glmmTMB are used only in the downloadable real-data code, each behind a check that it is installed.
  • Changed from the outline: the real-data section was to be built on the graded model’s extra information, with its explanation left open. A check added while drafting (agreement between halves) sides with the linear model, so the section now presents the extra information as a claim the halves don’t bear out, and problem 6 asks why the graded model makes it. The heaping widget was dropped (four widgets); heaping is counted in both tables instead.
  • Held back: the ends as a separate response process (a choice of the end), heaping as a response style, DIF for continuous items, interval responses, and a held-out comparison of these models on the lesson’s tables.
  • Data: the EPI-A items aren’t quoted. The climate-risk items are summarized from Karlsson et al. (2023), with one item quoted from the deposit’s variable label (the data are CC BY 4.0).
  • Solutions: held back in solutions/continuous-responses.qmd (not published).