What can bedtime sound offer? From experience to outcome evaluation

Feeling relaxed and sleeping better

A digital bedtime experience may offer a quiet transition from a busy day. To understand its value, first specify the intended support: comfort, an easier bedtime routine, or a sustained reduction in sleep disturbance. Each calls for different measures.

Personal experience is worth recording. Evaluating whether a particular type of bedtime content adds a benefit also requires asking: compared with what?

The USleep trial compared three categories of bedtime audio with everyday health content in the same app. Self-reported sleep disturbance improved in all four groups, without a clear additional advantage for sleep audio on the primary outcome. This takes product evaluation beyond asking whether people changed after use.

What did the four groups use?

All content was delivered through the Unmind app. The three sleep-audio groups were asked to use one track nightly, before sleep, while falling asleep, throughout the night or when returning to sleep after waking. They could repeat a track or choose another within their assigned category. Missing a night or two was acceptable, and no minimum listening duration was prescribed.

The control was neither a waitlist nor an instruction to do nothing. Participants also received app access, content, instructions and study reminders. It provided a comparison for platform experience, frequency and study duration. Reflection and relaxation content might indirectly influence sleep, so the trial examined what sleep-specific content added to this everyday content.

The comparison did not match every minute or circumstance: controls could read, content lengths differed and use did not necessarily occur at the same time. It is best understood as a same-platform active control, rather than an inert condition eliminating every aspect of the experience.

Can the four-week improvement be credited to sleep audio?

The main analysis followed an intention-to-treat approach, retaining eligible participants with valid baseline data regardless of engagement or follow-up completion. Mixed-effects models compared each audio group's change from baseline to four weeks with Daily Boost.

Sleep-disturbance scores fell significantly within every group. But a before-and-after change is different from an additional change relative to another group. The primary between-group results below come from Table 2; positive Hedges' g values favor sleep audio.

The estimates were very small and every interval crossed zero. The trial did not establish that these three audio categories outperformed the active control on the primary sleep-disturbance outcome. This was not an equivalence trial, and it does not show that nobody gained anything. It does challenge attributing all improvement after use to the specific ingredients of sleep audio.

Without a no-app group, the shared improvement cannot separately identify effects of everyday content, expectations, repeated measurement, natural fluctuation or other influences. An active control narrows the question; it does not test each explanation individually.

Why retain a signal in a secondary outcome?

Self-reported sleep duration increased by about 53 minutes in the Stories group and 33 minutes in the control, a difference of roughly 20 minutes. Table 2 reports p = .042, g = −0.26 and a 95% confidence interval of [−0.51, −0.01]. The direction for this duration measure favors Stories.

This is a signal worth checking, rather than a reason to say that no outcomes differed. Duration was one of many secondary outcomes, assessed with a single self-report item, and the main text and Table 2 do not report a multiple-testing adjustment for this set of outcomes and group comparisons. Crossing the conventional significance threshold once does not establish a robust duration benefit. No actigraphy or sleep monitoring confirmed how much longer people actually slept.

Product evaluation should preserve such detail without promoting it above the primary result. A subsequent trial could specify sleep duration as its main question, plan the sample and measurements accordingly, and examine whether the finding recurs.

What do playback records tell us?

The researchers did not assume everyone used the app nightly. Platform records captured openings, playback and completions. A sleep-audio track counted as completed after at least 5 minutes of playback; that neither required finishing the track nor established that someone fell asleep. This measurement rule differs from prescribing a minimum listening time: one defines the record, while the other concerns instructions to users.

A separate per-protocol analysis required at least 12 uses over four weeks; 141 participants met that threshold. It can supplement understanding of more frequent users, but cannot replace the main 495-person intention-to-treat analysis. Controls exceeded sleep groups on some usage-count measures. Appeal, actual use and change in sleep therefore need separate observation.

Longer playback does not necessarily mean greater benefit. Someone may close a track quickly; another may already be asleep while it loops. Defining a successful user from listening time alone mixes a recording convention with the outcome of interest.

What limits the interpretation?

Missing follow-up requires assumptions. Eighty-seven people did not complete the four-week questionnaire. The main model assumes that missingness can be treated as random conditional on observed information. This does not mean dropout was entirely random, or reveal how every person who left was sleeping. The authors report similar findings from a sensitivity analysis using 30 multiply imputed datasets; we did not rerun the data.

Recruitment also ended early for budgetary reasons. The sample of 495 fell below the calculated minimum of 520 and the planned recruitment target of 600. This adds uncertainty about very small between-group differences; nonsignificance does not establish equal effects.

Researchers and analysts knew allocation, and follow-up ended at four weeks. Participants knew their task, and researchers and analysts knew allocation; Daily Boost users were not explicitly told they were controls. Recruitment involved paid online participants interested in better sleep. Four weeks marked the end of follow-up, leaving longer natural use, other populations and Chinese-language content untested.

The content and choices were specific. Each person was allocated one category, which might not fit their preferences. Nature sounds, noise types and music were not separate randomized arms. The trial therefore did not isolate a particular natural sound, pink noise or singing bowl. It also did not study generative AI support or test either Luoboli's game or Luoboli Sleep.

No clear advantage does not mean no discomfort. Among 335 feedback respondents, 12 reported lasting negative experiences, including 2 Daily Boost controls. Separately, 10 of the 408 follow-up completers met the criterion for reliable worsening of sleep disturbance, including 1 control. The first measure concerns reported experiences; the second concerns scale changes. They are different outcomes and cannot simply be added to count independently affected people. These observations deserve attention without assuming that a particular audio type caused them, or describing all content as risk-free.

Bringing the comparison into product evaluation

The following are proposals for testing, informed by the study, rather than findings from hands-on tests of our products.

Specify what kind of help the content is meant to offer

Is the aim a convenient pause after a busy day, a comfortable experience or less persistent sleep disturbance? Experience feedback can begin to address the first two; the third requires relevant sleep measures and timescales. Content choice, enjoyment and sleep improvement can be recorded in one study while being reported separately.

Give the comparison group a real experience too

To ask whether a sound category adds something, researchers could provide similarly accessible comparison content, reminders and attention, while examining differences in duration, timing and expectations. Studying an entire bedtime routine would require an appropriate additional comparison. A single control cannot answer every question.

Include non-use, leaving and discomfort

Alongside playback counts, evaluation could explore why people do not use or stop using content, record discomfort and next-day experience without adding nighttime disruption, and specify how missing follow-up will be handled. Satisfaction is one part of an experience; continuing use, easier sleep and persistent change each need their own evidence.

Research into bedtime digital experiences can start with a clearer question: what helps whom, in what respect, and compared with what? A transparent comparison makes a product's contribution easier to assess.

Placing bedtime experiences within digital wellbeing

This study turns “I felt better after using it” into a question about added value relative to a comparison. Comfort, willingness to use a service, sleep disturbance, and next-day experience all deserve attention, with evidence appropriate to each. Sound categories and visual style are starting points; clear aims and comparisons make their contribution assessable.