ePRO Reminders: Delivery Failure Is Not Participant Nonresponse

In this synthetic example, a daily electronic patient-reported outcome (ePRO) diary window closes at 22:00, the central dashboard shows no evening entry, and a reminder was dispatched at 18:00. The dispatch record does not justify recording participant refusal: it establishes neither receipt, awareness, nor the reason for noncompletion. First reconcile the assessment record, including any pending transfer. If no qualifying entry is found, record the assessment as not completed or unavailable under the study's definitions, with its reason unresolved. Separately record what is known about the reminder.

The practical sequence is to verify the assessment instance and window, inspect the reminder's actual event records, check current contact permissions and preferences, and use protocol-authorized follow-up when appropriate. Document a participant's explanation as reported, or retain an unknown reason if the evidence remains inconclusive. Handset receipts are neither a prerequisite for recording a participant's direct explanation nor proof that the person chose not to respond. Delivery failure may coexist with other barriers; identifying a messaging fault does not, by itself, prove why an entry is absent.

The following taxonomy is a proposed prespecification tool, not a regulator-issued classification or a validated instrument. Treat its stages as separate observations rather than a mandatory sequence: a participant can complete a diary without opening a reminder, and a contact preference can change after scheduling. For each stage distinguish observed success, observed failure, unknown, and not applicable. Unknown should also carry an explanation, such as telemetry unavailable, reporting delayed, or no matching event found. Those situations call for different investigations.

TrialKit provides one concrete configuration example: its help page describes email, mobile-app push, SMS, participant notification preferences, and repeat reminders that stop on diary completion or window expiry. That page does not describe delivery or open receipts available to study teams. This is a documentation limit, not evidence that the product lacks such capabilities. Ask the vendor to demonstrate the actual events, exports, and access available in the deployed study; do not generalize this example to every eCOA platform. TrialKit notification configuration.

A dashboard label needs its provider definition before it can support an operational conclusion. Record where the observation occurred: application server, messaging service, carrier, device, or assessment repository. A positive event at one boundary does not certify every later boundary. Conversely, a missing event can reflect incomplete reporting rather than a failed message. The matrix below summarizes the cited platform documentation; it is not a ranking of channel reliability.

Apple describes APNs as a best-effort service. Notifications can be delayed or not delivered. Undelivered messages may be stored for up to 30 days, subject to the expiration header; APNs stores only one notification per bundle ID and does not guarantee that repeated messages are preserved in order. These limits make an accepted push request insufficient evidence of timely participant awareness. Apple APNs documentation.

Firebase's reporting distinguishes Sends, Android-only Received and Impressions, and Opens for notifications received while the app was backgrounded. Reporting paths differ: aggregate Data API data can lag by up to five days and have gaps without backfill. iOS alert-notification delivery export requires appropriate app instrumentation, including a notification service extension. A missing dashboard event is therefore not conclusive evidence of non-delivery. Firebase delivery reporting.

For Twilio, sent means upstream-carrier acceptance. Delivered means carrier confirmation, with destination-handset confirmation where available; undelivered indicates a receipt reporting failure, including possible carrier filtering or handset unavailability. Read receipts are channel-dependent, with RCS and WhatsApp examples; an SMS delivery receipt is not a read receipt. Preserve this provider-specific meaning instead of translating every delivered status into 'participant notified.' Twilio message statuses.

With Apple's Protect Mail Activity enabled, Mail downloads remote content privately in the background when a message is received, rather than when it is viewed. An email tracking-pixel event can therefore occur without human viewing. Interpret open telemetry in light of the recipient client and enabled privacy features. Apple Mail privacy documentation.

Where Part 11 applies to a closed system, 21 CFR 11.10(e) requires secure, computer-generated, time-stamped audit trails for operator entries and actions that create, modify, or delete electronic records. Changes must preserve prior information; audit-trail retention must cover the required record-retention period and support agency review. This is not a requirement that every messaging service expose a handset receipt. 21 CFR 11.10(e).

FDA's October 2024 electronic-systems Q&A recommends risk-based audit-trail review. Q20 allows a person, system, DHT, or EHR to be the authorized originator, while identifying a participant who manually enters ePRO data as the originator of those answers. Q22 recommends recording DHT-to-repository transfer dates and times. The guidance does not prescribe a universal reminder-event schema or require every raw gateway response to sit inside the clinical assessment audit trail. FDA electronic-systems Q&A, Q12 and Q20–22.

EMA section 6.2 describes data-point-level audit-trail visibility and dynamic export for pattern review. Annex A5.1 distinguishes entry time from submission/transmission time and describes procedures for failed ePRO data transfers. These expectations concern trustworthy trial records; applying them to the reminder investigation means defining which event logs and metadata support the trial record, not relabeling every operational log as a clinical audit trail. EMA computerised-systems guideline.

A practical reminder evidence record can contain the following linked fields. These are proposed implementation choices to be justified by study risk and available interfaces, rather than a fixed regulatory minimum:

• Assessment reference: coded participant identifier, diary instance, expected window, and completion definition. Keep contact details in the appropriately controlled system.

• Permission and configuration: eligible channel and preference status at dispatch, suppression reason if applicable, reminder rule version, and planned local time with an unambiguous time-zone reference.

• Observed events: message identifier, provider, channel, raw event/status, source event time, time the study received the event, and a retained evidence reference. Distinguish reporting delay from message delay.

• Assessment reconciliation: original entry time and originator, transfer time, repository receipt, and any unresolved local record. Keep these separate from notification timestamps.

• Investigation and disposition: technical incident reference, contact attempt where permitted, participant-reported explanation if obtained, reviewer, reason-code version, and traceable corrections preserving earlier values.

Define access, retention, and export responsibilities with the vendor before the study starts. Preserve enough evidence to reconstruct a relevant decision without indiscriminately copying personal contact details, credentials, or entire service payloads into analysis datasets. The analysis dataset should contain the study-defined reason and necessary traceability; detailed operational evidence can remain in controlled linked records. A reviewer should be able to distinguish a system observation from an investigator conclusion or participant report.

FDA's October 2024 Core PRO guidance recommends prospective missing-data mitigation procedures, including completion monitoring such as reminders, in the protocol, and documenting missing-PRO reasons in the analysis dataset. Its scope is specified cancer trials, so it is an oncology anchor rather than a mandate for every therapeutic area. FDA's December 2023 DHT guidance separately recommends missing-data planning that can include monitoring, reminders, and outreach. Both are guidance recommendations, except where they cite binding requirements. FDA Core PRO guidance, section V.B; FDA DHT guidance, section IV.E.

Keep the factual reason for an absent assessment separate from assumptions used in statistical analysis. A technical outage does not automatically establish missing completely at random (MCAR), and a reported decision not to complete does not automatically establish missing not at random (MNAR). Assumptions of missing at random (MAR) likewise need justification in the analysis plan. No notification status assigns MCAR, MAR, or MNAR. Record the available evidence for the statistical team to evaluate under the prespecified analysis plan; do not turn an operational reason label into an estimand strategy or an automatic imputation rule. FDA 2009 PRO guidance, section V.D.

flowchart TD
  A["Expected assessment absent from repository"] --> B["Reconcile assessment identity, entry time and pending transfer"]
  B --> C{"Qualifying entry found?"}
  C -->|"Yes"| D["Record completion and transfer evidence"]
  C -->|"No"| E["Record noncompletion or unavailability; reason initially unresolved"]
  E --> F["Inspect reminder events; preserve failures and unknown states separately"]
  F --> G{"Follow-up permitted and indicated by study procedure?"}
  G -->|"Yes"| H["Record contact attempt and any reported explanation"]
  G -->|"No"| I["Document contact restriction or no-contact decision"]
  H --> J["Use study-defined reason supported by evidence, or retain unknown"]
  I --> J

The examples below are synthetic dispositions, not standardized analysis codes. A confirmed technical event and the reason the participant did not complete can both be recorded, and can remain different. Multiple contributing factors may be present; retain that uncertainty instead of forcing a single cause from the messaging log.

Return to the opening example. If the 18:00 event is only gateway acceptance, the initial record is 'assessment unavailable after reconciliation; reminder accepted; delivery unknown; reason unknown.' If later evidence confirms an outage, add that technical finding with its scope and time. If the participant independently reports choosing not to complete, preserve that explanation even if delivery telemetry is absent. Update the disposition through traceable correction rather than replacing the original observations.

FDA's 2009 PRO guidance describes protocol-specified safeguards for unsupervised diary entry and standardized administration processes. This is a legacy source supporting those principles, not a complete account of subsequent COA guidance. Before the reminder build, align the protocol, data management plan, and system specification on channels, timing, repeats, suppression on completion or expiry, fallback eligibility, quiet hours, and the owner of follow-up. Define local time, travel, and daylight-saving handling so a technically valid send is not mistaken for a timely reminder. FDA 2009 PRO guidance, administration and trial quality control.

Device strategy also belongs in this specification. For either bring-your-own-device (BYOD) or provisioned equipment, ask which platform events reach the study team, under which app configuration, and with what delay. Ownership of a handset does not establish receipt of a particular message. Treat device replacement, channel changes, and notification preferences as configuration inputs to test. The broader deployment decision is covered in eCOA Device Strategy: BYOD vs Provisioned vs Hybrid Models.

The following is a synthetic daily-diary specification with an 18:00–22:00 participant-local-time window. Its intervals and escalation threshold are illustrative, not validated schedules or regulator-endorsed cutoffs. An actual trial should justify burden, timing, channel permissions, and the instrument's recall requirements before adopting a similar design.

Acceptance testing should exercise each boundary: an accepted message with no receipt, a delayed receipt, a recorded failure, a participant opt-out, completion before a repeat, a window that expires before delivery, and an entry waiting to synchronize. Confirm that each produces the intended record and action. The expected result for missing telemetry is an explicit unknown state, not an invented success or a participant reason. Verify that the study can export and interpret the evidence it will later rely on.

Use consistent, neutral instructions that direct the participant to the assessment without suggesting an answer. A synthetic example is: 'Your daily study diary is available until 22:00. Please use the study app to complete it.' Avoid wording such as 'record how much your stiffness improved,' because it cues a direction of change. Review the exact message against the instrument's administration instructions and study procedures, and document the rationale for approved variations.

Separate the purpose of a technical alert from an assessment prompt so the recipient knows what action is needed. FDA's DHT guidance recommends alerts about problems such as low battery, poor signal, or data not recording/transmitting, and processes that help participants understand how to respond. A sync warning calls for technical troubleshooting; it is not a request for the coordinator to supply or reconstruct the participant's answers. Keep reminder escalation within prespecified burden and contact limits. FDA DHT guidance, alerts.

Use a proportionate operational investigation when failures recur or affect important assessment windows. Proposed review signals include a cluster of explicit provider rejections, an unexplained loss of previously available receipts, or a group of missing assessments associated in time with a service problem. Check reporting coverage and eligible-message counts before interpreting a pattern. An isolated failure can matter, while a cohort-wide problem is not automatically a reportable regulatory breach; assess its impact rather than assigning severity from message count alone.

EMA Annex A4.11 addresses security incident procedures, criticality assessment, corrective and preventive action where applicable, and timely service-provider escalation to the sponsor. A reminder outage is not necessarily a security incident. Apply the relevant performance-incident procedure and investigate possible security or data-integrity implications when evidence warrants it. Agree in advance who preserves logs, evaluates affected windows, and notifies the sponsor; do not describe every delivery interruption as a legally reportable event. EMA incident-management context.

Pugh and colleagues compared NRG Oncology trials with and without automatic emails to institutional research associates, not participant push notifications. Form submission within one month was higher with emails: prostate trials 79.7% versus 75.7%, and breast trials 59.2% versus 31.3%. Submitted forms included explanations of noncompletion. Patient-completion differences were generally not maintained after adjustment, although an EQ-5D association remained in the prostate comparison. The retrospective cross-trial design could not isolate email effects from confounding; it does not establish a causal benefit for participant reminders. Pugh et al., 2021.

Stone and colleagues' 2002 paper-diary study provides a historical caution: reported compliance exceeded objectively observed compliance. It predates contemporary smartphone messaging and provides no estimate of today's reminder delivery performance. The transferable lesson is to identify what a record actually measures, rather than treating a reported status as direct observation of participant behavior. Stone et al., 2002.

Check the scope of a participant's decision and current channel preferences before each permitted escalation. A reminder opt-out can suppress one channel without ending trial participation. A withdrawal may refuse all further assessments or only particular activities; do not infer the scope from a treatment-stop flag. Suppress prompts for activities or channels the participant has refused, and route uncertainty through the site's consent procedures. See Withdrawal From a Trial: Separating New Collection From Retained Data.

Changing reminder cadence or adding a channel should enter the study's change-control process, including impact assessment and appropriate testing. Whether a protocol amendment or ethics review is needed depends on the change and applicable requirements; a purely technical configuration adjustment is not automatically a protocol amendment. The wider decision belongs in EClinCloud eCOA Change Control.

A late reminder must not become an instruction to backdate or reconstruct an answer outside the study's assessment rules. Keep data-entry time separate from transfer time and reconcile genuinely completed offline entries before deciding that an assessment is absent. For those adjacent decisions, see Which Time and Record Count After an Offline eCOA Gap and PRO Recall Periods: When Daily and Weekly Answers Differ.

Assign review ownership across clinical operations, data management, biostatistics, and the vendor. The deliverable is a traceable disposition for each unresolved assessment: what was observed, what was reported, what remains unknown, and which action was authorized. Related guidance covers eCOA and EDC reconciliation, Part 11 evidence for eCOA, and endpoint risk monitoring.