Content Validity for Trial Endpoints: Using PFDD Guidance 2 to Identify What Matters

In clinical trial design, teams routinely make a critical early assumption: that an existing, widely published clinical outcome assessment (COA) questionnaire already captures the disease symptoms and daily impacts that matter most to patients. When clinical development leaders, biostatisticians, and outcome-assessment specialists plan registrational programs, the operational temptation is to pull a legacy instrument off the shelf, verify that it has published internal consistency coefficients, and embed it directly into the protocol. This practice skips the fundamental measurement foundation of clinical drug evaluation: establishing that the concept of interest measured by the endpoint is truly meaningful and important to the target patient population.

The decision addressed here is specific, bounded, and prior to instrument selection: which qualitative, quantitative, or mixed methods FDA recommends under its Patient-Focused Drug Development (PFDD) framework to identify what is important to patients regarding disease burden and treatment burden, so that the endpoint's concept of interest is grounded in authentic patient voice rather than assumed from a legacy questionnaire. Answering this question correctly determines whether downstream endpoint scores measure clinical benefit that patients actually care about, or merely generate statistically precise measurements of arbitrary questionnaire items.

This workflow operates before and feeds into downstream clinical outcome assessment steps. It is not the job of evaluating whether a named instrument is fit for purpose, modifying an existing scale, or compiling an evidentiary rationale under FDA PFDD Guidance 3 (Selecting, Developing, or Modifying Fit-for-Purpose Clinical Outcome Assessments). It is not the downstream job of choosing whether a patient-reported outcome (PRO), clinician-reported outcome (ClinRO), observer-reported outcome (ObsRO), or performance outcome (PerfO) is the proper measurement modality once the concept is fixed, as addressed in our COA modality selection guide. Nor is it the task of draft PFDD Guidance 4, which governs incorporating COA-based endpoints into trial protocols, defining estimands, and estimating clinically meaningful change thresholds. Finally, it does not evaluate electronic collection modalities, which we examine in our eCOA mode measurement comparability analysis. Completing a concept elicitation study does not establish that a named instrument is fit for purpose, does not qualify a tool, and does not constitute regulatory acceptance of a trial endpoint.

To place the elicitation job correctly, teams need to know where Guidance 2 sits in FDA's Patient-Focused Drug Development series. The series was issued pursuant to section 3002 of the 21st Century Cures Act and PDUFA VI commitments. FDA's series page, content current as of 23 October 2025, still lists four documents: Guidance 1 (who provides representative input), Guidance 2 (what is important), Guidance 3 (fit-for-purpose COAs), and draft Guidance 4 (COA-based endpoints and meaningful change).

That series page, and Guidance 2's own introduction, split the work as follows:

• Guidance 1 (June 2020): Collecting Comprehensive and Representative Input. Addresses sampling strategies, defining the target population, and operationalizing methods to ensure that collected patient-experience data are accurate and representative.

• Guidance 2 (February 2022; announced in Federal Register 87 FR 11077): Methods to Identify What Is Important to Patients. Details qualitative, survey, and mixed methods to elicit patient input regarding symptoms, impacts on daily living, burden of treatment, and disease management priorities.

• Guidance 3 (October 2025): Selecting, Developing, or Modifying Fit-for-Purpose Clinical Outcome Assessments. Addresses evaluating, selecting, modifying, or developing a specific named instrument and whether it is fit for its intended context of use in CDER and CBER reviews. Completing Guidance 2 elicitation does not complete this job.

• Draft Guidance 4 (April 2023 draft): Incorporating Clinical Outcome Assessments into Endpoints for Regulatory Decision-Making. Addresses collection, capture, storage, and analysis of COA data; methods to incorporate COAs into endpoints used for regulatory decision-making; and methods to define meaningful change in a COA-based endpoint. It remains draft; it is not ICH E9(R1) and does not, by itself, specify estimand or intercurrent-event strategy.

flowchart TD
  G1["PFDD Guidance 1 (June 2020)\nRepresentative sampling and population definition\nWho provides input; sampling is not automatic from interviews"] --> G2["PFDD Guidance 2 (February 2022)\nConcept elicitation and patient priorities\nWhat symptoms, impacts, and treatment burdens matter"]
  G2 --> G3["PFDD Guidance 3 (October 2025)\nFit-for-purpose COA selection and development\nMatching a named instrument to the concept of interest"]
  G3 --> G4["Draft PFDD Guidance 4\nCOA-based endpoints and meaningful change\nCollection, analysis, and interpretation of COA endpoints"]

  classDef doc fill:#fdfbf7,stroke:#5c242b,stroke-width:2px,color:#22252a;
  class G1,G2,G3,G4 doc;

Guidance 2 is current thinking, not a regulation. It does not establish rights and is not binding on FDA or the public. The word should means suggested or recommended, not required. An alternative approach may be used if it satisfies applicable statutes and regulations. Completing an elicitation protocol does not constitute FDA acceptance of a concept, a COA, or an endpoint.

Guidance 2's introduction states that it should not be viewed as providing detailed instructions on how to use particular methods or as a substitute for engaging subject matter experts. FDA recommends engaging patients and subject-matter experts—including qualitative researchers, clinical and disease experts, survey methodologists, statisticians, psychometricians, and patient-preference researchers—and encourages early interaction with the relevant review division. Citation of a scientific reference in the guidance is not FDA endorsement of that method for a particular study.

Guidance 2 section II says research to understand what matters most to patients should begin with a characterization of the disease or condition and currently available therapies. Before conducting studies in patients, literature reviews, consultation with relevant subject-matter experts, and other information sources should be used to develop targeted research questions and select appropriate methods.

Those preparatory sources commonly include:

• Published peer-reviewed literature reviews examining clinical pathophysiology, patient-reported symptoms, and health-related quality-of-life studies.

• Clinical practice guidelines and natural history studies defining primary disease milestones, organ involvement, and phenotypic heterogeneity.

• Structured consultations with treating clinical experts, subspecialists, nurse navigators, and multidisciplinary care providers.

• Public transcripts and summary reports from prior FDA Patient-Focused Drug Development public meetings, listening sessions, and patient advocacy workshops.

Background work frames questions; it does not lock the concept. Guidance 2 uses that literature and expert work to develop research questions and choose methods. It does not, by itself, establish what is important to patients, and it is not a substitute for elicitation from people living with the disease.

Guidance 2 also tells teams to consider whether the study sample reflects the target population, including the spectrum of disease and patient diversity, and points that sampling decision to Guidance 1. Guidance 1 distinguishes probability sampling, which is more likely to reflect the target population, from non-probability methods, which may not be representative. A completed qualitative interview set is therefore not, by itself, evidence that the sample represents the intended population. Convenience samples—people who are available, volunteer, or can travel to a meeting—are a listed Guidance 1 non-probability method whose results may not be generalizable.

Guidance 2 states that qualitative methods, quantitative methods, or mixed-methods research can be used to identify what is important to patients, independently or complementarily. Method choice should follow the research objective. There is no single preferred qualitative method for all uses and research questions. FDA will review the rationale and appropriateness of the method chosen. There is not a universal number of interviews and focus groups; it depends on the research objective and, for certain methods, on achievement of saturation.

Each methodological family serves distinct, complementary evidentiary objectives:

• Qualitative methods such as interviews and focus groups are typically used to obtain a deeper understanding of the patient experience in patients' own words. Guidance 2 also lists other qualitative options in Appendix 1 (Delphi methods, observation, facilitated patient meetings, open-ended surveys) as options, not a required menu.

• Quantitative methods such as surveys collect quantifiable data and apply statistical methods to summarize, describe, compare, or relate measures of patient experience. A survey can quantify concepts already generated; it does not, by itself, discover them.

• Mixed methods combine qualitative and quantitative approaches in a single study or program of inquiry. Guidance 2 describes sequential designs (qualitatively driven or quantitatively driven) and concurrent designs. Its qualitatively driven sequential example uses focus groups and interviews first, then a survey to understand prevalence of the generated concepts in a larger population. A survey-first design is not a shortcut around open-ended concept generation when the concepts themselves are unknown.

When choosing between individual interviews and focus groups, match the method to the question. One-on-one interviews (structured, semi-structured, or unstructured) explore topics in depth at an individual level; Guidance 2 notes they are useful for sensitive topics and for diseases or conditions with heterogeneous symptom presentation. Focus groups elicit a range of experiences through participant interaction and can capture a range of perspectives in a shorter time frame. Sample sizes between 5 and 10 participants are common, but there is no set number; groups that are too large may fragment into simultaneous conversations and reduce individual engagement. Sensitivity of the topic (for example, separating sexes for sexual function) is one listed reason the number of groups may vary.

FDA does not have a single recommended administration method for interviews and focus groups. They may be conducted in person or remotely, including by computer or telephone. The method should be appropriate for the target population, study characteristics, and study objectives, and FDA will review that rationale.

Whatever method is chosen, Guidance 2 lists operational controls that belong in the protocol: pilot-test the interview or discussion guide and the planned length; select and train interviewers or moderators; and design questions so they can be administered without leading. Those controls are recommended practices, not a certified FDA training curriculum.

How questions are framed is critical to collecting unbiased patient input. Guidance 2 states that spontaneous responses are ideal; prompts may be needed but should avoid leading the participant. Leading questions include or imply the desired answer and may produce biased or misleading results and miss unexpected insight.

Approaches to avoid leading questions include, but are not limited to:

• Do not suggest an answer. Guidance 2's boxed example steers a peripheral-artery-disease patient toward walking distance as 'most important'; the replacement prompt asks how the disease impacts the patient and what the patient would like to see improve.

• Do not assume you know what the participant is thinking or feeling.

• Do not ask questions that cast judgment or imply a preferred response. Guidance 2's boxed example recasts 'why are you not treating your condition with medication?' as what the patient considered when deciding whether to treat the condition with medication.

• Control priming: order generally should flow from general to more specific. Screening questions can themselves prime; placing a detailed symptom inventory before open elicitation can cause participants to reuse the screener's vocabulary.

The same fence applies to surveys. FDA discourages incomplete, leading, unclear, double-barreled, and double-negative questions. Guidance 2's double-barreled example asks how embarrassed and self-conscious a respondent has been because of a condition—two concepts in one item. Priming occurs when information in an earlier question causes respondents to adjust later responses; order generally should flow from general to more specific, and screening questions themselves can prime. Closed-ended questions can omit important concepts that patients would have named in their own words. A survey whose response options already list the sponsor's preferred symptoms is not elicitation of what is important; it can at best quantify concepts already generated. The first two rows of the table below paraphrase Guidance 2's boxed examples; the remaining rows apply the same framing rules to closed-ended lists and to priming from a pre-administered inventory.

FDA's 2009 PRO guidance defines saturation as the point when no new relevant or important information emerges and collecting additional data will not likely add to the understanding of how patients perceive the concept of interest. Guidance 2 Appendix 4 restates that interviews are typically conducted until no new important concepts emerge from the last set of interviews with a diverse group of patients, and that use of the saturation principle depends on the research objective. There are no set criteria for how saturation may be evaluated.

Appendix 4 gives one example approach, not a required algorithm: concepts reported in the first 25 percent of planned interviews are compared with the next 25 percent, then with subsequent quartiles. If saturation has not been reached by the initial planned sample, or more information is needed in certain patient groups, more interviews may be needed. Table 7 in that appendix is a saturation-grid example summarizing successive focus groups; the N=6 per group / 24 total patients in that table is an illustration, not a recommended sample size. Before assuming saturation, Guidance 2 says to review participant demographics to assess whether the participants represent the intended target population as much as possible.

Neither Guidance 2 nor the 2009 PRO guidance recommends a universal interview or focus-group number. The 2009 guidance states that FDA cannot provide recommendations for the number or size of interviews or focus groups for establishing content validity; sample size depends on the completeness of information obtained from the transcripts, and the number of patients is generally not as critical as interview quality and patient diversity relative to the anticipated trial population. A small convenience sample that stopped hearing new themes is not, by itself, evidence that the concept of interest is comprehensive for the trial population.

An existing, widely published questionnaire does not automatically have content validity for this trial's population, condition, and treatment. Publication count and society endorsement are not substitutes for documented patient input.

FDA's December 2009 PRO guidance, still listed as current, remains the public definition of content validity for PRO instruments used to support labeling claims. Content validity is the extent to which the instrument measures the concept of interest, supported by qualitative evidence that items and domains are appropriate and comprehensive relative to the intended measurement concept, population, and use. It is specific to the population, condition, and treatment to be studied. The 2009 guidance is nonbinding, like the PFDD series. It still supplies three operating rules for this elicitation step:

• Item generation should include input from the target patient population. Without adequate documentation of patient input, a PRO instrument's content validity is likely to be questioned. Documentation of patient input in item generation, together with later evaluation of understanding through cognitive interviewing, can contribute to evidence of content validity.

• Other types of validity or reliability will not overcome problems with content validity. The 2009 guidance states that content validity should be established before other measurement properties are evaluated. High test-retest reliability, internal consistency, or responsiveness cannot rescue an instrument that measures an incomplete or irrelevant concept.

• With existing instruments, it cannot be assumed that the instrument has content validity if patients were not involved in development. New qualitative work using open-ended interviews or focus groups can provide that documentation. The 2009 guidance notes this is particularly important if a review of the instrument content gives cause for concern, for example if symptoms known to be common in the trial population are missing.

The same handoff is stated in FDA PFDD Guidance 3 (Selecting, Developing, or Modifying Fit-for-Purpose COAs). Guidance 3 states that identification of concepts of interest that are both appropriate for, and important to, a given target population in CDER and CBER decision-making is described in Guidance 2, and that such identification may involve qualitative research with patients, caregivers, and clinical experts. If important concepts have already been identified in previous studies based on patient, caregiver, clinical-expert, and other input, sponsors should reference and summarize that prior work when justifying their choice of concept. That is a documentation duty, not a license to skip elicitation when the match to this population and context of use is unshown. Concepts that have the potential to demonstrate a clinically meaningful effect within the planned trial time frame, and construction of the COA-based endpoint, are later jobs.

Guidance 3 footnote 33 also moves the former 'content validity' label. How well a measure reflects all important aspects of a concept of interest was previously referred to as content validity; the 2014 Standards treat validity as a unitary inference about score interpretation for a proposed use. That terminology shift belongs to named-instrument evidence under Guidance 3. This article stays on eliciting the concepts themselves under Guidance 2. Completing elicitation is not the entire validity argument for a named COA.

Patrick, Burke, Gwaltney, Leidy, Martin, Molsen, and Ring (Value in Health, 2011) is the ISPOR PRO good-research-practices task-force report on eliciting concepts for a new PRO instrument. Guidance 3 cites Patrick et al. 2011a for qualitative interviews or focus groups that identify meaningful aspects of health. Part 1 covers elicitation of key concepts; Part 2 covers cognitive interviews that assess respondent understanding of a draft instrument. The 2011 reports are ISPOR good practices, not FDA or EMA regulations. They do not replace Guidance 2, they do not set a universal sample size, and they do not make a completed concept-elicitation study into fit-for-purpose evidence for a named scale. Cognitive interviewing of draft items belongs to named-instrument work under Guidance 3, not the open discovery of concepts under Guidance 2.

Typical self-report techniques may not be appropriate for all target populations and may need to be tailored. Guidance 2 section VI covers usability of materials for people with physical, sensory, intellectual, or communication differences; limited attention span; pediatric drawing and props; and caregiver presence:

• Pediatric populations: Guidance 2 says young children may have limited attention span, so interviews may need to be conducted in shorter segments, and that elicitation can use drawing activities and props.

• Physical, sensory, intellectual, or communication differences: materials and administration should be tailored so respondents can participate. Guidance 2 also notes limited attention span as a reason to plan shorter interview segments.

• Caregiver presence: for respondents who have a caregiver, the caregiver generally should not be present during a patient interview (for example, they may be asked to sit outside the room). If presence is needed for comfort, the caregiver could sit behind the respondent to minimize influencing the interview verbally or nonverbally. The protocol should document caregiver presence or assistance and how patient versus caregiver data will be collected and reported so the source of the information is clear.

• Avoiding proxy reporting: for patients who are unable to self-report, eliciting what behaviors caregivers observe (including things the patients tell them) can help avoid proxy reporting—reporting from the caregiver as if they were the patient. Guidance 2 states that proxy reporting can lead to inappropriate inferences and may not reflect what a patient is thinking or feeling.

• Language, dialect, and culture: questions should be culturally sensitive. Poorly translated instruments can prevent comparable data. Guidance 2 says translatability of questions should be assessed if they will be used in multinational and multicultural studies.

• Social media: Guidance 2 says social media may be used to collect qualitative or quantitative data, but requires an appropriate research design, careful source selection because platforms differ, methods that address the inability to verify patient identity or diagnosis, data-quality checks including bots, and privacy and ethics controls. Social-media listening is not a representative sample of the trial population and does not replace Guidance 1 sampling.

A completed Guidance 2 elicitation identifies candidate concepts and meaningful aspects of health. The PFDD series still requires later work before a named instrument or endpoint is justified. The table below marks those handoffs. Screening and exit interviews inside a trial (Guidance 2 Appendix 5) can add depth on symptoms, functioning, expectations, and trial experience; they are not a substitute for pre-protocol elicitation of the concept of interest.

Completing elicitation under Guidance 2 documents what is important to patients about burden of disease and burden of treatment. It does not select whether the measure should be a PRO, ClinRO, ObsRO, or PerfO—a decision addressed in our COA type selection guide. Nor does it establish whether a named instrument is fit for purpose under FDA PFDD Guidance 3. Guidance 3 describes eight components that should be considered for an evidence-based fit-for-purpose rationale; different trials and contexts of use might call for fewer or more components than those in Table 2. Draft Guidance 4 remains the later methods document for turning a COA into an endpoint and defining meaningful change. Elicitation is not qualification, not labeling, and not regulatory acceptance.