Mixed methods approach

Fatoumata Fofana, Senior Research Associate, Statistics & Psychometric
Benoit Arnould, Senior Director
Alexia Marrel, Associate Director
Antoine Regnault, Research Director
All from Patient-Centered Outcomes, Mapi HEOR and Strategic Market Access

Mixed methods research can be defined as “research in which the investigator collects and analyzes data, integrates the findings, and draws inferences using both qualitative and quantitative approaches or methods in a single study or program of inquiry.”1
Mixed methods have been an active topic of research in many fields including educational research, sociology, psychology, and health sciences, and have led to the development of multiple approaches to address various research questions. Mixed methods have the advantage of being able to use the strength of an existing method to overcome the weaknesses of another method and thus strengthen the overall findings of the study. They can potentially be used and applied in clinical research and specifically in patient-centered outcomes research. However, until recently, the mixed methods approach has not been explicitly acknowledged in clinical research.
Mapi has extensive experience in both quantitative and qualitative research. Recently, we combined this expertise and explored several applications of mixed method approaches. Among the various possible applications, Mapi has identified three research questions pertaining to outcome research that mixed methods research could contribute addressing:

• Saturation
• Exploration and documentation of content validity of PRO measures
• Outcomes research in the context of rare diseases

In addition, Mapi has identified the operational challenges that represent the integration of mixed methods research within phase I/II clinical trials and has developed tailored solutions to address them.
Saturation
One crucial aspect of qualitative research is saturation.2 Saturation is the point when no new relevant or important information emerges and collecting additional data beyond this point will not add to the understanding of how patients perceive either the concept of interest or the items in the questionnaire under investigation.3 Current methods to demonstrate saturation are mainly based on the qualitative researchers’ judgments. Therefore, auditability and replicability of those methods cannot be validated while those two features are critical to strengthen the trust in results; that is key in clinical research. Therefore, Mapi is exploring the use of mixed methods research to document saturation in a more reliable way. A PhD program was launched in 2012 with the collaboration of the University of Nantes on the application of mixed methods research to investigate the perception of patients in clinical research.4-6
Content validity
Mapi has extensive experience in developing and validating PRO questionnaires. Mapi is constantly striving to utilize innovative methods to integrate patient viewpoints in relation to a specific medical condition or in the evaluation of a new drug. Therefore, one aspect Mapi is focusing on is the content validity of PRO questionnaires. Assessment of content validity provides empirical evidence that the content of items and domains of an instrument are consistent with a patient’s perspective and his/her experience.7 Content validity of a PRO endpoint is necessary to meet U.S. regulatory requirements for the development of PRO measures.3,8 Qualitative research is the state-of-the-art tool to support content validity.7 Beyond this traditional approach, there are two applications of mixed methods research that can support content validity:
A quantitative approach, such as item response theory (IRT), is increasingly used to complement cognitive debriefings.9
The integration of cognitive debriefings within phase I/II clinical trials.
Rare diseases
In the context of rare diseases, the generation of robust evidence on patient-centered outcomes is jeopardized by a scarcity of available data. These small patient populations need to be studied more carefully than other broader ones. This raises issues in evidence generation in general, but it is even more crucial for patient-centered research. When combined, lack of specific questionnaires, limited validity of generic questionnaires, and insufficient power severely limit the documentation of the patient’s and caregiver’s perspective in most orphan drug regulatory dossiers.
Due to the unique importance of the patient’s perspective in rare diseases, Mapi has a very specific interest in orphan drugs and the challenges they face during clinical development. We advocate that a mixed methods approach could address these challenges of patient-centered outcomes evaluation in rare diseases. Indeed, mixed methods research approaches combine standardized analysis warranted by quantitative research necessary for evidence generation and the richness of information on patient experience that can be collected, even in a small group of individuals, in qualitative research.10
Challenges of Mixed Methods Research within Clinical Trials: Operational Considerations
Utilizing words, pictures, and narrative to add clear meaning to numbers could be one of the main justifications for proposing a mixed methods research approach within phase I/II clinical trials. This approach raises questions about various operational aspects of a clinical trial such as: In what stage in clinical planning should mixed research be envisaged? What level of information should be included in the study protocol and what documents should be appended? What training should there be for the sites? Who should interview patients? What is the best interview technique? Because the analysis of the interviews is not performed on site, how will the data information be transferred offsite? How do we manage confidentiality while accessing the patient? How do we best manage patients’ verbatim reports, etc.?11
Mapi’s ambition in this context is twofold. The first is to contribute to innovation in methodology in clinical trials and the second is to assist the industry in the practical aspects of such innovation.
Mapi is very enthusiastic about exploring and expanding mixed methods research in patient-centered outcomes. However, these mixed methods research inquiries are recent in the patient-centered outcomes field. We are proud to be leaders in this development, searching the best way to emphasize the patient’s perspective among the diversity of ideas and perspectives across the mixed methods research landscape.
References
1. Tashakkori A, Creswell JW. Editorial: Exploring the nature of research questions in mixed methods research. Journal of Mixed Methods Research 2007;1:207-211.
2. Guillemin I, Marrel A, Lambert J, Beriot-Mathiot A, Doucet C, Kazoglou O, et al. Patients’ Experience and Perception of Hospital-Treated Clostridium difficile Infections: a Qualitative Study. Patient 2014;1:1-9.
3. U.S.Department of Health and Human Services, Food and Drug Administration, Center for Drug Evaluation and Research, Center for Biologics Evaluation and Research, Center for Devices and Radiological Health. Guidance for industry. Patient-reported outcome measures: use in medical product development to support labeling claims. Available at: http://www.fda.gov/downloads/Drugs/GuidanceComplianceRegulatoryInformation/Guidances/UCM193282.pdf
4. Fofana F, Bonnaud-Antignac A, Guillemin I, Marrel A, Beriot-Mathiot A, Regnault A. A mixed method approach to saturation: applying partial least square regression to qualitative data. Presented at: ISOQOL 20th Annual Conference; October 9 2013; Miami, FL, USA.
5. Fofana F, Bonnaud-Antignac A, Guillemin I, Marrel A, Beriot-Mathiot A, Regnault A. Applying partial least square regression to further explore concept structure and interviewee profiles in qualitative research. Presented at: ISOQOL 20th Annual Conference; October 9 2013; Miami, FL, USA.
6. Fofana F, Bonnaud-Antignac A, Regnault A. A mixed method approach to help demonstrate saturation in qualitative research: applying Partial Least Square regression to qualitative data. Presented at: 1st Mixed Methods International Research Association (MMIRA) Annual Conference; June 27 2014; Boston, MA, USA.
7. Brod M, Tesler LE, Christensen TL. Qualitative research and content validity: developing best practices based on science and experience. Qual Life Res 2009;18:1263-1278.
8. Patrick DL, Burke LB, Gwaltney CJ, Leidy NK, Martin ML, Molsen E, et al. Content validity–establishing and reporting the evidence in newly developed patient-reported outcomes (PRO) instruments for medical product evaluation: ISPOR PRO Good Research Practices Task Force report: Part 2—assessing respondent understanding. Value Health 2011;14:978-988.
9. Gorecki C, Lamping DL, Nixon J, Brown JM, Cano S. Applying mixed methods to pretest the Pressure Ulcer Quality of Life (PU-QOL) instrument. Qual Life Res 2012;21:441-451.
10. Arnould B, Symonds T, Houyez F, Regnault A. Can mixed methods research be the solution to the challenges of patient-centered outcomes research in the context of rare diseases? Presented at: ISPOR 17th European Annual Conference.
11. Marrel A, Calado F, Gruenberger JB, Fournie X. Mixed methods research within clinical trials: Operational considerations. Presented at: ISPOR 17th European Annual Conference.