Abstract

Organizations conducting clinical research can benefit from establishing quality and risk management systems to measure performance, using objective metrics to measure efficiency, safety and quality. The development of metrics involves identifying the values of an organization and the goals that express those values, developing measures to assess meeting those goals, and building infrastructure to capture data to support the metrics and develop adequate and timely response to drive improvement.

Keywords: Quality Assurance (QA); Quality Control (QC); Risk-Based Quality Management (RBQM); Key Risk Indicators (KRIs); Key Performance and Quality Indicators (KP-QI); Metrics

Quality Management Systems (QMS)

Quality Management Systems (QMS) are based on a methodology to produce reliable and high quality end results as a systematic deliverable by developing verifiable standards for processes that can control for variation [1]. In order to sustain QMS procedures must be developed and documented, then implemented and updated throughout a life cycle of a project (Figure 1). Training of sponsors, contract research organizations (CROs) and site personnel must be conducted on QMS, and computer systems utilized in quality management must be validated.

Monitoring of clinical sites and technical facilities should be conducted on site or by using centralized monitoring techniques for data management and quality control (QC), and internal and external audits performed by independent auditors [1]. Quality Control (QC) and Quality Assurance (QA) systems together constitute key quality systems that are parts of Quality Management Systems (QMS) as shown on Figure 2 [2,3].

Quality Management Systems should be aimed to achieve the following:

1. Overall Quality Control (QC) plan
2. Sampling plan to be used (i.e. reserved samples, if applicable)
3. Data source to be used for QC at each operational stage: test methods should be scientifically sound and written procedures established for standards validation and verification of processes with pre-defined specifications (i.e. packaging, shipping, preparation and dispense of investigational product (IP))
4. Metrics to be documented
5. Acceptable quality levels
6. Management of compliance according to the study protocol, Standard Operating Procedures (SOP), and Good Clinical Practices (GCPs)
7. Resolution of system problems based on lessons learned prior and to the end of the study
8. Reduction of data queries
9. Identification of ways to reduce cycle times for various processes
10. Ensure data integrity throughout course of study and that data collected are the data required by the protocol
11. Ensure the accuracy and consistency of data from entry into the case report forms (CRFs) to final datasets and presented in final study report
12. Deal with issues of nonconformity, while carrying out clinical trials
13. Deliver an accurate and complete final study report

Quality Control (QC)

Purpose of Quality Control (QC) is to ensure compliance with SOPs, FDA, sponsor’s protocol and local regulatory bodies; verify staff training of GCP, SOPs and federal/local regulatory requirements; develop and use Quality Improvement (QI) program within organization by utilizing standardized forms and checklists to ensure complete and accurate documentation; conduct periodic internal reviews to ensure compliance and address deficiencies found through internal QC or sponsor monitoring visits that should be corrected [2,3].

Documentation for each study should be reviewed periodically and should include the following information:

1. Who performed each task
2. Degree of delegation of authority by Principal Investigator (PI)
3. How and where data were recorded
4. Verification of protocol adherence
5. How study test article accountability and dispensation are/were maintained
6. Communications between sponsor and PI
7. IRB documentation, communications and approvals

Process validation
Where the results of a process cannot be fully verified by subsequent inspection and test, the process shall be validated with a high degree of assurance and approved according to established procedures. According to ICH GCP, Title 21 CFR 820.75, when changes or process deviations occur, the sponsor shall review and evaluate the process and perform revalidation where appropriate.

Process change control
All changes are initiated as improvements but they can have the potential to “in”validate a valid process. According to ICH GCP, Title 21 CFR 820.70 (b) all changes should be verified or where appropriate validated before implementation and these activities documented. Procedures for changes to a specification, method, process or procedure should be established and maintained.

Corrective and Preventive Action (CAPA) Plans
When discrepancies are identified the CAPA plans should be implemented, which include the following:

1. Potential problems should be identified and steps taken to avoid them
2. Discovery of a problem should trigger an immediate corrective action
3. Development of a plan to prevent recurrences. A re-evaluation of the system should be performed to ascertain how the problem occurred (Root Cause Analysis)
4. Documentation to avoid any questions from an auditor

Quality Assurance (QA)

Independent review/examination of the work/trial and all those planned and systematic actions that are established to ensure that the trial is performed and the data are generated, documented (recorded), and reported in compliance with Good Clinical Practices (GCPs) and the applicable regulatory requirements [2,3].

A rationale for a risk-based approach to clinical trial conduct
Traditionally, sponsors understood the FDA expected frequent (every 4-6 weeks, depending on the rate of enrollment or specific issues identified at the site) onsite monitoring visits. In most cases assuming 100% source data verification (SDV) was needed for all trials, regardless of study design or complexity (Table 1). Even with such high level of monitoring intensity, neither the integrity of data nor investigator performance improved, but rather triggered increased reactive response to problems identified with site performance [4]. Forecasting monitoring intensity for clinical trial has become increasingly complex as regulatory requirements continue to expand and differ between different countries and regulatory agencies, despite attempts at harmonization [4].

In 2013 the Food and Drug Administration (FDA) and other regulatory bodies published complete administrative and regulatory requirements on risk-based, centralized monitoring of clinical trials, and clinical research professionals are still learning how to adapt to this new concept [5-7]. Risk-based monitoring is an adaptive approach to clinical trial monitoring that directs monitoring focus and activities to the evolving areas with greatest need, which have the most potential to impact subject safety and data quality. This guidance draft was subsequently finalized and aims to achieve the following [5]:

1. Building Quality by Design (QbD) into clinical trials
2. Proactive, early and ongoing risk assessment
3. Focus on critical process and critical data
4. Key performance indicators
5. Adjustments in monitoring based on issues and risk
6. On site monitoring versus centralized/remote monitoring (i.e. source data and electronic case report forms (eCRFs) can be assessed remotely) and analysis of data trends not detected by on-site monitoring
7. Standard checks of range, consistency, completeness of data, unusual distribution of data between sites
8. Targeted on site monitoring for data abnormalities or higher frequency errors, protocol violations, or drop-outs rate relatively to other sites
9. Data quality of real-time data entry (e.g. missing data, inconsistent data, data outliers, deviations
10. Statistical analyses of study data to identify sites that are outliers, evaluate subject data for plausibility and completeness
11. Analyses of site characteristics, performance metrics (high screen failure rates, eligibility violations, delays in reporting data), characteristics correlated with poor performance or noncompliance

As the industry’s utilization of risk-based monitoring continues to increase along with the development and expansion of Risk-Based Quality Management (RBQM) systems, the need for the integration of these two concepts becomes apparent [8-10]. The premise behind risk based monitoring (RBM) is that monitoring quality can be improved by leveraging existing data intelligence. This, in turn, calls for more robust quality assurance (QA) systems focused and efficient resource utilization and allocation at the clinical site level. RBQM is the proactive identification and mitigation of risks (Table 1). By combining these two concepts, the inherent risks identified upfront can feed into the design of the risk-based monitoring plan (Quality by Design, QbD). RBQM at site level requires development of well-defined and relevant metrics, key performance and quality indicators (KP-QI), as well as a solid process for review and follow-up of the identified signals (Table 1). Both aspects need to be supported by robust information management as well as training and cross-functional communication strategies.

RBM uses a systematic approach to risk identification, providing a structured process for assessment, control, review and re-assessment of risks throughout a lifecycle of the project. It allows sponsors to focus monitoring efforts on source documents, critical data elements, and compliance with regulatory requirements and protocol adherence to ensure research subjects safety and to verify data quality and integrity at a participating sites [9,10].

Initiating the RBQM process
The concept of RBQM begins with review of study protocol, a literature research and analysis of risks associated with investigational product (IP). The RBQM development team should include representatives from study team, safety, monitoring, statisticians, and data management [8-10]. Assembled team should conduct in-depth review of factors to consider: know and anticipated risks based on IP’s risks/benefits profile, study procedures, complexity of study protocol design, disease/condition under investigation, phase of study conducted (i.e. first in human versus later phases of clinical development program), vulnerability of study population, study duration and assessment of study personnel at the site level (i.e. experience in clinical research, expertise in a specific therapeutic filed under study, personnel turnaround, workload, site performance, rate of enrollment, GCP compliance history, access to site’s electronic medical records and data quality).

Next is initial risk assessment and designation by RBQM development team (i.e. low, medium, high risk) based of probability and impact. Figure 3 illustrates a quality risk planning and management process that includes assessment of study protocol for complexity with focus on critical processes and critical data, development of key performance indicators, building quality by design (QbD) into clinical trials, training of research personnel, early and ongoing risk assessment throughout life cycle of the trial, implementation of quality management strategies, and adjustments in monitoring based on issues and risks identified.

RBQM proposed approaches
Although the framework can vary between different clinical trials, risk profile should be established and risk analysis performed before the trial begins and adjustments should be made throughout the trial as new risks possibly could emerge.

Prioritization and risk mitigation strategies should be implemented across several dimensions:

1. Protection of study subjects - rights and integrity, safety
2. Credibility of data and results
3. Stratified according to knowledge of investigational product Customized approach should be utilized depending on:
4. Protocol complexity
5. Therapeutic indication and nature of endpoints, population co-morbidities, concomitant medications
6. Administration of investigational product (timing of administration, dose, formulation, route)
7. Complexity of study procedures and data points collection, nature of intervention
8. Vulnerability of the study population

Different people have different tolerance for risks. Tolerance should be balanced against rewards and can be affected by individual bias. During RBQM development and subsequently implementation team meetings information regarding tolerances should be communicated and discussed to find common grounds, so decisions can be made to reduce and/or accept risks.

Where risks are to be mitigated, the methodology adapted from conventional GCP should be utilize to determine type of monitoring visits needed: intensive, regular or reduced on-site monitoring and/or central monitoring, targeted site visit focused on primary endpoint variables, safety issues or major protocol deviations.

Risks must be communicated to all stakeholders/decision makers as well as results and new information (i.e. new pre-clinical data, long term mutagenicity/carcinogenicity animal data, updated Investigator Brochure, protocol amendments) and ongoing review (i.e. Data Monitoring Committee Meeting report, Audit report/concerns).

If a risk occurs, it has consequences such as study protocol re-design and/or investigational product re-design (e.g. changes in device specifications or dose, route and regimen of administration for study drug/biologic) maybe needed. If risk realizes, it can cause delays and scope changes. Risks must be controlled and process put in place for actions needed if risks are identified, especially for high risks. However, there may be implications for low risks such as multiple minor deviations can affect data integrity. Process of risk assessment should be documented (risk management measures/risk indicators) with review of the measures need to be taken to mitigate the risks or implement contingency plans as necessary, as shown on Figure 4.

To effectively implement RBQM, it is essential to identify study-specific risks, develop multidisciplinary strategies and processes to target those risks; and incorporate a holistic analytical and operational approached focused on centralized monitoring and action plans.

Developing the monitoring plan based on Quality by Design (QbD) approach
When developing a monitoring plan, it is important to define which activities will be performed remotely and which ones will be completed on site. Remote monitoring should include review of regulatory documents (i.e. CVs, licenses, etc.), financial disclosures, IRB correspondence, study supplies and IP accountability logs, site communications, monitoring reports, action items, and documentation of site personnel training records and study logs. The focus of “on site” monitoring should include site’s standard operation procedures (SOPs), informed consent documentation, source data verification (SVD), adverse events and serious adverse events (AE/SAE) documentation and reporting within required timeframes, confirmation of subjects eligibility, IP accountability, study endpoints, regulatory and protocol compliance, and any site performance issues. The monitoring plan should be broad enough to include some flexibility, thresholds/triggers that would require review and actions, and study risks implementation plan.

Utilization of a centralized metrics dashboard will aid quick identification and assessment of emerging key risk indicators (KRIs).

Key Risk Indicators (KRIs)

In an operational context a key risk indicator is a metric that provides information on the level of exposure to a given operational risk which the organization has at a particular point in time. KRIs must have an explicit relationship to the specific risk whose exposure it represents. Controls need to be implemented to reduce/mitigate a given risk exposure [11]. While developing KRIs, the factors influencing study quality and integrity should be assessed [11,12]. Among such factors to consider are the following:

1. Well-designed protocol
2. Data Safety Monitoring Plan (DSMP) which is study specific
3. Mix of centralized and on-site monitoring practices
4. Electronic case report forms (eCRFs) versus traditional case report forms (CRFs) review
5. Study objectives
6. Complexity of the study protocol
7. Therapeutic area, indication
8. Critical data
9. Patient safety
10. Qualifications, experience and training
11. Site clinical research experience in specific therapeutic field
12. Level of experience, education and training of Investigators, research staff and monitors

Also critical data and processes should be defined such as:

1. Regulatory agency and ethics committee approval
2. Standard Operating Procedures
3. Study protocol complexity
4. Data supporting primary and secondary objectives
5. Data critical to subject safety: SAEs and events leading to investigational drug/device discontinuation
6. Data critical to trial design and statistical endpoints (efficacy and safety outcomes)
7. Adherence to eligibility
8. Informed Consent process, prior to any study related procedures
9. Documentation of administration of investigational agent or treatment procedures
10. Training/education of study personnel
11. Adequate monitoring and audits

Implementation of RBQM. An academic site perspective
Site should take a proactive approach if participating in a clinical trial where remote monitoring based on RBQM approach will be utilized. The first step is to request to review protocols in draft stages and perform risk assessment. If this approach is implemented the site will no longer have the “routine” monitoring visit, but targeted monitoring visits. If risks drive visits, then the goal is to have a site that has evidence of low risk. What was routinely done by the monitor will now need to be done by the site personnel with increased reliance on technology. Therefore, timely data entry and query resolution are critical. Monitors will focus on source data review with regards to:

1. Quality of source data
2. ALCOA principles - Attributable, Legible, Contemporaneous, Original, Accurate data are recorded for research purposes
3. Protocol compliance
4. Evidence of investigator involvement
5. IP Accountability
6. Laboratory reports

Beyond Quality and Compliance Management to Developing Core Competencies in Clinical Research

Personnel training
To enable personnel (i.e. Principal Investigators, co-Investigators/sub-Investigators, study coordinators, research nurses, ancillary services such as blood bank personnel and institutional Investigational Drug Services (IDS)) to perform the assigned functions, all involved parties should have appropriate education and documentation of training on GCP, study protocol and SOPs, as applicable per specific study protocol and based on their roles and responsibilities on the study. Prior experience with investigational product is desired to prepare IP and be familiar with QC principles and acceptable methods for complying with the regulatory requirements of cGMP, GCP, etc.

Investigator’s oversight
Principal Investigator (PI) must personally conduct the study and provide an oversight for all activities performed within given study protocol, which includes among other responsibilities review of screening data with formal, documented approval for enrollment, assessment of diagnostic reports and all adverse events with assignment of causality [13]. PI also must review all protocol deviations and determine the impact on human subject protection, safety, data quality and integrity [13].

Site facilities and equipment
Facilities and equipment can be crucial to study conduct and must meet the following requirements:

1. Sufficient space, required environment (e.g. sterility control/verification), appropriate construction
2. Appropriate lighting, ventilation, heating, cooling, plumbing, washing, and sanitation
3. Appropriate air handling systems (e.g., laminar flow hoods) to aid in preventing contamination and cross-contamination of investigational product
4. Appropriate equipment: avoid contamination of the product; not reactive, additive, or absorptive with product; is properly maintained, calibrated, cleaned, and sanitized at appropriate intervals following written procedures
5. Complex preparation requirements for biologics (process = product): sensitivity to heat, temperature, and pressure. Handle and store to prevent degradation or contamination
6. Written SOPs: handling, review, acceptance criteria and control of components used in combination product
7. Components should be segregated, labeled, until examined or tested, as appropriate, and released for use in production or to be used at site level
8. Keep records of receipt and use of study supplies

Defining organizational metrics
Each organization should define specific indicators of progress towards its goals in the course of identifying meaningful metrics. These indicators can include both traditional metrics that typically assess activities, services, or the time elapsed between events, and value-based metrics that measure an aspect of the quality of the service or activity [11-14].

Clinical research sites provide a standard set of site performance data to potential sponsor during feasibility assessments for potential participation in a clinical trial. If tracked and analyzed, these data can be valuable measure of site performance.

For example, the process of clinical trial conduct can be evaluated by developing metrics for each stage from protocol development through completion of a clinical research project. The timelines and financial aspects of conduct can be assessed using traditional metrics, as well as by measuring value-based metrics, which are designed to capture the quality and integrity of the process [14].

Institutional Review Boards (IRBs) track submissions and approval dates for study protocols, study accrual rates, protocols violations, and other data for regulatory compliance purposes. This information can be used to improve participating in a clinical trial site performance. Similarly, administrative data collected for study start up activities such as contract review time frame, budget review and approval, time to enrollment of the first patient can be utilized to assess efficiency and feasibility of study conduct.

Traditional metrics are usually quantitative and reflect study personnel, financial indicators, time intervals and other resources. They can be useful for identifying trends related to issue of budgets, supplies, personnel turnarounds, etc. These metrics would not provide an insight into reasons for a duration, turnaround and/or prolongation of activity or study completion (Table 2).

The value-based metrics include measures of study quality, satisfaction, regulatory approval timelines, and study milestones completion. They reflect progress towards the objectives and focus of capturing process and outcomes [14,15].

Conclusion

Besides being a new expectation by regulatory agencies under Good Clinical Practices, Quality by Design (QbD) and Risk-Based Quality Management (RBQM) concepts are receiving attention on a world-wide basis. As the industry's utilization of risk-based monitoring continues to increase along with the development and expansion of the area of RBQM, the need for the integration of these two concepts becomes apparent. The premise behind RBQM is that monitoring quality can be improved by leveraging existing data intelligence. This, in turn, calls for more robust quality assurance (QA) systems focused and efficient resource utilization and allocation at the clinical site level. RBQM is the proactive identification and mitigation of risks. By combining these two concepts, the inherent risks identified up front can feed into the design of the risk-based monitoring plan QbD. RBQM requires development of well-defined and relevant metrics, key performance and quality indicators (KP-QI), as well as a solid process for review and follow-up of the identified signals. Both aspects need to be supported by robust information management as well as training and cross-functional communication strategies.

Implementing knowledge gained from metrics collected can be a challenging task which requires organizational commitment. However, having data to support the implementation as an effective tool to both enhance value of the organization’s initiatives and demonstrate capabilities.

The future of RBQM is about adopting a data driven approach to trial management using key risk indicators to determine monitoring intensity, and new processes to enhance patient safety and improve quality of clinical research conducted. Quality management based on remote and risk-based monitoring methods, which relies on technologies for proactive data assessment, rapid statistical analysis for detection and resolution of problems, and continuous trend analysis to identify and resolve issues should improve quality and compliance in clinical trials that utilize RBQM approach.