HRPP Quality Assurance Team

1. The types of PAM Reviews that are conducted are:
   • Routine PAM Review that is conducted as a quality assurance activity and randomly chosen, to ensure that human research studies are conducted according to regulatory requirements and the IRB approved protocol.
   • For-Cause PAM Review that is based upon a complaint, a concern or an IRB required review to ensure that the study is being conducted according to regulatory requirements and the IRB approved protocol.
2. A written notification will be sent to the Principal Investigator (PI) that will outline the type of review to be conducted and provide information about the process that will be followed for the review.
3. A representative from HRPP will contact the PI approximately five (5) business days to schedule a visit to conduct the review and arrange a time for the initial meeting with the PI and research team in order to review the audit schedule, procedures, and plan.
4. The review consists of:
   A.  A review of research documentation:
   • Source documents pertinent to the study.
   • Correspondence between the PI and the sponsor and/or the IRB.
   • Informed consent forms (ICFs).
   • Research subject records.
   B.  A tour of the facilities where study procedures are conducted, as well as interviews with the research staff.
   C.  A review of any other procedures necessary to understand the conduct of the study.
   D.  At the end of each review day on-site, there will be time for questions to be answered and requests for documents needed.
5. At the conclusion of the review, an exit interview will be held with the PI and the study team to discuss any findings and to answer any questions.
6. A factual report about the conduct of the study will be prepared by an HRPP Analyst and presented to the IRB Executive Committee for review at a full board meeting and may be accepted as presented with no further action, a clarification memo sent, or a debriefing memo requesting a corrective action plan.  The report and the supporting documentation will be included in the PI’s profile within eIRB for future reference.  A copy of the report is available on request by the PI.

Post Approval Monitoring (PAM) Routine Review

A Routine Review is an essential part of the oversight responsibilities of the Rutgers University Human Research Protection Program (HRPP) and an important quality assurance measure. Research studies are selected by a random process. Each study element is reviewed including the conduct of the study at the site, the IRB Review standards and the HRPP administrative process.

PAM Routine Review Process: How is a study selected?

Each HRPP Analyst will obtain a list of all ongoing, IRB-approved human subjects research studies from the Rutgers University HRPP electronic Institutional Review Board (eIRB).  eIRB includes studies conducted by students, staff and faculty at Rutgers Schools and Rutgers Health Sciences programs, irrespective of performance site. A representative number of studies will be scheduled for routine review based on the current total of IRB-approved studies. All studies will be categorized by the type of research involved and the potential risk to the safety, welfare, or rights of study subjects.

Each HRPP Analyst selects a minimum of two (2) to four (4) studies a month to review.  All human subject studies are eligible for PAM review irrespective of sponsor, performance site or which IRB reviews the study.

When selecting a study for PAM Routine review, below are some of the factors considered:

• What is the study's risk level?
• Was the protocol approved with any conditions?
• Which studies involve an FDA-regulated product (e.g., drug, device, or biologic)?
• Does the study's PI hold an FDA Investigational New Drug (IND) or Investigational Device Exemption, with clinical trials.gov requirements?
• Who initiated the study (Investigator, Sponsor)?
• When was the last time a study at each school or performance site was selected for a routine review?
• Does this study recruit vulnerable populations?
• Has this study had any issues with enrollment (i.e., not enrolling, over enrolling, under enrolling)?
• What is the source of funding (grant/contract) and are there additional Sponsor requirements?
• How many studies is the PI currently conducting?
• How are study records, drugs, devices tracked and secured?
• Does the study need a follow-up visit after a for-cause review to confirm that the corrective action plan is being followed?

PAM Review Preparation

How is a Routine Review scheduled?

The HRPP Analyst will notify the PI by sending a “Notification of a Routine Review memo” via Rutgers email. The HRPP Analyst provides (10) business days for the PI to respond to the routine review request.  If there is no response, the HRPP Analyst will begin a correspondence log to track the efforts of scheduling an initial visit. If the HRPP Analyst is unsuccessful with arranging the routine review, they will seek the support from the Executive Director of the Human Research Protection Program.

When the PI responds, the Analyst will arrange a mutually convenient time for the initial meeting and then a time with the study staff to review of research records including but not limited to:

• Self-assessment questionnaires based upon the type of study
• IRB submissions, approvals, reportable events, SAEs, protocol deviations
• Sponsor, regulatory and subject binders/files
• Subject research record and/or medical records, as necessary
• Electronic data, where applicable

PAM review guidance, checklists, manuals and self-assessment questionnaires are available on the Rutgers Research website within the HRPP Toolkit and are used by the Analysts during the review.

One or two days before the review, the HRPP Analyst will contact a member of the study team to confirm details (i.e., date, time, and place) and to answer any final questions the team may have. This communication is followed up by an email to summarize the conversation.

What are the responsibilities of the HRPP PAM Team?

The HRPP Analyst shall conduct a preparatory review of each study document within the full eIRB submission. This review of study materials will be documented in chronological order and will later be compared to the PI’s eIRB study file.

The study will be reviewed for compliance with 45 CFR 46 (DHHS), NJ State law and 21 CFR 50, 56, 312, 600, 812, and ICH GCP -E6 (if applicable), the research site standard operating procedures, as well as Rutgers University institutional policies and procedures.

The HRPP Analyst will meet with study staff and review the on-site research study files to verify completion of the following items:

• Initial submission, with IRB approval letter
• Protocol and amendments/modifications, protocol deviations with IRB approval letters
• Consent process documentation and consent forms with IRB approval letters
• Continuing reviews, with IRB approval letters
• Reported adverse events
• Correspondence (e.g., IRB debriefing letters, investigator responses to the IRB, emails)
• IRB approval from all collaborating sites/institutions
• Delegation of authority logs, training and education of staff related to the study
• All source documents
• Correspondence with study monitors and/or clinical research organizations (CROs) and study sponsors.
• The HRPP analysts will contact the research pharmacist to review all records regarding the dispensation of investigational/study drugs, if applicable.
• The Medical Records Department for access to subject records and may need PI notification to the Chief/Executive Director to gain access to the specific set of subject records associated with the source documents that support the conduct of the study.

What are the responsibilities of the Principal Investigator and the Study Team?

Prior to the review, the PI and study team are responsible for the following items:

• Securing meeting space (i.e., office or conference room) for the on-site document review
• Complete the self-assessment questionnaires
• Ensure the regulatory binder and subject binders are complete and up to date
• Ensure that original eligibility checklists, ICFs, CRFs, and all source documents for each subject are organized in an appropriate format to be reviewed
• Ensure completed case report forms (CRFs) and research files for each subject
• Ensure that all source documentation for all subjects is complete

It is essential that all electronic source documentation is printed or provided in an appropriate electronic format in preparation for the review.

PAM Routine Review Visit

What happens during the initial interview?

The on-site review of the requested study begins with an initial interview. The PI is encouraged to invite his/her co-investigator(s), study coordinator(s), and any other personnel named on the protocol.

At the initial interview, the HRPP Analysts will address the following questions. Questions may vary according to specific study details:

• How is your study conducted?
• How are subjects recruited?
• What is the informed consent process?
• What staff is involved in study activities, and what are their roles and qualifications?
• How many subjects were approved by the IRB; enrolled or completed the study?
• What source documents are available to support the conduct of the study?
• Is this the only performance site?
• Where are the original signed and dated consent forms stored?
• Have there been any serious adverse events/unanticipated events, or protocol deviations and have they been reported to the local IRB and sponsor (if applicable)?
• Where are study medications/investigational drugs/investigational devices stored and dispensed (if applicable)?
• Has there been a lapse in IRB approval?  If so, why?

What happens during the tour of the facility?

The HRPP Analyst will request a tour of the facility to observe where the research is being conducted and location of study records. The PI, co-investigator, or study coordinator should be able to answer the following questions:

• Where are the study records and ICFs secured and stored as outlined in the protocol?
• Where does the consent process take place?
• Where are physical exams and/or study procedures conducted?
• Where are study medications/investigational drugs/investigational devices stored and dispensed (if applicable)?
• Are there any temperature/storage logs for the study medications/ investigational drugs/investigational devices?
• Where is the study equipment located?
• When were study equipment calibrated?
• Where are the computer stations and source documents located?  Is the area private and secure?

What study documents will be reviewed?

The PI shall be asked to provide a space for the HRPP Analysts to review study records and arrange for research staff to be available during the routine review to assist and provide answers to questions regarding the conduct of the study. At the end of each day, study staff will be asked to clarify any questions that arise during the review. Examples of documents to be reviewed are as follows:

• Regulatory Files with appropriate IRB Approval documents:
  • Initial IRB review submission
  • Protocol, consents, advertisements, with all amendments/modifications
  • Continuing reviews, reportable events, protocol deviations
  • Correspondence (e.g., IRB debriefing letters, investigator responses to the IRB, emails)
  • Training and education of staff related to the study
  • Location of study documents (i.e., confidentiality, limited access, separate from study data)
  • Appropriate recruitment of subjects with respect to protocol-specific inclusion/exclusion criteria (i.e., enrollment and screening logs)
  • Signed consents equal the number of enrolled subjects
  • Content of signed consents corresponds to IRB-approved versions
  • Proper execution of each informed consent (i.e., signed before study activities) and documentation of the consent process
  • Determination of any enrollment during lapse in IRB approval
  • Source documentation of protocol-specific study procedures, interventions, study visits, follow-up, adverse events, and unanticipated problems (e.g., CRFs, computer generated study visit forms)

In the event the review identifies serious non-compliance, such as lack of oversight, deliberate falsification or omission, failure to follow the protocol and/or the requirements and determinations of the IRB, subject safety concerns, the HRPP Analysts will immediately notify the Executive Director of the HRPP who will review the findings and determine whether notification of the Institutional Official (IO) and the IRB of record is required.

What happens during the exit interview?

The exit interview will occur approximately within 72 working hours of the completion of the review. The PI must be present during the exit interview. The co-investigator(s) and/or study team member(s) are also strongly recommended to attend.

The HRPP Analyst presents any review findings and responds to any questions from the PI and study team. It is important at this point to discuss any concerns and provide opportunities for clarification. Any missing, incomplete, or incorrect data should be given to the HRPP analyst within ten (10) business days of the exit interview. If no communication is received regarding the missing, incomplete, or incorrect data, then the deviation will be cited in the final report. Immediate action by the Chief/Executive Director and notification of the Institutional Official would take place in the event of serious non-compliance, continuing non-compliance that may affect subject safety risks, or possible research misconduct.

What is included in the final PAM report?

The HRPP Analysts will complete the final PAM report within two (2) to four (4) weeks of the exit interview. The final report outlines the specific factual details of the review, including the dates of the review, any findings, observations and information provided during the PI and study team interviews.

The HRPP Senior Analysts will electronically send the PAM report to the IRB administrator who will place it on the next IRB Executive Committee agenda.

How can the IRB Executive Committee respond to the final PAM report?

If there were no findings, the PAM Report is placed on the IRB Executive Committee agenda to be reviewed, discussed and accepted. No formal written response by the PI will be required if the PAM review of the study is deemed “accepted with no further action” by the IRB Executive Committee. A notice will be provided to the PI of this action.

If there are any findings of non-compliance, then the PAM Report is placed on the next IRB Executive Committee agenda for review and the PI is notified. At the Executive IRB Committee meeting, the HRPP Analyst presents a verbal report, that is then reviewed, discussed and any required action is determined. A written response by the PI is required if the reviewed study requires a corrective action plan or clarifications by the IRB Executive Committee. A notice will be provided to the PI of this action.

The IRB Executive Committee has the responsibility to take immediate action if serious or continuing non-compliance is found, including suspension of the study and/or recommendation of closure. In the event a Rutgers University IRB is not the IRB of record, the Rutgers Executive IRB Committee acts as a designee of the IO in the protection of human subjects as required by Rutgers Federal Wide Assurance (FWA).

Overview

The purpose of the Requested (For-Cause) review is to determine if the rights and welfare of participants are protected according to research ethical standards, federal regulations, state laws, and Rutgers Policy and Procedures.

Requested (For-Cause) reviews are requested by the Institutional Review Board (IRB), the Institutional Official (IO), or Rutgers HRPP Executive Director as a result of a complaint or a concern about how a study was reviewed or conducted.

A requested (for-cause) reviews could range from a review focused on a specific concern, allegation, or complaint, to an in-depth examination of all components of a research study. The review may include all records and source documents, observations of processes, and interviews with investigators, research staff members, and participants.

Study Selection: How is a study selected?

The convened IRB Committee, IRB Chair, or HRPP Executive Director may instruct an HRPP Analyst to conduct a review in response to a particular concern, allegation, or complaint. The allegation, complaint or concern may come from the Rutgers University Compliance portal or directly to the HRPP/IRB inbox by Individuals who may provide their identity or remain anonymous that may include, but are not limited to, the following parties:

• a research participant or his/her family member
• a research team member
• faculty, staff or students at Rutgers University or the performance site

PAM Requested Review Preparation

How is a PAM requested review scheduled?

The Director of the HRPP Analysts or designee will notify the PI by sending a formal “Notification of a Requested Review” via Rutgers email. The memo outlines the reason(s) for the requested review and any action to be taken by the PI such as a hold on the research study. The PI is informed that a HRPP Analyst will contact him/her within approximately five (5) business days. If there is no response, the HRPP staff will begin a correspondence log to track the efforts of scheduling an initial visit. If the HRPP team is unsuccessful with arranging the requested review, they will seek the support from the Executive Director of the Human Research Protection Program.

When the PI responds, the Analyst will arrange a mutually convenient time for the initial meeting and then a time with the study staff to review the research records including but not limited to:

• Self-assessment questionnaires based upon the type of study
• IRB submissions, approvals, reportable events, SAEs, protocol deviations
• Research Protocol and Consent process and forms
• Sponsor, regulatory and subject binders/files
• Subject research record and/or medical records, as necessary
• Electronic data, where applicable

PAM review guidance, checklists, manuals and self-assessment questionnaires are available on the Rutgers Research website within the HRPP Toolkit and are used by the Analysts during the review.

One or two days before the review, the HRPP Analyst will contact a member of the study team to confirm details (i.e., date, time, and place) and to answer any final questions the team may have. This communication is followed up by an email to summarize the conversation.

What are the responsibilities of the HRPP QA Team?  

The HRPP Analyst shall conduct a preparatory review of each study document within the full eIRB submission. This review of study materials will be documented in chronological order and will later be compared to the PI’s eIRB study file.

The study will be reviewed for compliance with 45 CFR 46 (DHHS), NJ State law and 21 CFR 50, 56, 312, 600, 812, and ICH GCP -E6, etc. (if applicable), the research site standard operating procedures, as well as Rutgers University institutional policies and procedures.

The HRPP Analyst will meet with study staff and review the on-site research study files to verify the following items:

• Initial submission, with IRB approval letter
• Protocol and amendments/modifications, protocol deviations with IRB approval letters
• Consent process documentation and consent forms with IRB approval letters
• Continuing reviews, with IRB approval letters
• Reported adverse events
• Correspondence (e.g., IRB debriefing letters, investigator responses to the IRB, emails)
• IRB approval from all collaborating sites/institutions
• Delegation of authority logs, training and education of staff related to the study
• All source documents
• Correspondence with study monitors and/or clinical research organizations (CROs) and study sponsors.
• The HRPP analysts will contact the research pharmacist to review all records regarding the dispensation of investigational/study drugs, if applicable.
• The Medical Records Department for access to subject records and may need PI notification to the Chief/Executive Director to gain access to the specific set of subject records associated with the source documents that support the conduct of the study.  

What are the responsibilities of the Principal Investigator and the Study Team?

Prior to the review, the PI and study team are responsible for the following items:

• Securing meeting space (i.e., office or conference room) for the on-site document review
• Complete the self-assessment questionnaires
• Ensure the regulatory binder and subject binders are complete and up to date
• Ensure that original eligibility checklists, ICFs, CRFs, and all source documents for each subject are organized in an appropriate format to be reviewed
• Ensure completed case report forms (CRFs) and research files for each subject
• Ensure that all source documentation for all subjects is complete

It is essential that all electronic source documentation is printed or provided in an appropriate electronic format in preparation for the review.

PAM Requested Review Visit

What happens during the initial interview?

The on-site review of the requested study begins with an initial interview. The PI is encouraged to invite his/her co-investigator(s), study coordinator(s), and any other personnel named on the protocol.

At the initial interview, the HRPP analysts will address the following questions. This is not a comprehensive list, as the questions may vary according to specific protocol details:

• How is your study conducted?
• How are subjects recruited?
• Have there been any concerns or complaints by study subjects?
• What is the informed consent process and how is the process documented?
• What staff is involved in study activities, and what are their roles and qualifications?
• How many subjects were approved by the IRB; enrolled or completed the study?
• What source documents are available to support the conduct of the study?
• Is this the only performance site?
• Where are the original signed and dated consent forms stored?
• Have there been any serious adverse events/unanticipated events, or protocol deviations and have they been reported to the local IRB and sponsor (if applicable)?
• Where are study medications/investigational drugs/investigational devices stored and dispensed (if applicable)?
• Has there been a lapse in IRB approval?  If so, why?

What happens during the tour of the facility?

The HRPP Analyst will request a tour of the facility to observe where the research is being conducted and location of the study records. The PI, co-investigator, or study staff should be able to answer the following questions:

• Where are the study records and ICFs secured and stored as outlined in the protocol?
• Where does the consent process take place?
• Where are physical exams and/or study procedures conducted?
• Where are study medications/investigational drugs/investigational devices stored and dispensed (if applicable)?
• Are there any temperature/storage logs for the study medications/ investigational drugs/investigational devices?
• Where is the study equipment located?
• When were study equipment calibrated?
• Where are the computer stations and source documents located? Is the area private and secure?

What study documents will be reviewed?

The PI shall be asked to provide a space for the HRPP Analysts to review study records and arrange for research staff to be available during the routine review to assist and provide answers to questions regarding the conduct of the study. At the end of each day, the PI and/or study staff will be asked to clarify any questions that arise during the review. Examples of documents to be reviewed are as follows:

• Regulatory Files with appropriate IRB Approval documents:
  • Initial IRB review submission
  • Protocol, consents, advertisements, with all amendments/modifications
  • Continuing reviews, reportable events, protocol deviations
  • Correspondence (e.g., IRB debriefing letters, investigator responses to the IRB, emails)
  • Training and education of staff related to the study
  • Location of study documents (i.e., confidentiality, limited access, separate from study data)
  • Appropriate recruitment of subjects with respect to protocol-specific inclusion/exclusion criteria (i.e., enrollment and screening logs)
  • Signed consents equal the number of enrolled subjects
  • Content of signed consents corresponds to IRB-approved versions
  • Proper execution of each informed consent (i.e., signed before study activities) and documentation of the consent process
  • Determination of any enrollment during lapse in IRB approval
  • Source documentation of protocol-specific study procedures, interventions, study visits, follow-up, adverse events, and unanticipated problems (e.g., CRFs, computer generated study visit forms)

In the event the review identifies serious non-compliance, such as lack of oversight, deliberate falsification or omission, failure to follow the protocol and/or the requirements and determinations of the IRB, subject safety concerns, the HRPP Analysts will immediately notify the Executive Director of the HRPP who will review the findings and determine whether notification of the Institutional Official (IO) and the IRB of record is required.

What happens during the exit interview?

The exit interview will occur approximately within 72 working hours of the completion of the document review. The PI must be present during the exit interview. The co-investigator(s) and/or study team member(s) are also strongly recommended to attend.

The HRPP Analyst presents any review findings or concerns and responds to any questions from the PI and study team. It is important at this point to discuss any concerns and provide opportunities for clarification. Any missing, incomplete, or incorrect data should be given to the HRPP analyst within ten (10) business days of the exit interview.  If no communication is received regarding the missing, incomplete, or incorrect data, then the deviation will be cited in the final report. Immediate action by the Executive Director and notification of the Institutional Official would take place in the event of serious non-compliance, or continuing non-compliance that may affect subject safety risks, or possible research misconduct.

What is included in the final PAM Requested Review Report?

The HRPP Analysts will complete the final PAM report within two (2) to four (4) weeks of the exit interview. The final report outlines the specific details of the allegation, complaint or concern and how the review of how the study was conducted related to the allegation, complaint or concern. Also included are the dates of the review, any findings, observations, corrective actions underway, as well as information provided during the participant, PI and study team interviews.

The HRPP Analysts will electronically send the Requested Review Report to the IRB administrator who will place it on the next IRB Executive Committee agenda.

How can the IRB Executive Committee respond to the final PAM Requested Review Report?

If there were no findings, the PAM Report is placed on the IRB Executive Committee agenda to be reviewed, discussed and accepted. No formal written response by the PI will be required if the PAM review of the study is deemed “accepted with no further action” by the IRB Executive Committee. A notice will be provided to the PI of this action.

If there are any findings of non-compliance, serious non-compliance or continuing non-compliance, then the PAM Report is placed on the next IRB Executive Committee agenda for review and the PI is notified. At the Executive IRB Committee meeting, the HRPP Analyst presents a verbal report, that is then reviewed, discussed and any required action is determined. A written response by the PI is required if the study requires a corrective action plan or clarifications by the IRB Executive Committee. A notice will be provided to the PI of this action.

The IRB Executive Committee is constituted as required by Rutgers Federal Wide Assurance (FWA), including members of the university or performance site compliance officers, and has the responsibility to take immediate action if serious or continuing non-compliance is found, including suspension of the study, recommendation of closure and/or review of the PIs ongoing research studies. Remedial education and training in the protection of human subjects may also be required.  In the event a Rutgers University IRB is not the IRB of record, the Rutgers Executive IRB Committee acts as a designee of the IO in the protections of human subjects as required by Rutgers FWA.

PAM Resources

• (HRP-025) SOP - Directed Review (For Cause) Audits
• (HRP-025a) Initial Interview Routine or For-Cause Review
• (HRP-025b) Tour of the Facility for On-site Routine or For Cause Review
• (HRP-025c) Review of Study Documents
• (HRP-025e) Role of the Executive Committee for Review of Audit Reports

The regulations we use in the audit process include:

• Code of Federal Regulations 45CFR and 21CFR
• ICH GCP Guidelines
• Rutgers Policies and Procedures

• Rutgers University Policy for the Protection of Human Subjects
  Human Research Protection Program (HRPP) Toolkit
• Rutgers University Standard Operating Procedures (SOPs)
  Human Research Protection Program (HRPP) Toolkit
• 45 CFR 46 DHHS Policy for Protection of Human Subjects
  45 CFR 46 | HHS.gov
• 21 CFR 50 FDA Protection of Human Subjects
  eCFR :: 21 CFR Part 50 -- Protection of Human Subjects (federalregister.gov)
• 21 CFR 56 FDA Institutional Review Boards
  eCFR :: 21 CFR Part 56 -- Institutional Review Boards (federalregister.gov)
• 21 CFR 312 FDA Investigational New Drugs
  CFR - Code of Federal Regulations Title 21 (fda.gov)
• 21 CFR 600 FDA Biological Products
  CFR - Code of Federal Regulations Title 21 (fda.gov)
• 21 CFR 812 FDA Investigational Devices
  CFR - Code of Federal Regulations Title 21 (fda.gov)
• NJ C.26:14 Access to Medical Research Act 2008
  S1757 2R (state.nj.us)
• International Conference Harmonization (ICH) Good Clinical Practice (GCP) Guidelines-E6  2018
  https://www.ema.europa.eu/en/ich-e6-r2-good-clinical-practice
• Research Site SOPs for the Conduct of Human Subjects Research
  Note: Specific to the research site

• ICH GCP-Essential documents for the conduct of a clinical trial
• FDA Inspection References: Resources used by FDA investigators and inspectors in their daily activities
• OHRP: Compliance Oversight Procedures for Evaluating Institutions (2009)
• (HRP-430) CHECKLIST - Investigator Quality Improvement Assessment
• (HRP-430a) CHECKLIST - Post Approval Monitoring Self-Assessment - Drug or Device Clinical Trial
• (HRP-430b) CHECKLIST - Post Approval Monitoring Self-Assessment - Participant File
• (HRP-430c) CHECKLIST - Post Approval Monitoring Self-Assessment - Biomedical Research
• (HRP-430d) CHECKLIST - Post Approval Monitoring Self-Assessment - Social Behavioral Research
• (HRP-430e) CHECKLIST - Post Approval Monitoring Self-Assessment - Humanitarian Use Device (HUD)
• (HRP-430f) SOP - Exit Interview Routine and For-Cause Review