Resident Research Using a Logic Model to Design, Implement and Evaluate a Program

1
Resident Research Using a Logic Model to Design,
Implement and Evaluate a Program

• Mary Jo Knobloch, MPH, CPH
• WREN Conference 2009

2
Association of American Medical Colleges Study

• Primary healthcare professionals need more
  training in basic research skills and increased
  opportunities to become involved in research
• Research education in the early years of medical
  training associated with continued interest in
  scholarly activity

3
Objectives

• To demonstrate use of a logic model as a
  framework for program development, implementation
  and evaluation
• To present preliminary results of 2008 and 2009
  assessment of first year residents

4
Current Setting

• Marshfield Clinic Residency Program
• Each program different in research learning
  approach
• 5 Dermatology
• 11 General Surgery
• 19 Internal Medicine
• 8 Medicine/Pediatric
• 11 Pediatric
• 2 Pharmacy

5
Logic Model Framework

• Basic Framework
• A living document
• Inputs
• Outputs Activities and Participation
• Outcomes-Impact
• Short
• Medium
• Long

6
UWEX Logic Model Site

• http//www.uwex.edu/ces/pdande/evaluation/evallogi
  cmodel.html

7
(No Transcript)
8
Short-term Outcome Measures Using Resident
Research Database

• THE NUMBERS -
• Number of resident projects
• Completed resident projects
• Abstracts, posters, manuscripts
• Number of active research mentors
• Diversity of project study designs
• QI and public health or advocacy projects

9
Resident Research DatabaseTracking the
Short-Term Outcomes
10
Medium and Long-term OutcomesUsing Assessment
Tool

• Increase in knowledge and skill level related to
  basic research methods and processes
• Increase in knowledge related to identification
  of study design and statistical methods
• Increase in skill level related to literature
  searches
• Change in confidence level related to
  understanding and implementation of research

11
Medium-Long Term Outcomes

• Continued interest in scholarly activity
• Use of evidence to guide practice
• Ongoing provision of multiple training
  opportunities

12
Results to Date

• Projects (the numbers)
• Assessment results (the learning)

13
Projects

• Incidence of Eosinophilic Esophagitis in MESA
• Relationship of Urine Cotinine Levels and Wound
  Implications in Patients Undergoing Reduction
  Mammoplasty
• Effects of HMR Weight Loss Intervention Program
  on Cardiovascular Disease Risk Factors
• Effect of Hyperglycemia on Outcomes in Acute
  Exacerbation of COPD

14
Projects

• CNS Blastomycosis in Marshfield Clinic System A
  Case Series
• A Pilot Project to Promote Early Literacy
• Cardiovascular Morbidity and Mortality among SLE
  Patients in a Rural Population-based Cohort
  ICTR project
• Severely Suppressed Bone Turnover and Atypical
  Skeletal Fragility
• Maja Visekruna, MD, Medicine/Pediatrics
• Faculty and Resident Perspectives on the
  Dermatology Integrative Competency Curriculum
• Modeling Clearance of Warfarin

15
Projects

• Implementation of a Resident Well-Being Survey
• Hamman-Rich Syndrome (Idiopathic Acute
  Interstitial Pneumonitis)
• Evaluating Clinical Compliance with IDSA
  Guidelines for Diagnosis and Management of Lyme
  Disease
• Incidence and Description of Anaplasmosis
• Rapunzel Syndrome in a 5 Year-Old (Case Report)
• Risk Factors Related to Recurrence of Umbilical
  Hernia

16
Projects

• Renal Cell Carcinoma Incidence Rates in
  Wisconsin
• A Retrospective Review of Patients Diagnosed with
  Stenosing Tenosynovitis (Trigger Finger and
  Thumb)
• Comparison of Wounds Before and After Sharp
  Debridement Using 16S DNA Sequencing
• Identification of Drug Therapy Problems through
  Medication Reconciliation in Heart Failure
  Patients
• The Role of Pharmacist Evaluation in Activation
  of Adult Asthma Patients (ICTR project).
• A Public Awareness Campaign about Insulin
  Dependent Diabetes

17
Projects

• Natural History of Idiopathic Neonatal
  Hypertension and the Long Term End Organ Effects
  in Children with Neonatal Hypertension
• Do Clinicians Document the AAP Recommended
  Clinical Information Used to Diagnose and Manage
  Otitis Media with Effusion?
• Defining Services and Referral Patterns for
  Infants with Positional Plagiocephaly within the
  Marshfield Clinic System
• Implementing a Pharmacy Continuing Professional
  Development (CPD) Project at Marshfield Clinic
• Factors Associated with Successful Matching and
  Residency Matriculation by Non-traditional
  Applicants to Dermatology Residency Programs
• Effects of Metabolic Syndrome on Disease State
  and Long-Term Outcomes in Patients with
  Colorectal Cancer

18
Projects

• Are Lytic Bacteriophages against Staph Aureus
  Found in the Anterior Nares of Humans?
• A Rare Presentation of Painless Aortic Dissection
  as Seizures
• Analysis of Dermatology Resident Self-Reported
  Successful Learning Styles and Implications for
  Core Competency Curriculum Development
• Caustic Ingestion in a 2-year old (Case Report)
• Assessment of Medical Residents Knowledge of
  Drug Interactions after an Educational
  Intervention

19
Combined Results for 2008 and 2009 First-Year
Residents(the learning)

• Assessment of attitude, confidence level and
  knowledge
• 33 residents

20
Attitude Questions

• To be an intelligent reader of the literature, it
  is necessary to know something about statistics
• 100 agree or strongly agree

21
Attitude Questions

• I often use statistical information in forming
  opinions or making decisions
• 51 disagree or strongly disagree
• Given the chance, I would like to learn more
  about biostatistics
• 75 agree or strongly agree

22
Level of Confidence Questions

• Finding clinical practice guidelines for patient
  care questions
• 57 say they have fair or little amount of
  confidence
• Exploring evidence using Cochrane Database
• 39 say little or no confidence

23
Confidence Questions

• Identifying factors that influence a studys
  statistical power
• 78 say fair, little or no confidence

24
Confidence Questions

• Interpreting P-value for a given result
• 70 say fair, little or no confidence
• 30 say a lot or complete confidence

25
Knowledge Questions

• Which of the following is true regarding
  p-values?
• 27 correct
• 73 incorrect
• Which of the following is NOT considered a
  vulnerable population?
• 38 correct
• 62 incorrect

26
Knowledge Questions

• Which of the following is an example of a
  testable null hypothesis
• 40 correct
• 60 incorrect

27
Knowledge Questions

• Which of the following is true with regard to the
  differences between efficacy and effectiveness
• 33 correct
• 66 incorrect
• Which one of the following measures of central
  tendency is least affected by outliers?
• 37 correct
• 63 incorrect

28
Knowledge Questions

• Which statement is not true about confidence
  intervals?
• 36 correct
• 63 incorrect
• The purpose of a double-blind or double-masked
  study
• 97 correct
• 3 incorrect

29
Knowledge

• These 2008/2009 first year residents had
  difficulty answering questions related to
• Study design
• Identifying discrete versus continuous variables
• Differences between efficacy and effectiveness
• Definition of bias
• Differences between mean, median, mode
• Sensitivity and specificity
• Definition of prevalence

30
Conclusions

• A program planning tool usually used for public
  health or intervention planning or grant planning
  can be used to set up and evaluate a resident
  research program
• Tracking systems must be in place
• Preliminary assessment of resident attitude and
  knowledge of basic epidemiology and biostatistics
  demonstrates the need for continued work in
  teaching research methods and skills

31
Questions?