Social and Character Development in Elementary School: The Effectiveness of the Making Choices Program

1
Social and Character Development in Elementary
School The Effectiveness of the Making Choices
Program
Preliminary Findings

• Mark W. Fraser, PI, School of Social Work,
  UNC-Chapel Hill
• Steven H. Day, School of Social Work
• Shenyang Guo, School of Social Work
• Alan Ellis, Sheps Center and School of Social
  Work
• Roderick A. Rose, School of Social Work
• Maeda J. Galinsky, School of Social Work
• Kim Dadisman, Co-PI, Center for Developmental
  Science, UNC-CH
• Dylan Robertson, Center for Developmental
  Science
• Tom Farmer, School of Education, Pennsylvania
  State University

This presentation was given at the School of
Social Work, University of Maryland, Baltimore,
MD, on April 9, 2009. Portions of this report
were presented at the annual meeting of SACD
Project grantees on June 13, 2008 in Washington,
DC
2
Agenda

• Theoretical Bases and programs
• Design and challenges
• Analytic strategies
• Analytic methods (skim see slides)
• Findings

3
Acknowledgments

• This project was support by a cooperative
  agreement (R305L030162) with the Institute of
  Education Sciences at the U.S. Department of
  Education (US DOE). Funding for the project was
  appropriated by the US DOE and the Centers for
  Disease Control and Prevention.
• We thank Paul Rosenbaum (U Penn), Ben Hansen (U
  of Michigan), and Matthias Schonlau (Rand Corp)
  for their consultation on methodological issues
  related to this presentation.

4
Teachers Talk about Making Choices

• Changes in Classroom Atmosphere
• Observable Differences
• in Student Behaviors
• Measurable Academic Achievement

5
Classroom Atmosphere

• I noticed that the classroom started working
  more as one big group instead of individuals.
• Gr.5 Sandy Grove Elementary,
• Hoke County

6
Observable Behaviors

• The students tend to be less critical of each
  other and more understanding of each others
  differences.
• Gr. 5 Sandy Grove Elementary,
• Hoke County

7
Academic Achievement

• The program uses excellent books to support
  the goals of being a good friend and not hurting
  others. I use them during Language Arts time.
  
• Gr. 4 Tommys Road Elementary, Wayne County

8
Observable Behaviors

• It provided a way for students to put their
  feelings into words.
• Gr.
  2, Bunn Elementary,
• 
  Franklin County

I am feeling really mad!
9
Academic Achievement

• My students spend more time on task. They
  seem less distracted by annoying behavior.
  
• Gr. 5 Scurlock Elementary, Hoke
  County

10
Make the right choice!

• Children are actually stopping and thinking about
  making the right choices, and I have heard a lot
  of children say to themselves,
• Make the right choice.
• It is great to hear.
• Kdg. Bunn Elementary, Franklin County

11
It made a difference with teaching children how
to deal with their feelings using better methods
rather than having tantrums or hitting.Kdg.
Bunn Elementary, Franklin County
Oh, boy! I need that Making Choices program.
12
Classroom Atmosphere

• This program provided a foundation on which we
  could build a classroom community.
• Gr. 1 North Drive,
• Wayne County

13
Children in my school need
social skills to make friends and deal with
interpersonal problems.
lessons that teach children respect toward
others and responsibility for their own actions.
a program designed to reduce disruptive
behavior and promote academic achievement.
Making Choices
14
Does social and character education work?

• Research Question

(i.e., is Making Choices effective?)
15
Intervention Research Perspective The Design and
Development Approach

1. Specify the problem and develop a program theory
2. Create and revise program materials
3. Refine and confirm program components (sequential
   experimentation perspective)
4. Assess effectiveness in a variety of practice
   circumstances and settings
5. Disseminate findings and program materials

Source Fraser, M. W., Richman, J. M., Galinsky,
M. J., Day, S. H. (2009). Intervention
research Developing social programs. New York,
NY Oxford University Press.
16
PROBLEM THEORYPerspectives on Conduct Problems
and Academic Achievement in Elementary School
Social and Character Development in Childhood A
Risk and Resilience Orientation

• Developmental risk perspective
• Ecological theory
• Social information processing theory

17
Eco-Developmental Risk Cascade
POINT Risk factors for poor developmental
outcomes vary over time. Lacking effective
intervention, the potential for poor outcomes
increases and cascades as function of complex
bio-social processes. To promote positive
outcomes, we must disrupt malleable risk
mechanisms.
18
Cognitive Mediation Model(in Developmental
Sciences)
Biological Predisposition
Biological Predisposition

• Parenting
• Monitoring
• Bonding

• Mental Processes
• Social knowledge
• Scripts
• Schema/skills

• Conduct Problems
• Conduct disorder
• Fighting
• Drug use

• Peers
• Deviancy training
• Contagion effect
• False consensus
• effect

• Sociocultural Context
• Stress/poverty
• Racism
• Street codes
• Acute/chronic stress

• Sociocultural Context
• Stress/poverty
• Racism
• Street codes
• Acute/chronic stress

Adapted from Dodge, K. A., Pettit, G. S.
(2003, p. 351). A biopsychosocial model of the
development of chronic conduct problems in
adolescence. Developmental Psychology, 39(2),
349-371.
19
Social Information Processing Theory SIP Skills
and Emotional Regulation as Malleable Mediators?
State the problem
Interpret social cues
Set goal(s)
Generate potential solutions
Encode social cues
Assess outcomes
Evaluate potential solutions
Select enact the best solution(s)
Social Knowledge Life experiences producing
scripts, schemata, skills, and beliefs
20
PROGRAM THEORY (specifies how a program is to
work)
21
Intervention Program Structure Such as Targeting
Unit Classroom Entire school Other (after
school, family) Curriculum Structure
Distinct activities Embedded in
curriculum Activities to address SACD Goals Such
as Character education Violence prevention/peace
promotion Social and emotional development Toleran
ce and diversity Risk prevention and health
promotion Behavior management
Social and Character Development Prevention Model
Behavior Positive Behavior Responsible
behavior Prosocial behavior Self-regulation Cooper
ation Negative behavior Aggression Minor
delinquency Disruptive classroom behavior
Social - Emotional Competence (mediator) Attitudes
about aggression Self-efficacy Empathy
School Climate (mediator) School
connectedness Victimization Feelings of safety at
school Parent involvement
Academics Academic competence School
engagement Grades Standardized test scores
Moderating Factors Child Family Community Gend
er Parenting practices Community risk
factors Socioeconomic status Home
atmosphere Social capital Race/ethnicity Risk
status Program School Prior test
scores/grades Fidelity Activities to promote
social and character development Intensity and
dosage Organizational structure
22
Social Development Model Perspective
23
PROGRAMS
24
The Competence Support Program
Social Dynamics Training for teachers
Social Skills Training for students
Group randomization Cohort 1 Hoke and Wayne
Counties (10 schools randomized to 5
intervention 5 control) Cohort 2 Franklin
County (4 schools randomized to 2 intervention 2
control)
Classroom Behavior Management Training and
Consultation for teachers
Developed by the program investigators, the
intervention simultaneously focuses on the
characteristics of children and on the classrooms
in which they learn. The intervention combines
three components.
25
Program Elements

• Making Choices Skills Training curriculum for
  students in elementary school. In-service
  training introduced teachers to the risks of peer
  rejection and social isolation, including poor
  academic outcomes and conduct problems.
  Throughout the school year, teachers received
  consultation and support (2 times per month) in
  providing lessons designed to enhance childrens
  social information processing and other skills.
  As a part of the Standard Course of Study, the
  program was integrated into routine class
  instruction.
• Classroom Behavior Management provided teacher
  consultation on classroom management strategies
  designed to strengthen engagement in
  instructional activities.
• Social Dynamics Training provided teacher
  consultation on classroom contexts, social
  groupings, and interactional patterns that can be
  used to reinforce academic achievement and
  prosocial behavior.

26
Theory of Change Making Choices
Core 1
Core 2
Core 3
Core 4
Core 5
Random Assignment
Application of Making Choices by Teacher or
Counselor
SIP skills of the Children in the School
Impact on Social Engage-ment and Peer Rejection
Training the Teacher or Counselor
Impact on Disruptive Behavior and Academics

• Test the degree to which the intervention is
  delivered as intended, e.g., specific activities

• Assess implementation of training
• Assess if teacher acquires skills from
  training/supervision

Characteristics of the Teacher or Counselor
Characteristics of the Children and the Classroom
Note. In a randomized trial, you must figure out
a way to measure each of the core elements.
Treatment as Usual Control Condition
27
Make Program Manuals

• From risk mechanisms, mediators, and logic models
  to the design of a program
• Specifying program activities that target the
  malleable mediators and have cultural congruence
• Example Making Choices

For a discussion of issues in the development and
use of treatment manuals, see Galinsky, M. J.,
Terzian, M. A., Fraser, M. W. (2006). The art
of group work practice with manualized curricula.
Social Work with Groups, 29(1), 11-26.
28
Warning It is easy to under estimate the
difficulty of developing a program manual.
29

• That Sunk Feeling

If you start in the wrong place, it usually does
not help to dig deeper!
Source Don Moyer, Harvard Business Review
(October, 2004, p. 160)
30
Start with theory and research, plus practice
experience
How to begin in the right place

• Develop a template for each lesson or session

31
Grade 2
Lesson 2
Overview
Standard Course of Study
Prep Materials
Activity 1
Review
Prop
Answers
Process Tip
32
Avoid labeling
Scenarios
Activity 2 Write About It!
33
Develop all worksheets and artwork
34
(No Transcript)
35
(No Transcript)
36
Pete the Penguin Poster for Grade 2
37

• Sample Lesson
• Activities
• from
• Making Choices

38
Gr. 3 Lesson - Intentions

• SYMBOLS

39
Intentions Mean or Friendly?
40
GOAL SETTING

• GOAL Something a person wants or something a
  person wants to see happen.
• RELATIONSHIP GOAL Goals that involve wanting to
  get along with another person.
• Grade 4 Lesson 6

41
Are these Relationship Goals?
(thumbs up or thumbs down)

• I want to make an A on my math test.
• I want to play more often with my friend.
• I want a new video game for my birthday.
• I want to eat out at a restaurant for dinner.
• I want to become friends with the new student.
• I want to join in the basketball game at recess.
• I want to sit with Jose on the bus.
• I want to be in the class play this fall.
• I want to stop getting upset when friends ignore
  me.

42
GOAL SETTING

• Set a relationship goal for these situations
• I was playing basketball at recess with some
  friends. Terrell, who is not very good at
  basketball, asked if he could play with us.

43
Set a Relationship Goal

• Denise just made me really upset. She tried to
  pick a fight with me by saying things that are
  not true. I am feeling angry with her right now.

44
Set a Relationship Goal

• Yesterday, my mom gave me a really cool pen that
  writes in all different colors. When I brought
  it to school this morning, Stacey asked me if she
  could borrow it. Last time I let Stacey borrow
  something she lost it, but if I say no she might
  get angry with me.

45
EVALUATION DESIGN Cluster Randomized Trial with
Ten Schools Randomly Assigned to Treatment (j5)
and Control (j5) Conditions Cohort Design
Intervention provided in grades 3, 4, and 5

• Prior Studies
• Single-group qualitative trial of MC intervention
  (8th grade girls)
• Two-group cluster randomized trial at classroom
  level in one middle school (6th grade only)
• Two-group cluster randomized trial at classroom
  level in one school (3rd grade)
• Two-group, MCSF intervention randomized trial
  (11 sites, 3rd 4th grade)
• Cohort sequential study by classroom in two
  schools (3rd grade)
• (Current) Two-group cluster randomized trail at
  14 elementary schools

46
SAMPLE
47
Two Overlapping Samples
Grade 3 n571
Grade 4 n557

• Grade 3-4 Sample
• 3rd and 4th graders
• 10 schools
• Any consented students on a 3rd or 4th grade
  roster
• Changeentrants-leavers

Grade 5 n433
n414
n370

• Grade 3-4-5 Sample
• 3rd, 4th, and 5th graders
• 9 schools
• Only consented students on a 5th grade roster
• Changeaddition of entrants

One treatment school was reorganized into a
different building and dropped the program
between 4th and 5th grade students from that
school were excluded from the 3-4-5 sample.
48
Equivalence of Intervention and Control Groups on
Selected Child, Family, and School Attributes
Grade 3 Cohort 1
49
Difference in School-Level Academic Performance
Percentage at Grade Level
Test results for 2005-06 and 2006-07 are based on
a revised accountability model and are not
comparable to those from previous years.
50
Sample sizes vary because pretest measures were
collected from different respondents (teachers,
students) at different times. SLA and Peer
assessment pretest were collected from students
at the end of 2nd grade. CCC and ICST were
collected from teachers at the beginning of 3rd
grade. SLASkill Level Assessment (SIP skill
HOME Scale adaptation by Dodge, 1980).
CCCCarolina Child Checklist (Macgowan et al.
2002 Research on Social Work Practice).
ICST-Interpersonal Competency Scale Teacher
(Xie et al., 2002, Social Development)
51
(No Transcript)
52
Causation in Research Design Randomization Is
Supposed to Produce the Counterfactual
Note. We let the control group serve as evidence
for what would have happened counter to the fact
of participation in intervention (the stat
class). Randomization is supposed to create
equivalence or balance between the intervention
(taking the stat class) and control (not taking
the stat class) groups. But it didnt. On several
observed and an unknown number of unobserved
measures, the intervention and control group
schools differ.
53
Four evaluation challenges

• Selection Bias Covariates are not balanced
  between treated and control groups
• Missing Data No baseline data on enterers and
  lost data on leavers constant churning of
  sample
• Rater Effects Outcome ratings were made by the
  same teachers within grades, but different
  teachers over grades 3, 4, and 5
• Piecewise analyses change scores within grade
  level
• Treatment Contamination/History High
  intervention content in control schools

Note. Student Citizen Act (SL 2001-363) was
passed into law by the Legislature in 2001. The
Act required local boards of education to develop
and incorporate character education instruction
into standard curricula. Local boards of
education began implementation in the 2002-2003
school year.
54
MEASURES
55
Site-Specific Outcomes

• Skill Level Assessment Activity (SLA) Based on
  the Dodge Home Scale (1980), the SLA uses
  students responses to questions about
  hypothetical social situations. After viewing
  picture scenarios, students answer questions
  measuring different aspects of social information
  processing skill encoding (a.78), goal
  formulation (a.76), and response decision making
  (a.80).
• Carolina Child Checklist (CCC) The CCC is a 35
  item teacher questionnaire that yields factor
  scores on childrens behavior including social
  contact (a.90), cognitive concentration (a.97),
  social competence (a.90), and social aggression
  (a.91).
• Interpersonal Competence Scale-Teacher (ICST)
  The ICST is an 18-item teacher questionnaire that
  yields factor scores on childrens behavior
  including aggression (a.84), academic competence
  (a.74), and popularity (a.78).
• Peer interpersonal assessments Peer
  interpersonal assessments were used to examine
  classmates perceptions of participants social
  and behavioral characteristics including
  aggression (a.92), prosocial skills (a.84 ),
  and internalizing behavior (a.67 ).

56
Summary of Data Collection Occasions
57
Minutes of Skills Training Instruction in 3rd and
4th Grades by Student
Below benchmark 19 Above benchmark
81 (overall n571)
58
ANALYTIC PROCEDURES
59
Analytic Procedures Flow Chart for Use of
Bias-Correcting Statistical Methods
Multiple Imputation of Missing Data (The
imputation models employed both predictor and
outcome variables, but the analysis models
employed imputed missing values for predictor
variables only). 50 imputations for each outcome
variable
Estimation of propensity scores using
Generalized Boosted Modeling (gbm) -- aims to
optimize balance on observed covariates between
treated and control groups
Optimal pair matching using propensity scores
estimated by gbm
Optimal full matching using propensity scores
estimated by gbm
Piecewise change score HLM analysis using
propensity score weighting (propensity scores
estimated by gbm)
Heckman sample selection Model (Predictors of the
selection equation are similar to the input of
gbm)
Post-pair-matching with regression adjustment
Post-full-matching with Hodges-Lehmann aligned
rank test
Dose (efficacy subset) analysis) using Abadie et
al. Matching estimator
60
Procedures for multiple imputation of missing data

• Test for MCAR (Little, 1988) confirms models are
  not MCAR.
• Assumption of MCAR not required if imputation
  model is informed (i.e., data may be missing at
  random)
• A diagnostic stage identified models that
  resulted in 99 relative efficiency for all
  analysis variables.
• 50 simulations (copies of the raw data set)
  generated using MI.
• DVs and predictors both used in imputation
  imputed DVs discarded after imputation (MID
  procedure von Hippel, 2007).

61
Missing Data Diagnostics Proportion without
Missing Data and Proportion of Missing Data Points
62
General Procedure for Propensity Score Analysis

• Step 2Analysis using propensity scores
• Analysis of weighted mean
• differences using kernel or local
• linear regression (difference-in-
• differences model of Heckman
• et al.)

• Step 1 Logistic regression
• Dependent variable log
• odds of receiving treatment
• Search an appropriate set of
• conditioning variables
• (boosted regression, etc.)
• Estimated propensity scores
• predicted probability (p) or
• log(1-p)/p.

• Step 2 Analysis using propensity scores
• Multivariate analysis using
• propensity scores as weights

• Step 3 Post-matching analysis
• Multivariate analysis
• based on matched
• sample

• Step 2 Matching
• Greedy match (nearest neighbor
• with or without calipers)
• Mahalanobis with or without
• propensity scores
• Optimal match (pair matching,
• matching with a variable number
• of controls, full matching)

• Step 3 Post-matching analysis
• Stratification
• (subclassification)
• based on matched
• sample

63
Estimating propensity scores

• Need relevant conditioning variables
• Obtain best logistic regression (i.e., best
  functional forms) however, no way to know
• Used Multiple Additive Regression Trees (MART) to
  run logistic regression. Rand Generalized Boosted
  Modeling (gbm) Aims for best balance on observed
  covariates between treated and controlled groups.
  Iteration stops when the sample average
  standardized absolute mean difference (ASAM) is
  minimized.

64
Example of gbm output Does gbm reduce the
difference between treated and control schools?
Point After using gbm propensity score weights,
all pretreatment differences are ns.
STRtreatment group LTRcontrol group ASAM
average standardized absolute mean difference
between treatment and control cases pretreatment
covariates red solid diamonds p-values before
use of gbm weights (if below line then
significant) black diamond outline p-values
after weights applied
65
Predictors of the propensity score model
Note. Predictors vary by outcome variable.
Following the convention of propensity score
analysis, we did not include predictors that are
highly correlated with the outcome variable.
66
Propensity score weighting

• When estimating the treatment effect, can use
  propensity scores as sampling weights.(Hirano
  Imbens, 2001 McCaffrey et al., 2004 Rosenbaum,
  1987)
• Suppose p is the propensity score of receiving
  treatment. Then
• Average treatment effect for the treated (ATT)
• control weight p/(1-p)
• treatment weight 1
• Average treatment effect for the population
  (ATE)
• control weight 1/(1-p)
• treatment weight 1/p

67
Post-optimal-matching analysis

• For the matched sample created by optimal pair
  matching, regress pairwise differences in Y
  between treated and control cases on pairwise
  differences in X vector between treated and
  control cases (Rubin, 1979). In doing so, use the
  intercept of the regression to estimate the
  treatment effect and its p-value as a
  significance test.
• For matched sample created by optimal full
  matching or optimal variable matching, use the
  signed-rank test of Hodges and Lehmann (1962) to
  estimate the average treatment effects.

68
Dose analysis using Matching estimator

• The dose analysis evaluates the outcome
  difference between a dose group (i.e., low,
  benchmark, or high) and a comparison group using
  Matching estimator developed by Abadie et al.
  (2004).
• Under the exogeneity assumption, this method
  imputes the missing potential outcome by using
  average outcomes for individuals with similar
  values on observed covariates.
• The estimator uses the vector norm (i.e.,
  xv(xVx)1/2 with positive definite matrix V)
  to calculate distances between one treated case
  and each of the matched multiple nontreated
  cases, and chooses the outcome of the nontreated
  case whose distance is the shortest among all as
  the predicted outcome for the treated case.

69
Comparing model features
Note. Regression models include covariates age at
baseline, female, black, white, latino, primary
caregiver education, income-to-poverty ratio,
primary caregiver fulltime employment, father in
household, and midyear change in teacher.
70
Comparing model features
71
FINDINGS
72
Findings Treatment effects measured by changes
in the 3rd Grade (g34)
73
Findings Treatment effects measured by changes
in the 4th Grade (g34)
74
Findings Treatment effects measured by changes
in the 3rd Grade (g345)
75
Findings Treatment effects measured by changes
in the 4th Grade (g345)
76
Findings Treatment effects measured by changes
in the 5th Grade (g345)
77
Findings Growth curve and dose models (g345)
Low Confidence Do Not Cite
78
Findings of Efficacy Subset Analysis Treatment
effects measured by changes in the 3rd and 4th
Grades (g34)
79
Findings of Efficacy Subset Analysis Treatment
effects measured by changes in the 3rd, 4th, and
5th Grades (g345)
80
Findings of Efficacy Subset Analysis Treatment
effects measured by changes in the 4th Grade
using subsets of Grade 3 exposure (g34)
81
Findings of Efficacy Subset Analysis Treatment
effects measured by changes in the 4th 5th
Grades using subsets of Grade 3 exposure (g345)
82
Summary

• From different methods of analysis, a pattern of
  small, cumulative program effects emerges across
  grades 3, 4, and 5. These analyses exclude one
  poorly performing that was dissolved in third
  year of the study.
• Positive cumulative effects on
• Social competence including
• Prosocial behavior and
• Skill in regulating emotions
• Internalizing behavior
• Relational aggression
• By HLM and efficacy subsets, promising effects
  observed on
• Academic competence
• Aggression

83
Focuses on Program Development and Steps in
Intervention Research
Focuses on (Selection) Bias-Correction
Statistical Methods
For a description of Making Choices and copies of
sample lessons, see http//ssw.unc.edu/jif/makingc
hoices/
84
References
Abadie, A., Drukker, D., Herr, J. L., Imbens,
G. W. (2004). Implementing matching estimators
for average treatment effects in Stata. The Stata
Journal 4(3), 290-311. Fraser, M. W., Day, S.
H., Galinsky, M. J., Hodges, V. G., Smokowski,
P. R. (2004). Conduct problems and peer
rejection in childhood A randomized trial of
the Making Choices and Strong Families programs.
Research on Social Work Practice, 14(5),
313-324. Fraser, M. W., Galinsky, M. J.,
Smokowski, P. R., Day, S. H., Terzian, M. A.,
Rose, R. A., Guo, S. (2005).Social
information-processing skills training to promote
social competence and prevent aggressive behavior
in third grade. Journal of Consulting and
Clinical Psychology, 73(6), 1045-1055. Fraser,
M. W., Nash, J. K., Galinsky, M. J., Darwin, K.
E. (2000). Making choices Social
problem-solving skills for children. Washington,
DC NASW Press. Fraser, M. W., Richman, J. M.,
Galinsky, M. J., Day, S. H. (2009).
Intervention research Developing social
programs. New York, NY Oxford University
Press. Galinsky, M. J., Terzian, M. A.,
Fraser, M. W. (2006). The art of group work
practice with manualized curricula. Social Work
with Groups, 29(1), 11-26. Guo, S., Fraser, M.
W. (in press). Propensity score analysis
Statistical methods and applications. Thousand
Oaks, CA Sage Press. Hansen, B. B. (2007).
Optmatch Flexible, optimal matching for
observational studies. R News, 7,
18-24. Heckman, J. J. (1979). Sample selection
bias as a specification error. Econometrica, 47,
153-161. Heckman, J. J. (2005). The scientific
model of causality. Sociological Methodology, 35,
1-97.
85
Hirano, K., Imbens, G. (2001). Estimation of
causal effects using propensity score weighting
An application to data on right heart
catheterization. Health Services and Outcomes
Research Methodology, 2, 259-278. Hodges, J.
L., Lehmann, E. L. (1962). Rank methods for
combination of independent experiments in the
analysis of variances. Annals of Mathematical
Statistics, 33, 482-497. McCaffrey, D. F.,
Ridgeway, G., Morral, A. R. (2004). Propensity
score estimation with boosted regression for
evaluating causal effects in observational
studies. Psychological Methods, 9,
403-425. Rosenbaum, P. (1987). Model-based
direct adjustment. Journal of the American
Statistical Association, 82, 387-394. Rosenbaum,
P. (2002). Observational studies (2nd ed.). New
York Springer-Verlag. Rubin, D. B. (2008). For
objective causal inference, design trumps
analysis. The Annals of Applied Statistics,
2(3), 808-840. Rubin, D. B. (1979). Using
multivariate matched sampling and regression
adjustment to control bias in observational
studies. Journal of the American Statistical
Association,74(366), 318-328. Rubin, D. B.
(1979). Using multivariate matched sampling and
regression adjustment to control bias in
observational studies. Journal of the American
Statistical Association,74(366), 318-328. Von
Hippel, P. T. (2007). Regression with missing Ys
An improved strategy for analyzing multiply
imputed data. Sociological Methodology, 37(1),
83-117.
Note. CCCCarolina Child Checklist ICST
Interpersonal Competency Scale - Teacher
86
Source Fraser, M. W., Richman, J. M., Galinsky,
M. J., Day, S. H. (2009). Intervention
research Developing social programs. New York,
NY Oxford University Press.