Reinventing CS Curriculum and Other Projects at The University of Nebraska - PowerPoint PPT Presentation

1 / 40
About This Presentation

Reinventing CS Curriculum and Other Projects at The University of Nebraska


Computer-Aided Education ... Summer Institute with Center for Math, Science, and Computer Education ... Computer-Aided Education. Leen-Kiat Soh, Hong Jiang, ... – PowerPoint PPT presentation

Number of Views:43
Avg rating:3.0/5.0
Slides: 41
Provided by: ncw


Transcript and Presenter's Notes

Title: Reinventing CS Curriculum and Other Projects at The University of Nebraska

Reinventing CS Curriculumand Other Projects
atThe University of Nebraska
  • Leen-Kiat Soh
  • Computer Science and Engineering
  • NCWIT Academic Alliance
  • November Meeting 2007

  • Reinventing CS Curriculum Project
  • Placement Exam
  • Learning Objects
  • Closed Labs CS1, CS2
  • Educational Research
  • Computer-Aided Education
  • I-MINDS (Computer-Supported Collaborative
  • ILMDA (Intelligent Tutoring System)
  • Affinity Learning Authoring System

Placement Exam
  • The primary purpose of the placement test
  • Place students into one of CS0, CS1, and CS2
  • Our approach emphasizes both pedagogical contexts
    and validation of the test
  • Placement exams we researched
  • Not used as a pre- and post-test
  • Do not explicitly consider pedagogical contexts
    such as Blooms taxonomy
  • Results not used to improve course instruction
  • No formative or summative analyses available

Reinventing CS Curriculum
Placement Exam
  • 10 major content areas
  • based on ACM/IEEE Computing Curricula 2001
  • Functions, sets, basic logic, data structures,
    problem solving, representation of data, etc.
  • addressed in the CS0 and CS1 courses
  • students knowledge are tested at multiple levels
    of competency based on Blooms Taxonomy
  • First five (25 questions) address prerequisite
    skills second five (25 questions) represent the
    topics students are expected to know after
    completion of CS1

Reinventing CS Curriculum
Placement Exam
Blooms Taxonomy
1. Knowledge/Memory
2. Comprehension
3. Application
4. Analysis
5. Synthesis
6. Evaluation
Reinventing CS Curriculum
Placement Exam Statistics
  • Degree of difficulty (mean)
  • The percentage of test takers who answer the
    question correctly
  • Too easy or too difficult not a meaningful
  • Targeted mean for each question is between 0.40
    and 0.85
  • Item-total correlation
  • Shows the strength of the relationship between
    the students response to a question and their
    total score
  • A good question should have a strong positive
    correlation between the two
  • 0.3 is generally regarded as a good target, 0.2
    is acceptable
  • Frequency of response for the choices
  • Unpicked choices are not providing any
    discrimination and should either be modified or

Reinventing CS Curriculum
Placement Exam Reliability Validity
  • Internal Consistency Reliability
  • A measure of item-to-item consistency of a
    students response within a single test
  • Cronbachs alpha statistic 0 1
  • Results show 0.70 to 0.74, which is acceptable
    for research purposes
  • Goal is to obtain 0.80 or higher
  • Content Validity
  • Determined by expert opinion by CSE faculty
  • Predictive Validity
  • Determined by correlating a students total score
    on the placement test with his/her exam scores in
    the course
  • E.g., 0.58 for Spring 2004

Reinventing CS Curriculum
Placement Exam Implementation
  • Duration 1 hour
  • 50 questions
  • 10 content areas
  • 5 questions in each area
  • Each question is classified into one of the
    Blooms competence level
  • Students are not informed of the competence
  • The presentation order is by the competence level
    within each content area
  • knowledge first, then comprehension, and so
  • Placement recommendation cutoffs
  • Greater than or equal to 10/25 ? CS1
  • Greater than or equal to 35/50 ? CS2
  • Otherwise ? CS0

Reinventing CS Curriculum
Placement Exam Some Results
  • Pre-Post comparisons
  • T(63) 11.036, plt.001 highly significant
  • Instructional effectiveness of the CS1 validated
  • Significant predictor of total test scores in CS1
  • Tests predictive validity

Spring 2004 session
Reinventing CS Curriculum
Placement Exam Some Results
  • Students who scored 48 or better vs. students
    who scored less
  • A one-way ANOVA found a significant difference
    between these two groups on total course points
  • F(1,64) 4.76, p. lt 0.5
  • Students who scored higher on the placement test
    received a higher grade in the course
  • Pre-Post Test
  • Overall Test T(68) 11.81, p lt 0.001
  • Individual Blooms category All show highly
    significant results (p lt 0.001)
  • Greatest improvement on knowledge questions
    t(68) 8.27, p lt 0.001)

Reinventing CS Curriculum
Learning Objects
  • Development of web-based learning objects on
    Simple Class and Recursion
  • Small, stand-along chunks of instruction
  • SCORM compliant (Shareable Content Object
    Reference Model)
  • Operating within Blackboard Course Management
  • With extensive tracking for data collection

Reinventing CS Curriculum
Learning Objects
  • Tutorial component

Reinventing CS Curriculum
Learning Objects
  • Tutorial component

Reinventing CS Curriculum
Learning Objects
  • Real-world examples component

Reinventing CS Curriculum
Learning Objects
  • Practice exercises component

Reinventing CS Curriculum
Learning Objects
  • Assessment component

Reinventing CS Curriculum
Learning Objects
  • Self-paced, with learner control of additional
  • Extensive, elaborative feedback for remediation
    and instruction
  • Tracking System
  • Student outcomes and time-spent data captured in
    real time
  • Provides data on students problems and progress

Reinventing CS Curriculum
Learning Objects Some Results
  • No significant difference between lab and
    learning object instruction
  • Evaluation results showed positive student
    response to the learning objects
  • Modular, web-based learning objects can be used
    successfully for independent learning and are a
    viable option for distance learning

Reinventing CS Curriculum
Closed Labs
  • Closed labs have multiple advantages
  • Active learning through goal-oriented problem
  • Promote students cognitive activities in
    comprehension and application
  • Some evidence that students test performance
  • Facilitates cooperative learning

Reinventing CS Curriculum
Closed Labs Design
  • Lectures
  • 2-hour laboratory (16 weeks)
  • 20 30 students per lab
  • Provide students with structured, hands-on
  • Intended to reinforce and supplement the material
    covered in the course lectures
  • Majority of the time allocated to student

Reinventing CS Curriculum
Closed Labs Design
  • A set of core topics are based on
  • Lecture topics
  • Modern software engineering practices
  • Computing Curricula 2001
  • We developed 5 components for each laboratory
  • Pre-Tests
  • Laboratory Handouts
  • Activity Worksheets
  • Instructor Script
  • Post-Tests

Reinventing CS Curriculum
Closed Labs Design
  • Pre-Tests
  • Students are required to pass an on-line test
    prior to coming to lab
  • May take it multiple times
  • Passing score 80
  • Intended to encourage students to prepare for the
    lab and test their understanding of the lab
  • Questions are categorized according to Blooms

Reinventing CS Curriculum
Closed Labs Design
  • Laboratory Handouts
  • Lab objectives
  • Activities students will perform in the lab
    (including the source code where appropriate),
  • Provide references to supplemental materials that
    should be studied prior to the lab
  • Additional materials that can be reviewed after
    the student has completed the lab

Reinventing CS Curriculum
Closed Labs Design
  • Activity Worksheets
  • Students are expected to answer a series of
    questions related to the specific lab activities
  • Record their answers on a worksheet (paper)
  • Questions provide the students with an
    opportunity to regulate their learning
  • Used to assess the students comprehension of the
    topics practiced in the lab

Reinventing CS Curriculum
Closed Labs Design
  • Instructor Script
  • The lab instructor is provided with an
    instructional script
  • Includes supplemental material that may not be
    covered during lecture, special instructions for
    the lab activities, hints, and resource links
  • Space for comments and suggestions

Reinventing CS Curriculum
Closed Labs Design
  • Post-Tests
  • During the last ten minutes of each lab, students
    take an on-line test
  • One-time-only
  • Another measure of their comprehension of lab
  • Questions are categorized according to Blooms

Reinventing CS Curriculum
Closed Labs Some Results
  • Study 1 To determine the most effective pedagogy
    for CS1 laboratory achievement
  • Participants 68 students in CS1, Fall 2003
  • Procedures
  • Structured cooperative groups had prescribed
    roles (driver and reviewers)
  • Unstructured cooperative groups did not have
    prescribed roles
  • Direct instruction students work individually
  • Randomly assigned the pedagogy of each lab
  • Used stratified random assignment to assign
    students to their cooperative groups within each
  • Based on ranking of the placement test scores for
    this course (high, middle, low)

Reinventing CS Curriculum
Closed Labs Some Results
  • Study 1, Contd
  • Dependent Measures
  • Total laboratory grades
  • Combined worksheet scores and post-test grades
    for each lab
  • Although some students work in groups, all
    students were required to take the post-test
  • Pre-Post-Test measuring self-efficacy and
  • Taken during the first and last week of the
  • Adapted 8 questions taken from Motivated
    Strategies for Learning Questionnaire by Pintrich
    and De Groot (1990)
  • Returned a reliability measure (Cronbachs alpha)
    of .90 with a mean of 3.45 and standard deviation
    of .09 good reliability

Reinventing CS Curriculum
Closed Labs Some Results
  • Results of Study 1
  • Both cooperative groups performed significantly
    better than the direct instruction group (F(2,66)
    6.325, p lt .05)
  • Cooperative learning is more effective than
    direct instruction
  • No significant difference between the structured
    cooperative and unstructured cooperative groups
  • 6 out of 8 questions showed statistically
    significant changes in student perceived
    self-efficacy and motivation

Closed Labs Some Results
  • Study 2
  • Same objective revised motivation/self-efficacy
    tool, additional qualitative feedback revised
  • Participants 66 students in CS1, Spring 2004
  • Results
  • Both cooperative groups performed better than the
    direct instruction group (F(2,64) 2.408, p lt
  • Discussion
  • Similar conclusions

Computer-Aided Education
  • Studies on the use of Computer-Supported
    Collaborative Learning (CSCL) tools
  • structured cooperative learning (Jigsaw) vs.
    non-structured cooperative learning
  • CSCL vs. non-CSCL
  • Studies on the use of Intelligent Tutoring System
  • ITS vs. Lab
  • ITSLab vs. Lab
  • Studies on the use of authoring tools
  • Affinity Learning Authoring System
  • How authoring tools impact learning
  • Graphical vs. non-graphical

Ongoing Work
  • Summer Institute with Center for Math, Science,
    and Computer Education
  • Teaching multimedia computing to student-teachers
  • NSF Advanced Learning Technologies Project
  • Intelligent Learning Object Guide (iLOG)
  • Developing SCORM-standard metadata to capture use
    characteristics of learning objects and student
  • Developing software to automatically capture and
    generate metadata to tag learning objects
  • Creating SCORM-compliant learning objects for
    CS0, CS1, CS2

Ongoing Work 2
  • Renaissance Computing
  • Joint curricular programs with other departments
  • School of Biological Sciences
  • School of Music
  • College of Agricultural Sciences and Natural
  • Digital Humanities
  • Multi-flavored introductory CS courses
  • Object first vs. traditional
  • Multimedia, Engineering, Life Sciences, Arts

NCWIT Academic Alliance Focus
  • Renaissance Computing
  • Multi-flavored introductory CS courses in
    conjunction with joint curricular programs with
    other departments (that have larger female
    populations) to promote more female participation
    in CS
  • Computer-Aided Education
  • Online learning objects for K-12 teachers to help
    them expose their students to computational
    thinking and real-world IT applications
  • Collaborative writing (via I-MINDS) for secondary
    female students on the use of CS paradigms to
    solve real-world problems
  • Reinventing CS Curriculum
  • Use placement exam as pre- and post-tests for
    future studies on learning performance of female
  • Use cooperative learning in labs to recruit and
    improve retention of female students

  • Rich Sincovec, CSE Department Chair
  • Reinventing CS Curriculum Project
  • Leen-Kiat Soh, Ashok Samal, Chuck Riedesel, Gwen
  • Computer-Aided Education
  • Leen-Kiat Soh, Hong Jiang, Dave Fowler, Art

  • UNL
  • College of Education and Human Sciences
  • Center for Math, Science, and Computer Education
  • J.D. Edwards Honors Program (CSBusiness)
  • Extended Education and Outreach (AP Courses)
  • Department of History, School of Biological
    Sciences, School of Music, etc.
  • Bellevue University (I-MINDS)
  • University of Wisconsin-Madison ADL Co-Lab
    (learning objects)

  • Reinventing CS Curriculum
  • Framework
  • L.-K. Soh, A. Samal, and G. Nugent (2007). An
    Integrated Framework for Improved Computer
    Science Education Strategies, Implementations,
    and Results, Computer Science Education,
  • Learning Objects
  • G. Nugent, L.-K. Soh, and A. Samal (2006).
    Design, Development and Validation of Learning
    Objects, Journal of Educational Technology
    Systems, 34(3)271-281

Publications 2
  • Reinventing CS Curriculum, Contd
  • Placement Exam
  • G. Nugent, L.-K. Soh, A. Samal, and J. Lang
    (2006). A Placement Test for Computer Science
    Design, Implementation, and Analysis, Computer
    Science Education, 16(1)19-36
  • Structured Labs Cooperative Learning
  • J. Lang , G. Nugent, A. Samal, and L.-K. Soh
    (2006). Implementing CS1 with Embedded
    Instructional Research Design in Laboratories,
    IEEE Transactions on Education, 49(1)157-165
  • Soh, L.-K., G. Nugent, and A. Samal (2005). A
    Framework for CS1 Closed Laboratories, Journal of
    Educational Resources in Computing, 5(4)1-13

Publications 3
  • Computer-Aided Education
  • Computer-Supported Collaborative Learning
  • L.-K. Soh, N. Khandaker, and H. Jiang (2007).
    I-MINDS A Multiagent System for Intelligent
    Computer-Supported Cooperative Learning and
    Classroom Management, to appear in Int. Journal
    of Artificial Intelligence in Education
  • Intelligent Tutoring System
  • L.-K. Soh and T. Blank (2007). Integrating
    Case-Based Reasoning and Multistrategy Learning
    for a Self-Improving Intelligent Tutoring System,
    to appear in Int. Journal of Artificial
    Intelligence in Education
  • Affinity Learning Authoring Tool
  • L.-K. Soh, D. Fowler, and A. I. Zygielbaum
    (2007). The Impact of the Affinity Learning
    Authoring Tool on Student Learning, to appear in
    J. of Educational Technology Systems

To Probe Further
Write a Comment
User Comments (0)