Doing Qualitative Data Analysis on Language Learning and Teaching - PowerPoint PPT Presentation

1 / 48
About This Presentation

Doing Qualitative Data Analysis on Language Learning and Teaching


Doing Qualitative Data Analysis on Language Learning and Teaching : – PowerPoint PPT presentation

Number of Views:214
Avg rating:3.0/5.0
Slides: 49
Provided by: Presentat81


Transcript and Presenter's Notes

Title: Doing Qualitative Data Analysis on Language Learning and Teaching

Doing Qualitative Data Analysis on Language
Learning and Teaching
  • ?????????????

?? ?????????? ???????? ???
????????????? 98????????? III (10/24/2009)
Before we start
  • One BIG TRUTH or many small truths?
  • Do you believe there is one single correct
    perspective or there are multiple perspectives
    that may be all true?

Before we start
Doing qualitative data analysis is like solving a
jigsaw puzzle!
Doing Qualitative Data Analysis on Language
Learning and Teaching
  • Why qualitative research?
  • What are qualitative data?
  • Types of qualitative data analysis
  • Data preparation and management
  • Data analysis process
  • Categorization, display, and interpretation
  • Quality of qualitative analysis

1. Why qualitative research?
Quantitative Research Qualitative Research
Assumptions about reality Single can be broken down into parts studied Multiple can only be studied holistically
Purpose of research To generalize, predict, and posit causal relationships To contextualize and interpret
Research questions Deductively formulated Inductively formulated
Research design Begins with a hypothesis and set methodology Evolves over time and is affected by the data gathered
Typical data A large, random sample data include numerical indices involving tests or responses to surveys A small, purposeful sample data include field notes, interviews, recorded activities, and documents
Data analysis Statistical Interpretive
(adapted from McKay, 2006, p. 7)
1. Why qualitative research?
  • Strengths of qualitative data
  • naturally occurring
  • focused and bounded phenomenon embedded in a
    particular context
  • rich and holistic with strong potential for
    revealing complexity
  • typically collected over a sustained period (this
    allows us to go beyond snapshots of what or
    how many to how and why things happen as they
  • with strong emphasis on peoples lived experience
    (their perceptions, assumptions, prejudgements,

(Miles Huberman, 1994, p.10)
Can you also point out weaknesses of
qualitative data?
1. Why qualitative research?
  • A qualitative approach is a person-centered
  • Qualitative research is designed to
  • explore the complexities and problems of the
    complicated social world
  • understand the patterns and purpose in our
    behavior and provide insights that will enrich
    our understanding
  • have personal engagement with the lived world

(Richards, 2003, pp. 8-9)
2. What are qualitative data?
Types of qualitative data Examples
Samples of learner language - Spoken oral presentations, public speeches, discussions in pairs or groups, role plays, online chats - Written compositions, notes, responses to questions, blogs, emails, online discussions
In-class activities and interactions - observation notes - audio/video recordings
Reports from learners/ teachers - responses to open-ended questions in questionnaires - interviews - journals - learning / teaching histories - self-revelation using think-aloud - feedback given to students
Teaching materials - textbooks, supplementary materials
3. Types of qualitative data analysis
Types of QDA
Ethnographic analysis
Content analysis
Linguistic analysis
Discourse analysis
Interaction analysis (can be subsumed in discourse analysis)
Meaning / theme
Form (linguistic features)
Form and function in a particular context
Interaction/communication pattern and strategy use
3. Types of qualitative data analysis
  • Your research purpose will determine what type(s)
    of analysis to be used.
  • Examples
  • 1) to identify and understand students errors
    in the use of the past tense
  • 2) to identify and understand students problems
    in making requests
  • 3) to understand how students solve problems in
  • 4) to understand how students perceive the
    effectiveness of a new learning program
  • 5) to find out why students keep silent to
    teachers questions
  • 6) to find out why students feel anxious in
    foreign teachers classes
  • 7) to compare how students write persuasive
    essays in Chinese and in English
  • 8) to understand how foreign teachers view
    Taiwanese students language learning attitudes

4. Data preparation and management
  • Data organization
  • Data selection
  • Data transcription

Example 1
Example 2
4. Data preparation and management
  • Transcribing data
  • Is it necessary to transcribe all the interviews
    or recordings of interactions?
  • How detailed or tidy does a transcription need
    to be (e.g., linguistic forms, errors/slips of
    the tongue, gap fillers, pauses, paralinguistic
    features, non-verbal behavior, gestures, eye
    gaze, contextual information)?
  • How should the data be transcribed into segments,
    such as lines, sentences, and paragraphs?
  • How and when should the data be translated if the
    data are not recorded in the target language?
  • What tools can be used for more efficient data
  • transcription?

Example 1A
A students oral narrative
  • Ill talk about one simple event that I went to.
    I guess that is one thing that I feel really
    surprised. That is when last time we had Moon
    festival. During the moon festival we all
    together went to have a barbecue. But I normally
    thought when I come to this event, I only think
    that I will just have something that they want me
    to do, not to think of something very interesting
    to do during the barbecue. I first find out one
    thing that just existed in my imagination before.
    That is seeing somebody who can play guitar very
    well and sing in front of us, because that image
    only exist in my imagination, because I dont
    think that I will meet someone who can just
    perform that well when I am in the university. So
    this really makes me feel that I really enjoy my
    university life .

Example 1B
  1. uh..Ill talk about one simple ..umeventthat I
    went to
  2. I guess..uhI guess that is one thing that I feel
    really surprised
  3. That is whenuhlast time we hadMoon festival
  4. During the moon festival we..we all together went
    to have a barbecue
  5. But I...normally I thought when I come to this
  6. I only think that I will just have someuh
  7. or something that they want me to do
  8. not to think of something very interesting toto
  9. souhdurin.. during the barbecue
  10. I first find out one thing that..that I.
  11. that just existed in myin myimagination before
  12. that isseeing somebody who canuhplay guitar
    very well
  13. and sing in front of us
  14. Because that image only my
  15. Because I dont think that I will
    someone who can just perform that well
  16. uhwhen I am in the collin the uniuniversity
  17. So..uhthisreallyuhmakes me feel that
  18. I reallyuh enjoy my university life

  1. Helen Well, we need to tell them ((sees
    John coming into the office))
  2. Hi, John, how // are ? you?
  3. John // Hi
  4. Sue Hi // John ?
  5. John // Hi Sue
  6. Helen Wel?come to the CELLAR.
  7. John Thanks.
  8. Helen You had a pretty tough summer, uh?
  9. John (.) Yeah ?, you heard?
  10. Helen Yea?h ?
  11. Sue Yeah?, I heard // of your story too.
  12. John // Oh, yeah??
    Everybody knows ? heh, heh ((chuckles))
  13. Helen Yeah, youve been heard of EVERYwhere
  14. John Yeah ? ((smiles but feels embarrassed))
  15. Sue ((laughs))
  16. Helen ((sees John have a book with him)) So, you
    wanna sell back your book here?
  17. John No ((chuckles)), I came to uh drop
    this book to um Eun-young
  18. Helen Ok. ((The telephone rings and Helen
    answers the phone))
  19. John Ill put my name on it for the first

Example 2
  • Symbols used in the transcription
  • . - indicates a stopping fall in tone at end of
  • , - indicates a continuing intonation
  • - indicates an extension of the sound or
  • ? - indicates a rising intonation at tend of
  • ! - indicates an animated tone
  • - indicates no interval between adjacent
  • // - indicates overlap of two utterances
  • ? - indicates quieter than the surrounding talk
  • - indicates a rising shift in intonation
  • ? - indicates a falling shift in intonation
  • (.) - indicates very slight pause
  • (( )) - indicates other voice qualities or the
    analysts observation notes
  • Capital initial - indicates start of utterance
  • Capital letter - indicates heavy accent and

Example 3
A Group discussion on a gender issue
  • Carrie ????????????,??????????,?????????,???????
  • If you think this is treating women unfairly,
    asking men to do all the housework would imply
    the same. Dont you think its weird?
  • May ?????,???????????
  • Why is it weird? Dont you think its righting
    the wrong?
  • Carrie ????????,????????,???????????
  • Does that mean men will come back to dominate
    women in 10 years? And will this become a vicious

Example 4
Interview data
  • ??? ?????????! ???????????????????????????????
  • Student A It was really a torture in the
    beginning! The schoolwork had already been too
    heavy to make me breathe. Then the teacher added
    this time- and effort-consuming job to us. I
    almost gave up in the first and second
    discussions. However, when I survived to the
    third discussion, I started to feel it was fun.
    Everyone was expecting to read others postings
    and getting their feedback. Frankly speaking, I
    think I did learn a lot both in English writing
    and the information searching skills on the
    Internet. I hope we can still continue this

5. Data analysis process
(Seidel, 1998)
  • Notice read carefully, disassemble, identify,
    and code
  • Collect sort and sift
  • Think search for patterns relationships, and
    make sense of data
  • Doing QDA is an iterative, progressive,
    recursive, and holographic process.

5. Data analysis process
Data display interpretation
Figure 6.1 Aspects of analysis (Richards, 2003,
p. 271)
5. Data analysis Categorization
  • (Merriam, 1998, p. 179)
  • Devising categories is largely an intuitive
  • but it is also systematic
  • and informed by the studys purpose,
  • the investigators orientation and knowledge,
  • and the meanings made explicit by the
    participants themselves.

5. Data analysis Categorization
  • Guidelines for category construction
  • (Merriam, 1998, pp. 183-184)
  • categories should reflect the purpose of the
  • categories should be exhaustive
  • categories should be mutually exclusive
  • categories should be sensitizing
  • categories should be conceptually congruent

5. Data analysis Categorization
  • Essential features of an adequate category
  • (Richards, 2003, p. 276)
  • Analytically useful
  • When used, does it contribute anything to
  • Conceptually coherent
  • Does it make sense in terms of the conceptual
    framework within which interpretation will be
  • Empirically relevant
  • Can it be mapped onto the data?
  • Practically applicable
  • Is it possible to specify criteria that can be
    used to assign data bits to the category?

5. Data analysis Categorization
  • Levels of analysis (Strauss Corbin, 1998)
  • Open coding breaking down the data for the
    purpose of categorizing, conceptualizing and
  • Axial coding looking for patterns and
    concentrating on organizing the data, based on
    the axis of a category. It involves relating
    categories to subcategories and making
    connections between categories.
  • Selective coding identifying a central category
    or explanatory concept in terms of which other
    categories can be refined and integrated

Example 1
Analysis of Online Discussions
Participants Discussions
Interaction patterns
Exchange structure- IRF
Thinking skills
Feedback type questions vs. comments
Feedback function information exchange vs.
relationship building
Responses to Questionnaire
Perceived learning effectiveness
The relationship between the students online
interaction and their learning effectiveness
The relationship between the students online
interaction and their learning effectiveness
Focus group Interviews
Analysis of thinking skills
Example 2
Signal words / Question stems Cognitive operations
Memory questions who, what, where, when naming, defining, identifying, designating, yes or no responses
Convergent thinking questions why, how, in what ways? explaining, stating relationships, comparing and contrasting
Divergent thinking questions imagine, suppose, predict If ... then ... , How might ... , What are possible consequences... predicting, hypothesizing, inferring, reconstructing
Evaluative thinking questions defend, judge, justify/ What is your opinion ...? Why do you think is better? Valuing, judging, defending, justifying choices
(Blanchette, 2001 Ciardiello, 1998 Gallagher
Aschner, 1963)
Example 3
Analysis of Reasoning Performance
Type of fallacies Definition
Failure to follow Reasons irrelevant or contradictory to a point
Faulty evidence Facts, experiences and authoritative sources falsely used to support a point
Overgeneralization Generalizations made without giving representative or sufficient reasons/evidence
Shifted focus Reasons/evidence deviating away from a point to a less significant one
Straw person Failure attack due to a misinterpretation of the opponents views
(Ennis, 1996)
Analysis of Politeness Strategies
Example 4
Thematic Analysis of Students Views of Gender
Example 5
  • Mens reluctance to give up power to women
  • Mens control over women as sex objects
  • Mens search for independence
  • Womens perceived subordination to men
  • Womens emotional dependence on men
  • Womens longing for love
  • Womens longing for physical beauty
  • Womens quest for self-liberation
  • Womens conflicts between the traditional and
    modern self

5. Data analysis Display
  • Display refers to visual format that presents
    information systematically, so the user can draw
    valid conclusions and take needed action.
  • (Miles Huberman, 1994, p.91)
  • Common displays of qualitative data
  • Quotations
  • Vignettes / narratives
  • Tables / figures
  • Excerpts / samples of written works/documents
  • Pictures / images

Content/thematic analysis of interviews
Example 1
  • Womens conflicts between the traditional and
    modern self
  • Doris In the end of the episode, its still the
    mother who holds the childs hand after they got
    out of the car, not the father. Although womens
    status is enhanced and we can do things men do,
    we still have to do the work that we were
    previously designated to.
  • ????????,???????,????????,???????????,???????????
  • Nancy I think this commercial speaks for women
    in the contemporary society. On one side, you
    want to be independent, but on the other side you
    still want to be cared for. For me, I can drive
    and Im good at directions. Cars represent a
    symbol of independence to me. But at the same
    time, I still want to be a little girl taken good
    care of by men.
  • ??????????????????????????????,?????????,????,???

Example 2
Ethnographic analysis of three classes
Class A Class B Class C
Purpose of Use To facilitate students writing process and evaluate their writing performance at the initial stage To facilitate students writing process To grade students essays and replace the human grader
Functions Emphasized Both the grading system and the writing resources/ editing features The writing resources and editing features The grading system
Requirements Grading Policy Students were asked to write multiple versions on assigned topics at home. They were not allowed to turn in their essays to the teacher until they gained a score of 4 given by the program. Students were asked to write drafts on assigned topics in class and develop them into better organized essays at home. The scores given by the program were not important. Students were asked to write multiple versions on assigned topics at home. The scores they obtained from the program accounted for 40 of their final grades.
Duration of Use 4 months 1 month 4 months
Post-grading Feedback Consultation The teacher gave individual written feedback and held in-class discussions on students essays. The teacher gave individual written feedback and provided after-class consultation with students when needed. The teacher gave no feedback to students essays and little consultation in class.
Ethnographic analysis of a students email
Example 3
To Whom When For What Purposes
Lawyer Feb. 1999 Jan. 2001 - to request advice for Green Card application - to request action or assistance - to check progress of the application process - to make an appointment - to respond to the lawyers questions
Administrative Personnel Administrative Personnel Administrative Personnel
a) in his work place Nov. 1999 Feb. 2000 - to request information/assistance regarding his employment visa application
b) in different universities Oct. 2000 May 2001 - to request information/assistance regarding his Ph.D. study application
Professors Nov. 2000 Dec. 2001 - to make an appointment - to request recommendations - to request permission - to request simple information - to show his academic interests - to give thanks
Friends (mostly Taiwanese who did not live close to him) May 1999 Dec. 2001 - to send personal care and greetings - to share life experiences - to give thanks - to request information/assistance - to give information - to express disagreement
Example 4
Discourse analysis of a students email requests
Request Types Linguistic Forms of Request Acts Examples
Requesting advice (a) Please Imperative (b) Want Statement (c) Interrogative - (If you have time/If you know the answer), Please let me know/tell me. - I need your suggestion/advice. - What should I do? - Do I need to contact ?
Requesting action (a) Please Imperative (b) Query Preparatory - Please review this file for me. - Can/Could you send xxx to me?
Checking work progress (requesting a result or a confirmation) (a) Just Want Statement (b) Interrogative - Just want to know/ask if you ? - Have you sent the form to ?
Making an appointment (requesting a meeting) (a) Want Statement (b) Query Preparatory - I want/hope to make an appointment with you . - Can we find sometime next week to talk about ?
Example 5
Table 4.1 Structures of the students oral
Structure Elements Level Groups Structure Elements Level Groups Abstract Orientation Orientation Orientation Action Action Action Evaluation Result Coda
Structure Elements Level Groups Structure Elements Level Groups Abstract Complete Complete Incomplete Complete Complete Incomplete Evaluation Result Coda
Structure Elements Level Groups Structure Elements Level Groups Abstract With evaluative devices Without evaluative devices Incomplete With evaluative devices Without evaluative devices Incomplete Evaluation Result Coda
High (N3) O12 Y 1 1 0 8 3 1 6 Y Y
High (N3) O14 N 3 2 0 9 4 2 13 Y Y
High (N3) O15 Y 0 2 0 8 1 0 4 Y Y
Middle (N3) O8 Y 0 4 0 3 8 1 5 Y Y
Middle (N3) O10 Y 4 4 0 9 2 0 5 Y N
Middle (N3) O11 Y 2 1 0 2 7 0 2 Y N
Low (N9) O1 N 0 0 0 4 8 9 0 N N
Low (N9) O2 Y 6 3 0 2 2 0 8 Y N
Low (N9) O3 Y 1 3 2 4 2 6 2 Y N
Low (N9) O4 Y 1 1 0 3 1 0 9 N Y
Low (N9) O5 N 6 3 0 0 0 0 10 N N
Low (N9) O6 N 5 1 5 0 0 0 13 N N
Low (N9) O7 Y 4 9 3 3 1 2 8 Y N
Low (N9) O9 Y 5 3 0 1 0 0 1 N N
Low (N9) O13 Y 1 8 0 1 0 2 7 Y N
Discourse analysis of online debate
Example 6
Partial agreement
  • Dear Hedy
  • According to your statement, imitation is
    important when learning an L2. I agree with your
    point of view, and your examples are quite true.
    However, i have a few questions to the examples
    you mentioned above. We learn L1 by imitating
    from the videos or tapes that we heard, or from
    the teachers and parents...etc, but when we learn
    an L2, sometimes you cannot "acquire" it that
    easily. What if the teachers or parents don't
    speak the L2 at all? What if the language is not
    spoken by many people and there are no programs
    or tapes or books to learn at all? We speak L1
    everyday, we speak it everywhere, there is only a
    little time to actually imitate an L2, and even
    if there is a plenty of time, there might be
    errors when we are practicing it. I would like to
    know what's your point of view on this problem.

Indirect disagreement
Counter- argument
Mitigated request for response
Content analysis of students attitudes toward
online discussions
Example 7
Improving writing fluency

Figure 4.2 A students perceived improvement in
5. Data analysis Interpretation
  • Data interpretation making sense of data
  • To explain
  • how things work how people do things
  • why things are working or not working
  • how things can be made to work better
  • To relate findings to theories or develop new
  • To compare the present study with existing
    empirical studies and make new contributions

Figure 3.2 Analytical Framework of Email
Lets review the data analysis process
(Richards, 2003, p. 271)
6. Quality of qualitative analysis
  • A qualitative study can be judged in three
  • Process What data collection and analysis
    methods are used to ensure the validity and
    credibility of the findings?
  • Product How adequately are the data represented
    and interpreted? What contribution does the study
  • Position What does the researcher bring to the
    study in terms of his/her orientations,
    standpoints, perspectives, qualifications, and
  • (Richards, 2003, p. 293)

6. Quality of qualitative analysis
  • Descriptive validity
  • Interpretive validity
  • Theoretical validity
  • Strategies to enhance validity
  • Triangulation
  • Constant comparison
  • Member checks
  • Long-term or repeated observations
  • Peer examination
  • Clarification of the researchers biases
  • Rich, thick description
  • Credibility (internal validity)
  • Transferability (external validity)
  • Dependability (reliability)

(Richards, 2003, pp. 285-287)
Final Reminders
  • the human element of qualitative inquiry is
    both its strength and weakness
  • its strength is fully using human insight and
  • its weakness is being so heavily dependent on
    the researchers skill, training, intellect,
    discipline, and creativity.
  • The researcher is the instrument of qualitative
    inquiry, so the quality of the research depends
    heavily on the qualities of that human being.
  • (Patton, 2002, p. 14)

Final Reminders
  • But research is inherently imperfect, and we
    would support the line of argument that multiple
    perspectives and methods increase the likelihood
    of reaching good explanations.
  • (Jaworski Couland 1999, p. 37)
  • we need to ensure that there is sufficient
    evidence and sufficient kinds of evidence. the
    link between the evidence we present and the
    interpretations we derive from that evidence must
    be robust.
  • (Richards, 2003, p. 283)

  • Jaworski, A. Coupland, N. (1999). Introduction
    Perspectives on discourse analysis. In A.
    Jaworski N. Coupland (Eds.), The discourse
    reader (pp. 1-44). New York Routledge.
  • McKay, S. L. (2006). Researching second language
    classrooms. Mahwah, NJ Lawrence Erlbaum.
  • Merriam, S. B. (1998). Qualitative research and
    case study applications in education. San
    Francisco, CA Jossey-Bass.
  • Miles, M. B., Huberman, A. M. (1994).
    Qualitative Data Analysis (2nd ed.). Thousand
    Oaks, CA Sage.
  • Patton, M. Q. (2002). Qualitative evaluation and
    research methods (3rd ed.). Thousand Oaks, CA
  • Richards, K. (2003). Qualitative inquiry in
    TESOL. New York Palgrave MacMillan.
  • Seidel, J (1998). Qualitative Data Analysis. The
    Ethnography v5 Manual, Appendix E. Available
    online at http//
  • Strauss, A., Corbin, J. (1998) Basics of
    Qualitative Research (2nd ed.). Newbury Park, CA

Questions Comments
  • Your questions and comments are welcome.

If knowledge is worth having, it is worth
sharing. Deborah Cameron
??? Dr. Chi-Fen Emily Chen?????????? ????????
Website http//
Write a Comment
User Comments (0)