Tutoring

EFFECTIVENESS OF TUTORING

One-to-one human tutoring has been shown to be a very effective form of instruction. The average tutored students achieved performance gains about two standard deviations (thus usually referred to as the 2 sigma effect) above the average students in conventional classrooms with a class size of 30 students (Bloom, 1984). A meta-analysis of school tutoring programs reported superior academic performance of tutored students in 45 of the 52 studies. The average effect size was modest (.4), but it varied from study to study with the largest effect size reaching 2.3 (Cohen et al., 1982).

Tutoring effectiveness varied depending on a number of features of the tutoring program. The effect was larger when mathematics was taught as compared to reading and also when lower level skills were assessed. The effect was also larger in structured tutoring programs and also in programs of shorter durations. However, age difference between the tutor and the tutee did not seem to matter. Providing tutors with pre-training did not increase the effect sizes either (Cohen et al., 1982).

Tutoring produces effects not only on the cognitive level, but also on the affective level. In addition to higher achievement scores, tutored students developed positive attitudes toward the subject matter covered in the instruction. Tutoring also benefits both tutors and tutees. Although the amount of improvement was smaller than that of the tutee's learning, tutors' achievement test scores also improved in 33 out of the 38 studies (Cohen et al., 1982). This effect is often called learning by teaching (Bargh & Schul, 1980) or the tutor learning effect (Roscoe & Chi, 2007).

WHAT TUTORS DO

The tutoring session is predominantly controlled by tutors. Tutors, not the students, are the ones who set the agenda, introduced the subtopic, and/or proposed problems to solve. Tutors also speak first, take more turns, and make more statements than tutees (Chi, Siler, Jeong, Yamauch, & Hausmann, 2001; Graesser, Person, & Magliano, 1995; McArthur, Stasz, & Zmuidzinas, 1990). Given their dominance of the tutoring session, it is only natural to assume that tutoring is effective because of what tutors do. This leads to the conjecture that tutors would undertake an array of sophisticated pedagogical moves during tutoring. Research on tutoring, however, shows that although certain moves are frequently undertaken by tutors, they rarely undertake other, often more sophisticated pedagogical moves in spite of the conjectures and expectations that they would do so.

Feedback and Explanation. Earlier research on human and computer tutors highlighted the role of tutorial feedback (Anderson, Boyle, & Reiser, 1985; Anderson, Conrad, & Corbett, 1989; McArthur et al., 1990; Merrill, Reiser, Merrill, Landes, 1995; Merrill, Reiser, Ranney, & Trafton, 1992). For example, Merrill and colleagues (1995) studied natural human tutoring in computer programming and found that tutors kept the students' problem solving on track by providing ongoing confirmatory feedback and setting new goals. When errors were made, tutors immediately drew students' attention to them if students did not notice their errors first. Later studies of tutoring, however, reported that tutorial feedback was not only infrequent but also unimportant to student learning (Graesser et al., 1995; Chi et al., 2001). Because the majority of the studies emphasizing the role of tutorial feedback came from studies of tutoring procedural skills (e.g., programming language, algebra, and geometry), it is likely that the value of immediate feedback on errors might be most useful in the procedural domain where following the right problem solving steps is important. However, there is considerable variability in how tutors handle errors even in the procedural domain. Some tutors provide explicit feedback, sometimes even telling students how to solve a problem (McArthur et al., 1990), whereas other tutors give very subtle and indirect responses to student errors (Fox, 1990; Lepper, Woolverton, Mumme, & Gurtner, 1993).

Another frequent tutor move during tutoring is giving explanations. Graesser et al. (1995) examined the tutoring of algebra and research methods and reported that one of the noticeable tutor behaviors was providing explanations along with specific examples that connected student understanding to concrete real-world examples. Chi and colleagues (2001) analyzed various tutor statements made by novice biology tutors in a naturalistic tutoring session, such as: (a) giving explanations, (b) giving feedback followed by short corrective explanations when the feedback was negative, (c) reading text sentences aloud, (d) making self-monitoring comments (e.g., “I don't know whether this will help you”), (e) answering questions that students asked, (f) asking content questions, (g) scaffolding with generic and content prompts, and (h) asking comprehension gauging questions (e.g., “is this starting to stick?”). Of the eight types of tutor statements they coded, the most prevalent tutor statement was explanations. Giving explanations is also frequent in the procedural domain (Anderson et al., 1989; McKendree, 1990), but the prevalence of explanation might vary depending on the nature of the content domain. Conceptual domains such as biology are more conducive to tutor explanations than procedural domains such as algebra. In addition, the role of tutorial explanations seems to be limited as they only correlated with shallow measures of students' learning (Chi et al., 2001) or not at all (VanLehn, Siler, Murray, Yamauchi, & Baggett, 2003).

Socratic Technique and Error Diagnosis Rare. Although tutors dominate the tutoring dialogue, this does not necessarily mean that they employ sophisticated tutoring techniques. One such technique that is absent in typical tutoring is Socratic technique. Socratic technique refers to a tutoring method in which the tutors lead students to discover their own misconceptions and construct understanding through a line of questioning instead of giving out information to students directly (Collins & Stevens, 1982). Socratic tutoring appears to produce superior learning outcomes compared with didactic tutoring (Rosé, Moore, VanLehn, & Allbritton, 2001), but it is rare not only in typical tutoring session involving novice tutors (Graesser et al., 1995) but also when tutors have years of experience (VanLehn et al., 2003).

Another tutoring technique that has often been attributed to tutors is the ability to perform a detailed and accurate diagnosis of student understanding especially with respect to errors and misconceptions. An accurate diagnosis and monitoring of students' misunderstanding is essential for tutors to be able to decide how and when to deliver the feedback and explanations and also to tailor their moves to the needs of individual students. Chi, Siler, and Jeong (2004) analyzed how accurately novice biology tutors diagnose and monitor the status of students' misunderstandings. The results indicated that tutors could assess only what students did not know against the normative understanding that the tutors were teaching and were dismal at diagnosing the students' alternative understanding or misconception. Additional evidence indicated that even experienced tutors did not engage in detailed error diagnosis and monitoring (McArthur et al., 1990; Putnam, 1987). It seems that tutors have some sense of what students understand and are aware of the general level of their competence, but are insensitive to the specific errors or misconceptions students have.

REASONS FOR TUTORING EFFECTIVENESS

In spite of the initial belief that tutors would play a critical role in tutoring, a closer examination of tutor behaviors revealed that their contribution is rather limited. This led to a realization that tutoring effectiveness needs to be examined from a broader perspective that considers not only what tutors do but also what students do in the context of interactive tutorial dialogue.

Tutorial Dialogue. To examine the potential mechanisms responsible for the tutoring effect, Graesser et al. (1995) studied the tutoring dialogues of unskilled tutors as they tutored algebra and research methods in psychology and examined the extent to which tutoring dialogues manifest components that have been emphasized in contemporary pedagogical theories and intelligent tutoring systems. These components were: (a) active student learning, (b) sophisticated pedagogical strategies, (c) anchored learning in specific examples and cases, (d) collaborative problem solving and question answering, (e) deep explanatory reasoning, (f) convergence toward shared meanings, (g) feedback, error diagnosis and remediation, and (h) affect and motivation. Of these, only three components were prominent in a typical tutoring session: collaborative problem solving and question answering, explanatory reasoning, and anchoring in the context of specific examples.

Because most of the sophisticated techniques were underdeveloped or virtually non-existent in typical tutoring, Graesser and colleagues (1995) postulated that the tutoring effect would come from the use of localized strategies embedded within tutorial dialogues. They identified frequent dialogue patterns between tutors and tutees in the following five broad steps (p. 504):

1. Tutor asks a question.
2. Student answers question.
3. Tutor gives short feedback on the quality of the answer.
4. Tutor and student collaboratively improve the quality of the answer.
5. Tutor assesses student's understanding of the answer.

The first three steps of tutoring dialogue roughly correspond to the initiate, response, and evaluation cycle of the classroom dialogue in which the teacher initiates the dialogue typically by asking questions, the students respond, and then the teacher evaluates the responses. There are two extra steps in tutoring dialogue, however. Graesser and colleagues (1995) postulated that these extra steps of tutoring dialogue provided the advantage of tutoring over classroom instruction as tutors and tutees collaboratively construct knowledge during these steps.

Tutor-centered, Student-centered, and Interactive Hypothesis. Chi and colleagues (2001) conceptualized that tutoring effectiveness can be attributed to either what tutors do, what students do, or what tutors and tutees do together, and formulated three corresponding hypotheses: a tutor-centered, student-centered, and interactive hypothesis. The tutor-centered hypothesis states that tutors' pedagogical tactics are responsible for the effectiveness of tutoring and predicts that tutors not only employ sophisticated pedagogical moves during tutoring but also that their moves would make a significant contribution to students' learning. The student-centered hypothesis, derived from the findings that students' generative and constructive activities such as self-explanations aid learning (e.g., Chi, de Leeuw, Chiu, & Lavancher, 1994), states that tutoring is effective because it provides more opportunities for students to be generative and constructive. It predicts that students would be active during tutoring and that their active construction (e.g., question asking, self-explanations) would correlate with their learning outcomes. Lastly, the interactive hypothesis states that the key to tutoring effectiveness is not what the tutors or the students do alone, but the interaction between them. Tutorial interaction is critical because it elicits constructive responses from students. The hypothesis predicts that tutor and student would be interactive and that students' constructive and interactive responses (e.g., responding to tutor questions and scaffolding prompts) would foster learning more so than constructive but non-interactive responses (e.g., self-explanations) or interactive but non-constructive responses (e.g., acknowledgements).

In their analysis of naturalistic tutoring by novice biology tutors, Chi and colleagues (2001) found evidence for all three hypotheses. As for the tutor-centered hypothesis, tutors provided extensive amount of explanations, which was significantly correlated with student learning outcome. However, explanation was the only tutor move significantly correlated with student learning and did so only with shallow measures of learning. As for the student-centered hypothesis, they found that although tutors dominated the tutoring session, students were still more active during tutoring than in the classroom. Students asked far more questions during the tutoring sessions (e.g., about eight questions per hour) than in the classrooms (e.g., less than one question per hour according to Graesser et al., 1995). They also found that some of the students' constructive moves were correlated with learning outcomes, but they were all interactive moves (i.e., responses to scaffolding prompts and comprehension gauging questions), lending support to the interactive hypothesis as well. As for the interactive hypothesis, although tutors were not very interactive (after all, they dominated the tutoring session), students were. They never ignored tutors and always responded to prompts and questions from tutors. Tutorial interaction was also essential to student learning in that interactive construction was correlated with learning outcomes, whereas non-interactive construction was not.

Tutoring is a complex process. On the surface, it appears that tutors play a critical role in making it so effective. However, tutors in general do not use sophisticated tutoring strategies such as the Socratic technique or error diagnosis. In addition, even the tutorial moves frequently used by tutors such as explanations only made a limited contribution to students' learning. A deeper look at the tutoring session suggests that what students do in response to tutorial interaction also plays an important role. Being in a one-to-one situation with a tutor gives students more chances to engage in active learning and to be constructive, but it is not merely students' construction that mattered. Student constructive responses elicited by tutor moves played a more important role in their learning than self-initiated constructive responses.

MAKING TUTORING MORE EFFECTIVE

Although tutoring is already a quite effective form of instruction, there is still room for further improvement. As Graesser and colleagues (1995) noted, many of the components that have been emphasized in contemporary pedagogical theories are either poorly executed or rare in typical tutoring situations. Thus, one obvious way to improve the effectiveness of tutoring is by training tutors in these missing pedagogical components. Although efforts to teach tutors to use specific strategies can be challenging, it seems that some of the strategies such as scaffolding seem to be relatively easy to implement (Chi et al., 2001).

Alternatively, tutoring can be made more effective by making tutors do less instead of trying to make them do more. Chi and colleagues (2001) also manipulated the kind of tutoring strategies tutors were permitted to use. In order to make tutoring less didactic and promote a more interactive style of dialogue, tutors were asked to refrain from giving explanations and feedback. They were encouraged to prompt the students instead. The results showed that students were just as effective at learning the materials even when tutors were suppressed from giving explanations and feedback. Even in the absence of tutor explanations and feedback, students were able to learn from a greater amount of scaffolding episodes as well as by taking greater control of their own learning by reading more.

COMPUTER-BASED INTELLIGENT TUTORS

Computer-based intelligent tutors began to appear in the mid 1980s. These Intelligent Tutoring Systems (ITSs) heavily relied on the cognitive model of competence about how students learn and solve problems (e.g., Anderson et al., 1985; Anderson, Corbett, Koedinger, & Pelletier, 1995). A detailed cognitive model of students allowed computer tutors to trace students' problem-solving behaviors closely and provide directive step-by-step feedback to ensure that students stayed on the right paths. There are now well-tested tutors of algebra, geometry, computer languages, physics, and electronics (e.g., Koedinger, Anderson, Hadley, & Mark, 1997; Lesgold, Lajoie, Bunzo, & Eggan, 1988; VanLehn et al., 2005). According to one estimate, they produce learning gains of approximately 1.0 standard deviation units compared with students learning the same content in a classroom (Corbett, Anderson, Graesser, Koedinger, & VanLehn, 1999) and are actively implemented and used in schools in the United States (Aleven & Koedinger, 2002; Koedinger et al., 1997).

In spite of their success, existing ITSs are often too rigid, allowing only one strategy for problem solving and providing perhaps too much scaffolding in the form of error correction. They are also limited in promoting deep learning, failing to help students to articulate reasons behind the problem-solving procedures and to apply what they learn to more qualitative problems (Aleven & Koedinger, 2002; Graesser, VanLehn, Rosé, Jordan, & Harter, 2001; Ohlsson, 1986; VanLehn et al., 2000). Several attempts have been made to remedy these shortcomings and make computer tutors as competent as expert human tutors. One approach, based on the research showing the importance of students' own construction activities, is to provide more opportunities for students to self-construct their understanding (Aleven & Koedinger, 2002). Another approach is to endow ITSs with natural language capability so that computers could engage in tutorial dialogue with students using natural language, thereby eliciting collaborative knowledge construction more actively (Graesser et al., 2001; Person, Graesser, Kreuz, & Pomeroy, 2001; VanLehn et al., 2000). The preliminary results from these attempts look promising. Students who explained their steps during problem-solving practice with a computer tutor learned with greater understanding compared to students who did not explain steps (Aleven & Koedinger, 2002). Students tutored by an ITS with natural language enhancement also outperformed students tutored with an earlier version of ITS without the enhancement (Graesser et al., 2001).

INSIGHTS INTO IMPROVING CLASSROOM INSTRUCTION

The tutoring effect demonstrates that most of the students have a potential to reach a high level of learning. In spite of its effectiveness, tutoring has not been used as actively and widely as it should be. The main reason has been the cost. There are ways, however, to circumvent this limitation and introduce the benefits of one-to-one tutoring into classroom instruction. One traditional solution is to use peer tutors. Because the majority of tutors are novices and yet produce significant effects, pairing students with other students of either same-age or cross-age across grades can be an effective alternative to using professional tutors. Peer tutoring can also help those students who serve as tutors since tutors themselves engage in active knowledge construction processes as they tutor (Roscoe & Chi, 2007). Peer-tutoring also brings other benefits such as community building and integration of students with diverse cultural and linguistic backgrounds (O'Donnell, 2006; Webb & Palincsar, 1996).

Benefits of tutoring can also be obtained by combining collaborative learning with tutoring. For example, Chi, Roy, and Hausmann (in press) examined the effectiveness of collaborative observation of tutoring against several other instructional conditions such as one-on-one tutoring and collaboration without observation. The results showed that students learned to solve physics problems just as effectively from observing tutoring col-laboratively as the students who were being tutored individually. It seems that learning conditions such as collaborative observation of tutoring can serve as a promising alternative to one-to-one tutoring while providing all the same benefits.

Research on tutoring has demonstrated and reinforced the idea that students learn best if they construct knowledge for themselves rather than being told the knowledge. Tutoring seems to be a particularly effective form of instruction because it facilitates active knowledge construction within the interactive context of tutorial dialogues. Regardless of whether teachers implement tutoring into the classroom, classroom instruction can benefit greatly by finding ways to elicit more active construction from students in interactive contexts.