Formative feedback in intro CS

This essay was selected for the 2019 UC Berkeley Teaching Effectiveness Award. The definitive version can be found on the GSI Teaching & Resource Center website. Below is the author’s version.

Students learn computer science (CS) by doing. In CS 61A, a highly-rated introductory CS course at UC Berkeley, students are introduced to new concepts in lecture, go hands-on to learn the solution process in lab and discussion with guidance from teaching assistants, and synthesize multiple ideas on their own or with a partner in coding homeworks and projects. Throughout these activities, students experience aspects of CS such as programming, debugging, program tracing, and analysis while building a sense of community and belonging through small-classroom environments supported by numerous course tutors. However, these activities do not explicitly prepare students for taking high-stakes paper exams, which have been identified as a significant source of stress in the course. Furthermore, to improve the efficiency of their learning, students receive real-time assistance from instructors during lab, discussion, and office hours. While this assistance significantly reduces frustration, it can also shortcut learning when too much help is given causing students to struggle with exams as they become reliant upon the guidance.

I believe introductory CS instructors can maintain their current goal of minimizing student frustration with real-time feedback while also supporting comprehensive student learning by highlighting misconceptions through targeted, formative feedback. In Fall 2016, as a means of providing such formative feedback, I introduced 10-minute paper quizzes in my 40-student discussion sections. Quizzes serve as a check on student understanding after synthesizing and applying ideas in homework but before taking a high-stakes exam. Quiz questions are designed to model what students can expect to see on the exam and highlight key ideas from the previous week’s discussion section. Inspired by the active learning literature, students first take the full 10 minutes to complete their quiz individually and commit to an answer, then break out into small groups to discuss their solution process, before regrouping as a class to summarize their ideas. This sparks lively discussion as students are forced to confront their misconceptions and reflect on their solution process.

After some polishing, my quizzes were deployed to 400 students in CS 61A Summer 2017. While student feedback reported the group discussions as generally helpful, not all discussions were equally effective. From classroom observations, students would often miss small but crucial details. Furthermore, data collected from the course dashboard suggested that students tended not to revisit past assignments. In order to encourage students to review their quizzes, I used Gradescope, an online grading platform, to scan, grade, and share individual quiz feedback with students the same day they took the quiz. Quizzes were graded on completion, not correctness, to encourage students to focus on improvement rather than points. Our exam questions often test multiple concepts and skills, so within each rubric item I explicitly described the embedded learning goals, potential misconceptions, and problem-solving strategies. Students were encouraged to use the rubric to guide their learning and clarify questions from discussion.

Iterating on student and course staff feedback from the summer, I deployed quizzes to 1,400 students in CS 61A Fall 2017. While the detailed rubrics helped students identify areas for improvement, there was no guidance to point students to relevant resources and additional practice problems for students to check their understanding. To support these needs, I developed an online study guide accompanying each quiz. Each study guide included a summary of the quiz’s key ideas, problem-solving strategies, and resources for further review, as well as 3–6 problems of increasing difficulty for students to check their understanding. End-of-semester student evaluations suggested that this quiz-study guide system was well-received by students and that it served as an effective use of time in discussion section.