Subgoals, Problem Solving Phases, and Sources of Knowledge: A Complex Mangle

TL;DR

This study analyzes how middle school students develop computational thinking skills, focusing on abstraction in programming, by examining problem solving processes and instructor-student interactions in a real workshop setting.

Contribution

It introduces a situated cognition framework for documenting communication during problem solving and provides a microgenetic analysis of abstraction teaching in a programming environment.

Findings

Problem solving phases are nested with context-switching.

New ideas facilitate transitions between problem solving phases.

Planning is a non-linear, iterative process.

Abstract

Educational researchers have increasingly drawn attention to how students develop computational thinking (CT) skills, including in science, math, and literacy contexts. A key component of CT is the process of abstraction, a particularly challenging concept for novice programmers, but one vital to problem solving. We propose a framework based on situated cognition that can be used to document how instructors and students communicate about abstractions during the problem solving process. We develop this framework in a multimodal interaction analysis of a 32-minute long excerpt of a middle school student working in the PixelBots JavaScript programming environment at a two-week summer programming workshop taught by undergraduate CS majors. Through a microgenetic analysis of the process of teaching and learning about abstraction in this excerpt, we document the extemporaneous prioritization of subgoals and the back-and-forth coordination of problem solving phases. In our case study, we identify that (a) problem solving phases are nested with several instances of context-switching within a single phase; (b) the introduction of new ideas and information create bridges or opportunities to move between different problem solving phases; (c) planning to solve a problem is a non-linear process; and (d) pedagogical moves such as modeling and prompting highlight situated resources and advance problem solving. Future research should address how to help students structure subgoals and reflect on connections between problem solving phases, and how to help instructors reflect on their routes to supporting students in the problem solving process.

Full text

Computational Thinking; Education; Situated Cognition; Debugging; Abstraction; Problem Solving

††copyright: rightsretained††doi: 10.1145/3287324.3293712††conference: the 50th ACM Technical Symposium on Computer Science Education; February 27–March 2, 2019; Minneapolis, MN, USA††journalyear: 2019††submissionid: 1040††ccs: Social and professional topics Computing education

1. Problem and Motivation

Educational researchers have increasingly drawn attention to how students develop computational thinking (CT) skills (Lye and Koh, 2014; Weintrop et al., 2015), including in science, math, and literacy contexts (Pérez, 2018; Basu et al., 2017; Jacob and Warschauer, 2018). A key component of CT is the process of abstraction. Indeed, abstraction is recognized as a threshold concept (Rountree et al., 2013): a generative idea that once learned provides “a qualitatively different view of subject matter within a discipline.” The process of creating abstractions interleaves multiple problem solving phases: planning, building, and monitoring. In addition, creating abstractions requires attending to subgoals. Finally, pathways to learning about abstractions are structured by complex tools and driven by a variety of sources of knowledge, including perception, testimony, reasoning, and memory (Chinn et al., 2011). Our purpose in this paper is to stitch these elements into a framework that can be used to document how instructors and students communicate about computational thinking.

2. Background and Related Work

Our framework integrates constructs from the learning sciences, computer science educational research, and human computer interaction. Our point of departure is situated cognition, a theoretical framework that inextricably connects learning to interactions between tools, cognition, bodies, and communities of practice (Jordan and Henderson, 1995). Increasingly, other researchers have studied CT teaching and learning from a situated perspective (Lewis, 2012; Lewis and Shah, 2015; Flood et al., 2018a, b). Specifically, our framework connects with prior work on how problem solving involves a balancing act between exploration, building, and monitoring (Basu et al., 2017; Rountree et al., 2013). In addition, we recognize that developing abstractions requires a gradual process of specifying and prioritizing goals and subgoals, including refactoring a previous route by developing new subgoals (Basu et al., 2017). The epistemic actions that advance problem solving are stretched across tools in the environment (e.g., editor, syntax checker, stepper tool, code reference sheet) and multiple sources of knowledge: perception, memory, reasoning, and testimony. Each interaction between a source of knowledge and a tool requires the learner to cross a gulf of execution, where they try to figure out how to express their idea to a tool in the environment, and attend to the outcome by crossing a gulf of evaluation, where they try to figure out the tool’s response (Norman, 2002). Recognizing that causes of failure in programming emerge from complex connections between proximal and distal events (Ko and Myers, 2005), we assembled the framework above to comprehensively integrate multiple constructs and offer a new lens on the pathways students take to learn foundational computer programming concepts.

3. Approach and Uniqueness

There is surprisingly little microgenetic, multimodal qualitative research on how young students program in naturalistic learning environments. For this paper, we selected a 32-minute long sample of a middle school student working in the PixelBots JavaScript programming environment at a two-week summer programming workshop taught by undergraduate CS majors. Using rich observational data including several camera angles and screen recordings, we transcribed the student’s activities, repeatedly watched video, connected our observations to constructs noted above, and gradually developed our framework. This methodology blends interaction analysis with the constant comparative method (Jordan and Henderson, 1995; Glaser, 1965). In this case study, the student solves the problem of programming the PixelBot to paint three jagged lines, horizontally spaced 3 tiles apart. Our purpose is to document the details of one student’s problem solving process, rather than make generalized statements about learning processes. We hope that this research will allow for more rigorous experimental studies to come.

4. Results and Contributions

We identify the student’s key challenge of coordinating subgoals and problem solving phases, and then identify how the student navigates this space by coordinating resources in the environment.

4.1. Coordinating Subgoals and Problem Solving Phases

The focal student’s programming process involves completing three subgoals—writing code to paint a jagged line, writing code that moves to the next line, and orchestrating this code in two functions—each of which entails navigating three phases of problem solving: planning, building, and monitoring. The student prioritizes subgoals and problem solving phases in response to syntax and logic bugs, instructor prompting, and focal sources of knowledge.

The student begins by planning the trajectory of the PixelBot through the jagged line, building using only API movement instructions. She silently monitors without running the code, and then deletes it, re-building with painting actions and a loop. The student then attempts to monitor the resulting PixelBot action by running the code, but quickly stops the program before it advances enough to show the corresponding PixelBot action. The student continues building but a drag-and-drop attempt leads to an error message in the text editor. Her subsequent monitoring and re-building process, which generates more syntax bugs, involves trying to interpret error messages and create symmetry between brackets, parentheses, and quotes.

An instructor walks over and introduces a different monitoring process: comparing the broken syntax in the editor with correct syntax on a handout. Over six minutes, the student foregrounds the subgoal of writing the jagged line code through three additional subgoals: writing the PixelBot trajectory, adding color, and adding a loop. New information from the environment motivates the selection of subgoals. When errors are identified, the student takes immediate action rather than continuing with her current task. The student transitions rapidly between planning, building, and monitoring. In addition, within a single problem solving phase, we see a range of sources of knowledge deployed. For example, the student’s monitoring approach of perceiving error messages and reasoning about symmetry contrasts with the instructor’s monitoring approach of perceiving and reasoning about connections between the handout and editor. After resolving the syntax bugs, the student identifies the subgoal of developing the jagged line function as complete despite having painted several squares the wrong color. Not until this logic bug is flagged in the correctness check at the end of the session does the student re-foreground the subgoal of monitoring the jagged line function.

Throughout this process, the participants extemporaneously structure how to navigate the problem space, selectively managing subgoals. Their pathway is neither linear nor premeditated: planning arises throughout, contingent on syntax bugs that arise, logic bugs not yet noticed, moments of refactoring, and moments of recalibration after subgoals are judged complete.

4.2. Coordinating Resources

How does the work of pursuing subgoals across problem solving phases unfold? This section describes a lower-level, moment-to-moment coordination of media across people and tasks.

The student utilizes multiple sources of knowledge to coordinate resources in the environment. For example, while monitoring the jagged line code, the instructor foregrounds a process for syntax verification by comparing the code token-by-token with an example from a handout: “Look at this [code reference sheet] and then compare every sort of word like ‘function’, ‘function’, […]” The student looks back-and-forth between the screen and the handout, perceiving and reasoning about similarities between the two. These sources of knowledge thus bridge two resources in the environment: the handout and editor. This process is repeated as the student debugs a missing paint instruction while working on moving to the next line. As the student steps line-by-line through the code, her gaze moves right and left between the code editor and the corresponding PixelBot action. Attention to different parts is variable: she steps quickly through parts she has already thoroughly vetted, and slows down, even stopping, when she arrives at code that corresponds with the dispreferred PixelBot action. The student again perceives and reasons about two resources (editor and PixelBot actions).

While monitoring the jagged line code using the instructor’s token-by-token syntax verification strategy, the student runs into a repeat loop which is not documented. The instructor provides expert testimony and suggests clicking the button to insert the repeat template. The student coordinates the two snippets of repeat code in the editor: one serving as the template, and one serving as the target to be fixed, but applies the same process of token-by-token comparison as she did earlier with the code reference sheet. Later, while monitoring the code to move to the next line, the student draws on her memory and uses the same strategy of inserting the repeat template to check the syntax of the program. The student calls upon this strategy in two different instances to help cross the gulf of evaluation and propose a fix for each syntax bug.

The coordination of resources in the learning environment helps the student cross the gulf of evaluation and execution. Before the instructor foregrounds the syntax verification strategy, the student’s method for resolving each syntax bug was to read the error message, interpret the problem, and propose a fix. This process presents a wide gulf of evaluation as the student needs to operationalize the error message by converting a description of the problem into an applicable fix based on prior knowledge of program syntax. The instructor’s method for syntax verification relies on the same sources of knowledge, perception and reasoning, but uses the handout to present a more accessible affordance to cross the gulf of evaluation.

4.3. Conclusion

Through a microgenetic analysis of the process of teaching and learning about abstraction, we document the extemporaneous prioritization of subgoals and the back-and-forth coordination of problem solving phases. Participants navigate these tasks using multiple resources and sources of knowledge to cross gulfs of execution and evaluation. Future research should address how to help students structure subgoals, reflect on problem solving techniques, and recruit productive sources of knowledge as an ensemble process.

Acknowledgements.

This material is based upon work supported by the Sponsor National Science Foundation https://doi.org/10.13039/100000001 under the Grant No. Grant #1607742, Grant #1612770, and Grant #1612660. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.