Codespec

Designing and building a scaffolded learning experience for novice programmers

Overview

In 2021, there were two ways to practice programming online: block-based editors or writing the code from scratch. No platform successfully bridged the gap between these two modes.

Academic research confirmed “current introductory computer programming instruction and assessments may fail to scaffold the acquisition of [programming skills” [14], especially when it came to serving both novice and experienced learners with the same tool.

There was no one-size-fits-all solution.

So, a computer science (CS) education researcher and I created a web application that allowed learners to solve coding problems using 5 research-based problem types of increasing difficulty.

CS educators affirmed the tool’s pedagogical potential and learners praised the scaffolded design.

Where: After winning first prize and $10,000 at the 2021 University of Michigan Learning Levers competition, we continued to develop the tool at Carnegie Mellon University (2022-2024).

My Role: As the sole designer and lead developer, I partnered with an academic researcher to design 705 interfaces (186 Django templates and 519 Vue.js components) and build 486 backend endpoints (spanning authentication, course management, problem authoring, administrative tools, and a comprehensive REST API).

Five Research-Based Problem Types

Prior computer science education research served as the foundation for the 5 problem types we sought to combine, each with their own advantages and disadvantages.

Pseudocode

Solving a problem using plain English blocks that represent the underlying code.

  • Strength: planning a solution ahead of time reduing cognitive load for learners [16]

  • Weakness: advanced learners may find these exercises pointless for easy problems

Parsons (Code Blocks)

Solving a problem using mixed-up code blocks

  • Strength: allows learners to focus on concepts instead of syntax. Block-based problems have also been found to maximize engagement [5, 15]

  • Weakness: learners may struggle if the expected solution doesn't match their own mental model

Faded Parsons (Fill-in-the-Blank Blocks)

Solving a problem using mixed-up code blocks with blanks

  • Strength: these exercises have been found to be "significantly more effective for teaching pattern comprehension and pattern application over code-writing and code-tracing exercises"

  • Weakness: expected blank values may not match learner's mental models

Fix Code

Solving a problem by fixing errors in buggy code

  • Strength: provides opportunities to practice code reading and code tracing

  • Weakness: error messages may be unintuitive and/or intimidating to novice learners

Write Code

Writing code from scratch

  • Strength: allows learners to use total creative freedom to solve the problem

  • Weakness: greatest risk for syntax issues (forgetting a colon at the end of a function), semantic errors (using the wrong data type), or typos (misspelling a variable name)

Our Value Proposition: Strong Scaffolding

I analysed 7 competitor block-based programming practice platforms. None of them supported a fluid progression of problem solving modes from pseudocode to writing code from scratch, confirming the uniqueness of our offering.

Prioritizing Learner Agency Through Tab-Based Design

I explored several options for combining the 5 problem types into 1 learning experience:

Alternating problem types - learners would see and solve each problem type 1 at a time in order to complete the overall exercise.

  • Pros: Least cluttered UI, kept learners focused on the material, avoides having to name the problem types (which could result in unfamiliar labels)

  • Cons: Took away all agency from the learner and potentially increased frustration, forcing beginners to attempt problem types that were too difficult and advanced learners to complete trivial exercises

Dropdown toggle - learners would use a dropdown to switch between modes

  • Pros: Minimized additional controls in the UI, provided learners complete agency over which problem types to attempt

  • Cons: Only displayed the current problem type and obscured all possible options (learners may not notice the dropdown and miss the other choices), problem type labels such as "Parsons" are likely unfamiliar to many learners

Problem type tabs - each problem type is displayed in the interface, from easiest to hardest (left to right)

  • Pros: Provided complete agency to learners, displayed all options at once, minimized clicks required to change the problem type

  • Cons: Potential for unfamiliar tab labels, busiest/most cluttered interface that could be overwhelming to some users

My co-founder and I ultimately chose the tab-based design because it potentially offered the greatest learner agency, clearest display of all options, and quickest way to switch between problem types.

1- vs 2-Column Layout: Why Not Both?

Historically, Parsons problem libraries such as JSParsons have displayed block-based problems using a 2-column layout: mixed up blocks on the left and the solution area on the right.

Pros: Reduced drag-and-drop distance between the two work areas (offering the best usability/accessibility), afforded easy comparison between used and unused blocks

Cons: Limited problem authors to short blocks or risked obscuring block content to learners

Given the limitations of the traditional 2-column layout, I explored a single column version of block-based problem types.

Pros: Allowed for longer blocks without hiding any of the content, condensed unused block area kept drag-and-drop distance low

Cons: Mixed up blocks are less scannable, interface may introduce vertical scrolling for long block-based solutions

Neither option seemed like a clear winner, so for the first round of usability testing I offered learners both options and the ability to switch between them.

Usability Testing: Round 1 Insights

I prepared a clickable prototype of the interface. My co-founder and I then tested it with 7 participants who ranged from novice programmers to PhD computer science education researchers.

What went well?

Grid lines support indentation: To reinforce the idea of indenting blocks, my co-founder encouraged use of a grid in the solution area, which appealed to participants.

"These gridlines are interesting, I guess they help you indent the [blocks] that you drag [and drop]."

Strong preference for 1-column layout: 5/7 learners preferred the single column layout, as it afforded better readability of the code blocks, especially for longer solutions.

"I already couldn't see everything to the right. And now I can't even see everything all the way down. Like, I can't see everything. So it's hard to understand that code."

Where did the design fail?

Pseudocode indentation: Our design enforced proper order AND indentation for all block-based problem types, including pseudocode. But one participant pointed out:

"When I'm writing pseudocode, indentation doesn't matter quite as much."

We decided to leave the indentation requirement despite this feedback, as my co-founder and I saw pedagogical value in getting learners to think about indentation earlier in their planning process.

Unfamiliar tab names: As expected, participants struggled to understand the "Parsons" and "Faded Parsons" tab names.

"I have no idea what [Parsons] means."

I proposed the plain language alternatives of "Blocks" and "Fill-in-the-Blank".

Implementing Accessible Drag-and-Drop

One of the biggest technical challenges I faced when building the interface was making the problem interface keyboard accessible. Vuedraggable, the NPM package I selected for the primary drag-and-drop interaction, did not offer an out-of-the-box solution for keyboard users.

So I modeled a custom approach after the recommendations found in "4 Major Patterns for Accessible Drag and Drop". Users press the space bar to select a block, enter to move it to the solution grid, delete to move a solution block, arrow keys to move a block around the solution grid, and space bar again to release it.

Coming Soon: More Development Highlights

I'm still putting this case study together. Up next I'll talk about:

  • Designing the Parsons problem data architecture

  • Tracking problem solving activity with click stream analytics

  • Developing an intelligent hint algorithm

Thanks for reading!