Blog cataloging research process building a code visualizer tool. For CREU-CRA research grant.

View the Project on GitHub

Week One


Welcome to my blog, documenting the research/coding process of building a code visualizer. This CREU project is a research project related to the redesign efforts of University at Buffalo (UB)’s undergraduate introductory computer science courses (CSE115 and CSE116). Our faculty advisor and two of the student researchers are a part of the introductory computer science course redesign team.

Project Team:

In brief, the redesign team is a departmental effort to streamline the introduction to computer science courses. The end goal is to revamp our school’s computer science curriculum by improving course structures, incorporating more applications of the topics covered, and solving logistical issues that come with an increasing computer science student body. Ultimately, the hope is that any student, regardless of major, will be able to take the first introductory course (CSE115), gain interest in computing, and understand the fundamentals necessary to continue and succeed in the field of computer science.

As a team, we decided a few goals over the course of the Fall 2017 semester, to ensure that the project stays on track. While coding and building progress occurs on the code visualizer, Katie and I (Heeba) will focus on conducting research in tandem.

Week Two

Design Discussion

This week, we discussed over-arching design of the code visualizer. Below is the “final” diagram, at the end of whiteboarding: Visualizer Planning Diagram

There were many questions we had to answer, going into this project:

We have decided that the interpreter would exist on a browser frontend, in order to make it more accessible – in other words, an indiviudal would only need a browser and internet access in order to utilize the visualizer. The interpreter would deal in JavaScript, for a few reasons; namely, the new introductory computer science courses would introduce students to a few languages, including JavaScript. As of now, there is a plan to use this code visualizer for the CSE115 introductory course in Sprig 2018, and then publish in the summer.

For the minimally viable product, variables are the first step (to be able to interpret and visualize). As far as gauging the impact of the interpreter, testing will need to be conducted. A/B testing within education will be interesting, however–we also have some ethical concerns to deal with, such as how we go about getting approval to collect data.

Week Three

Research: jGRASP

This week, we also started looking at jGRASP, an existing system that includes a lot of program animation. To begin research in this area, I read through some papers, including Testing the jGRASP structure identifier with data structure examples from textbooks.

What is the jGRASP IDE? It is an IDE that includes dynamic viewers for Java, specifically intended to generate abstract visualizations of data structures such as linked lists, stacks, queues, heaps, hash tables, and binary trees. The purpose of these new viewers is to provide fine grained support for understanding data structures, an important concept in programming and computer science.

This paper talks about the jGRASP IDE, and how the integration of dynamic data structure visualization and the ability to step through source code in either debug or workbench mode provides a “unique and promising environment for learning data structures.” In short, the jGRASP structure identifier, which is used to automatically identify data structure objects and generate appropriate visualizations, has been tested with examples from twenty different data structure textbooks. The results of the testing indicate that the structure identifier is extremely robust.

The discussion of this paper is promising, as it indicates the potential of what we can do with our code visualizer. Of course, dynamically visualizing the various states of data structures is much more complicated than visualizing variables (our first step), but there is an implication that the visualization does in fact aid the user in understanding such concepts.

Coding: Architecture Discussion

On the code/building side, this week involved architecture discussion; see the below images. Architecture 1

We’ve decided to only change code in editor if you start interpretation, and to use the same interpreter object as the state of the program. When a user is “stepping into” code, the code state cannot change. For mathematical expressions, we will want to show the steps to get to the final value.

Architecture 2

Week Four

Research: Dynamic Program Visualizations

This week included research around Dynamic Program Visualizations, specifically in regards to Dynamic Program Visualizations – An Experience Report, conducted out of Auburn University (see paper here). This report actually detailed the authors’ experience using dynamic program visualizations in their Java-based introductory computer science courses during the Spring ‘13 semester. The visualizations were constructed by both faculty and students using the new viewer canvas in jGRASP. Students and faculty had access to the software, and program visualizations were used in the classroom as well as in lab activities and programming projects.

The focus of the report was on how helpful the program visualizations were in aiding students’ understandings of their programs. There are a lot of parallels with this report, and the future plans for this code visualizer–i.e., wanting to use it in the classroom next semester to help students understand their code through visual representations. The reports’ positive findings – that “…direct observations in the lab and individual meetings with students to discuss their work indicated that students could indeed build dynamic visualizations that, in conjunction with the debugger, helped them understand and correct bugs in their programs much more effectively than using the debugger alone…” makes me hopeful for the success of our interpreter.

Coding: Language Feature Support and Implementation; Prototype Out

The prototype is now out! Check out this video, running through it here.

On the coding/building side, work includes discussion about use of a linter, specific language features to support, and the implementation of functions and prototype.

As far as linter goes, we will want to support the JavaScript language feature up to around halfway through CSE 115. We will need to provide feedback on interpreter about language features we don’t support; the Mozilla developer pages has a good reference for everything we’ll want to cover in our interpreter. As far as implementation goes, we want to traverse through symbols and find valid syntax; we should throw an error on the first invalid syntax instance.

In terms of language feature support, ideally we want to have full support for:

We will consider external functions as 1 step: we will not step through it within our interpreter. For calling external functions, we should try to implement function calls in a way that allows us to have inbuilt JavaScript libraries for free.

Week Five

Research: Dynamic Object Viewers for Data Structures

Considering the direct parallels of the research featured in last week’s blog post (Dynamic Program Visualizations – An Experience Report), I decided to look at more research out of Auburn University with respect to this report. The paper Dynamic Object Viewers for Data Structures (link found here) is also from Auburn University and by the same authors. This paper included experiments to determine whether visualization improved students’ grades and understanding. The results of this paper found there to be a statistically significant improvement over traditional methods of visual debugging. As the authors mention in their abstract, although many visualization techniques have been shown to be effective, they still are not widely adopted. The main reasons for this include:

  1. A lack of methods of automatic visualization generation
  2. A lack of integration among existing visualizations
  3. A lack of integration with basic IDEs (integration development environment).

It is our hope that the interpreter will at least address reason number one, which then relates to two and three. Since the interpreter and visualizer will live in a browser, it makes it much more accessible as a tool. Students don’t have to worry about adding it to their IDE, and indeed might find it to be a “natural” addition to their workflow, since all they would need to do is open the tool in a browser.

Coding: Visualizer, Variable Capturing, and Editor Updates

Visualizer: At this point, a p5.js canvas has been embedded into the visualizer. Unfortunately, you cannot copy text from the canvas, as of now. However, it’s a lower-level issue, and the group is okay with focusing forward. There is also now variable/naming representations (in the form of boxes). Although there’s room for changing layout, this is mostly concrete. The next steps include to start designing for future representations, like arrays, scope, etc.

Variable Capturing: Right now, we have capturing of variable names on assignment. We are currently working through all the of stranger edge cases and possibilities, while making sure it’s still scalable. Also working to connect the back-end with the visualizer (front-end).

Code Editor: For the code editor, the line numbers are partially functional. We will continue to work on linking the API to the code editor. Additionally, progress has been made on fixing issues when the window is resized.

Week Six

Research: Memory Diagrams

There has also been a lot of research on the use of static images of program execution (e.g., memory traces) and its effects on learning. This week, we looked at some of the papers from that area of research, specifically with regards to Memory Diagrams. The paper, Memory Diagrams: A Consistant Approach Across Concepts and Languages (link found here)

This paper, from the Ithaca College computer science department, discusses memory diagrams and their usefulness in demonstrating concepts. As the paper denotes, hand-drawn memory diagrams are frequently used in computer science to demonstrate new programming concepts and support students’ understanding of program functionality. However, these diagrams often vary among courses, instructors, and languages, which can confuse students moving through the curriculum. The main argument of this paper is that consistent memory diagrams throughout a curriculum will not only reduce confusion, but provide a foundation of understanding that helps a student throughout the transitions among courses, instructors, and levels of difficulty.

From this paper, it is clear that there needs to be a level of “standardization” in the diagraming/visualizations of our interpreter. It is definitely to our advantage that the project is being conducted in conjunction with our CSE deparment’s course redesign committee. More discussion will need to occur on the choices of representation, and how we might help those representations live on past the introductory courses.

Coding: Documentation, Visualizer State; MVP Out

The prototype is now out! Check out this video, running through it here.

Coding progress this week includes discussion about testing, deployment, and general workflow needs. Some changes need to be made to the visualizer structure, to make the user experience more intuitive. Additionally, work is being done with regards to the visualizer state; the console needs to know this state, so we need to figure out how to better link the console with the visualizer. There is also progress on checking for multiple-statements on each line, although research needs to be done on how best to do this, since it gets complicated.

Week Seven


Automated construction of memory diagrams for program comprehension (


Beta Collaborators Clear Visualizer Break out of code run Increase Speed Code text resizing

Week Eight

Research: Trace-Based Teaching

Trace-based teaching in early programming courses here

Coding: Landing Site, Beta-Testing and Collaboration

The landing page for the code visualizer is up! Check it out here Visualizer