Neurodiverse Computational Thinking

A National Science Foundation Project

Neurodiverse Education Resource Center & Arizona State University

Introduction

Students with disabilities, particularly those with autism, experience unequal outcomes in STEM education and employment. Despite frequently reported strengths and interests in STEM disciplines, individuals with autism often do not receive the support—those related to communication, transitions, and flexibility—needed to succeed. In working with teachers, parents, and employers, this project will create pathways to position students with autism spectrum disorder for success in school and also the larger community.

This NSF-funded project commences with three workshops in Fall 2021, during which a pilot group of supported teaching fellows will explore the potential of custom-designed, wearable musical instruments to embed and teach computational thinking concepts to neurodiverse middle school students. An initial set of instruments, described below, have been designed by co-principal investigator Prof. Seth D. Thorn of the School of Arts, Media & Engineering at Arizona State University. During these three workshops, teaching fellows will explore these instruments while co-designing curriculum, exploring ensemble activities, and proposing new designs and instrument modifications and elaborations. We propose the following themes for these initial workshops, but will adapt these to the recommendations of our teaching fellows:

Workshop 1: Playful exploration of wearable instruments.

Workshop 2: Curriculum development.

Workshop 3: Social affordances and ensemble activities.

Instruments and demonstrations provided by Prof. Seth D. Thorn, Technical Design Lead.

Project Investigators


  • AME: Xin Wei Sha (Principal Investigator)

  • AME: Seth Thorn (Co-Principal Investigator)

  • Teachers College: Mirka Koro (Co-Principal Investigator)

  • Teachers College: Margarita Pivovarova (Co-Principal Investigator)


Wearable / Embeddable Musical Instruments

Equipment

Teaching fellows will receive a workshop kit:

Rainstick

What CT concept(s) does it embed?

  • counting / indexing

  • sorting / scrambling

  • clocking (synchronous / asynchronous)

  • probability

How does it work?

Like an acoustic rainstick, this computational version contains a finite number of "grains" that trickle from one end to the other when the rainstick is tilted. The more vertical the tilt, the faster the grains fall.

In the simplest version, the grains fall at a linear rate. The student records a short piece of audio that is arbitrarily sliced into a number of individual grains. The student can dynamically change this number into fewer or more grains, so that the grains become longer or shorter in duration, respectively. As the rainstick is tilted, these grains will play back in ascending or descending order, according to the state of the rainstick.

Variations can be introduced. For instance, the playback order of the grains can be randomized using a scramble function, or sorted back into ascending/descending order. The timing can be altered so that the grains play back asynchronously rather than synchronously. A set of timing delays can be added to better simulate the physics of an acoustic rainstick, so that the grains encounter multiple "twigs" as they trickle down, multiplying the sound generation. A probability parameter can also be added determining the likelihood of a grain encountering a twig.

Playful improvisation is enhanced by having students record their own sounds "into" the rainstick. They can use speech, singing, musical sounds, and foley effects using everyday objects.

How can it be playfully used in group ensemble activity?

Rainsticks can be divided between two students: one pours their rainstick into someone else's rainstick. The other then pours it back. (This is also easy to implement telematically). This dyadic instrument could be extended to a group involving multiple individuals (3+), with a finite set of "grains" being poured around, with unexpected sonic transformations occurring after each exchange, or other creative modulations.

Wearable Jazz

What CT concept(s) does it embed?

  • arrays / indexing

  • iteration / recursion

  • use of division / remainder (modulo) in iteration

How does it work?

Students are shown a simple melody player that generates a chromatic scale when the hand is raised and lowered. Raising the hand climbs up the scale; lowering it climbs down. Students then hear a different version that plays the familiar pentatonic scale, ubiquitous in folk and pop music. This new version transforms the basic chromatic scale (0, 1, 2, 3, 4) by using a lookup table that takes those numbers as input and outputs a new set of numbers (0, 3, 5, 7, 10).

This instrument can be enriched in many ways that makes it very engaging and fun to play. For instance, accompanying harmonies can be used the shift the scales into a new key by adding a fixed offset (e.g., 0, 3, 5, 7, 10 --> 1, 4, 6, 8, 11). Bass tones and duplicate melodies can be used as well, or drum beats added after a series of notes are played to enrich the instrument. Students can also record their voice and "melodize" it using a familiar autotune effect that is widely used in contemporary pop music, increasing affordances for playful improvisation and experimentation. A student can create a rich musical ensemble instrument that they activate by spinning around, raising the hand, or twisting the arm, generating harmonies, melodies, and drumbeats all at once.

How can it be playfully used in group ensemble activity?

The rich ensemble instrument can be divided among several students, so that one plays the drums, another the melody, and another changes the harmony. Overall tempo or other transformations can be an effect of averaging the movements of the group as a whole.

Glissandi Catapult

What CT concept(s) does it embed?

  • linear / logarithmic relationships

How does it work?

An exponential increase in frequency is needed for humans to hear a linear increase in pitch. This presents an opportunity to explore different curves (exponential functions) while listening to a ramp between two pitches. The traditional term for this musical effect is the Italian word "glissandi." Students listen to a few examples, then begin producing their own.

How can it be playfully used in group ensemble activity?

Raising or lowering the arm determines the slope of the glissandi. Touching the screen while rotating the sensor "winds up" the catapult, which determines its duration when the screen is released.The instrument is polyphonic, so that multiple rising and falling glissandi with different durations and curves can overlap in time.

At particular pitch ranges and curves, glissandi begin to sound like familiar sci-fi effects ("laser blasts"). Moreover, thick polyphony generates interesting and unique perceptual results. Students will have fun activating this chaos.

Musical Shoes

What CT concept(s) does it embed?

  • signal vs. noise

  • high-pass and low-pass filtering

  • thresholds / events

How does it work?

A pedometer detects footsteps by isolating spikes in acceleration signals from noise and other variations. Students will use an interface that offers simple filtering and threshold operations to carefully "tune" a sensor worn on the foot to detect footsteps. Students can then use this signal to create different sounds as they walk around the room. Pre-recorded samples of walking on gravel, snowpack, or in puddles of water can be used, or footsteps can trigger individual musical tones or play bits of musical melodies. Students also have the option to record their own sounds and parse them into smaller "grains" (as with the rainstick) that can be triggered by footsteps. Parameters can be manipulated to modify these recordings.

How can it be playfully used in group ensemble activity?

Students can walk around together to playfully explore the different sounds their footsteps are making. It is also possible to have everyone's shoes be related to the same sound source. A measure of relational activity, such as the elapsed time between the most recent footstep and the last one, can set the musical pitch of an instrument, which individual footsteps trigger. This relational feature enlivens the instrument whether it is used in a group or by an individual. Musical shoes should also work well telematically - both sonically and conceptually - by imparting the sense that everyone is walking in the same space.

Bop It

What CT concept(s) does it embed?

  • counting / indexing

  • sorting / scrambling

  • clocking

  • probability tables

How does it work?

Students record commands into the computer, such as "raise your hand," "twist your arm," "kick your foot," or "spin around." The commands that are chosen must be related to the kinds of gestures or movements the sensor can detect, which is contingent on its placement. The order of commands occurs through either predetermined random sorting or a probability table. Students design how the gameplay occurs by deciding on the relation between the number of commands, the length of gameplay (based either on events or a clock timer), the rate of tempo increase, and the placement of the sensors.

In this activity, the sensors can also produce musical responses not only when commands are executed and fulfilled, but also in response to motion generally. The bop-it game then becomes a kind of "score" for a unique musical piece.

How can it be playfully used in group ensemble activity?

There are many ways to implement this game. In one iteration, students will record their names into the computer along with the commands. The computer announces a person's name prior to the command. Alternatively, each player could be assigned one or two particular gestures, so that when a specific command is heard, the individual associated with the gesture performs it. Thus, students are able to design their own gestures. For instance, someone may wish to wear the sensor on the foot, another atop the head, another on the wrist, etc.