Cascade started with a simple frustration: I wanted to share the pieces I was learning on piano, but recording yourself playing isn't always easy. Lighting, camera angle, nerves. What if the music could speak for itself, visualized as those satisfying falling notes you see in YouTube piano covers?

The Idea

The concept was straightforward on the surface: parse a MIDI or sheet music file, animate each note as a colored bar falling toward a piano keyboard, and let the user export the result as a video. But "straightforward on the surface" is usually code for "deceptively complex underneath," and Cascade was no exception.

I'd been wanting to build a native Swift app for a while, and the Apple Student Swift Challenge felt like the right forcing function. The competition gives you one Swift Playground to impress the judges: no external servers, no third-party packages beyond what ships with the SDK. Everything had to live in a single self-contained package.

Building It

The core challenge was rendering. I used SpriteKit to animate the note bars in real time, synced to AVFoundation for playback. Parsing MusicXML (the format most sheet music exports to) turned out to be the messiest part. The spec has dozens of edge cases around ties, grace notes, repeats, and transpositions that I slowly hunted down one by one.

Playback speed control came next. I wanted users to be able to slow the animation down to learn a passage, or speed it up to preview a whole piece quickly. Threading this through the SpriteKit timeline while keeping the audio in sync required more care than I expected, with a few late nights spent debugging desync artifacts.

The export pipeline was the final piece. SwiftUI's VideoToolbox let me write frames directly to an AVAssetWriter, capturing each rendered frame of the SpriteKit scene and encoding it to H.264. The result: a shareable mp4 that looks exactly like what you see in the app.

Submitting

Getting everything to fit inside a Swift Playground App was its own exercise in constraints. Assets had to be bundled carefully, and I had to be ruthless about what features made the cut. The submission window is tight and there's no "just push a patch." Whatever you submit is what the judges see.

I'm proud of what Cascade became in that window: a clean, native app that does one thing well. Whether it impresses Apple's reviewers or not, building it taught me more about the Apple frameworks in a few weeks than months of tutorials ever had.

Built with Swift, SwiftUI, SpriteKit, AVFoundation, and a lot of late nights. Jason Fang, 2026