I was working on video rendering on Android devices – reading through and mashing together libraries and example code I found. The task as it’s currently structured has 6 main classes interacting:
An extractor to present encoded frames of video from a file
A decoder to decode each frame
An animator to keep this happening regularly and check if it’s time to show a new frame
A view which is the interface element the video will be shown in, and which I’m using as a convenient repository for all the other action (this seems to be pretty standard approach)
A renderer which talks to OpenGL, the rendering API
I’ve been bodging everything together in one prototyping/sketchpad project. My approach was to keep it just-about-functional for my one test case (displaying three overlaid alpha-blended video elements) as I swapped in and out components and ported bits from Java to Kotlin.
I definitely learnt a lot from that, but today I tried a different tack.
Instead of accomplishing a fairly complete task messily and crudely, I took the single component I wanted to understand and placed it in a blank, fresh project. Then I tested it!
I got the class, TimeAnimator, to report all salient variables to GUI elements. Like debug text but organised so I can see the situation at a glance.
I realised I couldn’t make it do what I thought it had been doing, which was fire events at a steady framerate equal to the video framerate (of 30fps).
I shuffled through variants of the class and learnt what they do.
After a bit, I realised none of them did what I wanted. I went back to the library I was studying and finally twigged that the controlling framerate/timing information came from the decoded video data, which was merely being checked (at a frequency much faster than the framerate) by the TimeAnimator, which is not meant to run anywhere near as slow as 30fps.
So far, so trivial. But I might have kept playing with the original code for ages without actually challenging my assumptions, distracted by all its moving parts and the fact that it looked somewhat like the final product I wanted.
I will definitely be reusing this technique of isolating the smallest component that I don’t understand, and “prickin’ it and frickin’ it” as my favourite rapper Guru might say.
Hope to be back here soon to discuss what this video functionality is actually for!
Are you new to blogging, and do you want step-by-step guidance on how to publish and grow your blog? Learn more about our new Blogging for Beginners course and get 50% off through December 10th.
WordPress.com is excited to announce our newest offering: a course just for beginning bloggers where you’ll learn everything you need to know about blogging from the most trusted experts in the industry. We have helped millions of blogs get up and running, we know what works, and we want you to to know everything we know. This course provides all the fundamental skills and inspiration you need to get your blog started, an interactive community forum, and content updated annually.
Warm analogue it ain’t. I knew when I started coding my synth-sequencer, Foldy, a few months ago, that it’d be harshly digital and crude sounding. I was inspired by tracker software as well as by two old PC-only music programs, Drumsynth and Hammerhead (which were the basis of my beat-creating project last year).
I’m releasing it today and calling it version 1.0. It works, but some iffy design decisions mean I won’t keep developing it.
That said, the code quality is a step up from my last release, the experimental art tool MoiréTest. I was able to go back and make big changes in Foldy, without the whole thing crumbling, which is always a good sign.
For the rest of this post I’ll explain what the program does, then what questionable decisions I made and how I would do it again.
(To try it yourself, download Foldy.jar from here and double click on it. If that doesn’t work try the further instructions in the readme.)
Foldy takes in a musical sequence, which you can type into a box in the app window. Notes are numbered as MIDI notes, where A=440 is at 69, and notes range from 0 to 128, and separated by commas. A rest is -1.
(By the way, did you know that, incredibly annoyingly, there is no industry standard for numbering the octaves of MIDI notes? The frequencies are agreed on, but one manufacturer’s C3 is another’s C4… how sad. This doesn’t impact Foldy though, I just work from the frequencies.)
The speed that notes are played is altered using tempo and beat subdivision controls. All the other parameters in the window modify the sound of individual notes. Only one note can play at a time. This kept things a bit simpler though, with the Java Sound API, opening another output line or mixing two together wouldn’t be much harder.
I was going to include a choice of mathematical curves, possibly Bezier curves, for the amplitude envelope, out of a perverse desire to avoid the bog-standard Attack-Decay-Sustain-Release model, which is suited to a keyboard instrument where a note is attacked, held and released. I was thinking this synth could be more percussive, inspired by the basic sample-playback model of drum machines and trackers (a type of sampler software originally made for Amiga computers and associated with the demoscene).
Unfortunately I didn’t finish the Bezier stuff, but in any case it probably wasn’t suitable. (For one thing, Bezier curves can easily have two y values for one x value.) In fact, I didn’t do any extra envelope options, partly because envelopes typically drive filters or modulations, but these are not allowed by my architecture. If there’s an obvious v1.1 feature, extra envelope curves is it.
One feature that did make it in is “wave-folding”. To get more complex waveforms, I cut a sine wave at a certain amplitude, and invert anything above that amplitude. This can be done multiple times to add a lot of harmonics.
However, this is a restrictive technique with a distinctive grinding, mechanical sound. All we’re doing here is shaping a waveform which is then repeated exactly at the period of the note frequency. The ear instantly picks up the lack of complexity.
I remember when I was a teenager, having the following bright idea: if I can see that the recorded waveform from my bass consists of repeated bumps, can’t I just sample one of those and repeat it/change the speed of it to get any bass note I want?
This is the basic concept of wavetable synthesis. However, when done as simply as that, it sounds completely artificial, not at all like a bass guitar. The sound of any real instrument has complexities like propagating resonances, changes in pitch, string rattle and other distortions/energy loss.
(E.g. listen to the low note in this sampled bassline – it’s starts really sharp, then reverts to normal. That’s because plucking of a stringed instrument raises the pitch of the note momentarily, especially on an open string – I think this was an open E string on the original sampled recording, just it’s been pitched up here.)
Foldy has no capability for such modulations. I could try put them in, but here we come up against the compromises I made at the start.
Because I was afraid that rounding errors would mount up and give me grief, I decided to keep everything as whole numbers, taking advantage of the fact that digital audio ultimately is whole numbers: a series of amplitudes or “samples” each expressed as, for example a 16bit or “short” integer. (Most studios mix at 24bit these days, but say CD audio only goes up to 16bit precision.)
This informed the basis of the synth. Desired frequencies and tempos are approximated by a wavelength and a subdivision length expressed in whole samples. 44100 samples per second might seem fairly precise, but for musical pitches, it isn’t. So I found a compromise that bounded pitch error to about 20 cents:
Foldy tries to fit multiple wave cycles within a whole number of samples, for example 3 cycles in 401 samples. This gives a bit more precision, because the wavelength is 401/3 = 133.667 samples, in between the 133 and 134 wavelengths that are all I could get otherwise.
I then use these bits of audio, which I call “chunks”, and which could contain a single cycle or a handful of cycles, in the same way I was using single wave cycles originally. So every note would contain hundreds of them. Then I decided I could reuse this division to store amplitude envelopes – I gave each chunk a starting amplitude, and interpolated between these. (Of course, this is redundant at the moment because my overall envelopes are merely a linear interpolation from maximum to zero! But with a curved envelope, the result would be to store the curve within a few dozen or hundred points, with straight lines from point to point.)
Ugh… I don’t even want to write about it anymore. It wasn’t well conceived and caused me a lot of hassle. It precluded any of the more intriguing synthesis techniques I like, such as frequency modulation, because pitch in this system is fixed for each note (and imprecise).
Long story short, when I opened up the source code of Drumsynth recently, I realised that… it just uses floats and gets along fine. For modulation, it simply keeps track of phase as another float. I should’ve done that.
(That said, I think Drumsynth’s sound quality is far from pristine. This isn’t from rounding errors, I’m certain, but from not doing more complex stuff like supersampling. But, that’s out of my ability level right now anyway.)
Using floats, I still would have had trouble with the timing for the sequencer, probably… but that would have led me to the realisation that I was biting off too much!
It’s not a complete loss. I really enjoyed trying to calculate sine waves while sticking to integer arithmetic . I found out about Bhaskara‘s approximation, implemented it, and then found some really nice code using bitshifts to do a Taylor Series approximation of a sine wave. (I wish I had the chops to come up with it myself!)
Reading the source of Drumsynth also completely changed my approach to the GUI code. I originally had all of the classes that make up the synth – Note, Chunk, Sequence and so on – also be GUI elements by inheriting Java Swing component classes. I think I picked this up from some book or tutorial, but it’s obviously not good. It breaks the basic principle of decoupling.
Drumsynth blew my mind with its simplicity. There are no classes as it’s written in C, an imperative language. The synthesis is just one long function! I almost didn’t know you could do that, having spent a year studying Java and OOP. But given that the app is non-realtime (meaning that there is a third of a second pause to calculate the sound before you can hear it)… this is the sensible approach. Logically, it is one long straight task that we’re doing.
So I ripped out the GUI code from my main classes, and stuck it into one class called Control. Drumsynth’s GUI is even more decoupled: it’s written in a different language – a Visual Basic form that calls DLLs to access the synth functions!
(Yes, I know this is pretty out-of-date inspiration – heck Drumsynth even cheekily uses INI files for configuration though they were officially deprecated – but I think the lesson on directness and decoupling stands.)
My overall lessons from this project are:
Do normal stuff rather than trying to reinvent things.
Find exactly what draws you to a project and make that the focus. E.g. with this I would’ve been better off making something smaller and more conventional but which allowed me to try some unusual FM stuff.
Even though I’ve so, so much further to go, I kinda like low-level stuff. I mean, okay, nothing in Java is actually low-level, but still I was dealing with buffers, overflows, even endianness! Those are fun errors to fix.
Read other people’s code!
Even more generally, there’s a kind of tricky question here. This project showed me that it’d be a huge amount of work to approach the quality level of some of the audio programming frameworks out there such as JSFX, VST/Synthmaker, or JUCE. If I’m interested in actually programming synths for musical purposes, I should use one of those.
On the other hand, these are all coded in C or C++ (maybe with another abstraction layer such as EEL scripting language in the case of JSFX). If I really want to understand fundamentals, I should learn C.
But, it’s not very likely I’ll get a job doing high performance programming of that sort, considering the competition from those with as much enthusiasm as me for flash graphics or cool audio, but much more chops! I’m at peace with that – I quit music to get out of a profession that is flooded with enthusiasts.
I coded up a prototype recently in a new realm: video compositing in real time. It was enjoyable for a few reasons, one of which was the language I used.
Processing has been around for a while. It’s designed to give instant access to graphics programming. It does that by exposing one-line functions that draw to a window without pretty much no configuration needed, by locking in a folder structure (e.g. images you want to work with go in a “data” subfolder), and tying everything together in a simple dev environment.
It’s ostensibly aimed at visual artists and students, and it fits in a lineage of what we could call “beginner languages” that intentionally hide complexity. Because I’ve now had a variety of encounters with such languages, including QBasic in my teens, a previous failed attempt to learn Processing, and a bit of a tool around with Racket earlier this year, I thought I’d try pin down some thoughts on them.
Let’s maybe take a wide perspective on my whole encounter, from the first thought of “Maybe I could use Processing?”
I DuckDuckGo-ed it, and a nice .org site popped up. Wow! First impressions were really good. The whole vibe of the community and ecosystem is cheerful and welcoming.
There are pleasing non-jargon phrases all over the front page: “Creative Applications”, “Sketchpad”, “Visual arts”, “books available” – and yet it also feels like extensive technical work has been done, with links to implementations in multiple languages, and the obligatory Github profile, and many sponsors and partners.
I’m gonna go through this experience both from my current point of view, and an imagined one of a total beginner programmer. Both of these would give the Processing site, and related ones like Happy Coding, a hearty
The default installation format is a portable one. That might be a hassle to some. Or maybe it’s actually good. Unzip it and go, no permissions or shortcuts or whatevs.
The IDE loads showing a crisp splash screen with generative vector patterns like the main site. It’s perfectly decent, except for one thing which the docs make a lot of fuss about. Each code file is called a “sketch” – and the folder where these are kept is insistently referred to as “the sketchbook”. This caused me about ten minutes of trying to figure out if the sketchbook was a file format, and figuring out that it was automatically placed in a different location than where I’d unzipped the IDE. Calling it “sketches” would have made that clearer.
But for our absolute beginner, it’s not so bad: the IDE opens with a blank file, and you can copy in some example code and click the run button, and it works straight away. That’s REALLY nice.
This is what I really want to explore today. The trade-offs of simplification.
I think there’s an interesting conflict, in that simplification can make the first steps easier – which again is super valuable – but the next steps harder.
Both times I tried out Processing, I had this sinking feeling of near-betrayal as the light fun of the early steps sank into a morass of what feels like second-guessing the developers.
And I had it years back with the natural-language interactive fiction system, Inform 7, which I gave up on.
Making first steps easier is an illusion – one gets the feeling of accomplishment without really understanding what’s going on.
That can be all right. Computers hide inner workings from us all the time. The issue is finding the right balance of illusion so that the first peek behind the curtain doesn’t shatter fragile confidence.
This tendency of mine clashes with the carefully-laid illusions of beginner languages. It took me say twenty minutes and half a screen of code to display one layer of a candle flame and show it, chuffed, to my dad. Implementing my actual goal of compositing a few layers with filtering and interactive controls, took a whole day, much of which was spent in frustration as the seemingly generous affordances of the language turned to cobwebs.
Here were some things that tripped me up after my first joyful success:
drawing off-screen required trawling the none-too-voluminous developer docs to figure out the difference between a PImage and a PGraphics object -I’m guessing working on an off-screen a buffer wasn’t considered a fundamental use case for people coming from a visual arts background
a Movie object acts like an Image, but its width and height fields are set at 0 until the first call to its read() method (making some initialisation stuff harder)
some graphics functions are overloaded so that they can take either the red green blue components of the desired colour or a single component as a greyscale value – but not consistently
drawing to an off-screen buffer has to take place inside the single set draw() method, otherwise it randomly will draw to the main window in flickers
And there was more, but this is boring. Once I figured them out, it was okay. Even the rather arcane pipeline I ended up with – a Movie’s read() setting a flag that an update is required, which is checked in draw() which if necessary calls loadPixels(), updatePixels(), beginDraw() and endDraw() on the various buffers, and the actual draw calls and filter calls, in the correct order – is more or less understandable. (I didn’t say optimal!)
It’s just that… there’s a let-down from the initial hey presto! to what feels like debugging a somewhat complex system you don’t understand. Now there’s nothing wrong with that process. That’s how to learn. But could the journey from delight to disappointment be mitigated?
It was that feeling of having waded in a bit beyond my depth and being swept away by uncontrollable factors, that turned me off Processing and Inform 7 before.
However… I don’t think Processing could be much better. The only ways to smooth my experience would be tiny tweaks: if someone rewrote a tutorial, tweaked a line in an example, and so on. I see why people talk of community as a central aspect of high-quality software.
My conclusion is that despite the inevitability of harsh contact with system complexity, beginner languages are great – but don’t start one, join an existing one.
Community building is also behind those good vibes that are maybe even more important than the technicalities, when welcoming interested newbies.
Oh, and yeah, what did I actually code? Glad you asked. Taking some tips from an awesome visual effects tutorial by an enthusiastic fellow called Lendon Bracewell, I made some candle flame effects for a votive shrine app I’m planning. The idea is for this to become a fast prototyping/design tool when I’m assembling my app’s candle animations and effects. Check it.
Today I finished up a coding project, which was to make a simple synthesiser written in Java. I noted though, as I dashed off a quick readme file and uploaded the repository for the last time, a distinct sense of anticlimax, even disappointment. The app didn’t work out that well. Instead of writing an account of what I was trying to make and how I set about it, then, this evening I want to reflect on ways to avoid that letdown, specifically with side projects where you have total creative freedom.
This is about accepting and optimising for the way my brain works. Which is: it creates lots and lots of ideas at the start of a creative process. Here are some of the ideas I had for my synth:
doing additive synthesis by mixing together multiple instances of my basic instrument
doing additive synthesis by creating custom waveforms containing the desired harmonics
generating just intonated and 5-limit tunings
generating tunings from two overlapped harmonic series
generating 12-tone equal temperament and quarter-tone (24-tone) tunings mathematically
using the stuttering/echoing sound of buffer underruns as a musical effect
adding frequency modulation synthesis
having the synth be about a limited subset of sounds like bells, marimbas and bass
avoiding certain assumptions from the MIDI standard, e.g. removing note duration in favour of focusing on note onsets only (an idea influenced by my study of African-derived musics which have this emphasis)
using the synth in some kind of persistent, low-key online game as a way for players to leave melodies for each other to find
using rules to generate different melodies from the same basic info (say an array of integers), for example with different tunings or in different modes and scales
generating scales, chords and perhaps melodic cells or fragments geometrically, using relations I know from my study of music theory back in the day (e.g. a chord is every odd member of a scale is an approximation to 7 stacked fifths in a chromatic space generated using the twelfth root of two)
making an interface to expose these options interactively
And so on. The obvious common factor among all of these, I would say, is that there is no common factor. They are an extremely heterogenous bunch suggesting a whole lot of varied perspectives. Some are from a coder’s perspective (fixed-point calculation), some are about synthesising technique, some reflect my own limitations of ability, some are more like observations about the nature of music. Many have a distinctly contrarian flavour.
You can probably see the problem. There is no way to ever succeed at a project “defined by” such a list. And I’m not talking about success defined externally, but even just personal satisfaction. There are far too many requirements, many of which are contradictory.
The ideas there which are more rooted and achievable have another issue: they are technical challenges only, which makes them arbitrary. There’s no way to know if the solution arrived at is good, because there’s no agreed-upon way to measure performance. Should my bezier curve envelope allow two values for a given x input (which an unconstrained quadratic bezier can easily have), or forbid such curves? There’s no right answer because I haven’t defined what I’m trying to do.
This is in stark contrast to the school projects I’ve done for my higher diploma course: making a banking app, or an e-commerce website. Even if the work could get boring with those, the remit is clear and it’s satisfying when such a system comes together.
How did I manage to put a few weeks into my synth without realising that I hadn’t fixed upon a goal?
Or do I even need a goal? There’s nothing wrong with just fiddling about with stuff for fun, there’s even a fancy phrase to make it sound more official: “stochastic tinkering“. However, I know that I get my fun as a programmer in quite a specific way – by turning stuff that doesn’t work into stuff that does. When definitions are too loose, there’s no way to decide whether something works.
I’ve come up with a few pointers on how I might avoid this looseness in future.
The first is to design something conventional with a fictional, “normal” user in mind. This was how my school projects got done. This is good because convention (shopping carts in an e-commerce site, account selection in a banking site) guides you. This leverages the part of my mind that can’t stand to be wrong, and that likes tidying up: as long as the project fails to meet conventional expectations, I’ll be nettled into improving it.
However, finding the motivation to develop without the ego boost of originality would be hard for me. I know from the experience of finishing up my synth that work done just for the sake of appearing competent to strangers who come across my Github profile, isn’t very sustaining. The school projects had the virtue of being compulsory.
The second solution is to design something I’d like to use. This is… hard actually. It requires some self-awareness and honesty. I made a synth because I thought it’d be cool… or so I thought. Yet if I truly believed synths are cool, I’d probably have used one in the last few months; I haven’t done any synth-based music-making though in that time (despite having dozens of software synths installed on my computer). My conclusion is that I find synths to be a pleasantly intricate subject for mental distraction, but that I don’t actually have much desire to use them.
And similarly with the pixel art app I made before the synth. I like thinking about pixel patterns and generating them, yet if I liked making pixel art I’d be making some.
So, thinking honestly about one’s interests and requirements isn’t all that easy.
A third approach is to make something new, but very small. This worked well with something I made last year, an interactive sine wave visualiser. I actually made use of it, just for a second, while working on my synth to help me think about differentiating sines.
I’ve read advice to programmers about making tools that do one thing very well, and I can see the sense of it.
A fourth thing that has worked well for me is collaborating. When I’m working closely with others, my desire to appear right is a strong motivator. The hard part though for a side project is putting the energy into finding collaborators and the contradictory twin fears of not being good enough versus working with someone I feel is holding me back.
Those are actually familiar negative thoughts from my musician days.
Well, that’s what I wanted to write. Conclusion: even though my brain likes nothing better than lashing out idea after idea, finding the right one takes courage and deliberation. And it seems likely that good project ideas will combine a couple of the following: doing one thing only; being conventional; solving a real problem I have; being collaborations.
The idea popped into my head a few weeks ago. It won’t leave the prototype stage because this combination of technologies – pure HTML, CSS and JS (although do I use one library to synthesise sounds) doesn’t robustly support what I wanted.
I aimed to turn a piece of text into an instrument, where moving the cursor over any letter which corresponds to a musical note – so, ‘c’, ‘d’, ‘e’, ‘f’, ‘g’, ‘a’, ‘b’ – would pluck the letter like the string of a harp, and play that note as a sound!
At the time, I was thinking of a few possibilities
adding audio feedback for testing web pages, so that a developer/designer could hear if an element was malformed or missing information (aspects which are often invisible to the eye)
sonification, a concept which I think is rapidly going out of date as people realise its enormous limitations, but which was about turning reams of data into continuous sound patterns that would somehow reveal something within the data, but which I think were usually just third-rate electronic music or else showed no more than a good graph could, and basically made clear that the humanities PhD system sucks in people who’d be better off elsewhere… sorry I seem to have gotten into a rant here
simple enrichment and adding magic to the experience of visiting a webpage
That last is out of favour for web design nowadays. Instead, minimalism, accessibility and function are the buzz words. Fair enough… but also ominously envisaging the web as merely where stressed and harried folk get updates from a corporate or government source, staring down at their little phone screen.
Well. My little toy isn’t going to do anything to overturn that paradigm. Still, let’s take a short tour of the challenges in making it work.
My script caused each paragraph to detect when the mouse was over it. Then it cut the paragraph into one-character chunks and placed each of these single letters into a <span></span> HTML tag, so that it became its own card-carrying member in the Document Object Model.
Not very elegant at all! Also, despite span tags being supposedly invisible, throwing in so many of them causes the paragraphs to twitch a little, expanding by a couple of pixels, which wouldn’t be good enough for a production page.
However, it works. I set each of the single-letter chunks to play a synthesized tone when the mouse goes over them, and that’s it. Also, when the mouse leaves that paragraph’s zone, I stitch the letters back together, the way it was.
The downsides are that any HTML tags used to format or structure the text tend to get damaged by the process. Usually resulting in piles of gibberish, or text disappearing cumulatively. It would be possible to improve that, but with a lot of manual work. And, the browser’s attempts to be clever by healing broken tags here actually cause a lot of difficulties.
Defining some new kind of object that held the text and knew the location of each letter, would be a better bet.
However, I’m turned off this avenue of enquiry for the moment, because dealing with audio in browsers is a pain. Not for the first time, musical and sensual uses of technology have been left in the gutter while visuals get all the investment.
Where was I? Yes, the second problem is that Google Chrome made it standard that web sites can’t play sound upon loading up, but only after the user interacts with them. Well meaning, but really shit for expressivity – and quite annoying to work around. My skillz are the main limitation of course, but even trying out two libraries meant to fix the issue, I couldn’t make my audio predictably avoid error messages or start up smoothly.
No tech company would forbid web pages from showing images until the user okays it. But sound is the second class citizen.
When I know my JS better, I’ll hopefully find a solution. But the sloppy timing issue is discouraging. Some demos of the library I used show that you can do some decent stuff, although the one I experimented with took a good idea – depict rhythm as a cycle – and managed to fluff it with two related interface gripes. They made the ‘swing’ setting adjustable for each bar of a multi-bar pattern – pointless and unmusical. And they made the sequencer switch from bar to bar along with the sound being played – theoretically simple and intuitive, but – especially with the above-mentioned time imprecision of web interfaces – actually resulting in loads of hits being clicked into the wrong bar. (And if I say a drum machine is hard to use, it probably is – I’ve spent so much time fooling around with software drum machines I ought to put it at the top of my CV.)
But what am I saying! That demo’s way more polished than mine.
Perhaps even a little too polished! Visually anyway. All of the examples on that site are rather slick and clean-looking, perhaps because, I believe, the whole library has some affiliation with Google.
Ah, I’m being a prick now and a jealous one too, but one scroll down that demos page would make any human sick. The clean grids. The bright colours. The intuitive visualisations – yes, technology now means that you too can learn music, it’s just a bit of fun! Practice, time feel, listening, gear, human presence – nah!! And then the serious, authoritative projects – generated-looking greyscale, science-y formal patterns and data…. bleh.
And I’ll be using standard web technologies – JS, SVG – to make anything I come up with 100% reusable by non-programmers.
I was at a humanist wedding recently, my second this year. The readings the couple chose were short excerpts from J.R.R. Tolkien. It was a little disconcerting, but actually made total sense. To choose as a “sacred text” something that you quote all day long (they were obsessive Lord of the Rings fans). Something that you return to in times of turmoil.
Now, extensively customised weddings are par for the course these days, as is the observation, which could be made about almost anything, that LotR fandom resembles a secular religion. What this (completely lovely) ceremony really made me think of, is how I could do similar things in my own life.
If you’ve read my posts about hip hop, you’ll surmise it means a lot to me. Particularly since last summer when I spent a few months mostly on self-reflection, I’ve accepted that my personal conception of hip hop will be a sustaining force for the rest of my life. Basically, I think of hip hop as a martial arts move, a judo flip executed on society and the music industry, that smuggles deep wisdom inside the given structures of capitalism, masculinity and race. The catchy “gems” that stick with you, although camouflaged as egotistical boasts, are fully intended as self-help and spiritual sustenance for anyone who’s ready to receive them. This is true.
And it goes beyond hip hop. Somehow, quitting music performance helped make it clearer to me, that jazz and gospel are spiritual resources in my life. Monk, Nina, the Staple Singers, Paul Desmond, Charles Brown, etc. are presences I can always come back to.
My point? By all means, should we roll our own spirituality! Now, I’m not thinking of organising any weddings anytime soon. However: I know it’s late for a Christmas post, but isn’t that a ritual that could totally be expanded to include something personally meaningful? My first thought was, put some jazz photos on that tree. That’d be cool. Then I remembered that, as usual when I think I have great ideas, others are doing it already.
A good friend of mine has been making a determined effort for a few years now, to create the perfect Christmas jazz playlist. And it works! (I’m not expecting you to bookmark this 11 months in advance, BTW.) Chrimbo and jazz music combine just fine.
Back to that wedding for a second, the music for the ceremony was provided by Dublin duo, Moon and Son, and they played strictly jazz standards throughout, and it worked too.
My takeaway? Popular music has changed irreparably from the restructuring of the industry after MP3s and streaming. It will never again sound like its 20th century peak of effort and sophistication (I believe). Part of the reason I quit playing was this eclipse in cultural relevance. But – it cannot be denied that jazz, blues, gospel, hip hop and more have always had spiritual force, and people still feel it and maybe will become more and more conscious of it. So I bet that among my social class, the globally mobile, mostly white middle class, we’ll see more ritual use of black music, subtly, somehow, every year.
Three and a half months ago I started a software development course, to build a new career as a programmer. I’ve also quit playing music. As we come to the start of a new year I’m very happy with these choices and with the prospects ahead.
One of the things I’ve loved for a long time about programming was the culture of sharing knowledge. It directly influenced the tone of this blog. So, while I don’t know whether I’ll continue writing about programming on Drum Chant or make a new blog for it, today I want to try my hand here anyway at a software postmortem.
Following the time-honoured format (which I first saw on gamasutra.org I think, and also used on this blog before), I’ll introduce my project, discuss what went right in the production, what went wrong, and what lessons can be learned about software development.
I developed my first app in Java this month. It’s a number-guessing game simulating a lottery draw. I call it Gambler’s Delight. I had previously worked on a similar brief for a group project in school which I enjoyed enormously. So I decided to remake that idea on my own, using a more elegant, manageable design.
The game affords multiple rounds of attempts to win the lottery. In each round the user fills out a lottery ticket of 1-3 lines, where each line is a guess at the 6 random lottery numbers which will be drawn. When the user decides to play their ticket, the application draws the 6 random numbers and tells the player how good their guesses were, awarding fictional cash prizes depending on performance. Finally, the app keeps a record of all the rounds played and displays this when the user requests to end the play session.
So, it’s rather like the Irish National Lottery‘s digital play platform. I didn’t refer to this when designing my game, but my effort converges on the same concepts.
You may be already seeing some of the design issues I faced:
how to make sure the player chooses six unique numbers
how to have automatically chosen (“quick pick”) numbers
given that some lines are active and ready to play, some not, how to deal with the transitions from inactive to active and back
My own learning goals for the project were to get my head around Java’s Swing library, which provides platform-independent graphical user interface (GUI) widgets, as well as some intermediate programming concepts like interfaces, events, exceptions, and of course the object-oriented programming principles of encapsulation, inheritance and abstraction. (The fourth technique always mentioned with that group, polymorphism, didn’t really come into my app.)
Or, to put it crudely, I wanted to make a simple game but:
split it out intelligibly into classes representing real-world objects,
use the premade Swing classes more or less as they’re meant to be used, and
keep a handle on the structure and proliferation of my code.
What Went Right
A clean GUI: I designed the look of my program in one blast of inspiration, which luckily was then realisable in one of Swing’s layout manager classes, the euphoniously named GridBagLayout.
I decided that each line of 6 numbers would be built as 6 text fields, similar to a licence key or credit card number entry form. And I was able to have all necessary user actions available on this one, small window, in a layout based on the number 3.
Encapsulation: as you can see in the source code, I split the functionality of my game into 9 classes and and an interface. Many of these are satisfactorily neat and conceptually self-contained. Even the bigger ones use some basic principles to hide their data, i.e. private variables. Also, I followed the correct convention for passing arrays – copying their contents into a new array before passing that – to avoid giving the requesting class access to a private array in cases where this matters, e.g. the LottoTicket’s getResults() method.
Keeping control of my code: I was pleased that by the end of this project, I still knew clearly what everything did and where to look for any particular functionality. I could skim my code and know what I was looking at thanks to nice variable and method names, whitespace and my reasonably clear class structure. I’m an ultra verbal thinker so writing lines like “history.updateWith(ticket.getResults());” that do what they sound like they do in approximately readable English, pleases me no end.
Use of Java classes: the built-in classes I used – probably around 12 or so – were rewarding to study and build around. Of course they are, they’re made by top people! The experience of using things like Swing’s InputVerifier or Container classes, say, is of initial simplicity giving way to great depth and flexibility. The big but logical inheritance hierarchies of these components are inspiring. I’d love to make something someday as worked-out and usable as these. The only downside of my plug-and-play approach is that I ended up using a good few classes quite superficially. I can see how later it’d be all about digging down deeper and overriding parts of fundamental classes to put my desired custom behaviours in more unified containers that match their purpose more elegantly.
Use of events: I used a good few of Java’s event types and wrote one of my own, so now I have at least some insight into event-driven programming.
Bullet-proof UX and validation: I’m proud of this achievement even though conceptually, behind the scenes, it could’ve been cleaner. My interface instantaneously reacts to invalid input with an appropriate error message and an updated count of how many lines are ready to be played. The only exception is, I allow the user to leave fields blank for smoother navigation. It’s impossible to leave an unplayable input in a field, and impossible to play a line that doesn’t validate correctly (e.g. has blanks). There are two subtleties to this. First, I had to use multiple kinds of events to make sure and cover all interactions: DocumentEvents, FocusEvents, ActionEvents… and then pass the appropriate EventListener classes back and forth when building all the objects (lines, fields, verifiers) in my game. That’s overly complex for sure. On the plus side, I had to take care of weird player behaviour like going back and deleting numbers from previously finished lines. That involves some slightly tedious code but the result is robust.
Minimising hardcoding: I kept crucial gameplay variables in a separate LottoRules class and made sure that they only need to be changed once to correctly change the game’s behaviour. However, I chickened out of following this to its proper conclusion of allowing customisable line lengths and number of lines per ticket – the game logic can handle this, but the user interface wouldn’t dynamically adjust.
What Went Wrong
Having no target audience: the game was inspired by a brief for a college project. It doesn’t fulfill any real user’s needs and therefore there’s no point continuing to develop it. Honing in on excellent design becomes arbitrary in this situation. For my next project I’ll come up with something that I, and hopefully many people, definitely want.
Insufficient use of exceptions and defensive programming tactics: changes in one part of the code could easily cause crashes or undetermined behaviour because I don’t check the validity of values or handle wrong values (apart from the user input discussed above). That said, I think I’m right to move on without polishing this more.
Hardcoding: this point also contravenes good programming practice. The UI I made isn’t adaptable to different line lengths/amounts, and has some per-pixel hardcoding. But here I was coming up against the fact that system-independent GUI design is hard. Java’s Swing layout managers are pretty flexible and wide-ranging, but with that comes a lack of predictability and precision. I think next time I might try a different GUI technology – AWT, or something web-based – just to keep learning new stuff.
Lack of reusability: although I made one class explicitly for future use, (TextNiceties which deals with plurals and counting words) I mostly failed at designing for reuse which is, so my software engineering teacher says, the holy grail. Partly this is due to the trivial but finicky task: much of what I wrote is for the necessities of this particular game. Then there was the spreading of GUI code throughout my classes, which I’ll discuss below, but which obviously makes my code specific to one task.
Refactoring is hard: “restructuring existing computer code—changing the factoring—without changing its external behavior” as Wikipedia defines it, was challenging. I already had a working prototype at the start of this project, from my school group work. Unfortunately I tended to unthinkingly take elements of the previous design into my new one. And I also assumed that problems were already solved without noticing that with changed design assumptions, my previous techniques were now invalid. So, I made two failed attempts at restructuring my GUI code before settling on a pattern of passing a window (JFrame) object into my various classes so they could paint themselves onto it.
Deciding what object should have what responsibilities: I had the concept of an overarching “app” creating a lotto “ticket” containing “lines” which each contained number “fields”. Reasonable enough, but the question of which object should know about which other objects was hard! This applied to the UI creation, the validation, and the game state changes… I have tons to learn here.
Over-engineered interface: I allowed the user to invalidate a completed line of numbers by clearing text fields. This adds nothing, it’s merely a standard text input convention. I think an excellent design would intentionally limit such possibilities in the interests of clarity and simplicity. I note that the Irish National Lottery interface:
uses a graphical grid of numbers that are either selected by a single left click, eliminating duplicates automatically
doesn’t allow deselection (deletion) of numbers, so lines are simply always valid once completed – and bright colours are used to indicate the change to valid
therefore never has e.g. the first line invalid while subsequent ones are playable, an edge case I had to write a fair bit of overly-involved code to deal with
I’ll stop there. That’s a lot of chat about a learning project, but hey I’m proud of it and I sank a few days into it. Nice one if you read through it all and perhaps I’ll be back soon with a cooler project. A friend of mine gave me the idea of doing something that queries a web API, and I also think I want to do something with non-trivial calculations (maybe some geometry/graphics) and file handling.
*EDIT* Oh, one last addition – I think I know now what the solution is for organising the line, field and ticket objects and their drawing code. Sticking closer to the paradigm from John P. Russell’s excellent beginners Java book, I would go back to having each major element subclass a Swing component such as a JPanel, and draw itself in its constructor. And, I would structure the lines like a group of radio buttons: first a LottoLine would be created, and it would be passed into the constructor of all its LottoFields.
Ah well, there’s always more improvements to make. Still happy to draw a line under this project, cause it works and the code is readable.
A good friend sent me some clips of The Carbonaro Effect recently. It’s a hidden camera magic show with 5-minute segments, perfect for YouTube, of unsuspecting people in public places momentarily believing the seeming impossibilities manifested by creator Michael Carbonaro’s sleight-of-hand abilities (plus a lot of set and prop design).
Without intruding on anyone’s right to watch mindless entertainment as sheer relaxation, I want to explore what makes these little clips so compelling. Come and enjoy some magic with me!
In this trick, Carbonaro buys a Singapore Sling at a cocktail bar, and starts pulling out some of the tacky ornaments that normally go in such drinks, commenting on them and getting the attention of a young woman alone in the next chair up. Without in any one moment getting too utterly implausible, he removes a ludicrous quantity of objects, while keeping the girl at the edge of engagement and disbelief with his smooth, slightly dopey patter. She’s already murmuring “that is so cool” by the time he takes out an egg with a tribal mask pattern printed on it – just about plausible, for a second anyway, in a place called the “Tiki Bar”.
Getting the timing seemingly just right, he cracks the egg and a live green budgie flashes out, flaps around and settles on the rim of the glass (more of a vase, textured and opaque). The girl freaks out, and Carbonaro, still in his guileless persona, slips away to “wash [his] hands”.
In another trick worked in a cosmetics shop on a more sceptical woman who recoils at least twice but is drawn in by Carbonaro’s patter and intimate vocal tone, he apparently transmutes a chicken wing, buffalo sauce and blue cheese sauce into a lovely-smelling soap for men. Again, the timing and pacing are great, with a sealed plastic tub instead of an egg being opened for the climax.
little stunts. What makes them entertaining isn’t so much the
trick, though, as the reaction of the mark, how it’s attained, and
the highly emotive nonverbal story it tells. Carbonaro enlists a
massive cultural context in bedazzling these women.
He initially comes on like a man who probably wants something from them, a phone number or a sale, in a mainstream, commercial setting. He then focuses on objects that women might associate with a respite from the predations of men or commerce, that might be oases safe from the sleaze, belittlement and bleakness of club culture and consumerism. Cocktails and handmade cosmetics appeal to the young child in us who loves potions, secret ingredients and sensual pleasure.
inside these feminine-coded zones of release, innocence and magic,
using their childlike appeal to bypass emotional defenses and
scepticism… and then sinks home a dose of joy and wonder: what if
the little bit of magic allowed by mainstream culture, the small
luxury you turn to when life is tough, really was magic; and the
silver-tongued charmer really only intended you to bring you innocent
Seeing someone filled with childish joy is great TV, all the more so if she’s a good-looking woman like many of Carbonaro’s targets are, and even more so if she starts off plainly presented or emotionally guarded so that we get some ugly-duckling thrills. But what grabbed me emotionally and had me pondering these clips is something deeper. Archetypally, these two tricks tell a story of female magic transfiguring the male!
In the cocktail
trick, a bar-propping potential sleazebag (Carbonaro himself, in
character) is transformed into a beautiful exotic pet to be adored.
In the cosmetics one, the grossness and rankness of men, as we can
presume the woman has encountered it – their barbecues, TV
watching, beer guzzling, etc. (which are of course the masculine
culturally sanctioned zones of release and indulgence) – are
transformed into purity and heavenly scent.
At the deepest level, these tricks appeal to a female and perhaps feminist desire: that men would wash their hands of sin. The image of Carbonaro washing his hands caps both the tricks.
(This appropriates the century and a half of investment in images of washing as a moral, prestigious act – a domestic magic, in fact – that is detergent/soap advertising. Something that since Victorian times has intertwined race, gender roles and colonialism with our domestic lives).
It’s female power that accomplishes the miraculous transformations, symbolised by the yonic closed tub (which is heated – think of the phrase “bun in the oven” to make the link between furnaces and wombs) and the opaque cocktail glass, as well as the strictly feminine coding of cocktails, handmade cosmetics, ingredients and fancy things.
Female powerredeems the male. Hard
to think of a more loaded narrative than that!
so I’ve riffed pretty hard off these two little videos. To restate
what I think is happening:
Carbonaro gets deep inside our mainstream capitalist/retail/advertising culture which is hard on women but which a) gives them small zones of respite and pleasure and b) holds out the hope that, as desirable females, they might persuade their menfolk to be morally purer; Carbonaro then delivers an ephemeral, impossibly perfect realisation of these painfully felt desires for sanctuary and redemption, which more typically just sell e.g. washing powder.
Carbonaro’s skill is in how intimately he inserts himself into submerged, but emotionally charged parts of our culture. Different aspects of his persona subtly undercut each other: openly gay in real life, he can take on a stereotypical gay engagement with rituals of femininity or a stereotypical gay cosmopolitanism; as a prim white guy he can deliver science-y patter his trick needs – while remaining unthreateningly kind-of-dumb throughout.
(And then, to be sure, a lot of the time there isn’t any subtext for me to chin-strokingly analyse; most of his clips are just I-can’t-believe-he-fell-for-it gags or gross-out.)
It’s very smart; but I’m not claiming this is woke entertainment. Carbonaro appropriates and manipulates tropes and interactions from advertising and retail, but he’s 100% participating in capitalist distribution systems himself – these clips are ads for his TV show on TruTV, owned by WarnerMedia. They are formatted for the exploitative and opaque adtech ecosystem of YouTube.
More fundamentally, the fantasy Carbonaro sells us in these clips, of redeeming the everyday, is predicated on that everyday being the bleak, inequitable cultural mainstream of Western capitalism and authority systems.
The setup of the series does nothing to challenge that mainstream. It grants no agency to the participants, even though their reactions form so much of the entertainment value (and are invariably used for the video thumbnails). They’re not credited – I don’t know if they’re even well paid – and by the nature of the show they don’t consent beforehand. In our contemporary #content-driven culture, privacy, renumeration and control of one’s depiction are lost values.
At times, the racist and sexist society which forms the background for magical tranformation, reappears in the painfully deferential way some participants address the white Carbonaro: “You think I’m crazy, don’t you”, or “I don’t mean nothing by it”. Even with careful editing and selection of takes, the fantasy of upending privilege and inequality is fragile.
very idea of a magician
is retrograde and creepy. The
personality type that wins acceptance by painstakingly practised,
performance, and maintains a cloak of mystery around its methods, is
perhaps the most defensive and emotionally unavailable there is. I
should know, I’m a jazz musician.
So overall, I agree with the judgement of my friend who sent me these videos in the first place – they are indeed a trashy, addictive dopamine rush. But it seems to me it takes a lot of heart to make something so emotionally resonant. The videos certainly grabbed me enough that I had to sit down and pen this. Carbonaro made me ponder how I might make art or music that respects the heat of moral desire in people, the fervent secret wish for the world to be redeemed in a moment’s magic.
A satisfying practice session can involve many subtasks. I’ve been using the music production program, Reaper, to conveniently manage some of these. In this post I’ll go through my setup. It’s a work in progress. Eventually, I want to have a friendly and supportive digital environment for my creative mind, something to help sustain the musical work I’m doing and minimise clicking around on the computer.
My setup uses one free VST plugin, some drum samples I found for free, three plugins that came with Reaper, the webcam software that was bundled with my (Dell, Windows) laptop, and Reaper itself. An unlimited licence to Reaper costs €60 for personal or small business use, that’s the only thing I paid for. Here’s what it looks like in action:
It took me a while to figure out the arrangement of screen space, so I’ll go through it bit by bit. The aim was to minimise mouse clicks and maximise time with my hands on my bass. This setup is what I leave running as I play.
These are the basic track controls for the recording of my bass. Sometimes I use monitoring i.e. listening to the bass sound as it comes out of Reaper through my speakers, rather than my bass amp – but usually not. Using monitoring would allow use of effects, but there’s still perceptible latency (in the low two digits milliseconds) which I don’t like. I record everything and throw it out after. I keep my amp plugged into my soundcard all the time. I suspect this habit of recording everything may have led to some recent slight corruption errors on my hard drive, because recording involves constant drive access and I left it running for a few hours at a time more than once, by accident. So I put a recording time limit of 45 minutes in my default project options.
My teacher in Amsterdam years ago, David de Marez Oyens, recommended using the waveform of recorded bass as a visual aid to check one’s playing, but I only realised how powerful it is recently. Seeing the waveform instantly gives information on note length and attack, timing and perhaps most of all dynamics. The consistency of my playing has improved from routinely having the waveform on the screen.
The webcam image of my lovely self provides a check on my posture and particularly hand position (especially fretting hand wrist angle and finger curvature). As I’ve had health issues in the past from bad technique, this is a bit of a godsend.
Reaper has a handy tap tempo function so I can click here to change the project tempo (i.e. if I want a slightly different metronome tempo).
Assuming I pressed record at the start, this shows how long my practice session has lasted.
Transport controls to start and stop recording, say if I’m listening back to myself or whatever. Eh, my point is that I don’t allow any of the other windows to cover this up.
This is a cool little thing I discovered recently. You can “expose parameters”, or as I like to say “expose the knobs”, which means putting in a little dial in the track control which will control a parameter in one of the track’s FX plugins. In this case, this little dial controls what pattern my drum sequencer is on – here 0, which is an empty pattern and so plays nothing. But I can load up the sequencer with various patterns like a dance beat, claves, hip hop beat or whatever, and choose between them with this knob, without having to keep the sequencer window open.
Track controls for the drums and metronome, if I need to adjust levels or whatever.
I have lost probably about ten electronic tuners in my life. I just leave them behind routinely at gigs. So a digital solution is nice to have. Reaper’s standard “ReaTune” plugin works grand for bass once you turn up the window size to 100 milliseconds to allow for those big fat bass wavelengths.
For drums and click I use the bundled plugin “JS: MIDI Sequencer Megababy” which is a nice piece of software. It takes a bit of learning as it uses a lot of keyboard shortcuts and some of its design choices aren’t immediately evident, but it’s great and minimises the clicks needed to input a rhythm (because you don‘t have to put in a new MIDI item). The controls could be easily used to manage polymetrically related click tempos (“okay put the metronome once every two and half bars of 4”).
This purple horizontal bar is the click rhythm, in case I wanted to throw in a clave or something here. I could similarly display the current drum machine sequence, but it would take more screen space than this single bar, and also I don’t want my practising to be derailed by drum programming. For the same reason, I haven’t prioritised ease of adding or replacing drum samples – another rabbit hole.
To summarise, this setup lets me have the following functions available at all times as I play: Tuner Metronome Drum machine with preset beats Waveform visualisation Video of myself
The plugins I use are: JS: MIDI Sequencer Megababy (Cockos) ReaTune (Cockos) shortcircuit (Vember Audio) (a nice sampler)
Another function I haven’t tried yet would be putting in sound files to play along with (in full or looped). I used to use Audacity for this but it’d be easily done in Reaper.
The main downside from a user interface point of view is that each time after I change anything in Reaper or start recording, I have to click on the webcam software to open up that window again. Another thing is that changing tempo confuses things if done mid-recording and so necessitates a stop and a few clicks, although I could perhaps change some options to mitigate that.
Okay that’s it, I hope you enjoyed the tour. Feel free to comment about any software or configurations you use for practising!
Today’s post analyses a composition by Tim Follin from the soundtrack of a 1994 Sega Mega Drive game, Time Trax. (I found it on this sweet playlist.) I wanted to find out how it succeeds in being so improbably funky.
Chiptune music has been rising in cultural prominence with the predictability of any nostalgic trend. A mate of mine recently put me on to the quite expensively produced Diggin’ in the Carts, series, for example. I guess what’s fun about the music, beyond just hearing things from your childhood, is the musical meaning conveyed within harsh technical limits. Somehow, cheaply synthesised noises that don’t sound at all like brass, bass guitar, a string section, or whatever, can cheekily evoke just those things. So I want to examine that dialogue across the chasm of failed simulation, where the ludicrousness of the attempt at orchestral grandeur or, in this case, funk jamming, is part of the aesthetic.
You won’t remember this one from your childhood, because this game was never actually released and only a prototype of it emerged online in 2013. “The game is notable for its use of a relatively advanced sound driver designed by Dean Belfield for Follin,” segaretro.org tells us. I get the impression that this was a technical peak of sound design on the Megadrive.
Not to get too nerdy – let’s save that for later – but this style of synthesis is associated with, roughly, the 16-bit generation of consoles as opposed to earlier 8-bit. It is called frequency modulation synthesis and it tends towards a distinctive metallic, clanging, bell-like, brassy tone. (Which Tim Follin’s sound design actually disguises pretty effectively, at least until the heavy distorted riff sections.) You may also recognise the sound if you ever played MIDI files on a laptop with a cheap soundcard, like my Dell Latitude.
Let’s get to the music!
So, apart from the dinky sounds, we have a medium tempo funk-rock groove tune. The first thing I was curious about was the structure. As is typical for game soundtracks, this one is designed to loop interminably. However this isn’t really an issue either way as players were not likely to stay long on the ‘Mission Briefing’ screen where this track is played. (In this playthrough video the player spends 40 seconds.)
In any case, there’s a 90-bar structure lasting about 3 and a half minutes. The basic principle is one found in a lot of groove music – Wayne Shorter’s ‘Beauty and the Beast’ is my canonic example – which we could call “on or off”: you’re either in the groove or getting ready to get back in.
Follin uses key changes to shape his track. (He mentions his “fondness for random key changes” here.) As shown above, we get the excitement of going up a b3rd, then a gradual floating down to the home key – a good scheme for a track whose opening will be heard more than its ending.
The key change is smooth on a number of counts.
Okay, the lead line doesn’t voice lead, but in general its D major pentatonic melodic/modal colour is goes strongly to G minor (i.e it’s the elemental I major to IV minor alternation that has strong gravity in both directions). The (faint) bass voice ends on a D before a strong G bassline comes in, so that works. And the inner voices have a general upwards sliding of a semitone. Nice.
There are some other
nice applications of harmonic colour, suiting this rock/funk context.
In the intro we get some Cs, the b6 of the key, giving a suitably
earnest diatonic natural minor mood of cop show epic – after that
these are thrown out in favour of minor-7th/sus-chord funk colours.
There’s some strong use of the 9th, F#, in a couple of places e.g.
the brass pentatonic build.
(I recognise that my names for the instrumental sounds are arbitrary. I just don’t want to put quotations around “organ” and “elec. piano” for the whole blog post. Your interpretation of what the instruments are meant to be is just as valid.)
This ambiguity of instrumental sounds is crucial for what I consider the secret sauce to this track. I’m talking about the inner voices that comp all the various organ solos and can be most clearly heard in the breakdown at 3:00. Quietly, with a warm electric piano-like sound, they add some rhythmic action that interlocks nicely with the rest, and fills out the middle part of the sonic spectrum. At the end of bars 2 and 4, every time, they feature some bluesy parallelism of a type I discussed a long time ago on this blog. A bII I movement, and next time it’s bIII IV. How does the bII I fit so smoothly in a minor key? I think it’s really a bV IV – a classic blues side-slip – in the V key!
Here is where I’d normally show a little transcription (I did attempt one bar of it above – the dotted eighths in the second bar). However, these inner voices are incredibly hard to transcribe due to two peculiarities of the medium: overtones generated by FM synthesis; and the need to swap instrument sounds mid-flow to maximise channel use.
The first issue means that, although I think only two channels are used for these inner voices, we get a fleeting impression of triads in the passing chords – I believe I can hear the thirds. This is due to the way frequency modulation synthesis works] – it adds overtones called side-bands, in various proportions, to the original fundamental. As the 3rd and 5th (actually 3rd+2 octaves and 5th+1 octave) are part of the overtone series, frequency modulation can generate a kind of major chord. So, for instance, while the most audible line in the inner voices starts b7 b7 8, or D D E at 3:00, I think this is actually the 5th and that below there’s a b3 b3 4, G G A which is the fundamental.
What makes it trickier is the fact that Follin may not be keeping the sound (or “patch” in synthesiser terminology) consistent from note to note. In the intro, we note a changing modulation on the synth stabs. And if you look at the visualisation, you’ll see that a channel may switch from brass to bass instantly (e.g. when the groove kicks in at 1:05), or whatever. (Follins mentions this as a “basic trick” here.)
So, even though with a bit of hunting around on old-school, nerdywebsites I got some tools to extract MIDI data from the game files, I still can’t, after a decent effort, unambiguously notate it, because these lines might be transposed or changing timbre any time, and in any case they’re definitely using timbres with at least a strong 5th above the root.
What does all this mean musically? Just that it’s a full-sounding comping pattern with some sonic depth and mystery, and which, especially during those passing chords, is subtly but unmistakeably bluesy! Because blues uses that ambiguity between harmony and timbre all the time (so does electronic dance music, funk, jazz…), particularly for cliched parallel chord movements.
Let’s talk about the other sounds! Some commenters on Youtube have gotten into the nitty gritty of Follin’s techniques – in particular, his use of clipping/overdriving the signal to get otherwise impossible waveforms. I don’t know enough to comment there but I’ll just praise the sounds from a musical perspective and from what I can see in the visualisations.
Firstly, the very effective drum sounds are a single instrument/patch sounding at a high note for snare and a low note for kick, and a really high note for the hats. (Listen in the breakdown sections and you’ll hear the hat sound is kind of like a snare.) This is clear in the intro fill which sounds like it’s on the toms – but later those same notes function as a kick and snare in the main beat. While initially they sound more like toms than a kick or snare, in the mix they’re convincing. The beefy snare takes up some bass register quite effectively.
The distorted sounds later on, and the brass in the intro, are even less “realistic” but still sound good. I really like the bleep on beats 2 and 4 in the intro – here Follin uses a classic technique of “fake delay,” repeating the tone more quietly 3 16th notes later to give the impression of a classic tempo-synced delay effect. Then the bells/glockenspiel in the middle are a really nice timbral contrast. In fact timbral contrast is one of Follin’s main tools.
There are some cool sequencing tricks. The time feel changes from straight 16ths when it’s only hats, to swung 16ths when the groove kicks in. Also, there are some nice dynamic changes in instrumental sounds: the volume swell for the 2nd pads chord in the section starting 0:05, and the changing timbre of the synth stabs in the following section (accomplished by dialling in the degree of modulation of the carrier wave).
Of course, the centrepiece is the organ melodies. Although not very memorable as themes, they’re definitely funky, using tricks like staccato pedal tones, 32nd-note blues scale ornamentation, and (not idiomatic for organ, as I mentioned) pitchbends. As Follin says, “I also liked the playing styles used by folk musicians, all the twiddles and little arpeggios, which were again relatively easy to reproduce.” In general, these organ lines are built using rhythmic groups of 3 and either I minor pentatonic or V minor pentatonic shapes.
There’s one characteristic of the programming which is more to do with expediency – there’s a lot of reused material. The underlying drum pattern has no variations until it switches to a disco beat; the last minute is mostly just one riff in various orchestrations; and all of the organ bits use the same “answer” phrase in bars 3-4 and 7-8. As Follins recounts, these tracks were made by typing in notes in a text editor. I’d say this is why he copied and pasted a lot. It’s not a major problem functionally: the up-and-down the arc of dynamics keeps a meaningful directionality even though much of the groove stays unchanged for multiple sections. However, once you know about them, some of the 4-bar exact repeats (i.e. in the middle of the organ solo bits) become a little jarring.
This track was evidently made quickly, within the strictures of commercial production. Nonetheless it’s remarkably crafted, especially the sounds, which are not only skilfully programmed but gel together in a very fat “band sound.” And this was done without any mixing in the normal sense of applying EQ, compression, reverb. My personal yardstick is that I repeatedly found myself tapping my foot as I analysed it. No surprise that Follins states, “My own preference in my early teens (squashed by peer pressure) was for Quincy Jones.”
The actual game Time Trax, BTW, “is a straightforward platformer that sticks to the 16-bit platforming formula rather than innovate.” It’s clearly Follin’s composing work – which he says was something too nerdy and embarrassing to mention to friends and acquaintances at the time – that has kept it in the limelight. It’s nice to see that he’s only getting more recognition with the years.
I hope you enjoyed this jaunt into some different territory for the blog! If you have any insights into VGM or synthesis, feel free to comment!