Revisiting A Classic

I finished another programming project and I think it’s my strongest yet, thanks to me finally getting serious about testing! I called my app HuffmanRevisited because it implements the classic Huffman coding algorithm from 1951, and also because I had previously tried to program this algorithm a few months ago.

This time round, I coded it in Java, not JavaScript. It probably required about a solid week of work. And unlike my earlier attempt, I made a finished app. It can:

  • load a text file, encode (compress) it and save the encoded version, or else
  • load a previously compressed file, decode it and save the original content back as a text file

(You can skim my code here if you like.)

A few aspects of my approach helped make coding this an enjoyable task: I tried to specify it rigorously, I wrote a lot of tests, and I tackled a clear, well-known problem.

Before Christmas, I read a book – well, no, I skimmed a book after reading the prologue (which took multiple attempts) – called How To Design Programs. It’s an MIT textbook, you can read it here. I recommend it.

My paraphrase of the book’s prologue is: “precisely specifying what a program does is the hardest part of making it.”

Of course, I had encountered this sentiment in my Object Oriented Software Engineering module in the National College of Ireland. But the MIT textbook avoids the familiar paraphernelia of design patterns, verbose conventions and profuse diagrams. Instead, the book’s prologue challenges you to specify a problem by answering two questions: how is the data represented (in terms of data types that the machine can understand, and at all steps of the program)? and what are the functions (in terms of operations on those types)?

I mean, when I write it out, it’s dull and obvious. Yet, time and again I’ve found myself skimping on this specification step. Because, yes, it is the hardest part and one’s brain tries to escape it.

Even after my epiphany from reading the MIT book, I still evaded it. I specified most but not all of my Huffman encoding app before I started, thinking that the remainder was “simple”. But the simple part is never simple and if you haven’t thought it through, the solution will be incoherent even if it seems to work.

I failed to specify all of HuffmanRevisited, but at least I knew that this failure was to blame when I witnessed complexity mushrooming in front of my eyes as I tried to solve new, small problems that kept cropping up.

BTW, I’ll mention a couple of those little problems to see if you spot a pattern that kind of pleased me:

  • accessing individual bits from an array of bytes
  • packing an unpredictable amount of bits into an array of bytes
  • turning Java’s int and char types into a byte representation (not just casting which truncates them)
  • saving a compact representation of a binary tree containing characters in its leaf nodes (the ‘Huffman tree’)

Yeah… the pattern I spotted is that I’m doing low-level stuff and pushing somewhat against the limitations of Java. This is nice because I’ve been looking for an excuse to study a more low-level language!

The other thing that went really well with this project was testing. I’d written tests in JUnit before, but to be honest I was doing it to fulfil obligations in school assignments. Just like with specifying rigorously, I knew that tests are a great idea but was lazy about writing them.

I totally changed my tune once I had the framework up and running. (I used JUnit 5, Maven and NetBeans 11, and I mention this combination because I had no joy with JUnit 4 or with Ant.) I realised I’ve always done a lot of testing, but amateurishly: printing variables to the console all the time. That works okay… until your program starts to have a decent amount of methods on the call stack (i.e., functions that call functions that call functions that call functions…) and you spend your time trying to remember in which method did your gnomic text originate. Plus, all those print statements mess up your code.

Not to sound too much like a new convert, but using a test framework was a delight after all that. It’s much more organised. (And with my let’s say “wide-ranging” mind, I need all the organisation I can get!) It’s just like what I was doing before, except you only see debug text if there’s a problem, you get to choose any range of values you like to test, you can test as small or as large a section of the program as you like (as long as you’ve made the program itself reasonably articulated), and all of this business lives in separate files. Oh, and you get a cheerful screen of green when you pass your tests!

It’s enough to warm anyone’s heart

Okay, so specification and testing are non-negotiable aspects of real-world software development, but the last aspect I want to discuss can be more of a luxury: a clearly defined problem.

Until I get a start in a programming job, I can’t be sure, but my impression is that even communicating what a problem is, never mind completing a rigorous specification, can be hard in a typical business context.

However, I did this project for self-study so I got to choose exactly what to work on.

(I was helped in this by a comp sci book called The Turing Omnibus that a mentor very kindly lent me! It has a chapter on Huffman coding. The hook of the book, I would say, is that it conversationally introduces topics but doesn’t take you through every nuance. For example, unlike the Wikipedia article on Huffman coding, it mentions neither the need to pad bytes with zeros, nor any scheme for storing a b-tree.)

I was so glad I chose such a an old chestnut of an algorithm to implement! When I was refactoring my way out of that mushrooming complexity I mentioned earlier, the clarity of my app’s intention was a godsend.

Even better was the lack of edge cases. I could be certain my program had worked when it took a text file, compressed it into a smaller file, and then decompressed that encoded version into the exact same data I started with!

That’s intrinsically neater than some other areas I’ve boldly attempted, for example digital audio or vector graphics, where you need good control of sampling and rounding.

When I do go back to such complex topics, I’ll have a crucial bit of extra experience with the exact weapons needed to contain the ambiguity. Testing and full specification.

So, I’ll sign off there. My app could easily absorb some more effort. The next thing to work on would be efficiency. Do some profiling, and just comb through it for wastages. I can also think of some cool ways to present it, but no point hyping work I may not get around to.

Anyway, I’m pleased with it already. Jaysusin’ thing works, and the code structure is fairly sensible.

Thanks for reading. Take care in these hard times!

The header image “Massachusetts Institute of Technology Parking Lot, Memorial Drive, Viewed from Graduate Housing” by MIT-Libraries is licensed under CC BY-NC 2.0. I chose it because David A. Huffman might have seen such a view as he wrote the term paper in which he invented his compression technique.

Striving For 3D

This week I made some progress towards towards coding 3D rendering!

I remember when I was in my early teens and a bit bored on holidays at my grandparents, trying to code an image of a road with 1-point perspective. I asked my grandfather to show me how to load the version of BASIC he had on his ancient Amstrad PC (it was GW-BASIC).

Back then, I didn’t get the basic idea of 3D perspective, but it isn’t actually very difficult: if objects are in a space in front of you where X is across, Y up and Z forward (and you are at 0, 0, 0), dividing their X and Y coordinates by the Z coordinate will create the necessary distortion.

(It seems that, typically, a small number is added to the Z component first to reduce the strength of the distortion, otherwise things get madly stretched off screen when the Z approaches zero. I bumped into that problem when making my Pink Sparks demo the other week.)

The real issue is that, to keep the code organised, manipulations of 3D data are best done using matrices. This way, a command can become data. Instead of running some code for each manipulation (such as rotating or resizing a shape), you have one piece of code that obeys a data representation of the desired command. This data is a transformation matrix.

You can then conveniently store these commands, for example the operations “Rotate by 30 degrees around the X axis, mirror in a plane with the same orientation as the X-Z plane but 5 units above it, resize to 80% scale” could be represented as three 3×3 matrices.

If you use homogenous coordinates, which are like ordinary coordinates but containing an extra element by which all the others are divided, then the Z-divide for perspective correction can be represented by a matrix which copies the Z value into that extra, dividing element (typically called W).

But enough of my attempts at understanding linear algebra! Let’s talk implementation details.

As usual, I made my demos in JavaScript. Being able to trivially publish stuff on the web, and send no-hassle links to friends or relatives, makes this the most attractive choice.

However, I decided not to use WebGL, the graphics-card-accelerated renderer that now comes with all browsers. I’ve been having a good time with WebGL tutorials, but connecting up buffers and typed arrays introduces more places to make a mistake and lose time debugging. I’ll return to WebGL someday because the raw power, the basic idea of shader language, and the depth-buffer and texture manipulation capabilities all attract me deeply. But for now there was, again, a clear best option: HTML5 Canvas.

This is a graphics API with higher-level features such as line-drawing commands – precisely what I wanted for my demos!

The first one I made demonstrated linear transformations in two dimensions. As you can see if you click here, these all operate around the origin (the centre point where the two axes meet), or to put it another way they preserve the origin. I used setInterval(..) to make the animation – not a good choice as we’ll see in a sec.

Then, I made a demo of affine transformations – a larger category which includes the linear transformations, as well as translations (i.e. just moving shapes around with no distortion) and mixtures of linear transforms and translations. To show the way that affine transformations can occur around any point, I added some quick interactivity to let the user set the centre point and choose a transformation. I also used matrix multiplication to iteratively apply the same transformation to my shape.

Affine transformations -area-preserving squash, in this case, which incidentally describes hyperbolic curves.

Around here I started thinking of possibilities for game graphics:

  • a stick person who squashes a bit before and after jumping
  • a stick person who leans back before and at the end of a run (wind-up and breaking), done with a shear transformation
  • explosions setting off shockwaves that pass through numerous characters on a the screen, squashing them in its direction as it does so
  • interactive objects that cause the player character to change size, Alice In Wonderland-style

(I was definitely channeling some old Flash games from my teens… stickdeath.com, anyone? I think that was the URL.)

I’ll get back to these ideas in a second to discuss what I think would actually be the hard part about making them…

My final demo was in actual 3D. Still working off Greg Tavarre’s nice WebGL tutorials (though NOT following his convention for ordering matrix elements in a 1D array), I implemented homogenous coordinates and a Z-divide. My first attempt had an annoying error in the Y-axis. Turned out everything was working, I had just put my transformation matrices in the wrong order so the up and down bobbing was happening after perspective had been applied!

That’s what it looks like! Click here to see it hosted on my site.

If you look at the working demo, you may see the star seemingly spinning the wrong way, despite the perspective cues, a classic illusion. I think this is just a general fault of wireframe graphics.

BTW The animation here is handled with the preferred modern JS technique requestAnimationFrame(..).

All this demo-making begs the question: could anything here become reusable software?

The matrix stuff is eminently reusable. To make it convenient, I would need to make an engine or interface allowing a programmer to load geometrical data, transform it and display it, through well-documented, user-friendly functions, while hiding inner workings.

So the last thing I did this week was some design work on a personal 3D library. Eventually this should be in WebGL, but to test the design I might do it in Canvas and maybe just with wireframes. The crucial point is that geometry exists in all these different spaces before it’s fully processed:

  • object space, that is, vertexes positioned relative to the centre of the object they represent
  • world space, so now that object is positioned in a world
  • camera space, now the world is spun around to face the camera
  • screen space, now anything visible is referred to by the position it takes up on the screen (in this case, the rectangular Canvas on a webpage)

All of these have potential for interesting experimentation. What exactly defines an object is an open question – can an object be composed of others, and in what ways might those sub-objects be transformed? Once in camera space, what are the possibilities for fish-eye effects or non-Euclidean geometry? And of course screen space is the traditional domain of the visual artist, the flat sheet.

Well that’s some big talk on the back of a spinning star. Baby steps though!

7 Days

Last Monday I decided I would work on a different coding-related skill each day, for a week. These days I’m familiar enough with my own brain to know that swapping subject areas and deep-diving into topics suit my attentional style. Because motivation can be hard to come by when jobhunting and self-studying during pandemic restrictions, I thought I’d stimulate myself with novelty. It might even help me get psyched up to work on longer-term ambitions, like releasing my ritual Android app Candle Shrine, and also making a portfolio website.

Day 1 – C

On Monday I worked on C programming – an area I haven’t touched since 1999, ha! Yes as a kid I did a summer course which touched on some C. Anyhow, the aim was to set up the compiler and get console input and output. I used a build of Tiny C Compiler nabbed from the Tiny C Games bundle. My project is a simulation of life as experienced by our family cat, Goldie. Actually I’m pretty pleased with this, my sister played it through and enjoyed it. I was inspired by Robert Yang’s concept of “local level design“- a design aesthetic celebrating small-scale social meanings rather than top-down formalism. (This suited me because I don’t yet know enough C to write anything other than this ladder of if statements. Still, it works!)

Day 2 – Linux

The next mini-project was Linux shell commands. I used VirtualBox to dip my toes in -it lets you run any distro on emulated hardware, from a Windows desktop. It nearly locked up on my a couple of times, but in fairness my computer was running two OSes at once so I forgave it. It never fully crashed.

I’d hoped to get into shell scripting, which is the power user technique of saving listings of shell commands (a shell being a console for directly running programs or OS utilities, etc. – like the ol’ Command Prompt in Windows) as text files to be invoked as, effectively, little programs.

But all I had time for was to learn about 20 standard shell commands. However, I really liked this stuff. I can see why there’s a stereotype that devs use Linux. It’s rather satisfying to install stuff, edit files, and set up the file system via typed commands and not all that intimidating either.

Day 3 – Pink Sparks

See it in action here

This one was fun. I made a 3D particle demo – a spinning cube made of flying pink sparks. My focus here was to prove I could make a simple particle system, which indeed wasn’t hard, cribbing off Jonas Wagner’s Chaotic Particles and leveraging the extremely handy Canvas feature in HTML5. I also wanted to do 3D perspective, which was hard. However here I used the simplest possible version, a bare z-divide where x and y coordinates are divided by distance from the viewer. The proper way to do this involves matrices and transforms, but I’m not there yet.

If I get back to this the two things I’ll do are change it to defining line-shaped sources of sparks rather than point-shaped, and make some nicer data than a cube, like say a chunky letter ‘K’. This wouldn’t be particularly hard. Making a display of my initial fits with my interest in what I call ceremonial coding which I believe will be an emerging cultural field in years to come. As life goes online, we’re already finding the need to program and design software for celebrations and community rituals – an example being my graduation from my computer science course, which is being held on Zoom. I am certain that techniques from game design and aesthetics from digital culture will be important to create spiritual meanings and affirmations of identity on computers. My upcoming ritual app for Android phones expresses this conviction.

Again, Robert Yang’s post is very close to the spirit of this: “What if we made small levels or games as gifts, as tokens, as mementos?”

Day 4 – Huffman Coding

I hit a roadblock on Thursday. Huffman coding compresses text, by taking advantage of the fact that certain symbols (for example, ‘z’) occur far less often than others. To make a Java app implementing this was a meaty challenge, requiring binary buffer manipulations, a binary tree, sorting and file I/O. Still should have been achievable – but I let myself down by not rigorously figuring out the data representations at the start. This meant I threw away work, for example figuring out how to flatten the tree into an array and save that to disk, only to twig that the naive representation I’d used created a file far bigger than the original text file.

Though I was working from a textbook with an understandable description of the Huffman coding technique, that was nowhere near enough. I still needed to design my program and I failed to.

So I ran out of motivation as poor design decisions kept bubbling up. This was a stinger and a reminder that no project is too small to require the pencil-and-paper stage. On the plus side, I did implement a tree with saving and loading to disk, plus text analysis.

Hopefully I can reuse these if I come back to this. It’s a fun challenge, particularly the raw binary stuff and the tree flattening (although I don’t know yet if I want to store the tree in my compressed file, I think probably just the symbol table needed to restore the original.)

Day 5 – WebGL Black Triangle

In the spirit of Jay Barnson’s Black Triangles – though I’m sure it’s a million times easier these days!

Now this was pure fun. I used a tutorial to learn how to display graphics on a webpage using WebGL. I… frankly love the feel of OpenGL Shader Language. The idea of using a harshly constrained programming language to express some low-level color or geometry calculations, which is then compiled and run on your graphics card so you can feed it astronomical amounts of data for ultra-fast processing, is so satisfying. (Actually, especially the compilation process, and the narkiness of the parser where for example 1 is different to 1.0… it feels like you’ve loaded a weapon when you’ve successfully compiled at last.) I love graphical magic, but previously have been doing it at several removes, using wrappers on wrappers like Processing. I will definitely be doing more of this.

Day 6 – RESTful API

Another failure! I wanted to make a RESTful API demo as a bullet point on my CV, and host it for free on Heroku. But although I did some good revision on API design, when I got into implementation I totally got tangled up in trying to get libraries to work. Blehh!

I wanted my system to be standards-compliant so I tried using libraries that’d let me use JSON:API instead of raw JSON, which some people say is not good for APIs as it has no standard way to include hyperlinks which are, in fairness, central to the concept of REST (i.e. that instead of the client knowing to use certain hardcoded URLs, each response from the API includes fresh hyperlinks that the client can choose to follow).

But I got stuck when the examples for the library I chose wouldn’t compile because they required a new version of a build tool, Gradle, and despite trying some things off forums my IDE failed repeatedly to automatically install this.

They don’t call it “dependency heaven“!

If I get back to this I’ll use the build tooling I had working already for school projects. Life’s too short!

I wonder if the area of web services – so essential, stolid, bland – might be a natural home for rather pedantic personalities. The type who would make a typology of all things and publish it as a web standard. In any case, wading through some comments and blog posts and Wikipedia pages gave me a stronger understanding of state in web services than I had before.

Day 7 – Pathfinding in JS

Like the previous one, this project spun off into piles of research. But it’s all valuable stuff, I revised a lot of CSS and particularly the grid system, and got deep into JavaScript using this quite good book.

My plan was to implement a classic pathfinding algorithm, Dijkstra’s algorithm. But I wanted to have it so a little monster would chase your cursor around elements on a webpage! Well, as usual, I should’ve thought this through more. The fact is, web page elements are not intended to be processed as 2D shapes. HTML is semantic – web content is structured and manipulated as elements like paragraphs, headings, links that have meaningful relationships in the context of the document itself – with the final presentation of these elements done on the fly according to the user’s needs.

Anyway… my point is, I had to compromise to get this going. My original vision was of words arrayed around the page randomly, at different sizes, with a monster sprite threading his way around them.

My solution doesn’t have the words, just boring blocks, and though I think I could do words at a fixed size, having splashy words in different size could be quite a hassle.

As you can see, the paths go through diagonal choke points, something to fix

Nor did I get around to the monster although I have the sprite for when I do:

Heheheheheh.

But the thing that worked well for this mini-project was: web standards! In particular, I made the excellent decision not to hack CSS positioning from JS, but instead take the time to revise the CSS Grid system. Which, as you might imagine from the name, was actually perfect for this use case. Those numbered cells above are arranged by Grid.

Conclusion

That was fun. I might even do it again!

Golden Ratio Synthesis

I made a VST software instrument that uses the Golden Ratio to generate frequencies off a given note. You can download it here if you want to lash it into your music program and try it out. It’s unfinished though – more details below.

This didn’t come from any particular intention – it was a discovery I made while messing about in Synthedit many months ago. I was trying to make a normal additive synth where you mix the relative levels of each harmonic within a single note. But I found that if I took the frequency multipliers for the harmonics (which are normally just 1, 2, 3, 4 – so the second harmonic is twice the frequency of the first/root, the third is three times its frequency, fourth four times and so on) and raised them to a particular power around 0.6, a cool sound came out.

“Around 0.6” turned out to be 0.618034 – the “Conjugate Golden Ratio” (or one over the Golden Ratio).

Now it’s not possible to discover some “alternate harmonic series” because harmonics are a physical phenomenon: if you have a vibrating object with a fundamental frequency, whole-number multiples of that frequency can likely also form waves in it. So, each half of a guitar string vibrates one octave higher than the open string, and each third vibrates one fifth higher than that, and so on. Our sense of hearing subconsciously interprets the presence and tuning of harmonics as derived from physical properties: material and size and density. No other frequency series could have this same effect.

Nonetheless, the Golden Ratio seems more musical and harmonious than any other I could get by that exponentiating technique – actually it sounds like a jazzy chord. And it has what Gerhard Kubik calls “timbre-harmonic” aspects – like a Thelonious Monk chord, the harmony bleeds into the perceived timbre. My synth (on default settings) has a silky, bonky, dense timbre. (That territory between noise and harmony is where I like to be, musically. Check out the sounds I used for my drum programming experiment, for example.)

I could hear that it wasn’t in tune to equal tempered notes (nor to the non-equal tempered ratios found in the natural overtone series). But it was tuneful enough to sound concordant rather than discordant. If you download the synth and try the other ratios in the drop down menu you’ll hear the difference, I hope.

Here are the ratios: Golden Ratio conjugate on top, then normal harmonics, then the non-inverse Golden Ratio. You can see that the Golden Ratio conjugate results in a somewhat out of tune minor 11th chord – definitely jazzy! (The normal overtone series results in a dominant chord.)

Here are the ratios for exponents of: 1/Golden Ratio, 1, and the Golden Ratio

I whipped up some little riffs so you can hear the synth. It’s very digital-sounding, like additive synths generally are, and also reminiscent of the stacked, sometimes exotic overtones of FM synthesis at its icier end.

Note I didn’t sequence any chords in these – the “harmony” is from the voices of the synth. And there are no effects added.

I’ll evaluate the musical aspect at the end of this post. For now I want to discuss the synth-making software I used: Synthedit.

When I first started messing with production as a teen, the free synths I downloaded were mostly built in Synthedit. I soon got to know its characteristic signs – exuberant amateur graphics, slightly misplaced buttons and sliders due to the software’s drag-and-drop interface, and I guess a lack of quality. I remember one bass synth that was pitched way off A=440 – rank sloppiness. I used it anyway. The Flea, it was called.

Most freeware Synthedit VSTs were like that: knock-off bass synths or delay effects, easy and obvious stuff, frequently derided by snobs on forums.

Synthedit enabled a flood of low-quality, imitative software synths by lowering the barrier to entry. Instead of coding C++, you could (and can today) just drag and drop components, add in other people’s custom components, and instantly see/hear the result in your DAW interfacing with your MIDI gear and other FX.

I was blown away when I first did this a couple of days ago. I clicked export, set some easy options, and then couldn’t find the exported file. Irritated, I went back to REAPER, my production software – and there was my synth just sitting there! And it worked! And nothing crashed!

Having studied programming for the last year, I know how hard it is to make software like that. The default mode of enthusiast-made nerdy software is to fail aggressively until you figure out some subtle, annoying configuration stuff.

So, today’s post is a celebration of a great tool, very much like the one I did about Processing. Once again, I want to emphasise how great it is that people make such programming tools for beginners, where the hard and horrid configuration stuff is done for you.

This is priceless. It can change culture, like Synthedit changed bedroom production culture and marked my adolescence.

Amazingly, the program is developed by a single man called Jeff McClintock. He is active on the forum and from reading a few of his posts I get an impression of someone who takes any user’s difficulty as a sign to improve the program. I really admire that. And it shows in the robustness of the app (even the old free version I’m using).

To make a synth, you drag connections between “modules” that provide a tiny bit of functionality or logic. It’s like wiring up a modular synth. The downside is that, if you already know how to code, it’s a drag having to do repetitive fixes or changes that in a programming language could be handled with a single line. Also, when a module you want isn’t available, you are forced to make silly workarounds, download third party stuff or change your idea. In Java or Python you could just do it yourself.

All told, I enjoyed the experience of making Golden (so I have baptised my synth). The best part is having impressively reliable access to powerful, mainstream standards: MIDI and VST. That made it a lot more fun than my previous synth which took in melodies as comma separated values and outputted raw audio data. It was brilliant to have all the capabilities of my DAW – clock/tempo, MIDI sequencing, parameter automation – talking to my little baby.

The drag-and-drop interface builder is also great. Once again, amazingly, McClintock hides all the donkey work of making interfaces, the boilerplate code and updating and events. You just put the slider where you want it, then it works. The downsides are being locked into standard interface elements unless you want to go much more advanced. So, I wanted to have one envelope take the values from another at the flick of a switch, but I couldn’t. (I’m sure it can be done, but I couldn’t find it easily online. In general, the documentation for Synthedit is weak, and online tutorials scanty. I think that’s due to the narrow niche served – people nerdy enough to make synths, but not nerdy enough to code.)

Although I had a great time with Synthedit, I’d like to keep learning and do this work in a procedural or OOP language next time.

Let’s finish. Do I think this Golden Ratio thing has musical value? Yes, and I would like to use it soon in a hip hop beat or tracker music production. (It could also serve as root material for spectral composition, I strongly suspect.) Is my synth very good as is? No, the envelopes don’t work nicely for immediately consecutive notes (I should make it polyphonic to fix that) and I’m not happy with the use of….

Actually, I should quickly explain the synth’s features.

My beautiful interface, in resplendent “Default Blue”. I’m not even sure it’s possible to change skins without paying for the full version of Synthedit. Which is entirely fair – I got a lot out of this free version.

At the top are overall options: the choice of exponent, then various tuning knobs. “Exponent fine tuning” lets you alter the exponent, “Voice shift” is an interval cumulatively added to each voice, “Keyscaled flattening” is a hack-y tuning knob that applies more to higher notes. Use these to massage the microtonality into sitting better with other harmony/instruments.

Then there are two instances of the basic synth, as you can see, each with 8 voices you can mix. You can turn each one up or down with the little knob on its left end. You can also change its tone with the lowpass filter big knob.

The idea of the two synth engines in one was to be able to double voices at Golden Ratio intervals. Sorry if this only makes sense in my head, but I thought that these dank Golden Ratio sounds should be harmonised using their own kind of interval rather than standard fifths or thirds, so by selecting the interval in one synth instance’s drop-down box you can set it apart from the other by one of those intervals. Selecting “First overtone” with “Golden Ratio Conjugate” set in the Exponent menu will, therefore, displace the 8 voices of that synth instance upwards by a perfect fifth + 42 cents.

Finally, to create some simple motion within the sound, I use two ADSR envelopes for each engine and linearly interpolate between them. The bottom one directly affects the bottom voice, the top one the top voice (always voice 8 BTW – I wanted it to detect how many voices are in use but had to abandon it – one of those workarounds I was talking about) – and voices in between are blended between these two, unless you click the “Link Envelopes” switch in which case only the bottom envelope is used.

And each engine has an LFO which affects the exponent, and therefore has a greater effect on the higher voices.

… I can see why they say writing docs is hard! Hope you could withstand that raw brain dump.

As I was saying, this synth is rough, but hey I’ve seen rougher on KVR Audio so it’s a release.

I’ve been listening to SNES-era game soundtracks so I’m tempted to try make some dreamy, pretty melodies using Golden. I think it might also be good for some woozy house or hip hop.

If I was to develop the synth, the first thing to change would be the two envelopes idea – really it should be some more sophisticated morphing. I saw an additive synth where each voice had its own envelope but that’s too much clicking. Some intelligent system – interpolating but using a selection of curves rather than linear, or maybe something like setting percentages of each voice over time while overall amplitude is determined by a single envelope – would be nice.

It also badly needs some convenience stuff: overall volume and pitch, an octave select, polyphony.

I’m leaving Golden as a nice weekend project. I’ll come back when I have some chops in C++, I would think.

Well, thanks for reading if you made it this far. You get a “True Synth Nerd” badge! If you want to talk about the Golden Ratio or synths, get in touch 🙂 And don’t hesitate to try out the instrument.

Why Is Python Nice For Learners?

I’ve come back to Python a few times in the past year. Before September 2019 I’d never touched it (or any modern programming language, really). Right now I’m using it to build a 3D game level generator, and very much enjoying myself!

I feel I’m close enough to the start of the Python learning curve to give a personal perspective on why this language suits dabblers and students.

This is not a prospectus of Python’s features or design decisions – I don’t know enough to talk about those. Just a happy acknowledgement of what’s been making my life easier in the past couple of days.

I’ll compare it to Java, the language I know best, which is also probably the closest to a standard for Year 1 of comp sci courses.

  1. The look
    Instead of the curly brackets of C, C++, Java and other much-used languages, Python uses the amount of tabs at the start of a line to determine what “block” that line belongs to (and also the colon at the end of any line that declares a new block). There’s something reassuringly basic and solid about that, to a newbie. You don’t need to learn the little skill of counting brackets (nor do you need to use semicolons) while discounting whitespace. Instead, how it looks in your editor tells you what’s going on directly.
  2. Simple string manipulation
    Python’s elementary text manipulations are simpler than Java’s. When you’re starting out, I think that helps keep your mind on the actual task. For example, getting input from the console in Java generally requires creating a “Scanner” – a powerful bit of software for carving up and selectively serving text from an incoming stream. Those capabilities are irrelevant for basic text input. So students end up depending on something whose quirks and abilities they don’t understand. Including some unintuitive behaviour that I’ve seen even experts admit is confounding.
    Also on this point, Python requires you to convert all number types to strings explicitly, when printing them. Which can be annoying. However, this arguably makes the underlying reality clearer than performing the same conversions invisibly as Java does.
  3. It doesn’t force Object Oriented Programming
    We’re getting close to the topic of many boring flame wars on which paradigm is better, but let’s keep this to an inoffensive point: Python affords procedural, functional or OOP styles of programming. So if you have experience of either, you can start from there. And if you don’t, you’ll be saved the laborious and ritualistic aspects of standard Java style: private member variables, getters and setters, etc. – until you’re ready.
  4. Those data structures
    I first read about Python in Jay Barnson‘s delightful old piece, which I have returned to many times since, “How To Build A Game In A Week From Scratch With No Budget”. In it, he eulogises its list and dictionary data structures. And he’s damn right, they’re great. Instead of wrapping data in near-pointless objects – a practice so widespread in the Java world it has its own name, “anemic objects” – you’ll find yourself creating, stacking and passing around these built-in structures, as I’ve done pretty much throughout my level generator app.
  5. It’s mellowed by age
    This one cuts both ways. You can find articles on Medium arguing that Python is past its peak, and getting less fashionable or even relevant by the year. Yet, the advantages include having plenty of 3rd-party libraries (though TBH I haven’t dipped into these yet) and also a smoothing away of rough edges. For example, Python 3.x (available since 2008) has what’s called “new-style classes” that allow for conveniences such as property annotations, getting info on a class or method at runtime… I actually don’t understand that stuff yet, but the point is, whereas a few versions ago I would’ve had to use a special syntax to get “new-style classes”, now that happens automatically. One less thing to think about.
  6. You don’t have to split files
    Another really small thing that nonetheless will be felt by newbies: Python scripts can have multiple classes, or none, in one file (called a “module”). Whereas even a minor task in Java might push you to make a few classes, and so have to save a few different files.

The end result of all these things together is: even if you’re not very good at it yet, solving problems in Python feels fast. I don’t myself have the depth of understanding to explain this one. But ever since I first tried a maths problem or scripting text templates with it, I was pleased by that feeling of getting things done.

BTW, I wouldn’t call Python a “beginner language” in the same sense I applied to Processing a few posts back. Python doesn’t have every interaction carefully shepherded so as to hide complexity. Nor does it provide a simplifying framework for graphics or what-not. It’s a full-featured language (although with a favoured domain of data analytics, science, scripting, and stuff like that, for sure).

Last thing. For some reason, I pronounce Python as PIE-THON, rather than PIE-thn. Does anyone else out there do this? Let me know I’m not alone.

Inspired by “Permacomputing”

I read Viznut’s piece on “Permacomputing” last night and got all fired up. We do live in an age of unparalleled waste of computing resources. He is far from the first to discuss this – I recall Fabien Sanglard‘s and Derek Sivers’ polemics – but Viznut has the uncompromisingly ecological and long-term vision to contextualise and channel the typical hacker’s anger at wasted CPU cycles.

Viznut a.k.a. Ville-Matias Heikkilä.

That made me think of my recent graphics-heavy work, using large resolutions and high-level frameworks; and of the kind of work I see myself doing long term: “coding ceremonial space, presence, substance and light”.

My recent work….

Frankly, I felt indulgent. I think Viznut is right: our civilisation needs to make drastic adjustments to save itself from ecological collapse. And although those changes are gonna be harder than a few well-intentioned Westerners turning to gardening, common sense and intuition and aesthetics and the environmental statistics suggest that permaculture and related ideologies – emphasising resilience, adaptation, local conditions and lowered resource use – may hold more solutions than our current dependence on growth.

When I read Viznut’s weighty opinions I wanted to do some work in the vein of his competition-winning computationally minimal art. Work that uses no more computing power than needed for the goal at hand.

Viznut and his ilk are so much more learned than I am, as coders. They specialise in low-level hardware hacking, an esoteric and difficult topic. However, one skill of theirs is more generally applicable yet fits precisely in the permacomputing ideology. As Viznut puts it, “Optimization/refactoring is vitally important and should take place on all levels of abstraction.”

So I’ll get technical now and chat about how I optimised my most recent learning project: “Iridesce”, a raycaster!

(In case you don’t know what raycasting is, it’s an old and simple way of rendering a maze from a first-person perspective. Windows 95 used it in an iconic screensaver, and it featured in 90s games, most famously Wolfenstein 3D.)

My raycaster is written in JavaScript and so runs in a browser. Try it now if you like!

I used the profiling tool available in my browser, Slimjet (a clone of Google Chrome). This gives an indication of how much CPU time is spent on various aspects of running your site’s frontend code.

The pie chart at the bottom right shows the breakdown.

How did it go? Long story short, I was able to reduce 68% CPU usage to 20%! The things that worked were:

  • Lookup tables – so, instead of calculating the rainbow colour function 400 times per frame, I save all possible values of it into an array and access that instead. This was the biggest single saving and is a pretty classic technique, whenever you have more memory than CPU available. I was also already using a lookup table for the angles of the raycaster.
  • Using the HTML5 Canvas graphics API properly, in particular the ImageData objects which I was using to manipulate raw pixel data. Originally, from a vestigial memory of how raycasters are traditionally done, I was working with one-pixel-wide strips of data for every single segment of wall. This was entirely unnecessary and I saved a lot of CPU time by switching to a single large ImageData object covering the whole canvas. Incidentally, this necessitated some manual byte offset calculations, which felt pleasingly close to low-level.

And what didn’t have much effect:

  • Tidying up to remove nested if statements and repeated checkings of the same condition. This improved the legibility of the program but I don’t think it did much for performance.
  • Removing at least a dozen multiplications from loops. (Taking stuff out of inner loops is another classic optimisation approach.) I really thought this would make a difference, but I couldn’t see it in the stats.

So… the conclusion is pretty clear. Modern computers are stupidly fast at arithmetic. Manual low-level cleaning up doesn’t seem to change much. I’ll still do it for elegance. And I expect I’ll start seeing the performance benefits as I get better at implementing algorithms and doing my own profiling and benchmarking.

But more important than removing those multiplications and ifs, was checking which external functions are taking up time. Using the profiler I could spot that my colour calculations as well as the ImageData manipulations were actually taking up the most time. In both cases there was no need to call the function 400x a frame and I got massive improvements by fixing that.

So that was fun. Am I any closer to sustainable computing? A tiny bit.

Calculating byte offsets is getting into the realm of pointer arithmetic, a skill needed when programming simpler and older machines – increasing the range of systems I could do useful work on.

I learned about profiling and I did some quite major optimising, if mostly by fixing previous poor decisions.

And I got some clarity about what values are important to me, and a good dose of idealism. I’ll keep Viznut’s ideas of communally stewarded, resourceful not resource-intensive, locally appropriate, and aesthetic computing close to my heart as I decide how to direct my energies in projects and job-hunting.

There are other of Viznut’s ideas that I could elucidate through my own practice. How about developing the communal appreciation and understanding of technology in my own family home? Or working on feedback and visualisation of complex system state (I love interactive and live feedback from running systems and that’s how I tackle a lot of coding problems). Or the classic yet delicious challenges familiar from the 80s and 90s: making impressive graphics on slow processors! Perhaps even finding styles and tricks so that the imperfections of low-fidelity enhance the aesthetic affect.

Anyhow. Thanks for reading!

The Tiniest Example

Here’s a learning technique I just discovered.

I was working on video rendering on Android devices – reading through and mashing together libraries and example code I found. The task as it’s currently structured has 6 main classes interacting:

  • An extractor to present encoded frames of video from a file
  • A decoder to decode each frame
  • An animator to keep this happening regularly and check if it’s time to show a new frame
  • A view which is the interface element the video will be shown in, and which I’m using as a convenient repository for all the other action (this seems to be pretty standard approach)
  • A renderer which talks to OpenGL, the rendering API

I’ve been bodging everything together in one prototyping/sketchpad project. My approach was to keep it just-about-functional for my one test case (displaying three overlaid alpha-blended video elements) as I swapped in and out components and ported bits from Java to Kotlin.

I definitely learnt a lot from that, but today I tried a different tack.

Instead of accomplishing a fairly complete task messily and crudely, I took the single component I wanted to understand and placed it in a blank, fresh project. Then I tested it!

A boring screenshot

I got the class, TimeAnimator, to report all salient variables to GUI elements. Like debug text but organised so I can see the situation at a glance.

I realised I couldn’t make it do what I thought it had been doing, which was fire events at a steady framerate equal to the video framerate (of 30fps).

I shuffled through variants of the class and learnt what they do.

After a bit, I realised none of them did what I wanted. I went back to the library I was studying and finally twigged that the controlling framerate/timing information came from the decoded video data, which was merely being checked (at a frequency much faster than the framerate) by the TimeAnimator, which is not meant to run anywhere near as slow as 30fps.

So far, so trivial. But I might have kept playing with the original code for ages without actually challenging my assumptions, distracted by all its moving parts and the fact that it looked somewhat like the final product I wanted.

I will definitely be reusing this technique of isolating the smallest component that I don’t understand, and “prickin’ it and frickin’ it” as my favourite rapper Guru might say.

Hope to be back here soon to discuss what this video functionality is actually for!

Thoughts on Beginner Programming Languages

I coded up a prototype recently in a new realm: video compositing in real time. It was enjoyable for a few reasons, one of which was the language I used.

Processing has been around for a while. It’s designed to give instant access to graphics programming. It does that by exposing one-line functions that draw to a window without pretty much no configuration needed, by locking in a folder structure (e.g. images you want to work with go in a “data” subfolder), and tying everything together in a simple dev environment.

It’s ostensibly aimed at visual artists and students, and it fits in a lineage of what we could call “beginner languages” that intentionally hide complexity. Because I’ve now had a variety of encounters with such languages, including QBasic in my teens, a previous failed attempt to learn Processing, and a bit of a tool around with Racket earlier this year, I thought I’d try pin down some thoughts on them.

Let’s maybe take a wide perspective on my whole encounter, from the first thought of “Maybe I could use Processing?”

I DuckDuckGo-ed it, and a nice .org site popped up. Wow! First impressions were really good. The whole vibe of the community and ecosystem is cheerful and welcoming.

A nifty header style from the Processing website

There are pleasing non-jargon phrases all over the front page: “Creative Applications”, “Sketchpad”, “Visual arts”, “books available” – and yet it also feels like extensive technical work has been done, with links to implementations in multiple languages, and the obligatory Github profile, and many sponsors and partners.

I’m gonna go through this experience both from my current point of view, and an imagined one of a total beginner programmer. Both of these would give the Processing site, and related ones like Happy Coding, a hearty

The default installation format is a portable one. That might be a hassle to some. Or maybe it’s actually good. Unzip it and go, no permissions or shortcuts or whatevs.

The splash screen

The IDE loads showing a crisp splash screen with generative vector patterns like the main site. It’s perfectly decent, except for one thing which the docs make a lot of fuss about. Each code file is called a “sketch” – and the folder where these are kept is insistently referred to as “the sketchbook”. This caused me about ten minutes of trying to figure out if the sketchbook was a file format, and figuring out that it was automatically placed in a different location than where I’d unzipped the IDE. Calling it “sketches” would have made that clearer.

But for our absolute beginner, it’s not so bad: the IDE opens with a blank file, and you can copy in some example code and click the run button, and it works straight away. That’s REALLY nice.

This is what I really want to explore today. The trade-offs of simplification.

I think there’s an interesting conflict, in that simplification can make the first steps easier – which again is super valuable – but the next steps harder.

Both times I tried out Processing, I had this sinking feeling of near-betrayal as the light fun of the early steps sank into a morass of what feels like second-guessing the developers.

And I had it years back with the natural-language interactive fiction system, Inform 7, which I gave up on.

Making first steps easier is an illusion – one gets the feeling of accomplishment without really understanding what’s going on.

That can be all right. Computers hide inner workings from us all the time. The issue is finding the right balance of illusion so that the first peek behind the curtain doesn’t shatter fragile confidence.

I explore systems by trying odd things. For example, when I was checking out the Standard Vector Format for geometrical web graphics modifiable with JavaScript, I gravitated to abusing the “dashed line” capability to make weird art. (Overlaying super wide dashed lines to form swimming shapes and random pixel patterns.)

This tendency of mine clashes with the carefully-laid illusions of beginner languages. It took me say twenty minutes and half a screen of code to display one layer of a candle flame and show it, chuffed, to my dad. Implementing my actual goal of compositing a few layers with filtering and interactive controls, took a whole day, much of which was spent in frustration as the seemingly generous affordances of the language turned to cobwebs.

Here were some things that tripped me up after my first joyful success:

  • drawing off-screen required trawling the none-too-voluminous developer docs to figure out the difference between a PImage and a PGraphics object -I’m guessing working on an off-screen a buffer wasn’t considered a fundamental use case for people coming from a visual arts background
  • a Movie object acts like an Image, but its width and height fields are set at 0 until the first call to its read() method (making some initialisation stuff harder)
  • some graphics functions are overloaded so that they can take either the red green blue components of the desired colour or a single component as a greyscale value – but not consistently
  • drawing to an off-screen buffer has to take place inside the single set draw() method, otherwise it randomly will draw to the main window in flickers

And there was more, but this is boring. Once I figured them out, it was okay. Even the rather arcane pipeline I ended up with – a Movie’s read() setting a flag that an update is required, which is checked in draw() which if necessary calls loadPixels(), updatePixels(), beginDraw() and endDraw() on the various buffers, and the actual draw calls and filter calls, in the correct order – is more or less understandable. (I didn’t say optimal!)

It’s just that… there’s a let-down from the initial hey presto! to what feels like debugging a somewhat complex system you don’t understand. Now there’s nothing wrong with that process. That’s how to learn. But could the journey from delight to disappointment be mitigated?

It was that feeling of having waded in a bit beyond my depth and being swept away by uncontrollable factors, that turned me off Processing and Inform 7 before.

However… I don’t think Processing could be much better. The only ways to smooth my experience would be tiny tweaks: if someone rewrote a tutorial, tweaked a line in an example, and so on. I see why people talk of community as a central aspect of high-quality software.

My conclusion is that despite the inevitability of harsh contact with system complexity, beginner languages are great – but don’t start one, join an existing one.

Community building is also behind those good vibes that are maybe even more important than the technicalities, when welcoming interested newbies.

Oh, and yeah, what did I actually code? Glad you asked. Taking some tips from an awesome visual effects tutorial by an enthusiastic fellow called Lendon Bracewell, I made some candle flame effects for a votive shrine app I’m planning. The idea is for this to become a fast prototyping/design tool when I’m assembling my app’s candle animations and effects. Check it.

Till next time.

Side Project Satisfaction

Today I finished up a coding project, which was to make a simple synthesiser written in Java. I noted though, as I dashed off a quick readme file and uploaded the repository for the last time, a distinct sense of anticlimax, even disappointment. The app didn’t work out that well. Instead of writing an account of what I was trying to make and how I set about it, then, this evening I want to reflect on ways to avoid that letdown, specifically with side projects where you have total creative freedom.

This is about accepting and optimising for the way my brain works. Which is: it creates lots and lots of ideas at the start of a creative process. Here are some of the ideas I had for my synth:

  • adding harmonics by wave-folding
  • only using fixed-point arithmetic (inspired by early trackers and my general interest in retrocomputing)
  • using fast sine approximations
  • using sine lookup tables
  • stretched harmonics
  • harmonics detuned to simulate string mass
  • using Bezier curves as volume envelopes
  • doing additive synthesis by mixing together multiple instances of my basic instrument
  • doing additive synthesis by creating custom waveforms containing the desired harmonics
  • generating just intonated and 5-limit tunings
  • generating tunings from two overlapped harmonic series
  • generating 12-tone equal temperament and quarter-tone (24-tone) tunings mathematically
  • using the stuttering/echoing sound of buffer underruns as a musical effect
  • adding frequency modulation synthesis
  • having the synth be about a limited subset of sounds like bells, marimbas and bass
  • avoiding certain assumptions from the MIDI standard, e.g. removing note duration in favour of focusing on note onsets only (an idea influenced by my study of African-derived musics which have this emphasis)
  • using the synth in some kind of persistent, low-key online game as a way for players to leave melodies for each other to find
  • using rules to generate different melodies from the same basic info (say an array of integers), for example with different tunings or in different modes and scales
  • generating scales, chords and perhaps melodic cells or fragments geometrically, using relations I know from my study of music theory back in the day (e.g. a chord is every odd member of a scale is an approximation to 7 stacked fifths in a chromatic space generated using the twelfth root of two)
  • making an interface to expose these options interactively

And so on. The obvious common factor among all of these, I would say, is that there is no common factor. They are an extremely heterogenous bunch suggesting a whole lot of varied perspectives. Some are from a coder’s perspective (fixed-point calculation), some are about synthesising technique, some reflect my own limitations of ability, some are more like observations about the nature of music. Many have a distinctly contrarian flavour.

You can probably see the problem. There is no way to ever succeed at a project “defined by” such a list. And I’m not talking about success defined externally, but even just personal satisfaction. There are far too many requirements, many of which are contradictory.

The ideas there which are more rooted and achievable have another issue: they are technical challenges only, which makes them arbitrary. There’s no way to know if the solution arrived at is good, because there’s no agreed-upon way to measure performance. Should my bezier curve envelope allow two values for a given x input (which an unconstrained quadratic bezier can easily have), or forbid such curves? There’s no right answer because I haven’t defined what I’m trying to do.

This is in stark contrast to the school projects I’ve done for my higher diploma course: making a banking app, or an e-commerce website. Even if the work could get boring with those, the remit is clear and it’s satisfying when such a system comes together.

How did I manage to put a few weeks into my synth without realising that I hadn’t fixed upon a goal?

Or do I even need a goal? There’s nothing wrong with just fiddling about with stuff for fun, there’s even a fancy phrase to make it sound more official: “stochastic tinkering“. However, I know that I get my fun as a programmer in quite a specific way – by turning stuff that doesn’t work into stuff that does. When definitions are too loose, there’s no way to decide whether something works.

I’ve come up with a few pointers on how I might avoid this looseness in future.

The first is to design something conventional with a fictional, “normal” user in mind. This was how my school projects got done. This is good because convention (shopping carts in an e-commerce site, account selection in a banking site) guides you. This leverages the part of my mind that can’t stand to be wrong, and that likes tidying up: as long as the project fails to meet conventional expectations, I’ll be nettled into improving it.

However, finding the motivation to develop without the ego boost of originality would be hard for me. I know from the experience of finishing up my synth that work done just for the sake of appearing competent to strangers who come across my Github profile, isn’t very sustaining. The school projects had the virtue of being compulsory.

The second solution is to design something I’d like to use. This is… hard actually. It requires some self-awareness and honesty. I made a synth because I thought it’d be cool… or so I thought. Yet if I truly believed synths are cool, I’d probably have used one in the last few months; I haven’t done any synth-based music-making though in that time (despite having dozens of software synths installed on my computer). My conclusion is that I find synths to be a pleasantly intricate subject for mental distraction, but that I don’t actually have much desire to use them.

And similarly with the pixel art app I made before the synth. I like thinking about pixel patterns and generating them, yet if I liked making pixel art I’d be making some.

So, thinking honestly about one’s interests and requirements isn’t all that easy.

A third approach is to make something new, but very small. This worked well with something I made last year, an interactive sine wave visualiser. I actually made use of it, just for a second, while working on my synth to help me think about differentiating sines.

I’ve read advice to programmers about making tools that do one thing very well, and I can see the sense of it.

A fourth thing that has worked well for me is collaborating. When I’m working closely with others, my desire to appear right is a strong motivator. The hard part though for a side project is putting the energy into finding collaborators and the contradictory twin fears of not being good enough versus working with someone I feel is holding me back.

Those are actually familiar negative thoughts from my musician days.

Well, that’s what I wanted to write. Conclusion: even though my brain likes nothing better than lashing out idea after idea, finding the right one takes courage and deliberation. And it seems likely that good project ideas will combine a couple of the following: doing one thing only; being conventional; solving a real problem I have; being collaborations.

Harping On

I made an online toy in JavaScript, called TextHarp. Try it out (it needs a computer rather than a phone because it uses mouse movements).

The idea popped into my head a few weeks ago. It won’t leave the prototype stage because this combination of technologies – pure HTML, CSS and JS (although do I use one library to synthesise sounds) doesn’t robustly support what I wanted.

I aimed to turn a piece of text into an instrument, where moving the cursor over any letter which corresponds to a musical note – so, ‘c’, ‘d’, ‘e’, ‘f’, ‘g’, ‘a’, ‘b’ – would pluck the letter like the string of a harp, and play that note as a sound!

At the time, I was thinking of a few possibilities

  • adding audio feedback for testing web pages, so that a developer/designer could hear if an element was malformed or missing information (aspects which are often invisible to the eye)
  • sonification, a concept which I think is rapidly going out of date as people realise its enormous limitations, but which was about turning reams of data into continuous sound patterns that would somehow reveal something within the data, but which I think were usually just third-rate electronic music or else showed no more than a good graph could, and basically made clear that the humanities PhD system sucks in people who’d be better off elsewhere… sorry I seem to have gotten into a rant here
  • simple enrichment and adding magic to the experience of visiting a webpage

That last is out of favour for web design nowadays. Instead, minimalism, accessibility and function are the buzz words. Fair enough… but also ominously envisaging the web as merely where stressed and harried folk get updates from a corporate or government source, staring down at their little phone screen.

Well. My little toy isn’t going to do anything to overturn that paradigm. Still, let’s take a short tour of the challenges in making it work.

I used basic JavaScript mouse events to change elements in the webpage, modifying what’s called the Document Object Model; which is nothing more than how your browser perceives a page: as a hierarchy of bits containing other bits, any of which can be scripted to do stuff.

My script caused each paragraph to detect when the mouse was over it. Then it cut the paragraph into one-character chunks and placed each of these single letters into a <span></span> HTML tag, so that it became its own card-carrying member in the Document Object Model.

Not very elegant at all! Also, despite span tags being supposedly invisible, throwing in so many of them causes the paragraphs to twitch a little, expanding by a couple of pixels, which wouldn’t be good enough for a production page.

However, it works. I set each of the single-letter chunks to play a synthesized tone when the mouse goes over them, and that’s it. Also, when the mouse leaves that paragraph’s zone, I stitch the letters back together, the way it was.

The downsides are that any HTML tags used to format or structure the text tend to get damaged by the process. Usually resulting in piles of gibberish, or text disappearing cumulatively. It would be possible to improve that, but with a lot of manual work. And, the browser’s attempts to be clever by healing broken tags here actually cause a lot of difficulties.

Defining some new kind of object that held the text and knew the location of each letter, would be a better bet.

However, I’m turned off this avenue of enquiry for the moment, because dealing with audio in browsers is a pain. Not for the first time, musical and sensual uses of technology have been left in the gutter while visuals get all the investment.

There are two big problems with web audio. First, JavaScript doesn’t offer precise timing. I see precision, whether in first person computer games, input devices, or in this case reactivity of audio, as inherently empowering – inviting us as humans to raise our game and get skilled at something. Sadly, much of our most vaunted current technology crushes this human drive to excel and be stylish, with delays and imprecision: touchscreens, cloud services, bloated webpages…

Where was I? Yes, the second problem is that Google Chrome made it standard that web sites can’t play sound upon loading up, but only after the user interacts with them. Well meaning, but really shit for expressivity – and quite annoying to work around. My skillz are the main limitation of course, but even trying out two libraries meant to fix the issue, I couldn’t make my audio predictably avoid error messages or start up smoothly.

No tech company would forbid web pages from showing images until the user okays it. But sound is the second class citizen.

When I know my JS better, I’ll hopefully find a solution. But the sloppy timing issue is discouraging. Some demos of the library I used show that you can do some decent stuff, although the one I experimented with took a good idea – depict rhythm as a cycle – and managed to fluff it with two related interface gripes. They made the ‘swing’ setting adjustable for each bar of a multi-bar pattern – pointless and unmusical. And they made the sequencer switch from bar to bar along with the sound being played – theoretically simple and intuitive, but – especially with the above-mentioned time imprecision of web interfaces – actually resulting in loads of hits being clicked into the wrong bar. (And if I say a drum machine is hard to use, it probably is – I’ve spent so much time fooling around with software drum machines I ought to put it at the top of my CV.)

But what am I saying! That demo’s way more polished than mine.

Perhaps even a little too polished! Visually anyway. All of the examples on that site are rather slick and clean-looking, perhaps because, I believe, the whole library has some affiliation with Google.

Ah, I’m being a prick now and a jealous one too, but one scroll down that demos page would make any human sick. The clean grids. The bright colours. The intuitive visualisations – yes, technology now means that you too can learn music, it’s just a bit of fun! Practice, time feel, listening, gear, human presence – nah!! And then the serious, authoritative projects – generated-looking greyscale, science-y formal patterns and data…. bleh.

My next JavaScript project is an exploration of a visual style which I explicitly intend to raise a middle finger to those kind of polished, bland graphics. I’ll be taking lessons from my 90s/00s gaming past to experiment with pixel art but without the hammed-up console-nostalgia cutesiness.

And I’ll be using standard web technologies – JS, SVG – to make anything I come up with 100% reusable by non-programmers.

Thanks for reading!