Striving For 3D

This week I made some progress towards towards coding 3D rendering!

I remember when I was in my early teens and a bit bored on holidays at my grandparents, trying to code an image of a road with 1-point perspective. I asked my grandfather to show me how to load the version of BASIC he had on his ancient Amstrad PC (it was GW-BASIC).

Back then, I didn’t get the basic idea of 3D perspective, but it isn’t actually very difficult: if objects are in a space in front of you where X is across, Y up and Z forward (and you are at 0, 0, 0), dividing their X and Y coordinates by the Z coordinate will create the necessary distortion.

(It seems that, typically, a small number is added to the Z component first to reduce the strength of the distortion, otherwise things get madly stretched off screen when the Z approaches zero. I bumped into that problem when making my Pink Sparks demo the other week.)

The real issue is that, to keep the code organised, manipulations of 3D data are best done using matrices. This way, a command can become data. Instead of running some code for each manipulation (such as rotating or resizing a shape), you have one piece of code that obeys a data representation of the desired command. This data is a transformation matrix.

You can then conveniently store these commands, for example the operations “Rotate by 30 degrees around the X axis, mirror in a plane with the same orientation as the X-Z plane but 5 units above it, resize to 80% scale” could be represented as three 3×3 matrices.

If you use homogenous coordinates, which are like ordinary coordinates but containing an extra element by which all the others are divided, then the Z-divide for perspective correction can be represented by a matrix which copies the Z value into that extra, dividing element (typically called W).

But enough of my attempts at understanding linear algebra! Let’s talk implementation details.

As usual, I made my demos in JavaScript. Being able to trivially publish stuff on the web, and send no-hassle links to friends or relatives, makes this the most attractive choice.

However, I decided not to use WebGL, the graphics-card-accelerated renderer that now comes with all browsers. I’ve been having a good time with WebGL tutorials, but connecting up buffers and typed arrays introduces more places to make a mistake and lose time debugging. I’ll return to WebGL someday because the raw power, the basic idea of shader language, and the depth-buffer and texture manipulation capabilities all attract me deeply. But for now there was, again, a clear best option: HTML5 Canvas.

This is a graphics API with higher-level features such as line-drawing commands – precisely what I wanted for my demos!

The first one I made demonstrated linear transformations in two dimensions. As you can see if you click here, these all operate around the origin (the centre point where the two axes meet), or to put it another way they preserve the origin. I used setInterval(..) to make the animation – not a good choice as we’ll see in a sec.

Then, I made a demo of affine transformations – a larger category which includes the linear transformations, as well as translations (i.e. just moving shapes around with no distortion) and mixtures of linear transforms and translations. To show the way that affine transformations can occur around any point, I added some quick interactivity to let the user set the centre point and choose a transformation. I also used matrix multiplication to iteratively apply the same transformation to my shape.

Affine transformations -area-preserving squash, in this case, which incidentally describes hyperbolic curves.

Around here I started thinking of possibilities for game graphics:

  • a stick person who squashes a bit before and after jumping
  • a stick person who leans back before and at the end of a run (wind-up and breaking), done with a shear transformation
  • explosions setting off shockwaves that pass through numerous characters on a the screen, squashing them in its direction as it does so
  • interactive objects that cause the player character to change size, Alice In Wonderland-style

(I was definitely channeling some old Flash games from my teens… stickdeath.com, anyone? I think that was the URL.)

I’ll get back to these ideas in a second to discuss what I think would actually be the hard part about making them…

My final demo was in actual 3D. Still working off Greg Tavarre’s nice WebGL tutorials (though NOT following his convention for ordering matrix elements in a 1D array), I implemented homogenous coordinates and a Z-divide. My first attempt had an annoying error in the Y-axis. Turned out everything was working, I had just put my transformation matrices in the wrong order so the up and down bobbing was happening after perspective had been applied!

That’s what it looks like! Click here to see it hosted on my site.

If you look at the working demo, you may see the star seemingly spinning the wrong way, despite the perspective cues, a classic illusion. I think this is just a general fault of wireframe graphics.

BTW The animation here is handled with the preferred modern JS technique requestAnimationFrame(..).

All this demo-making begs the question: could anything here become reusable software?

The matrix stuff is eminently reusable. To make it convenient, I would need to make an engine or interface allowing a programmer to load geometrical data, transform it and display it, through well-documented, user-friendly functions, while hiding inner workings.

So the last thing I did this week was some design work on a personal 3D library. Eventually this should be in WebGL, but to test the design I might do it in Canvas and maybe just with wireframes. The crucial point is that geometry exists in all these different spaces before it’s fully processed:

  • object space, that is, vertexes positioned relative to the centre of the object they represent
  • world space, so now that object is positioned in a world
  • camera space, now the world is spun around to face the camera
  • screen space, now anything visible is referred to by the position it takes up on the screen (in this case, the rectangular Canvas on a webpage)

All of these have potential for interesting experimentation. What exactly defines an object is an open question – can an object be composed of others, and in what ways might those sub-objects be transformed? Once in camera space, what are the possibilities for fish-eye effects or non-Euclidean geometry? And of course screen space is the traditional domain of the visual artist, the flat sheet.

Well that’s some big talk on the back of a spinning star. Baby steps though!

Magic and Desire

A good friend sent me some clips of The Carbonaro Effect recently. It’s a hidden camera magic show with 5-minute segments, perfect for YouTube, of unsuspecting people in public places momentarily believing the seeming impossibilities manifested by creator Michael Carbonaro’s sleight-of-hand abilities (plus a lot of set and prop design).

Without intruding on anyone’s right to watch mindless entertainment as sheer relaxation, I want to explore what makes these little clips so compelling. Come and enjoy some magic with me!

In this trick, Carbonaro buys a Singapore Sling at a cocktail bar, and starts pulling out some of the tacky ornaments that normally go in such drinks, commenting on them and getting the attention of a young woman alone in the next chair up. Without in any one moment getting too utterly implausible, he removes a ludicrous quantity of objects, while keeping the girl at the edge of engagement and disbelief with his smooth, slightly dopey patter. She’s already murmuring “that is so cool” by the time he takes out an egg with a tribal mask pattern printed on it – just about plausible, for a second anyway, in a place called the “Tiki Bar”.

Getting the timing seemingly just right, he cracks the egg and a live green budgie flashes out, flaps around and settles on the rim of the glass (more of a vase, textured and opaque). The girl freaks out, and Carbonaro, still in his guileless persona, slips away to “wash [his] hands”.

In another trick worked in a cosmetics shop on a more sceptical woman who recoils at least twice but is drawn in by Carbonaro’s patter and intimate vocal tone, he apparently transmutes a chicken wing, buffalo sauce and blue cheese sauce into a lovely-smelling soap for men. Again, the timing and pacing are great, with a sealed plastic tub instead of an egg being opened for the climax.

They’re great little stunts. What makes them entertaining isn’t so much the trick, though, as the reaction of the mark, how it’s attained, and the highly emotive nonverbal story it tells. Carbonaro enlists a massive cultural context in bedazzling these women.

He initially comes on like a man who probably wants something from them, a phone number or a sale, in a mainstream, commercial setting. He then focuses on objects that women might associate with a respite from the predations of men or commerce, that might be oases safe from the sleaze, belittlement and bleakness of club culture and consumerism. Cocktails and handmade cosmetics appeal to the young child in us who loves potions, secret ingredients and sensual pleasure.

Appealing to the young child in us…

Carbonaro slips inside these feminine-coded zones of release, innocence and magic, using their childlike appeal to bypass emotional defenses and scepticism… and then sinks home a dose of joy and wonder: what if the little bit of magic allowed by mainstream culture, the small luxury you turn to when life is tough, really was magic; and the silver-tongued charmer really only intended you to bring you innocent delight?

Seeing someone filled with childish joy is great TV, all the more so if she’s a good-looking woman like many of Carbonaro’s targets are, and even more so if she starts off plainly presented or emotionally guarded so that we get some ugly-duckling thrills. But what grabbed me emotionally and had me pondering these clips is something deeper. Archetypally, these two tricks tell a story of female magic transfiguring the male!

In the cocktail trick, a bar-propping potential sleazebag (Carbonaro himself, in character) is transformed into a beautiful exotic pet to be adored. In the cosmetics one, the grossness and rankness of men, as we can presume the woman has encountered it – their barbecues, TV watching, beer guzzling, etc. (which are of course the masculine culturally sanctioned zones of release and indulgence) – are transformed into purity and heavenly scent.

At the deepest level, these tricks appeal to a female and perhaps feminist desire: that men would wash their hands of sin. The image of Carbonaro washing his hands caps both the tricks.

Washing his hands of sin…

(This appropriates the century and a half of investment in images of washing as a moral, prestigious act – a domestic magic, in fact – that is detergent/soap advertising. Something that since Victorian times has intertwined race, gender roles and colonialism with our domestic lives).

It’s female power that accomplishes the miraculous transformations, symbolised by the yonic closed tub (which is heated – think of the phrase “bun in the oven” to make the link between furnaces and wombs) and the opaque cocktail glass, as well as the strictly feminine coding of cocktails, handmade cosmetics, ingredients and fancy things.

Female power redeems the male. Hard to think of a more loaded narrative than that!

Okay, so I’ve riffed pretty hard off these two little videos. To restate what I think is happening:

Carbonaro gets deep inside our mainstream capitalist/retail/advertising culture which is hard on women but which a) gives them small zones of respite and pleasure and b) holds out the hope that, as desirable females, they might persuade their menfolk to be morally purer; Carbonaro then delivers an ephemeral, impossibly perfect realisation of these painfully felt desires for sanctuary and redemption, which more typically just sell e.g. washing powder.

Carbonaro’s skill is in how intimately he inserts himself into submerged, but emotionally charged parts of our culture. Different aspects of his persona subtly undercut each other: openly gay in real life, he can take on a stereotypical gay engagement with rituals of femininity or a stereotypical gay cosmopolitanism; as a prim white guy he can deliver science-y patter his trick needs – while remaining unthreateningly kind-of-dumb throughout.

Other desires he taps into are motherly fecundity (with an impossibly effective juicer); and cosmopolitan glamour (with an art supplies shop becoming a portal to Paris at night).

(And then, to be sure, a lot of the time there isn’t any subtext for me to chin-strokingly analyse; most of his clips are just I-can’t-believe-he-fell-for-it gags or gross-out.)

It’s very smart; but I’m not claiming this is woke entertainment. Carbonaro appropriates and manipulates tropes and interactions from advertising and retail, but he’s 100% participating in capitalist distribution systems himself – these clips are ads for his TV show on TruTV, owned by WarnerMedia. They are formatted for the exploitative and opaque adtech ecosystem of YouTube.

More fundamentally, the fantasy Carbonaro sells us in these clips, of redeeming the everyday, is predicated on that everyday being the bleak, inequitable cultural mainstream of Western capitalism and authority systems.

The setup of the series does nothing to challenge that mainstream. It grants no agency to the participants, even though their reactions form so much of the entertainment value (and are invariably used for the video thumbnails). They’re not credited – I don’t know if they’re even well paid – and by the nature of the show they don’t consent beforehand. In our contemporary #content-driven culture, privacy, renumeration and control of one’s depiction are lost values.

At times, the racist and sexist society which forms the background for magical tranformation, reappears in the painfully deferential way some participants address the white Carbonaro: “You think I’m crazy, don’t you”, or “I don’t mean nothing by it”. Even with careful editing and selection of takes, the fantasy of upending privilege and inequality is fragile.

Maybe also the very idea of a magician is retrograde and creepy. The personality type that wins acceptance by painstakingly practised, seemingly effortless performance, and maintains a cloak of mystery around its methods, is perhaps the most defensive and emotionally unavailable there is. I should know, I’m a jazz musician.

So overall, I agree with the judgement of my friend who sent me these videos in the first place – they are indeed a trashy, addictive dopamine rush. But it seems to me it takes a lot of heart to make something so emotionally resonant. The videos certainly grabbed me enough that I had to sit down and pen this. Carbonaro made me ponder how I might make art or music that respects the heat of moral desire in people, the fervent secret wish for the world to be redeemed in a moment’s magic.