The Code Barrier
Bret Victor has become a little bit of a hero for me. Previously an interface designer at Apple, it seems that Victor was successful at the tech giant, but realised that his vision and theirs didn’t quite fit together and he left in 2010. He’s now best known for his provocations toward the way we interact with our digital tools being all wrong. And that’s why I like him.
I don’t always agree with everything he says, but I think the sentiment is solid: interaction with our tools shouldn’t be mediated through a layer of verbose language. This is particularly pertinent where our tools are code and when artistry can be stilted by the coding experience. The video above speaks to this idea and suggests that bodily performance is a better way to express ourselves through the computer medium. Whilst the presentation ironically becomes a bit of a showcase for his cleverly self-programmed tools, his opening up of this debate outweighs any self-indulgence.
In the presentation, Victor uses the analogy of performing music. When you’re learning to play an instrument, you do need to learn a certain degree of music theory and have to come to terms with the basic mechanics of playing. But it generally isn’t long before your body memory takes over; it’s almost as if you’re playing without thinking about it – at least not through language. And this is how Victor believes we should be able to express our artistic intent through code – something I see as having links to VJing or other live audiovisual performance, where the lines between engineer, performer and artist become wonderfully blurred.
Where I think Victor misses a chance at exploring an important idea is when he forgets his own reference to gaming. Early on, he talks about video games being the first artform that could only have been realised by a computer. This is debatable, although he uses this idea to point out that most of what we currently do on computers, wastes the power of the medium by simply emulating existing analogue experiences such as painting – and I wholeheartedly agree.
And so this idea that computers are very good at simulating is where he resolves the discussion about gaming. I wouldn’t stop there. Taking the game experience further, I would suggest that playing is a far more interesting endeavour. Specifically, pretend play. There’s a good chance that this bias is coming from my PhD research; looking at the developmental milestones of children, but stick with me for a moment. Victor thinks that we should avoid language as much as possible by allowing body memory – or perhaps the body schema – to be the conduit for artistic expression. My own thought is that the mind has a far greater role in the artistic process than what is presented here by Victor, and it resides in a personal or semi-shared language of imaginative play.
During pretend play, there is a tangible back-and-forth between imagination and physical reality. You can see this easily when watching a child playing with toys and giving each its own name or special super power – it’s as though an unseen layer of imagination is placed upon the real experience. This is not specifically visual, but there is a creative topology which is tangible and sensory. Again, there is some PhD research spilling out here.
Returning to learning (and teaching) to code, how can we best facilitate play which leads to creative outcomes? Although it would be wonderful to have Victor develop his unique tools for the specific style of each individual, I work within the reality of limited software deployment in a university system. Under these conditions, I can’t see a better alternative than that of visual programming.
Learnable Programming is an excellent essay that Victor wrote in 2012, mostly discussing why Processing isn’t the best tool for learning to code. Although this is exactly the tool we use to introduce students to creative coding at our university, I agree that we’re not approaching the potential of expressive code in the right way. Victor makes the point that we should be able to see what is happening whilst coding; be able to observe the flow of data. Again, he creates some pretty impressive tools to do this (and neatly avoids saying visual programming is the answer), but in the end the presentation becomes a strange hybrid of visual and textual programming. I personally think we should lose the reliance on text entirely.
Visual programming – whilst an imperfect tool – gives the student an almost instantaneous opportunity to create. My limited experience with teaching is that it isn’t actually what is happening that excites the new student, but just the fact that they made something happen. The hello world moment is valuable in capturing their attention, yet textual programming languages sets the bar for newcomers far too high. We spend too much time trying to learn a language before even getting the simplest of outputs. We need to explain what function is, what syntax this particularly language requires, what IDE we’re using – and most often of all, why. And forget creativity; some students are so overwhelmed by having to learn to code that they can’t even imagine the next step of creating something expressive. And so we often end up with very limited outcomes and most disappointingly: very few students who enjoy their introduction to the world of code. We shouldn’t be surprised in the least.
The debate about which tools to use at university often devolves into my most hated of terms: industry standard. Sure, visual programming isn’t necessarily going to get you a job making the new million dollar iPhone app title. But it’s probably time we were a little more honest about this debate; studying interactive media arts isn’t going to get you a job being a programmer, no matter what tool we teach you. If Serious Programming is what you’re after, you’re better off doing a Computer Science degree. Trying to pretend that a creative institution is best placed to produce graduates that are highly paid computer operators feels like it’s missing the point entirely. I would rather see students engaged and expressive – if they choose to move on to explore other tools, then that is the best possible outcome I can imagine.
Just because something is difficult, does not make it a more worthwhile pursuit. I’m open to discussion about which tools might be best placed to introduce creative coding, but after discussion with others about the software available to us, Isadora seems like a good place to start. Like (supposedly soon-to-be depreciated) Quartz Composer and some other module-based visual programming systems, the learning curve is shallow. Plug A into B and see what happens. Plug that into C and see how it changes things. Each module is like a little engine of real-time processing and doesn’t require knowledge of syntax to start experimenting right away. Just like the child playing with a box of mismatched toys, creativity is not limited by procedure or language.
The simplicity of Isadora could lead into more complex visual languages, such as Max. I personally use Max and for this reason am probably bias toward it, but it does lend itself to a lot more extendability than Isadora. While it has a much steeper learning curve, it does allow someone with a keen interest to push the tool much further. And that’s the kicker: we need students that are already engaged to be willing to push their own creative experiences further – forcing them to jump hurdles right at the beginning of their journey is not the right way to do this.
At the moment, we’re at a turning point at university as to how best to teach interactive media. This is hopefully the beginning of an ongoing discussion and like Bret Victor’s talk, I don’t think what I’ve presented here is the most thorough argument against teaching textual programming languages. However, I do think it’s a debate that needs to be had, and the sooner we start exploring alternatives to engage creative students, the better.