Aesthetics, Ideas, Projects

Data. Yesterday I made a website for the club I’m starting: DATA. I’m excited. Fun things are in the pipe and hopefully we’re making real progress towards transforming Computation Arts into a community of “artists and hackers”.

Today I went to a ‘workshop’ at the Sensor Lab. An interdisciplinary design PHd student at Concordia wants to create “intelligent” wearables, ultimately for medical applications, but for now he’s just interested in exploring ideas. It’s not quite a workshop, it’s more of a discussion group centered around his dissertation. We sat in a circle and hashed out abstract technical concepts like “granularity” of computation, distributed vs. serial, mesh networks vs. point-to-point, using the intrinsic qualities of material, cameras vs. ‘ordinary’ 1-dimensional sensors, power-usage vs. speed and complexity, and embedding sensors and processors intelligently i.e. in seams, ruffles, and bumps intrinsic to the fabric. Now I’ve got somewhere to toss around the theoretical notions I’ve been playing with in my head and to learn more about engineering, the design process, and wearable computing on Fridays!

Earlier today I met with the TML’s “Ozone” software team. We’re transplanting the media-system that we built for Remedios Terrarium into the lab. Freida Abtan presented her latest work, which I’d already seen at Mutek. We talked about future Jitter development inspired by techniques that Freida has been playing with and JS’s new system for routing OSC data between networked computers. In July and August I’ll be working as part of Ozone on meaningful mapping for “state evolution” [I’ve discussed the ‘state engine’ before] and (i) modularizing existing: + (ii) exploring new: feature extraction algorithms. ‘Mapping’ seems to be the most important creative activity in this practice of building “reactive” systems. Being able to set-up mappings seamlessly between computers (thanks to [pattrstorage]) will do a lot to grease the creative process and my job in particular. I’m particularly excited about exploring techniques for extracting meaningful data from audio signals (mainly from microphones). We’re also taking time-lapse video of vines growing in the lab and I’ve been brainstorming techniques for using this video to monitor the state of the plants and thus the room. Using plants as long time-scale sensors.

In other news: Calculus II started today + I’m moving in with Navid Navab for a month on July 2nd + I’ll be participating in the 2008 Minds and Societies summer school “conference” for two weeks starting the 28th? (I think). The website is down as of my writing this. The theme (this year) is “minds in societies” (that is, social cognition, perhaps the most boring branch of cognitive science) and “societies of minds” (more interesting, though I hope I don’t have to listen to too much talk about the ‘extended mind hypothesis’). I’m in for some exciting experiences – at least in one branch of my life – this summer methinks.

Meta. I intended this blog to be the ‘adult’ precursor to my LiveJournal (which I’ve described as “teenage catharses”). Instead of giving myself an outlet for the sort of writing I assumed I would want to be doing, I found that I had cut off the outlet that was most important (or necessary). LiveJournal, I decided, is for ‘any old drivel that I don’t need to write anyway’ that only my friends would read, while ‘my blog’ is for ‘serious stuff that I should be writing’. After making this choice, I found that I stopped posting to my LiveJournal and seldom felt the need to post to this blog. I’ve since realized that one of two things is true: either (i) at this point in my life I want/need to express ‘any old drivel’ (and not necessarily ‘serious stuff’) or (ii) it takes writing ‘any old drivel’ to get the engines running, at which point I can consider writing ‘serious stuff’. Back when I wrote drivel, writing on the internet was something I enjoyed; it got me writing, it got me formatting whatever ideas or problems I had for a general specific audience. I want that back. So, I’ve switched up my workflow a bit so that LiveJournal is a part of my daily routine. My friends’ feeds no longer route to an aggregator (they route to my LJ friend page) and I’ve strategically placed links to remind me that there’s more to the internet than Gmail, Wikipedia, and RSS feeds (which now route to Gmail thanks to a Python script called rss2email). The catharses (or plateaux?) are back . Perhaps (ii) is true, the engines will start up, and the need for ‘serious’ posts will emerge, and I’ll have this blog waiting just for that.

Media Medium. Two of my classes, CART363 “The Advanced Languages of Programming” and CART454D “Enchantment, Matter, and Topological Media” require a final techno-art project. When I began brainstorming, immediately I wanted to extend what I was doing at the TML in January and February: that is, a dance film with camera-tracking controlled sound and Jitter visuals projecting on a semi-translucent screen stretched over a pair of dancers. Part of the idea was for the ‘media’ (visuals) to form a ‘medium’ (as in air, water) in which the dancers would be immersed. The sound was then meant to reflect the notion of ‘touch’ (a crude example would be the bzzt that signifies a hit in fencing). While the visuals came together to adequate effect, the sound served as more of a soundtrack than anything. Having spent a few weeks experimenting with camera tracking techniques and brainstorming mappings, along with Tim Sutton who did the sound, the project felt unfinished as it came time to film what we had for the EMPAC submission deadline.

So, I had this format in mind: four speakers facing inward with camera-controlled sound. The question, ‘how can I map the movement of players in a small space to sound tangibly and meaningfully‘. On the bus home a few nights ago, a concept and a technology synthesized: sound as a medium + the State Engine.

[What is the State Engine? From Yon Visell’s project page at Zero-th:

The software provides a general-purpose choreography system for responsive design, based on a continuous dynamics within a topologically general state space. This approach allows designers of responsive media to specify complex media dynamics in a way that is idomatically concise and expressive, encoding the delicate set of relations between the interactions of the users of such systems and the character of the range of responses that are elicited.

(I will describe a basic configuration.) The State Engine consists of a ‘token’ (particle) in a topological space. The token is given coordinates, distances from the vertices of the space (simplex). The token is pulled around, by ‘forces’ ‘exerted’ by the vertices, in a smooth and continuous manner. It’s behavior is designed to mimic physical dynamics, either those of a spring or gravity. You can then pump in sensor data, define generally how the token should respond to that data, and you’re given in return a vector of the coordinates of the token. What’s exciting about this system is how physical and life-like the output is. It yields ideal data with which to drive a ‘media medium’.]

And that’s exactly what I plan to do. I will place four speakers in a square facing inward. Each one will play a sine-tone from an independent oscillator. The frequency of each oscillator will then be mapped to a vertex of a tetrahedron in the State Engine. The range of frequency modulation will be quite low, the idea being that as the token navigates the space, the oscillators will subtly detune, introducing frequency beating. To obtain data to disturb the token, I will use camera motion analysis, specifically a Jitter patch that sees only moving objects and leaves a decaying trace after things have calmed down. The Engine will be configured so that the token will snap back to the center of the space in the absence of motion.

Thus, a user in the space (framed by the speakers and the camera’s scope) will be able to throw the oscillators out of tune by moving. He will have the sense of disturbing a water-like, or elastic medium. A sharp movement will introduce a chorus of high-frequency beating patterns. As the ‘trace’ of their movement fades and the token returns to the center of the tetrahedron, the oscillators will reach equilibrium. Additional parameters to determine in what manner the token will move are yet to be defined, but I expect to work this out as I’m building.

I’m thinking of this as a ‘phenomenological experiment’. My goal will be to make the media as “thick” and “tangible” as possible. Once the system is constructed, I expect I will be spending considerable time ‘tuning’ it. Code-wise, it’s a fairly simple project (snapping together pre-written chunks), but the State Engine is fussy and offers myriad configurations. I suspect that a range of distinct behaviors will be possible. Also, it’s likely that a bit of random, ‘irrelevant’ / peripheral data will make the behavior more interesting. This I expect to discover in the course of experimenting, and that’s the point!

Techno. I woke up in the middle of the night with an image of music I wanted to create. Rich, fuzzy, spectral layers of sound, like pastry, draped delicately over hard lumps; thick, chesty, and deafening.

techno [jpeg]

(I discovered the scanner at the lab. In fact, there are two. Exciting!)

Aesthesia. “blue tarnished pewter”: what I told her methylphenidate feels like. She replied: “it comes from…dry, narrow hallways in 1970’s office buildings with grey tweed walls and tinny speakers, or the first gulp of gin on a sad person’s couch with their TV on.”

That doesn’t follow for me. Clashing aesthetics. She’s the second person I’ve met who feels at home speaking this way; the other is my girlfriend. Both synaesthetes. Sure, I’ve met others who understand — a sort of, “yeah, I can hit that” — but only the two synaesthetes really seem to get it.

“blue and tarnished…The feeling is smooth and cold and tastes like mealy rust and glows with some un-nameable eerieness that is best described as blue.”

“Still clashing,” I reply.

I’m not a synaesthete, a mere aesthete; sometimes, there are moments when I really feel like we (me and synaesthete) are communicating something identical. There’s a glow in our eyes, a rush of adrenaline — our brain firings are matching up! But all too often, a misfire. Our worlds seem like they’d line up when viewed from the surface, but when overlayed, the inconsistencies show through the cracks too many and tinfoil smothers the scintillating spectral sunburst — it’s less magical than that (I just like the way that feels). Soft disappointment; a frame wrapped in velvet.

“would you ever expect anyone, regardless of ‘__sthete’ to match? They should be considered miracles to each other.” A perfect reply. Dreamtime.