Barbara Nerness (Music, Science, and Technology) | Die Kunst des Sommerpraktikums: The Art of the Summer Internship (ZKM Karlsruhe)

Image

The experience I had in Karlsruhe, Germany at ZKM in the Hertz Lab was full of artistic, academic, and personal growth. Since I was an undergraduate at UC Berkeley, I have wanted to participate in an exchange program abroad, especially in Germany. The location of my internship in Karlsruhe allowed me to reconnect with family, who are only an hour away in Rheinland-Pfalz, and discover a rich connection with students at the Hochschule für Musik, which has a very similar program to the one that I’m currently in at Stanford (CCRMA’s Masters in Music, Science, and Technology). Through my internship, I was able to explore the creative, studio, and technical sides of audio, and found myself gaining an international awareness of the field that I did not have before. I cannot thank my sponsors enough for supporting this experience, as I now plan to include study or work in Germany within my future plans. I will first cover the wide range of projects that I worked on while at ZKM, starting with the largest and ending with the smallest. Then I will briefly describe my personal growth resulting from living in Germany and how it has shaped my future goals. I hope you enjoy the read!

I spent ten weeks at Zentrum für Kunst und Medien (ZKM) Karlsruhe (Center for Art and Media), where I had one large coding project that took eight weeks to complete, and one large artistic project resulting in a show after my seventh week. The coding project consisted of writing Max/MSP patches to control a Mackie MCU Pro Universal Controller. The Mackie is a high end physical interface with faders and buttons that sound engineers use with digital audio workstations (DAWs) such as Pro Tools. Because a lot of guest artists use patching software such as Max/MSP or Pd, my task was to write Max patches to interact with the Mackie, controlling the faders, buttons, vpots, and infinity wheel. The complexity of the project came from the fact that the MIDI messaging system supported by Mackie is proprietary, and thus not publicly available. I found one engineer in an online Max forum who had reverse-engineered a different Mackie controller, which helped me to understand the nature of MIDI messaging, which uses hexadecimal message pairs, and their implementation on the Mackie. However, not all of the document I found was correct, so I had to do some of my own reverse-engineering. Because I had never worked with the MIDI protocol before, it took some time to understand and use the right Max objects. Yet once it clicked after the seventh week, I was extremely happy, and even ended up using these patches in my own personal video project that I will describe later. Customizing MIDI controllers is quite a common practice in the field of interactive audio, and I now feel confident implementing this in my own work. It will be a great skill to have going forward, as MIDI has been around since the 1980s and is still in wide use today. Lastly, my boss on the project gave me some guidance throughout on tailoring my user interface to meet the needs of sound artists, which is very important. I discovered a love of building tools for creative users, and have an idea for PhD research that utilizes these skills to build interfaces and instruments for people with learning or physical disabilities.  

My second large project was much more creative and artistic, as I had the opportunity to design visuals for the large summer event in Karlsruhe called Karlsruher Museumsnacht (KAMUNA) that brings together arts institutions, museums, and galleries in Karlsruhe for a one-night show across the city. Common to many large cities in Germany, museum nights celebrate the art and culture of the area and I love the concept. ZKM was a big participant with shows not only on-site but at three other galleries across the city. The sound from the off-site shows was streamed back to ZKM, where engineers and artists remixed and re-represented it live in multiple spaces. My task was to create visuals that interacted with this audio using an audio to video converter called the Synchronator. The Synchronator is actually a digital implementation of the effect that is achieved when sending an audio signal into the video input of an analog tube television. Unfortunately, using any random audio signal does not look good, as the frequencies that create interesting horizontal, vertical, and modulating shapes are dependent on the refresh rate of the screen (usually 50 Hz or 60 Hz). In order to find these ‘interesting’ frequencies, I performed sine sweeps, using both analog and digital equipment. I found that frequencies in the kilohertz range looked much better, and that combinations of these frequencies resulting in amplitude or phase modulation created very pretty effects. I used the amplitude and pitch (from the largest peak) of the live audio stream to further modulate the frequencies themselves, which created more movement for louder and higher frequencies. You can see a short documentation of the final show below​.

Image

Displaying the output on the huge LED wall on the cube outside of ZKM proved very difficult, as it is not a standard resolution or size, so I developed a backup plan with the show curator in case it did not work. This involved using eleven tube televisions as a sort of sculpture/installation in collaboration with a sound engineer who was already going to be performing only the bass frequencies in the ZKM parking garage. By showtime, both solutions were working, so I went ahead with both! The final result consisted of a live visual installation on the LED wall that reacted only to the music that could be heard nearby, and a large subwoofer/tube television sculpture/sound installation in the underground parking garage. The creative freedom on this project was extremely motivating, and with no guidance, I learned how to use the technology for an artistic purpose that combines both audio and visual art. I received great feedback from Ludger Brümmer, the director of the department, who may want to use my work in an upcoming show this November. In addition, an old friend working in film contacted me about buying some of my work for a music video after seeing some of my social media posts. With more time in August for my own projects, I was able to complete the work and have now started a new collaboration combining my love of audio and video.

In between my work on these larger projects, I also had the opportunity to support the guest artists working at ZKM across roughly eight studio spaces. In July I assisted Götz Dipper with the task, including building up one 8-channel studio, in order to prepare for his absence in August when I would take over. By August I was comfortable using all of the different multichannel interfaces, checking out equipment, and hooking up speakers. Although I had done this before, it was never on such a large scale. I also had the opportunity to greet guest artists when they arrived at ZKM and be a resource during their stay, which allowed me to get to know them and their work. I became friends with many who are all still in touch - an invaluable network of artistic researchers from France, Portugal, Germany, and more. As a highlight, at the end of my first week I had the unique opportunity to interview guest artist and professor Takuro Shabiyama for ZKM’s website, and I hope to share this with you once it is released. 

When most of the department was on vacation in August, I also had time to work on some of my own projects. I learned to use all three versions of ZKM’s spatialization software, Zirkonium, to finish a piece I started at Stanford earlier in the year and to provide a comparison report for the developer, Dan Wilcox, who will be working on the software in October. I submitted the finished audio for the piece to the CTM Festival in Berlin, which is a “Festival for Adventurous Music and Related Visual Arts.” My piece, titled “Resist,” uses an audiovisual performance to present issues of surveillance and protest, and fits perfectly with this year’s CTM theme, Persistence. I will also be submitting the work to another conference in Santa Barbara this fall, and to other festivals that I find. This was my first experience with 3D spatialization, and I received feedback from a guest artist along the way that allowed me to see space as a compositional tool. I also discovered a need among electroacoustic composers for easy-to-use spatialization software. Most composers with an engineering background design their own tools for spatialization, but those who cannot still want to write music for 3D audio environments. Thus, the tools for spatialization are mostly academic with limited support. I interviewed one guest artist on his needs and desires for spatialization software and communicated this to the developer working on the project at ZKM.  

Throughout the summer there were many other small projects and tools that I worked on, including documentation of the museum installations provided by the Hertz Lab for the Open Codes exhibition, and servicing a broken installation in neighboring city Trier. I could also view the ZKM museum exhibits for free while working there, which was incredible and inspiring for my own work. I plan to apply for artistic residencies there in the future now that I know what they are looking for. Another great experience was visiting Trier, the birthplace of Karl Marx, to fix an audiovisual installation in my second-to-last week. By this time I was comfortable with ZKM’s sound equipment and able to replace the audio interface to get everything working smoothly again.  

Lastly, an important area of growth for me was not just academic and professional, but also personal. Having worked and lived in the Bay Area my entire life except for travel, I was apprehensive about setting up a life in Germany for three months. However, after settling into my apartment and work routine, I found Germany a fantastic and easy place to live. Karlsruhe is a small city with a lot of biking culture, families, and a fair amount of arts and culture. I met and became friends with students in a very similar program to mine at CCRMA, called Musik Informatik, at the Hochschule für Musik, and we hope to foster an exchange between the two institutions. In addition, I met with Professor Stefan Weinzierl at the Technische Universität Berlin, where I learned more about earning a PhD in Germany, as well as opportunities for funding. I plan to apply there this fall as I apply to other PhD programs in the US and Canada, and could easily see myself living in Berlin.  

As for my language skills, I have improved enough to be competent in everyday situations, but not as fluent as I would have liked. My boss did a good job of speaking mostly in German to me, and only using English when I indicated I did not understand. If I were to spend more time in Germany, I would sign up for a course to provide a more structured learning environment and improve my skills more quickly. Additionally, I plan to continue practicing by listening to podcasts and skyping with friends I made there in German. Overall, I cannot thank my sponsors enough for the opportunity to work in Germany and discover a new way to combine my love of both art and science. Hopefully it is not too long before I have the chance to return!