OpenFrameworks basic soundwave example 1

This is a OF basic sound wave generator example made by Zach Lieberman.

Two Soundwaves overlapping with each other, using mouse movements to control each wave’s frequency variation.

Two waves are named “Carrier” and “Modulator”.  “mouseX”movement controls the carrie wave frequency.  “mouseY” controls the modulator frequency.

It’s interesting that when the 2 frequencys are very close to each other and mix them, we can hear certain beats.

A very basic example to research on the Oscillation of soundwaves.

MIT video: The Sounds of Music- Walter Lewin, Professor of Physics

http://video.mit.edu/watch/the-sounds-of-music-9042/

Very interesting  lecture by professor Walter Lewin to introduce basic knowledge of sound in a physics perspective. Most of the audiences in the lecture are children, but prof. Walter talked to them exactly in an “MIT level”, treat them like researchers.

For anyone who wants to study Audio-Visualization theory, this might be a good video to watch during lunch break or when cooking something : )

Some Articles about Visual Music

1)”Visual-Music Culture, Kerry Brougher, Visual Music (Synaesthesia in
Art and Music since 1900), Hirshhorn Museum/Moca, Thames and Hudson 2005″
http://klaresque.org/downloads/visual_music_culture.pdf

2) Foundations of a Visual Music, Computer Music Journal, Brian Evans

www.brianevans.net/cmj/cmj.pdf

3) “Lost in Translation: Christoph Cox on sound in the discourse of
synaesthesia”, Christoph Cox, Artforum, October 2005 (File Name:
SS_cox.pdf)

4) Where Abstraction and Comics Collide,  TATE ETC. issue 7, Esther Leslie

5) Digital Harmony of Sound and Light, Computer Music Journal, Bill Alves
(about Whitney brothers)

6) “Tones from out of Nowhere”: Rudolph Pfenninger and the Archaeology of
Synthetic Sound, THOMAS Y. LEVIN

7) Towards a Poetic of Visual Music, PhD Dissertation, EGS, Jerry L.
Hoslopple

http://www.europeangraduateschool.com/pdfs/jerry-holsopple-visual-music.pdf

Coverting Image data to Audio with OpenFrameworks

This is a OF skecth made by Zach Lieberman, can automaticly analyse and convert image patterns to audio output:

https://github.com/ofZach/avsys2012/tree/master/week1_audioAndSoundBufferTogether

When continuiously  pressing “a” “b” “c” on keyboard, various patterns will be generated and than a scan line appears and moves from top to bottom of screen, it analyses every pixel info of the image and automaticly triggers the sound during the linear scanning .

Another way to convert image to audio based on the same technique is to draw  the picture in Photoshop and save as “.RAW” (Photoshop Raw)file, then use a sound program (et. Adobe Audition) to open it!  Enjoy! 

OF on iOS Configuration

In this post, the whole process of installing OF as an iOS library will be showcased step by step, I did this for more accessibility to approaching the music project goal, the process might vary from one platform to another, but the concept is always to use OF as an library and a toolkit to help create a pixel drawing based canvas.

1. Go to OF Download: http://www.openframeworks.cc/download/, where listing 5 platforms which in compatible with OF: OSX,iOS,Linux,Windows and Android, click on the download

2. In the OF iOS package, there are bunches of example ready to open and run, I picked a example called “GraphicsExample”

3. Select the right dependency SDK and right simulator, then run, and the result will be like this:

From Analog to Quantized Notes

Sound Wave–>Digital Signal–>Pitch Note–>Standard Notation

Analog–>Digital–>FFT Analysis–>Note Analysis–Notation Rendering Spec

An Audio Library for OF—FMOD

Because I always focus on the audio compatibility of a tool, so I explored the addson of OpenFrameworks a little, which might help my Bravura Project to figure out a way to process audio signal better. After searching for suitable audio library for iOS development, I founded OpenFrameworks is compatible with FMOD and take FMOD as its audio port, then I want to spend more time on FMOD other than OpenAL or PortAudio.

FMOD not only provide a tool for designer, but also open up the source of library for audio programmer who working for the audio mixing, realtime rendering in game based or other entertainment oriented projects. The main products provided by FMOD are :

1. FMOD Studio—-an audio content creation tool for games, with a focus on a ‘Pro Audio’ approach.

2. The FMOD Ex Programmer’s API and Designer —-a world-leading library and toolkit for the creation and playback of interactive audio.

So I am going to dig into FMOD Ex API in the rest of this semester. THis is a super powerful and cross-platform audio library and toolkit, it provides API for Windows, Windows Phone, Mac OX, iOS, Android…because we are using the iOS as and OF is Mac OX based, so I just downloaded the iOS API.

In the IOS API package, they provided api, documentation, examples which are really helpful for FMOD beginners, and tools. Find a “pitchdetection” example and open it in the Xcode, and config the iphone device for testing, then run the program on ios, the screenshot is like:

Digging into the codes a little, the logic of this program can be summed up as in every 0.05 seconds, device detect the audio wave from microphone, and through FFT analysis, return the main function a major frequency of the current sound wave, then through the “frequency–note” mapping, return the interface with both the detected frequency and discrete note char.

From Processing Junkie to A Beginner of OF

1.How Processing Works?

Processing is a subset of Java, and use the Object-Oriented theory for class to build up a sketch,but the polyMorphism is something new to Processing users, but it is a very basic knowledge of Java.

Although Processing hides the complicated polyMorphism from us through their compiler, Processing is actually an engine running a Base Class (pApplet). Anything that you write, including classes, inside the Processing application automatically extends this base class that the Processing engine then runs. When you write draw() and setup() functions, you are actually re-defining the draw() and setup() functions that are in the processing base class.

2.Java Compiles Different from C++
In Java, every time compiling, the entire program is run through and changed into byte code. Then when running your program, a Java interpreter does runtime compilation to make the program work.

In C++,the compiler pre-processes program; Second, the compiler parses through code, making sure that all of your code makes sense, and breaking code down into parse-trees, which it then translates into Assembly;Thirdly, the Assembly is translated into machine-readable code inside object files; and Lastly, these Object Files are linked together to create .exe or .app file

3.My nightmare–Pointers!!!!
C++ differs from Java in that you need to explicitly state whether you are passing something by value or by reference. You can also define when a variable will behave as if it contains actual data and when a variable will only contain a pointer to data. You can even make an int that behaves like the Test class did above. This is all done with the & (referencing) and * (dereferencing) symbols.

Topics and Milestones

Set the milestones and topic of this semester’s independent study