Vector and Sampled Sound in Mathematica

An Introduction

Michael Pilat

Software Technology
Wolfram Research, Inc.

Sampled Audio

CD Audio, MP3s, etc.

Discrete sampling of a continuous waveform:

Good: Quality can be very high

Bad: Lots of data, even with modern compression

CD audio:
  2 Channels
  44100 samples / second
  16-bit resolution
  
176,400  bytes / second!

MP3 or AAC:

128,000 - 192,000 bits / second

Vector Audio

Discrete Events

(At least) Two dimensions of "vectorness"

Time    (speed of playback or performance)

Frequency / Pitch    (transposition, etc.)

Good: Very small representation; easy to manipulate

Bad: Quality can be hit-or-miss; limits to what can be represented

Rendering Vector Audio

Musical Instrument Digital Interface ( MIDI )

MIDI Music Association -- 1985

Events

Note On

Note Off

Controllers & Effects (Foot Pedal, Sustenuto, etc.)

Meta Events (Lyrics, Song Title, etc.)

Notes

Equal-tempered tuning in semitones

Notes are numbered 0 - 127

Note 60 - C4 / 261.6 Hz / "Middle C"

Note 69 - A4 / 440 Hz

New Primitive - SoundNote

Three Parameters

Note or Pitch

Duration or Start/End Time

Volume

Single Notes

In Mathematica, notes are numbered around Middle C, which is 0.

Sound[SoundNote[0]]//EmitSound

Notes can also be specified by name:

Sound[SoundNote["CSharp4"]]//EmitSound

Sound[SoundNote["B♭5"]]//EmitSound

Chords

Sound[SoundNote[{"C","E","G"}]]//EmitSound

Duration

The second parameter to SoundNote controls the duration:

Sound[SoundNote["C4",1/2]]//EmitSound

Sound[SoundNote["C4",3]]//EmitSound

Velocity

Velocity doesn' t always imply volume, and even when it does, it doesn' t necessarily apply only volume.

The "SoundVolume" option controls the velocity of SoundNotes:

Sound[SoundNote["C",.25,"SoundVolume"->#]&/@Table[t,{t,0,1,.1}]]//EmitSound

Rests

Using None as the "pitch" works like a musical rest:

Sound[SoundNote[None]]//EmitSound

Sequencing Notes

Sound[ ] as a container

Sound[{SoundNote["A5"],SoundNote["B5"],SoundNote["G5"],
SoundNote["G4"],SoundNote["D5"]}]//EmitSound

Note that the duration of a Sound object is implied by the duration of its contents...

Duration

Sound[{SoundNote["A4",1/2],SoundNote["B4",1/2],SoundNote["G4",1/2],SoundNote["G3"],SoundNote["D4"]}]//EmitSound

Like SoundNote, a new second argument to Sound specifies the duration:

Sound[{SoundNote["A4",1/2],SoundNote["B4",1/2],SoundNote["G4",1/2],SoundNote["G3"],SoundNote["D4"]},1]//EmitSound

Sound[{SoundNote["A4",1/2],SoundNote["B4",1/2],SoundNote["G4",1/2],SoundNote["G3"],SoundNote["D4"]},7]//EmitSound

Scaled[ ] can also be used:

Sound[{SoundNote["A4",1/2],SoundNote["B4",1/2],SoundNote["G4",1/2],SoundNote["G3"],SoundNote["D4"]},Scaled[.5]]//EmitSound

Nesting Sound

Perhaps one of the most power features of these enhancements, Sound[ ]s can be nested like lists in Graphics!

encounter=Sound[{SoundNote["A4",1/2],SoundNote["B4",1/2],SoundNote["G4",1/2],SoundNote["G3"],SoundNote["D4"]}];

Sound[{encounter,encounter}]//EmitSound

EmitSound

EmitSound[ ] has been extended to handle vector audio...

More Parameters

Explicit start and end times can be given for both SoundNote and Sound

Sound[SoundNote["C4",{1,2}]]//EmitSound

SoundNotes can be overlayed and combined:

Sound[{SoundNote["C4"],SoundNote["E4",{.75,2}],SoundNote["G4"]}]//EmitSound

Audio Import / Export in 6.0

MIDI Export

Export["ex.mid",Sound[SoundNote/@Range[10]]]

AU / AIFF / SND / WAV Import & Export Rewritten

Faster

Better

Stronger

FLAC

Vorbis

MP3

Wave64

Elements

Import["C:\\Documents and Settings\\mpilat\\Desktop\\taxman.flac",{{"SampleRate","SampleDepth","AudioChannels"}},"ElementRules"->True]

Sampled Audio

The new extensions to Sound[ ] also work with sampled audio:

taxman=Import["C:\\Documents and Settings\\mpilat\\Desktop\\taxman_short.flac"];

Sound[taxman,Scaled[.5]]//EmitSound

Sound[taxman,Scaled[2]]//EmitSound

( The quality will improve, I'm still tracking down a bug in the resampling code )

New Options

"SoundVolume" will effect both vector and sampled volume

"SemitoneShift" will shift vector and sampled sounds up or down (pitch shift) but preserve time

"TimeRange" is the equivalent of PlotRange for audio.

Fusion!

Sampled and Vector sounds can be combined:

a440=SampledSoundList[N@Table[Sin[2Pi*440 t],{t,0,.5,1/8000}],8000];

Sound[a440]//EmitSound

Sound[{SoundNote["A4"],a440,SoundNote["A4"],a440}]//EmitSound

a880=SampledSoundList[N@Table[Sin[2Pi*880 t],{t,0,.5,1/16000}],16000];

(s=Sound[{a440,a880}])//EmitSound

Playback of sampled and vector sounds can be simultaneous

Sound[{
Sound[taxman,{0}],
Sound[SoundNote[#,1/6]&/@Range[12],{0}]
}
]//EmitSound

Just like with vector audio, we can overlay two sampled sounds:

Sound[{
Sound[a440],
Sound[a880,{.25}]
}
]//EmitSound

Remember that a440 and a880 were at different sample rates!

Resampling extends to Import and Export too:

Export["ex.wav",s]

Import["ex.wav",{"WAV",{"SampleRate"}}]

Styling Audio

A note can be styled:

Sound[SoundNote["C",1,"Harpsichord"]]//EmitSound

The concept of styles and directives applies too:

Sound[{"Violin",SoundNote[#,.25]&/@Range[12]}]//EmitSound

The same "inheritance" concepts from Graphics apply:

Sound[{"Violin",SoundNote[0],SoundNote[1,1,"Harpsichord"],SoundNote[2]}]//EmitSound

Styles can also be scoped with Lists:

Sound[{"Violin",SoundNote[0],{"Harpsichord",SoundNote[1]},SoundNote[2]}]//EmitSound

Style[ ] works too!

Sound[Style[SoundNote[0],"Marimba"]]//EmitSound

MIDI percussion will also be supported (but is currently broken -- oops!)

More Features and Future Directions

Improved Sound visualization:

Import["C:\\Documents and Settings\\mpilat\\Desktop\\taxman_short.flac"]

This will include a visualization of any vector audio when present, too.

More support for MIDI style and controllers, tempo, etc.

In Future Versions:

MIDI Import

More DSP /  processing functionality

New playback systems

Even more Import / Export formats

??? ( E.g., please tell us what you want!)

Thank You!

The New Features of Sound[ ] and SoundNote[ ] are fully documented and most
features are already available in current 6.0 releases.

Questions?

mpilat@wolfram.com