Bringing things ‘to life’: a talking piano

March 29, 2010

The most familiar early synthetic voice was that of Sparky’s Magic Piano in 1947, popular in schools over a decade.  It sounded as though a piano was talking. The technique is called ‘vocoding’. In those days it was a pioneering project, the Sonovox, from Bell Labs.   Today, such software is available as freeware. A vocoder, with voice-synched animation can bring an inanimate object ‘to life’.  

 

The vocoder with voice-synched animation can bring an inanimate object ‘to life’.  However, they offer much more: a key practical benefit of a synthetic voice is its ease of production.  One only has to speak into a microphone or create some text to make an artificial voice.  A library of ‘answers’ or ‘lessons’ from the talking piano could be stored as a database of .wav/mp3 files. 

The piano sequence required a few stages:  a text-to-speech voice  allows unlimited amounts of text to be turned into speech. This voice was then played into a .wav-to-midi software converter (many free ones available) to get a .mid file. This was then played in software with the midi set to ‘piano’. The resulting file was unintelligible as speech but was a perfect musical translation of the voice file.  That piano .wav file was then imported to Zerius Vocoder software where it became the ‘carrier’ .wav file.  The voice .wav file became the Zerius vocoder’s ‘modulator’ file.  The levels were blended such that the software then made an intelligible ‘talking piano’.

The soundtrack then animated a still frame cartoon piano using CrazyTalk software. This multi-stage process is perhaps too fiddly for a single short piece like that above. However, the idea is to breathe life into a ‘character’ that can then speak whatever text or audio is put into the system.  That it works with text-to-speech voices lends it to batch production of material. Points to observe and discuss:

  • Bringing characters to life.
  • When and how to use artificial (robotic etc.) voices.
  • Voice-synched animated visuals.
  • Vocoder and wav-to-midi to create musical versions of speech or other sounds (industrial sounds, animals, nature, …etc).
  • Batch production from text to talking character
  • Additional audio realism effects (pitch, reverb, compression ..)

Audio effects & blue screen to make miniatures life-like

March 26, 2010

The clips below were shot at my desk. They demonstrate special effects techniqes that can make miniatures seem like the real thing.

The blue-screen in the above clips is of secondary importance to the audio effects. The car was recorded on a metal cutting board with a microphone attached so it picked up the actual sound of the toy. That .wav file was then imported to Zerius Vocoder software where it became the ‘modulator’ file. The sound of a real racing car became the ‘carrier’ .wav file. The software then transformed the tiny car sound into that of a full-size car. The same could have been done with the diesel but the toy locomotive’s metal tracks already sounded authentic enough that all was needed was to pitch-shift the sound down in Audacity for it to sound like a real train. More complex vocoding with speech may require more stages of processing. Points to observe and discuss:

  • Advantages of miniatures over live sets or animations.
  • Blue (chroma) screen.
  • Uses of moving vehicle shots in movies.
  • Audio special effects applications.
  • Other ways of making small look big (& vice-versa)

Creepy Movie in under 1 Minute

March 26, 2010

The movie below was created ‘out of the box’ when I first downloaded a demo version of Reallusion iClone to try it out. The goal was to see whether it was possible to create an audience reaction in under 1 minute.

The above was inspired by The Blair Witch Project, the most successful independent film  and most profitable US film of all time in terms of the ratio of production cost to box office sales. It took a ‘mere’ $35,000 investment and 8 days filming to create an 86 minute film. My objective was to take less than one minute to creep you out, with $0 additional cash outlay, using freeware, canned templates, clip art, a $10 royalty-free soundtrack I’d earlier purchased, ‘out of the box’ programming with no tutorials, in less than 1 day’s filming, audio, & editing. Points to consider and discuss:

  • Script: the first 21 seconds (40% of entire movie) are mere ‘suspense-building’ music.
  • Element of surprise.
  • similarities and differences: creepy v horror films
  • Theme: identification with people’s feelings about pets.
  • 3D voice-synch’d avatar with ‘canned’ dance routine.
  • surreal scene would be preposterous and probably downright silly with live actors & set.
  • Lighting to simulate fire environment.
  • Pitch-shifting and reverb of vocals to alter character.

Digital Techniques applied to Traditional Media

March 25, 2010

Digital media needn’t be all-digital. The new tools should allow us to use the old tools more conveniently and quickly. It would be hard to top J M Barrie’s 1904 stage play of Peter Pan for audience engagement. Tinker Bell was a mere lantern beam flashed around the stage. Yet, in the tradition of British Pantomime audiences hissed and booed at the villain and wept hysterically “I believe in fairies” to save the poisoned Tinker Bell. They still do. The video below uses a reading from a Librivox volunteer (archive.org) as audio and the traditional light beam to portray Tinker Bell:

A well-written story like Peter Pan clearly can work across old and new media.  I make no claim that the visual added much to the nice audio track. Indeed, the trick would seem to be not to drown it with embellishments. Points to look for and discuss:

  • Relative importance of script v media.
  • Blended media – using digital tools to speed production of old media.
  • Shadows instead of direct images 
  • Audio and text versions as powerful in themselves.
  •  The role of imagination.
  •  Selective adding of multimedia such as sound effects.
  • Simple props – plastic hook, hat, bottle and flashlight
  •  Building of suspense in drama.
  •  Getting us to ‘care about’ a character’s fate.
  •  One-man operation – no helpers; one person plays all characters and flashes the light.
  •  Use of separate audio track instead of live audio.

Live Action v Animation

March 25, 2010

This video contrasts live action and animation methods of putting a character into a historic scene.

28 seconds into the blue screen sequence above the scene is frozen as a still frame. This frame is imported to Reallusion’s CrazyTalk facial animation/voice-synchronisation software. The voice recorded later is then synch’ed in CrazyTalk to keep the main character talking. Live action is hard to do. But a short burst of it is enough to give the character context so that the animated face can carry on the script.

Realistic scenes can be assembled to draw on the best attributes of each type of filming. Live footage excels with sensory material –ie- like a Roman emperor eating something, which could be hard to animate.    Voice-synched avatars excel with long complex scripts and scenes. Principles and techniques to look for and discuss are:

  • The importance of bringing the past to life.
  • Voice-synched software for extended or complex scenes.
  • Blending characters into scenes through colour/contrast.
  •  Use of blur and colour de-saturation to match the image least-editable (the character was altered to match a faded 60s movie).
  •  Use of archival footage in place of live action.
  •  Potential to use miniatures/avatars instead of live character.
  •  Re-usability of avatar with voice-synch.
  •  Chroma (blue screen) key.
  • Re-usability of blue-screened actor to introduce other live sets (aboard ship etc).
  •  2D v 3D characteristics (live actor moved only laterally).
  •  Simple props: Elvis towel for toga, plastic sword, rubber Roman helmet with knight visor.

Movie-Making

March 23, 2010

This video attempts to give a feeling for the main theme of this Movie-Tom.Com site as well as numerous points for discussion in under 2 minutes:

The above sequence is what I term a ‘movie’. It is short and intended to serve an educational purpose, beyond what a text description could do. It embodies the adage “Show, don’t tell.”  It deliberately does NOT start out “Hi, I’m Dr Tom and in this video I’m going to show you … blah, blah, blah”. That’s a waste of bandwidth for the sort of thing much more efficiently said in text. Principles and techniques to look for and discuss are:

  • putting elements of message into a short video.
  • one man operation – no helpers.
  • involve viewers’ interest by letting them guess the purpose.
  • giving credibility to a character by showing him in a role.
  • identification with character by invitation to ‘cup of tea’.
  • elements of surprise.
  • transformation from live action to a voice-synched avatar/talking image.
  • avatar can follow on in a less-resource-intensive media by just changing audio.
  • re-usability of the avatar with different audio messages.
  • use of archival footage, which can be substituted.
  • audio prop of loud hailer to give illusion of size.
  • chroma-key (blue screen).
  • colour de-saturation and blurring effects to blend with background movies.
  • [educational uses are a separate discussion in this series]

The idea of such a movie is to give the overall ‘flavour’ of a topic in the shortest, least-resource-intensive manner possible. Therefore, additional  movie segments will be needed to cover the separate issue of the actual educational use and presumed benefits of such movies.

Finally, these movies are left ‘rough’. I’d spend far more time with finishing touches than in making the movies. I’d then be lying to teachers to tell them I did these quickly if in fact I’d used a pro studio and pored over them for hours or days fixing bits. The whole idea is to demonstrate that a theme movie, even an epic, might with digital methods be done quickly and cheaply enough to serve a teaching purpose.

Scary Dinosaur

March 1, 2010

A genuine movie-making challenge is to make monsters scary.  Guys dressed up in dinosaur costumes seldom scare any one. More often they appear comical. It is the element of suspense that creates the emotion. In Jurassic Park, Spielberg put great effort into making his T-Rex scary by not showing the creature until suspense had been built. He did this by first  showing its shape lit by lightning, the goat disappearing, and most importantly the vibrations in the cup from the heavy footfalls.

The scene above tries to make the dinosaur scary by making it look like a creature that’s been surprised eating its prey. It rears up and exerts its authority by roaring. 

Author Stephen King described the role of imagination in horror as: “What’s behind the door or lurking at the top of the stairs is never as frightening as the door or the staircase itself”. (Danse Macabre).

The scene above was created with puppetry using an old brown sock and a dinosaur finger puppet for the head. These were filmed against a sky/cloud  image on a computer screen. The foreground foliage was photographed out the window on a clear blue sky day. The blue sky therefore acted as a blue ‘chroma-key’ so the dinosaur appears behind. Because the trees are real they give a size perspective to the dinosaur. The voice was mine, growling into a microphone then pitch-shifted down.

Additional elements could be added to make it even scarier – use night-shot instead of daylight … have the ‘explorers’ whispering or laughing loudly ‘what dinosaur?’ … make a story line about the scene … this is the challenge.

The sinking of the Titanic.

March 1, 2010

Animation of another perennial movie blockbuster: the sinking of the Titanic.

Software was Australian Kahootz, with one of the extraterrestrial scenes used to simulate icebergs. Music from Brahms 6th. Sound of sinking made by flushing a toilet, reversing the track, then pitch-shifting down. All done from desktop.

Nautilus prepares to ram the USS Abraham Lincoln from 20,000 Leagues Under the Sea

March 1, 2010

Animation of another perennial movie blockbuster: 20,000 Leagues Under the Sea by Jules Verne

The supposed ‘sea monster’ Nautilus prepares to ram the USS Abraham Lincoln

Software was Australian Kahootz, with a zeppelin turned upside down and rust-coloured to create submarine . Music from commercial soundclip. All done from desktop. The rust colour was intended to match the spaceship set from LionHead Studios’ hollywood designer game The Movies.  Taken together, Kahootz and the Movies would allow an animation of much of the copyright-expired Jules Verne epic.

Zoom shot to Orion Nebula M42

March 1, 2010

Typical sci-fi shot of moving astronomical images. Utterly improbable given that even at the speed of light you couldn’t see this sort of movement. Warp drives and wormholes are pure sci-fi.

This is a real-life shot of the Orion Nebula M42. The movement is merely from using the zoom feature of a conventional digital movie camera on ‘night shot’ setting.  Useful technique for sci-fi or merely to highlight areas of sky.