Sample Code for a lip sync animation using LipSyncAS3 v1

Few years back I was using FlashAmp tool to create amplitude based lip sync for Flash characters to use in mini games and e-learning projects. basically it was a tool that allows to get the array of amplitude value from the mp3 voice over files and then using this array I could sync the movieclip containing lip  of the character to the mp3 voice being played. But this was never dynamic…that means if you change the mp3 file, you need to pre-process the audio file and update the array again.  Now things have changed.

With LipSyncAS3, similar effects can be achieved, but there is no more pre-processing required.  LipSyncAS3 is an easy to use development kit to integrate audio amplitude based runtime lip sync to your Flash characters/animations. The LipSyncAS3 does not require you to preprocess your audio MP3 files by any tool, rather it is based on amplitude based lip-sync generated in real-time. The LipSyncAS3 does not even require any server to render lips , so any animation/interactive solution using this can be delivered via CD/DVD for playing locally on a PC, and therefore is a good tool to be used in e-learning and CBT/WBTs.

Note that, as the lip sync is not based on visme or phoneme set , the lip sync is not accurate to details. However it is good enough to be added to small interactive projects, web animation, e-learning projects like SCORM/CBT/WBT where interactive lip sync is required for voice mp3.

The lip sync engine gives the designer/animators the freedom to design their own lip images with different positions in reference to the provided template, which is rendered in their Flash movies via without need to pre-process the voice audio in any manner.

Many projects like the talking animals/birds at www.mgreetings.com use this LipSyncAS3 to do the lipsync parts.

Following is what exactly you need to be able to do a basic lip sync project using this kit:

First step is to get the kit (Trial version) from http://patterngraphic.com/detailspage.php?product=PG001. This is a full working version with a trial version message appearing.

System Requirements

LipSyncAS3 is a purely Flash AS3 based solution. It requires your Flash movies to be published for at least Adobe Flash Player 9 or above. The Flash Movies must be compiled with ActionScript 3. To use the LipSyncAS3 kit you need Adobe Flash CS3 or above IDE to integrate to your Flash movies. You can also integrate your lip sync movies to Flex/Flash Builder applications.

The audio files to be used as voice must be MP3 files as Flash only supports MP3 files to be loaded dynamically.

To run your Flash movies using LipSyncAS3 needs your Flash movie to be compiled as a projector file (.exe in windows) to run on a desktop locally. To run it from web you can use the swf version with publish setting with networking privilege

LipSyncAS3 Development Kit consist of the following basic files and folders

Lip Engine.swf : This is the lip engine that has to be loaded into your movie. This engine does not have any visual parts visible in the stage. Rather it helps to run any lip pattern or the lip graphic you have designed to render in real-time.

LipPattern.swf: This is the graphic representation of the lip to suit your animated character, that is rendered on the screen. You can create your own custom lip set based on a template FLA provided. The lip pattern can have maximum of 10 lip positions as shown below where the lip position image set at the frame 1 of the timeline will refer to a lip in silent position. From frame 2 on ward the lip image must show the opened positions based on the amplitude ..like the frame 3 will have the lip more opened than the one in frame 2. similarly in frame 10 the lip will have the lip image with widest/longest open lip position.


Note that depending on the mp3 audio quality, voice type, you may need to fine tune your lip design to match the needs.

The final out put folder may contain similar files as shown in the image below:

Now the available methods in the LipSyncAS3 is as follows:

SetLipPattern(lipPatternSource:String, lipLicenseEmail:String, lipLicenseSerial:String, onLipPatternLoaded:Function) :void

This method is called at the beginning, to initiate the lip engine. This method does the following: • It validates provided serial number • Loads Lip engine • loads your custom lip pattern.

lipPatternSource:String is path to your lip pattern swf file path;

lipLicenseEmail:String is your registered email id while purchasing a license of this product.

lipLicenseSerial:String is your purchased license/serial number.

onLipPatternLoaded:Function is the method which you can use to trigger anything when LipPattern finishes loading

In case you want to try the product use “TRIAL” (case sensitive) as your lipLicenseSerial and in case of registered email id use any of your email id. Running a trial version will not affect any of the functionality. It will only add a message to your flash movie mentioning that this is a trail version. Note trial version is only for the purpose of reviewing product.

SetVoicePath(Mp3FilePath:String, onVoiceLoadComplete:Function, onVoicePlayComplete:Function, onID3Load:Function, onIOError:Function, onVoiceLoadProgress:Function ) :void

This method sets the mp3 file path which contains the voice for lip sync.

This methods accepts number of parameters out of which all but the first one are optional

Mp3FilePath:String is the MP3 file path

onVoiceLoadComplete:Function is a function where you define what to happen when MP3 load is complete.

onVoicePlayComplete:Function is a function where you define what to happen when voice playing is complete.

onID3Load:Function is a function where you define what to happen when voice ID3 details are loaded from the mp3 file. For example you can then call the GetID3Info():ID3Info to get the ID3 details like artist name, album name, year and so on.

onIOError:Function is a function where you define what to happen when mp3 file could not be loaded.

onVoiceLoadProgress:Function is a function where you define what to happen while loading of mp3 continues. For example you can check for total loaded bytes to show a loading progress.

PlayLip():void This methods start playing already loaded mp3 and along with it starts rendering the lip. If the mp3 was paused using PauseLip(), then it resumes the playback from the last paused position.

PauseLip():void
This methods stops the playback of mp3 and stops the lip movement.

RePlayLip():void
This methods start playing already loaded mp3 from the beginning.

SetBufferTime (seconds:Number) :void
The default buffer time for the sound object to load mp3 is set to 1000 miliseconds. You can change this by using this method.

Seek(milliseconds:Number):void
This moves the play head to a particular position (in miliseconds) in mp3 to play. This can be useful in creating progress bar for the playback to control where to start playing .

GetID3Info():ID3Info
This returns an ID3Info object for the playing mp3 file (if the ID3 details are present)

Example :
var tempID3InfoObj:ID3Info = MovieClip(LipLoader.content).GetID3Info();
trace(“ID3 Loaded”); trace(“artist : “+tempID3InfoObj.artist); trace(“track : “+tempID3InfoObj.track); trace(“comment : “+tempID3InfoObj.comment); trace(“songName : “+tempID3InfoObj.songName); trace(“album : “+tempID3InfoObj.album); trace(“genre : “+tempID3InfoObj.genre); trace(“year : “+tempID3InfoObj.year);

GetBytesLoaded():Number
This returns the number of bytes loaded for the mp3

GetBytesTotal():Number
This returns the total bytes (file size) for the mp3 being loaded.

SetVoiceVolume(volumeValue:Number)
This sets the volume of the target voice object for any particular mp3. Each time a new mp3 is loaded the volume sets to default value., and in that case you can set the volume to your desired level using this method.

SetVoicePan(panValue:Number)
This sets the pan value for the target voice object.

GetVoiceVolume():Number
This returns the volume value.

GetVoicePan():Number
This returns the pan value.

Now you can get the sample code files from


The sample zip archive contains one sample template FLA file with example usage of most of the available public methods. In addition a sample Pocast Reader application FLA file is also included in the archive, in which a character’s lipsync is rendered as per the pocast voice. Also there are 2 sample lip pattern FLA files, which you can use to create your own lips.

http://www.patterngraphic.com/samplefilesdownload.php?product=PG001 to download the zip archive with the sample files.

If you open the sample files archive you can see the following code which is self explanatory:

/*

SAMPLE TEMPLATE FOR LOADING AND CONTROLING LipSyncAS3 v1 Engine

------------------------------------------------------------------------

(c)2009, www.PatternGraphic.com, All rights reserved.

------------------------------------------------------------------------

This template provides most of the available public methods of the LipSyncAS3 engine.

You can use this sample template to create your own lip sync enabled movies.

You need to publish your movie for Action Script 3 and aFlash Player 9 or above.

*/

import flash.media.ID3Info;

//LIPSYNC CONFIG CODE

var lipPatternSource:String ="LipPattern/lip_pattern_puppet.swf";

var lipEngineSource:String = "LipEngine/lipsyncas3v1_lip_engine_network.swf";

var lipLicenseEmail:String = "support@patterngraphic.com";//Use your registered email id here

var lipLicenseSerial:String = "TRIAL"; //Use your purchased license serial number here.

//Create a Loader Movie clip to load the lip

var LipLoader:Loader = new Loader();

//Now attach the lip loader to stage (you can attach it to your character's face movie clip too)

this.puppet.addChild(LipLoader);

//Now position it where ever you want

LipLoader.x = 25;

LipLoader.y = 120;

//Now load the lip engine swf file into your loader

var bgURL:URLRequest = new URLRequest(lipEngineSource);

LipLoader.contentLoaderInfo.addEventListener(Event.COMPLETE, lipLoaded);

LipLoader.load(bgURL);

function onLipPatternLoaded():void {

//the method is to trigger something when lip pattern is finished loaded. helpful when lip pattern is a big file.

trace("Lip Pattern is finished loading")

}

function lipLoaded(event:Event):void

{

//the following command loads the specific lip pattern

MovieClip(LipLoader.content).SetLipPattern(lipPatternSource,lipLicenseEmail,lipLicenseSerial,onLipPatternLoaded);

}

//Now to control the lip sync add some buttons (you can use your own method to control it)

loadBtn.addEventListener(flash.events.MouseEvent.CLICK, LoadVoice);

loadBtn2.addEventListener(flash.events.MouseEvent.CLICK, LoadVoice2);

function LoadVoice(e:MouseEvent):void {

//the following command loads the mp3 containing the voice to the lip

resetProgressBar ();

MovieClip(LipLoader.content).SetVoicePath("mp3/demo.mp3", onVoiceLoadComplete,onVoicePlayComplete, onID3Load, onIOError, onVoiceLoadProgress );

}

function LoadVoice2(e:MouseEvent):void {

//the following command loads the mp3 containing the voice to the lip

MovieClip(LipLoader.content).SetVoicePath("mp3/0.mp3", onVoiceLoadComplete,onVoicePlayComplete, onID3Load, onIOError, onVoiceLoadProgress );

}

//Now to control the lip sync add some buttons (you can use your own method to control it)

playBtn.addEventListener(flash.events.MouseEvent.CLICK, PlayVoice);

function PlayVoice(e:MouseEvent):void {

//the following command plays the sound and starts the lip sync activity

MovieClip(LipLoader.content).PlayLip();

}

pauseBtn.addEventListener(flash.events.MouseEvent.CLICK, PauseVoice);

function PauseVoice(e:MouseEvent):void {

//the following command pauses the lip

MovieClip(LipLoader.content).PauseLip();

}

restartBtn.addEventListener(flash.events.MouseEvent.CLICK, RestartVoice);

function RestartVoice(e:MouseEvent):void {

//the following command pauses the lip

MovieClip(LipLoader.content).RePlayLip();

}

function onVoiceLoadProgress():void {

var bytestotal:Number = MovieClip(LipLoader.content). GetBytesTotal();

var bytesloaded:Number = MovieClip(LipLoader.content).GetBytesLoaded();

if(bytestotal>0 ){

updateProgressBar (bytestotal, bytesloaded, 0, 0);

}

}

function onVoiceLoadComplete():void {

trace("Loading of Voice Completed")

//GetVoiceLength():Number will return the length  of the loaded audio

var voiceLength:Number = MovieClip(LipLoader.content).GetVoiceLength();

trace(voiceLength);

//auto play from the begining of the newly loaded audio when the load completes

MovieClip(LipLoader.content).RePlayLip();

}

function onVoicePlayComplete():void {

trace("the voice playing completed!")

}

function onID3Load():void {

//GetID3Info():ID3Info  returns the ID3 object of the loaded voice

var tempID3InfoObj:ID3Info = MovieClip(LipLoader.content).GetID3Info();

trace("ID3 Loaded");

trace("artist : "+tempID3InfoObj.artist);

trace("track  : "+tempID3InfoObj.track);

trace("comment  : "+tempID3InfoObj.comment);

trace("songName  : "+tempID3InfoObj.songName);

trace("album  : "+tempID3InfoObj.album);

trace("genre  : "+tempID3InfoObj.genre);

trace("year  : "+tempID3InfoObj.year);

}

function onIOError():void {

trace("Some error occoured in loading voice!")

}

//------------------------------Additional

positionBtn.addEventListener(flash.events.MouseEvent.CLICK, GetPlaybackProgress);

function GetPlaybackProgress(e:MouseEvent):void {

//GetChennelPosition() returns the current channel position of the playing audio

var tempPosition: Number = MovieClip(LipLoader.content).GetChennelPosition();

trace(tempPosition)

}

function GetBufferTime():void {

MovieClip(LipLoader.content).GetBufferTime ();

}

function SetBufferTime():void {

//Note: default buffer time is set to 1000;

MovieClip(LipLoader.content).SetBufferTime (8000);

}

//----------------------PROGRESS BAR-----------------------------

this.addEventListener(Event.ENTER_FRAME, checkPlaybackProgress);

function checkPlaybackProgress(e:Event){

if(MovieClip(LipLoader.content) != null){

var tempPosition: Number = MovieClip(LipLoader.content).GetChennelPosition();

var voiceLength:Number = MovieClip(LipLoader.content).GetVoiceLength();

var bytestotal:Number = MovieClip(LipLoader.content). GetBytesTotal();

var bytesloaded:Number = MovieClip(LipLoader.content).GetBytesLoaded();

var voicelength:Number = MovieClip(LipLoader.content).GetVoiceLength();

if(bytestotal>0){

updateProgressBar (bytestotal, bytesloaded, tempPosition,voicelength );

}

}

}

function resetProgressBar ():void {

progressBar.progressBarPlayed.width = 0;

progressBar.progressBarLoaded.width = 0;

}

function updateProgressBar(BytesTotal:Number , BytesLoaded:Number, voicePosition:Number, voicelength:Number):void{

if(BytesTotal>0 ){

progressBar.progressBarPlayed.width = voicePosition/voicelength*progressBar.progressBarBG.width;

progressBar.progressBarLoaded.width = BytesLoaded*(progressBar.progressBarBG.width/BytesTotal);

} else{

resetProgressBar ()

}

}

resetProgressBar();

//---------------------------------------ADDITIONAL METHODS -------------------------------------

seekBtn.addEventListener(flash.events.MouseEvent.CLICK, Seek);

function Seek(e:MouseEvent):void {

//Note: default buffer time is set to 1000;

MovieClip(LipLoader.content).Seek(7312);

}

bytesLoadedBtn.addEventListener(flash.events.MouseEvent.CLICK, bytesLoaded);

function bytesLoaded(e:MouseEvent):void {

trace(MovieClip(LipLoader.content).GetBytesLoaded());

}

totalBytesBtn.addEventListener(flash.events.MouseEvent.CLICK, totalBytes);

function totalBytes(e:MouseEvent):void {

trace(MovieClip(LipLoader.content). GetBytesTotal());

}

setVolBtn.addEventListener(flash.events.MouseEvent.CLICK, setVol);

function setVol(e:MouseEvent):void {

MovieClip(LipLoader.content). SetVoiceVolume(5);

}

setPanBtn.addEventListener(flash.events.MouseEvent.CLICK, setPan);

function setPan(e:MouseEvent):void {

MovieClip(LipLoader.content). SetVoicePan(-2);

}

getVolBtn.addEventListener(flash.events.MouseEvent.CLICK, getVol);

function getVol(e:MouseEvent):void {

trace(MovieClip(LipLoader.content). GetVoiceVolume());

}

getPanBtn.addEventListener(flash.events.MouseEvent.CLICK, getPan);

function getPan(e:MouseEvent):void {

trace(MovieClip(LipLoader.content). GetVoicePan());

}
Advertisements

One thought on “Sample Code for a lip sync animation using LipSyncAS3 v1

  1. Hi there Friends,
    I’m suppose to submit some sort of animated promotional video to my buye on monday unfortunately the animation studio i was dealing with ditched me at last moment. I’m sure it’s none of your concern but i’m going to be really greatful if anyone can recommend a superior animation studio who are able to work at tight deadlines and have good history. Thanks in advance.
    Luv
    Merissa

Comments are closed.