Blender Underground

I made this same tutorial in the summer of 2008, do you think it is time for an overhaul?

Notes: I am leaving the original tutorial alone and this one is using Blender 2.57 and a little 3rd party software known as Papagayo (pronounced "PAPA GUY OH") from Lost Marble. It contains the Preston Blair phonemes by Gary C. Martin. Now the video posted below shows the finished lip sync, I could have added a BVH to the model but this is not what the tutorial is about. Also I moved her top teeth and you will notice it looks like she has dentures.



(Tutorial building in progress check back later)

PART I - SETTING UP THE WORKAREA

Get any human model for the Blender, here I am using a female that is not too low poly like before. This mesh will be easy enough to work with but you can apply this to any model... even cows. Hmm... talking cows.

You can arrange your windows however you like, just start with a 3D view, Timeline, Properties, and Outliner.



There are some tools you will need to add to get the job done:

1. Papagayo - A program that will take the audio and text and turn it into a DAT file. This file can be imported to the Blender using an addon.
2. Lip Sync Importer - If you don't find this under user settings in the addon section you can go to The Blender Wiki and download this addon.



What is really great about this, is Papagayo covers the different OS. Not just Windows and Linux but Mac as well. However, I had some trouble getting Papagayo to run in Linux so I cheated a bit and used WINE. But at least they are working on making it highly compatible.

PART II - DEFINE YOUR SHAPES

In the old tutorial I only positioned the mouth in only 4 key positions for sounds. But this time, I made all of them. You don't have to make all that is up to you. This is very important you must name your keys for the phonemes AI, O, E, U, etc, L, WQ, MBP, FV, rest. Don't confuse sounds with actual letters. These are phonics the sound made by speech. My name looks like this "FV AI etc etc AI etc etc MBP L AI etc".

Select your mesh and find this icon for "object data". Look for Shape Keys, and expand this box.



Select the plus to add the Basis (editing in basis will edit the model) keep adding the shape keys and use the Name: field to rename each one.

Now this part will consume most of your time. Select each key and reshape the mesh to conform to one of the Preston Blair phoneme series. When you are in object mode you can check your work using the Value: slider.



I am a full time student and I think I play more video games. So I am pretty sure I only spent 1 to 2 hours in total doing this over a 3 day period. Also this was a learning process you will notice I didn't make them perfect.



I had Josh (Jabhacksoul) make an attempt with a Make Human model and if you can see the clock he started at 8:45 worked for 15 minutes took a break and finished up at 11:25 with the total time of 35 minutes for 1 key. So my estimation is 4 to 6 hours for all 10 keys. But once these keys are done you never have to do them again for the same model.

PART III - NOW ON TO THE SUPER EASY LIP SYNC

Now we load a sound file into Papagayo. Then you need the text as well. For this fact, I decided to take the date story of a young lady, run it through a screen reader, and record the sound. Viola! Text and sound.

The sound file is NOT really used by Papagayo. The text is broken up over the file and you manually align the phrases and words to match. Phrases are separated by the line breaks. Make sure your FPS matches, I ran this with 24 and the Blender at 25 FPS. But the end it was so out of sync! So pay attention that your FPS match.



The video above was made by Jabhacksoul, it took about 45 minutes to align the lyrics.

There are 3 animated mouths that come with Papagayo. Once you are satisfied with your results press the export voice button and save the DAT.

Press T for tools in the 3D window of Blender. If you don't see this box you need to go to your user settings and activate it or install it.



The LipSync Importer by Yousef Harfous is fairly simple to use. I should mention it has a blinker to make eyes blink. I did not use that feature. I selected the object. set the smoothing values all to 3 and pressed "Plote Keys PLEASE".

--Insert Video Here--

I ran off an OpenGL render and noticed right away the teeth and mouth seemed silly but it works. Play with the settings, I am sure you can tweak them.

2 Glaring problems:

1- The top teeth are moving

2- It looks like you are trying to hit EVERY letter (sound) this is bad practise and it looks silly.
Just try to hit the main sounds and you'll get a more natural animation.

I agree with Steve. It certainly looks like she is about to spit out her dentures...

For older gentlemen you don't read so well or did you forget your glasses again?

Try reading my notes.

Also point 2 Steve, this is automated by the sound, so can't be helped. I guess it could be tweaked.

... also she never blinks...

Whoa! I won't comment on the teeth --unlike others who failed to read your comments-- but her mouth is the size of Kansas City! I wouldn't blame the poor guy looking disgusted, she probably swallowed his head, and I don't mean in the nice way!!

So get busy Evol Child I want to see how you did that?!?! Am I correct you started this on the 9th? I want times on how much effort was placed in this.

THANKS!! I am subscribing to this thread.

Yeah, I'm with you, Wacky.
I want to see the Tute too, even though the animation shown is a bit rough - I sure that you will polish it up just fine - hey, that's why we do things round here, if you got it perfect first time i would be suspicious !

Actually, I think with a better formed mesh that wouldn't be as noticeable. I think there is creasing going on in weird areas maybe as a result of poles?

And yes, I was mesmerized by her flapping dentures that I completely missed you acknowledging it.

Good work!!!

3D has to have 4D - time!!! and sound 5D:
http://filmsound.org/film-sound-history/

Let us help each other to make sound movies:
http://en.wikipedia.org/wiki/Sound_film
The first known public exhibition of projected sound films took place in Paris in 1900

That's what I am talking about. Only a few hours? Cool BEANS dude!!

I am giving this a try today. I will just use a make human rig.

THANKS!

If your gonna post on a public forum you should be open to constructive criticism.

Constructive criticism on an unfinished tutorial? The video was slop and I admitted that prior to the tutorial was/is finished. If you want art go to the showcase. I wasn't getting bent over the film, just that neither of you had read my comments.

And that glasses ploy is only going to go so far penix1.

Meanwhile I have Josh hard at work doing a video of his own in Windows. We are going to see if tweaking this will make a difference.

@Wacky: Yes she has a big mouth, I just nabbed the story off the internet. I wasn't aware there was a good way to get your head swall... oh shame on you! Yes, if you use a Make Human model and have all your software ready it should take only a couple of hours to make all the keys you need. The rest takes only minutes.

I am having some issues learning the new Blender, so for now I downloaded Papagayo and I can't get it to load the mp3 I have does it work with any audio or just wav?

No, that is something else I failed to mention. Import your mp3 into Audacity or other sound software and save it as a wav file format. I didn't have 'fun' working with papagayo because of its control issue. The scrub is a nice feature, but they shouldn't make it work in reverse.

Also be very careful with the order of editing! Match the length of the phrase first (green bar) and then the individual words. Resizing the green bar will reset all words. I used a music number to show that the audio doesn't effect the lip sync.

My video capture rate was too low for that video, I should have set it higher.

I am not as fast as JAB there, but I've got 3 keys done.

I have my DAT file ready. When I get a video, I will post a link.