Friday, March 28, 2008

How to make your own motion-capture-video in Maya

What we are about to do is: create a jmyron blob detection, sending data with oscP5 to a server (localhost or other IP address), reading these data with my sliBase plugin to move object in realtime into Maya.
To make it easy, we are about to create an home-made low-cost motion-capture-video system for Maya.
Take a look at this video
http://www.youtube.com/watch?v=GtxATrZCIYg
I used my motion-capture-video I developed in Java but you can do the same think using JMyron, oscP5 and Processing.

Go to www.processing.org

  • download Processing for your operating system
  • install processing
  • try to load an example to see if it works
Go to http://webcamxtra.sourceforge.net/
  • download the library for Procesing
  • install it following the instructions
  • if your are running Processing on a mac-intel you have to change a file
    • download and replace libJMyron.jnilib
Go to http://www.sojamo.de/libraries/oscP5/
  • download the library
  • install it
Open Processing.
Go to examples and try to load JMyron_simpleCamera to check it it works
if you see your webcam-video all it's ok! good! ^_^
otherwise, mail me what happens... ;)
Create a new project in Processing and save as "oscP5sendReceive" and copy this example:

/**
* oscP5sendreceive by andreas schlegel
* example shows how to send and receive osc messages.
* oscP5 website at http://www.sojamo.de/oscP5
*/

import oscP5.*;
import netP5.*;

OscP5 oscP5;
NetAddress myRemoteLocation;

void setup() {
size(400,400);
frameRate(25);
/* start oscP5, listening for incoming messages at port 12000 */
oscP5 = new OscP5(this,12000);

/* myRemoteLocation is a NetAddress. a NetAddress takes 2 parameters,
* an ip address and a port number. myRemoteLocation is used as parameter in
* oscP5.send() when sending osc packets to another computer, device,
* application. usage see below. for testing purposes the listening port
* and the port of the remote location address are the same, hence you will
* send messages back to this sketch.
*/
myRemoteLocation = new NetAddress("127.0.0.1",12000);
}


void draw() {
background(0);
}

void mousePressed() {
/* in the following different ways of creating osc messages are shown by example */
OscMessage myMessage = new OscMessage("/test");

myMessage.add(123); /* add an int to the osc message */

/* send the message */
oscP5.send(myMessage, myRemoteLocation);
}


/* incoming osc message are forwarded to the oscEvent method. */
void oscEvent(OscMessage theOscMessage) {
/* print the address pattern and the typetag of the received OscMessage */
print("### received an osc message.");
print(" addrpattern: "+theOscMessage.addrPattern());
println(" typetag: "+theOscMessage.typetag());
}


Execute it and when you click on the black window you can see the received message into the output-console in the Processing window.
Now, close Processing.
Go to http://slidertime.blogspot.com/search/label/_my%20downloads
  • download my latest sliBase plugin for maya (only for Windows)
  • install it
This plugin is only for Windows, but you can capture data from a macosx and send to windows through ethernet using oscP5 in Processing.

This is the system I have used for this tutorial:
Motion capture video system:
Macbook1,1 Intel black 2GB ram, 2GHz CPU with Leopard and integrated iSight
Processing 0135 BETA for MacOSX
JMyron 0025 for Processing
modded libJMyron.jnilib for mac-intel
oscP5 0.9.3 for Processing
Java version 1.5.0_13 (preinstalled in Leopard, you don't need to install)


Maya receiver videodata system:
Alienware Aurora M77 with 2GB ram, AMD X2 2GHz, Geforce Go 7900GTX with WindowsXP Home edition
Maya 8.5
sliBase plugin


Now connect your two machine with a network cable or a wifi or ensure that are both into the same LAN with the same IP address with the last number changed.
I've used a network cable (not crossed, if you don't have it. actually the net-card can exchange RX and TX automatically).
macbook with 192.168.0.1 and alienware with 192.168.0.2

Then, open Processing and reopen oscP5sendReceive example.

make these changes

BEFORE
myRemoteLocation = new NetAddress("127.0.0.1",12000);
AFTER
myRemoteLocation = new NetAddress("192.168.0.2",57110);

BEFORE
OscMessage myMessage = new OscMessage("/test");
AFTER
OscMessage myMessage = new OscMessage("/slibase");

BEFORE
myMessage.add(123); /* add an int to the osc message */
AFTER
myMessage.add(mouseX + " " + mouseY);

Now, run it.
Open maya, execute sliBase and open the script for OSC, as read on the instructions
Start it.
Try to move mouse on the black window in Processing and take a look in Maya, you can see something received into sliBase plugin.
If you can see your mouse coords changing into sliBase, it's all ok! ^_^ good!!!!
otherwise, mail me..... ;)

REMEMBER to run FIRST Processing, THEN start sliBase in Maya, THEN stop sliBase and THEN close Processing!!

Now, oscP5 works, JMyron works, sliBase works... we have to mix them!

Open Processing and create a new project and save it as "sliBaseJMyronOscP5"
and copy the following code:

import oscP5.*;
import netP5.*;
import JMyron.*;

JMyron m;
OscP5 oscP5;
NetAddress myRemoteLocation;

void setup() {
int w = 320;
int h = 240;
size(w,h);
m = new JMyron();
m.start(w,h);
m.findGlobs(1);
frameRate(25);
oscP5 = new OscP5(this,57110);
myRemoteLocation = new NetAddress("192.168.0.2",57110);
ellipseMode(CENTER);
}

void draw() {
m.trackColor(255,255,255,255);
m.update();
int[] img = m.image();
loadPixels();
for(int v = 0; v < width*height; v++)
pixels[v] = img[v];
updatePixels();

int[][] a;

a = m.globCenters();
if (a.length >= 2) { // send just first 2 blob
int[] b1 = a[0];
int[] b2 = a[1];
ellipse(b1[0],b1[1],10,10);
ellipse(b2[0],b2[1],10,10);
OscMessage myMessage = new OscMessage("/slibase");
myMessage.add(b1[0]+" "+b1[1]+" "+b2[0]+" "+b2[1]);
oscP5.send(myMessage, myRemoteLocation);
} else { // I need this because my sliBase crash if doesn't receive data from OSC
OscMessage myMessage = new OscMessage("/slibase");
myMessage.add("0 0 0 0");
oscP5.send(myMessage, myRemoteLocation);
}

}

public void stop(){
m.stop();
super.stop();
}




Now, run it.
Open Maya and launch the plugin and the OSC-MEL script.
You should see into the message four zeros.
Take 2 white little pieces of paper and try to show to your webcam.
You should see two little circles in Processing and the centers of these circles into the sliBase.
Now all is done.
Assign these values to any object attributes in Maya using sliBase and look what happens... :)

enjoy!



6 comments:

Anonymous said...

EhilĂ !!! Credo di non averci capito nulla... ma credo d'essere sulla strada buona. Ti spiego.
Vorrei processare in real-time un video in entrata in modo da ottenere un effetto simile a questo: http://setpixel.com/content/?ID=ghostlymirror
Ho installato processing e inizierò a studiarmi il codice... ma mi sfugge il meccanismo generale, o meglio... cosa mi serve in termini di software?

Anonymous said...

hi :D I wanted to ask if you know what to do when i try to start processing and a window displays saying processing cannot run cause it could not create a folder to store your settings.
I´m starting on this and i simply got astonished with the potencials.
If you could say something i would appreciate, if you don´t i appreciate it anyway.
By the way keep up with the good work
stay well

gabrielknight said...

hi. iam having a problem with sliBase_vo.2
first of all I click on maya v8.5 then click on window select setting preferences then plug in manager.
afterwards i click on browse load sliBase.mll it shows plugin manage then i click close
I open the folder named sliBase_vo.2
and drag sliBase_getOSC_windowmel
to maya v8.5 i get a box named slibase_getdata from OSC then i click start
but what happens is that the programme freeze and maya freeze and cant do anything so if you can help me with this problem and thanks (i use window sp 2 )

Anonymous said...

hi,
i have this message when executing oscP5sendReceive example.
"cannot find a class or type named oscP5"
and this lineis highlighted:
OscP5 oscP5;
thanks

DQvsRA said...

Hello.
I have a quastion about how to load a sliBase into Maya 2009? It's don't loaded, may be thats why i have error message
"Cannot find procedure "sliCmd_toggleOSC".
Start of trace: (command window: line 1).
sliCmd_toggleOSC (command window: line 1)."
in Maya, when try to start a sliBase.
Can you help me in this?

The_sLideR said...

UPDATE: due to an error by the HTML interpreter which was changing some C++ into HTML tags, this code was returning some syntax errors.
Now it has been fixed (I guess).
It should be everything OK.
Thanks for you patience and tests ;)