|
|
| (45 intermediate revisions by 2 users not shown) |
| Line 1: |
Line 1: |
| − | == Gesture Follower tutorial == | + | == What is the ''gesture follower'' ? == |
| | | | |
| − | The gesture follower is a set of Max/MSP modules integrated in the toolbox MnM of the library FTM. The development of the gesture follower is pursued with the general goal to compare in realtime a gesture with prerecorded examples. The comparison mechanisms we implemented, following and recognition are further explained in the next subsections. | + | The ''gesture follower'' is a ensemble of Max/MSP modules to perform gesture recognition and following in real-time. It is integrated in the toolbox MnM of the library FTM (see dowload). The general idea behind it is to get parameters from the comparison between a performance and an set of prerecorded examples. |
| | | | |
| − | == Workspace : overview ==
| + | The gesture follower can guess the two following questions: |
| | + | * which gesture is it ? (if you don't like black and white answers, you can get "greyscale" answers: how close are you from the recorded gestures ? ) |
| | + | * where are we ? (beginning, middle or end of the gesture) |
| | | | |
| − | Get an overview of the interface functions.
| + | == What is a ''gesture'' anyway ? == |
| − | [[Image:Example.jpg]]
| |
| | | | |
| | + | A gesture here can be any ''multi-dimensional temporal curve'', sampled at relatively low frequency compared to sound. With the current implementation in Max/MSP the frequency sampling period must be at least 1 milisecond, but typically, 10-20 milisecond is recommended. There are no upper limit (if you have time...). |
| | | | |
| − | == 1st STEP : Record gestures ==
| + | There is no technical limit for the dimension of the gesture data (or number of sensor channel) other than what your computer can afford as a CPU load (for example 20 is generally no problem). |
| | | | |
| − | Let’s start with two simple drawings : a triangle and a circle.
| + | In Max/MSP the data feeding the ''gesture follower'' can be taken from a list, for example: |
| − | [[Image:Example2.jpg]]
| |
| | | | |
| | + | * sound parameters (pitch, amplitude, etc) |
| | + | * mouse, joystick coordinates |
| | + | * parameters from video tracking (EyesWeb, Jitter, etc) |
| | + | * Wiimote |
| | + | * MIDI |
| | + | * any sensors data, etc... |
| | + | * any combination of the above (you said multimodal ?) |
| | | | |
| − | == 2nd STEP : Compare == | + | == Download, license and referencing== |
| | + | The ''gesture follower'' comes freely with the download of [[http://ftm.ircam.fr/index.php/Download FTM]], in the folder .../MnM.BETA/examples/gesture_follower/. Note that you must have FTM installed. |
| | | | |
| − | Draw a figure and then see how similar it is with your two referent drawings.
| + | The latest version, v0.3 can be downloaded here [http://recherche.ircam.fr/equipes/temps-reel/gesturefollower/gesture_follower_v0.3.zip v0.3] |
| − | [[Image:Example3.jpg]] | |
| | | | |
| | + | This software is intended for artistic work and/or scientific research. Any commercial use is reserved. Copyrights 2004-2007 IRCAM - Centre Pompidou. |
| | | | |
| − | == 3rd STEP : Observe ==
| + | If appropriate please cite the Real Time Interaction Team, IRCAM or reference the following article: |
| | + | F. Bevilacqua, F. Guédy, N. Schnell, E. Fléty, N. Leroy, [http://mediatheque.ircam.fr/articles/textes/Bevilacqua07a/ Wireless sensor interface and gesture-follower for music pedagogy], Proceedings of the International Conference of New Interfaces for Musical Expression (NIME 07), New York,NY, USA, pp 124-12, 2007. |
| | | | |
| − | Pay attention to the curves below. They represent the velocity in X and Y axis of the mouse trajectories. That give a useful temporal information on how you realize your drawing.
| + | == Getting help and disclaimer == |
| − | [[Image:Example4.jpg]] | + | Any type of feedback, problem, bug report, feature request are welcome and we will try our best to help you. |
| | + | Please post any message/questions directly to the [http://listes.ircam.fr/wws/info/ftm FTM list]. |
| | | | |
| | + | Nevertheless, ...this is work in progress!!! Use this software at your own risk. We do not assume any reponsability for possible problems caused by the use of this software. |
| | | | |
| | + | == References== |
| | | | |
| − | == Connection Avec EyesWeb XMI ==
| + | The messages used the various modules are listed here [http://recherche.ircam.fr/equipes/temps-reel/gesturefollower/gesturefollower-reference.01.pdf ref] |
| | | | |
| − | EyesWeb XMI, the open platform for real-time analysis of multimodal interaction, can be connected to Max/Msp throughout the OSC protocol (Open Sound Control). OSC is open, message-based protocol which was originally developed for communication between computers and sythesizers (cf. wiki).
| |
| | | | |
| − | == Test == | + | == Getting started == |
| | | | |
| − | <html>
| + | === LCD example=== |
| − | <head>
| |
| − | <title>jit.matrix</title>
| |
| − | </head>
| |
| − | <body bgcolor=#F7F7F7 link=#000000 vlink=#000000>
| |
| − | <table><tr><td><a href="index.html"><img src="images/jitter_smallest.gif" border=0></a><br></td></tr></table>
| |
| − | <table border=0 width=100%>
| |
| − | <tr>
| |
| − | <td bgcolor=#CCCCCC width=30% valign=bottom>
| |
| − | <font size=+3 face="Times"><b>jit.matrix</b></font>
| |
| − | </td>
| |
| − | <td bgcolor=#DDDDDD width=70% align=right><font size=+1 face="Times">
| |
| | | | |
| − | The Jitter Matrix!
| + | ==== 1st step : Record gestures ==== |
| − | </font>
| |
| − | </td>
| |
| − | </tr>
| |
| − | </table>
| |
| − | </p>
| |
| − | <font face="Times">
| |
| | | | |
| − | The <b>jit.matrix</b> object is a named matrix which may be used to
| + | Let’s start with two simple drawings : a triangle and a circle. |
| − | matrix data storage and retrieval, resampling, and
| + | [[Image:Example2.jpg]] |
| − | matrix type and planecount conversion operations.
| |
| − | </font>
| |
| − | <p>
| |
| − | <table border=0 width=100%>
| |
| − | <tr>
| |
| − | <td bgcolor=#CCCCDD><font size=+1 face="Times"><a href="group-attributes.html"><b>Attributes:</b></a></font></td>
| |
| − | </tr>
| |
| − | </table>
| |
| | | | |
| − | <table border=0 width=100%>
| |
| − | <tr>
| |
| − | <td bgcolor=#CCCCCC width=10%><font face="Times"><b>Name</b></font></td>
| |
| − | <td bgcolor=#CCCCCC width=5%><font face="Times"><b>Type</b></font></td>
| |
| − | <td bgcolor=#CCCCCC width=85%><font face="Times"><b>Description</b></font></td>
| |
| − | </tr>
| |
| − | <tr>
| |
| − | <td bgcolor=#D7CCCC valign=top><font face="Courier">adapt</font></td>
| |
| − | <td bgcolor=#DDDDDD valign=top><font face="Courier">int</font></td>
| |
| − | <td bgcolor=#D7D7CC valign=top>
| |
| − | <font face="Times">
| |
| | | | |
| − | Matrix adaptation flag (default = 0 if matrix arguments are present, otherwise 1)
| + | ====2nd sep : Compare ==== |
| − | When the flag is set, the <b>jit.matrix</b> object will adapt to the incoming matrix planecount, type, and dimensions.
| |
| − | </font>
| |
| − | </td>
| |
| − | </tr>
| |
| − | <tr>
| |
| − | <td bgcolor=#D7CCCC valign=top><font face="Courier">dimstride</font> <font face="Times">(get)</font></td>
| |
| − | <td bgcolor=#DDDDDD valign=top><font face="Courier">int list[32]</font></td>
| |
| − | <td bgcolor=#D7D7CC valign=top>
| |
| − | <font face="Times">
| |
| | | | |
| − | The byte stride per dimension
| + | Draw a figure and then see how similar it is with your two referent drawings. |
| − | </font>
| + | [[Image:Example3.jpg]] |
| − | </td>
| |
| − | </tr>
| |
| − | <tr>
| |
| − | <td bgcolor=#D7CCCC valign=top><font face="Courier">dim</font></td>
| |
| − | <td bgcolor=#DDDDDD valign=top><font face="Courier">int list[32]</font></td>
| |
| − | <td bgcolor=#D7D7CC valign=top>
| |
| − | <font face="Times">
| |
| | | | |
| − | The dimensions of matrix data (default = 1 1)
| |
| − | </font>
| |
| − | </td>
| |
| − | </tr>
| |
| − | <tr>
| |
| − | <td bgcolor=#D7CCCC valign=top><font face="Courier">dstdimend</font></td>
| |
| − | <td bgcolor=#DDDDDD valign=top><font face="Courier">int list[32]</font></td>
| |
| − | <td bgcolor=#D7D7CC valign=top>
| |
| − | <font face="Times">
| |
| | | | |
| − | The destination dimension end position (default = all <font face="Courier">dim</font> values minus 1)
| + | ==== 3rd step : Observe ==== |
| − | </font>
| |
| − | </td>
| |
| − | </tr>
| |
| − | <tr>
| |
| − | <td bgcolor=#D7CCCC valign=top><font face="Courier">dstdimstart</font></td>
| |
| − | <td bgcolor=#DDDDDD valign=top><font face="Courier">int list[32]</font></td>
| |
| − | <td bgcolor=#D7D7CC valign=top>
| |
| − | <font face="Times">
| |
| | | | |
| − | The source dimension start position (default = all 0)
| + | Pay attention to the curves below. They represent the velocity in X and Y axis of the mouse trajectories. That give a useful temporal information on how you realize your drawing. |
| − | </font>
| + | [[Image:Example4.jpg]] |
| − | </td>
| |
| − | </tr>
| |
| − | <tr>
| |
| − | <td bgcolor=#D7CCCC valign=top><font face="Courier">interp</font></td>
| |
| − | <td bgcolor=#DDDDDD valign=top><font face="Courier">int</font></td>
| |
| − | <td bgcolor=#D7D7CC valign=top>
| |
| − | <font face="Times">
| |
| | | | |
| − | Matrix interpolation flag (default = 0)
| |
| − | When the flag is set, the input matrix will be interpolated
| |
| − | when copied to the internal matrix.
| |
| − | </font>
| |
| − | </td>
| |
| − | </tr>
| |
| − | <tr>
| |
| − | <td bgcolor=#D7CCCC valign=top><font face="Courier">name</font></td>
| |
| − | <td bgcolor=#DDDDDD valign=top><font face="Courier">symbol</font></td>
| |
| − | <td bgcolor=#D7D7CC valign=top>
| |
| − | <font face="Times">
| |
| | | | |
| − | The name of the matrix (default = UID)
| |
| − | </font>
| |
| − | </td>
| |
| − | </tr>
| |
| − | <tr>
| |
| − | <td bgcolor=#D7CCCC valign=top><font face="Courier">planecount</font></td>
| |
| − | <td bgcolor=#DDDDDD valign=top><font face="Courier">int</font></td>
| |
| − | <td bgcolor=#D7D7CC valign=top>
| |
| − | <font face="Times">
| |
| | | | |
| − | The number of planes in matrix data (default = 4)
| + | ==== Connection Avec EyesWeb XMI ==== |
| − | </font>
| |
| − | </td>
| |
| − | </tr>
| |
| − | <tr>
| |
| − | <td bgcolor=#D7CCCC valign=top><font face="Courier">planemap</font></td>
| |
| − | <td bgcolor=#DDDDDD valign=top><font face="Courier">int list[32]</font></td>
| |
| − | <td bgcolor=#D7D7CC valign=top>
| |
| − | <font face="Times">
| |
| | | | |
| − | Maps input places to output planes (default = 0 1 2 3 ...)
| + | EyesWeb XMI, the open platform for real-time analysis of multimodal interaction, can be connected to Max/Msp throughout the OSC protocol (Open Sound Control). OSC is open, message-based protocol which was originally developed for communication between computers and sythesizers (cf. wiki). |
| − | </font>
| |
| − | </td>
| |
| − | </tr>
| |
| − | <tr>
| |
| − | <td bgcolor=#D7CCCC valign=top><font face="Courier">size</font> <font face="Times">(get)</font></td>
| |
| − | <td bgcolor=#DDDDDD valign=top><font face="Courier">int</font></td>
| |
| − | <td bgcolor=#D7D7CC valign=top>
| |
| − | <font face="Times">
| |
| − | | |
| − | Total byte size of matrix
| |
| − | </font>
| |
| − | </td>
| |
| − | </tr>
| |
| − | <tr>
| |
| − | <td bgcolor=#D7CCCC valign=top><font face="Courier">srcdimend</font></td>
| |
| − | <td bgcolor=#DDDDDD valign=top><font face="Courier">int list[32]</font></td>
| |
| − | <td bgcolor=#D7D7CC valign=top>
| |
| − | <font face="Times">
| |
| − | | |
| − | The source dimension end position (default = all <font face="Courier">dim</font> values minus 1)
| |
| − | </font>
| |
| − | </td>
| |
| − | </tr>
| |
| − | <tr>
| |
| − | <td bgcolor=#D7CCCC valign=top><font face="Courier">srcdimstart</font></td>
| |
| − | <td bgcolor=#DDDDDD valign=top><font face="Courier">int list[32]</font></td>
| |
| − | <td bgcolor=#D7D7CC valign=top>
| |
| − | <font face="Times">
| |
| − | | |
| − | The source dimension start position (default = all 0)
| |
| − | </font>
| |
| − | </td>
| |
| − | </tr>
| |
| − | <tr>
| |
| − | <td bgcolor=#D7CCCC valign=top><font face="Courier">thru</font></td>
| |
| − | <td bgcolor=#DDDDDD valign=top><font face="Courier">int</font></td>
| |
| − | <td bgcolor=#D7D7CC valign=top>
| |
| − | <font face="Times">
| |
| − | | |
| − | Thru mode flag (default = 1) When the flag is set, a matrix is output
| |
| − | when another one is received.
| |
| − | </font>
| |
| − | </td>
| |
| − | </tr>
| |
| − | <tr>
| |
| − | <td bgcolor=#D7CCCC valign=top><font face="Courier">type</font></td>
| |
| − | <td bgcolor=#DDDDDD valign=top><font face="Courier">int</font></td>
| |
| − | <td bgcolor=#D7D7CC valign=top>
| |
| − | <font face="Times">
| |
| − | | |
| − | The matrix data type (default = <font face="Courier">char</font><br>
| |
| − | Supported data types are <font face="Courier">char</font>, <font face="Courier">long</font>, <font face="Courier">float32</font>, or <font face="Courier">float64</font>.
| |
| − | </font>
| |
| − | </td>
| |
| − | </tr>
| |
| − | <tr>
| |
| − | <td bgcolor=#D7CCCC valign=top><font face="Courier">usedstdim</font></td>
| |
| − | <td bgcolor=#DDDDDD valign=top><font face="Courier">int</font></td>
| |
| − | <td bgcolor=#D7D7CC valign=top>
| |
| − | <font face="Times">
| |
| − | | |
| − | Destdim use flag (default = 0)
| |
| − | When the flag is set, the destination dimension's attributes are used when copying an input matrix
| |
| − | to an internal matrix.
| |
| − | </font>
| |
| − | </td>
| |
| − | </tr>
| |
| − | <tr>
| |
| − | <td bgcolor=#D7CCCC valign=top><font face="Courier">usesrcdim</font></td>
| |
| − | <td bgcolor=#DDDDDD valign=top><font face="Courier">int</font></td>
| |
| − | <td bgcolor=#D7D7CC valign=top>
| |
| − | <font face="Times">
| |
| − | | |
| − | Srcdim use flag (default = 0)
| |
| − | When the flag is set, the source dimension's attributes are used when copying an input matrix
| |
| − | to an internal matrix.
| |
| − | </font>
| |
| − | </td>
| |
| − | </tr>
| |
| − | </table>
| |
| − | <p>
| |
| − | <p>
| |
| − | <table border=0 width=100%>
| |
| − | <tr>
| |
| − | <td bgcolor=#CCCCDD><font size=+1 face="Times"><b>Messages:</b></font></td>
| |
| − | </tr>
| |
| − | </table>
| |
| − | | |
| − | <table border=0 width=100%>
| |
| − | <tr>
| |
| − | <td bgcolor=#CCCCCC><font face="Courier">bang </font></td>
| |
| − | </tr>
| |
| − | <tr>
| |
| − | <td bgcolor=#D7D7CC>
| |
| − | <font face="Times">
| |
| − | | |
| − | Equivalent to the <font face="Courier">outputmatrix</font> message.
| |
| − | </font>
| |
| − | </td>
| |
| − | </tr>
| |
| − | </table>
| |
| − | <br>
| |
| − | <table border=0 width=100%>
| |
| − | <tr>
| |
| − | <td bgcolor=#CCCCCC><font face="Courier">clear </font></td>
| |
| − | </tr>
| |
| − | <tr>
| |
| − | <td bgcolor=#D7D7CC>
| |
| − | <font face="Times">
| |
| − | | |
| − | Sets all matrix values to zero.
| |
| − | </font>
| |
| − | </td>
| |
| − | </tr>
| |
| − | </table>
| |
| − | <br>
| |
| − | <table border=0 width=100%>
| |
| − | <tr>
| |
| − | <td bgcolor=#CCCCCC><font face="Courier">exportimage </font><font face="Times"> [<i>filename</i> (<font face="Courier">symbol</font>)] { [<i>file-type</i> (<font face="Courier">symbol</font>)] } { [<i>use-dialog</i> (<font face="Courier">int</font>)] }</font></td>
| |
| − | </tr>
| |
| − | <tr>
| |
| − | <td bgcolor=#D7D7CC>
| |
| − | <font face="Times">
| |
| − | | |
| − | Export the current frame as an image file with the name specified by the first argument. The
| |
| − | optional second argument sets the file type (default = png). Available file types are
| |
| − | <font face="Courier">png</font>, <font face="Courier">bmp</font>, <font face="Courier">jpeg</font>, <font face="Courier">macpaint</font>, <font face="Courier">photoshop</font>, <font face="Courier">pict</font>, <font face="Courier">qtimage</font>,
| |
| − | <font face="Courier">sgi</font>, <font face="Courier">tga</font> and <font face="Courier">tiff</font>. An optional <i>use-dialog</i> argument of <font face="Courier">1</font> will open a
| |
| − | file dialog to allow you to enter the image file settings.
| |
| − | </font>
| |
| − | </td>
| |
| − | </tr>
| |
| − | </table>
| |
| − | <br>
| |
| − | <table border=0 width=100%>
| |
| − | <tr>
| |
| − | <td bgcolor=#CCCCCC><font face="Courier">exportmovie </font><font face="Times"> { [<i>filename</i> (<font face="Courier">symbol</font>)] } [<i>FPS</i> (<font face="Courier">float</font>)] [<i>codec</i> (<font face="Courier">symbol</font>)] [<i>quality</i> (<font face="Courier">symbol</font>)] [<i>timescale</i> (<font face="Courier">int</font>)]</font></td>
| |
| − | </tr>
| |
| − | <tr>
| |
| − | <td bgcolor=#D7D7CC>
| |
| − | <font face="Times">
| |
| − | | |
| − | Exports a matrix as a QuickTime movie. The <font face="Courier">exportmovie</font> message takes an optional argument
| |
| − | to specify a file name. If no filename is specified, a file dialog will open to let you choose a file.
| |
| − | <p>
| |
| − | The default <i>FPS</i> is 30. frames per second</p>
| |
| − | <p>
| |
| − | The default <i>codec</i> is <font face="Courier">raw</font>. Supported codecs are
| |
| − | <font face="Courier">raw</font>, <font face="Courier">cinepak</font>, <font face="Courier">graphics</font>, <font face="Courier">animation</font>, <font face="Courier">video</font>, <font face="Courier">componentvideo</font>, <font face="Courier">jpeg</font>, <font face="Courier">mjpega</font>, <font face="Courier">mjpegb</font>, <font face="Courier">sgi</font>, <font face="Courier">planarrgb</font>, <font face="Courier">macpaint</font>, <font face="Courier">gif</font>, <font face="Courier">photocd</font>, <font face="Courier">qdgx</font>, <font face="Courier">avrjpeg</font>, <font face="Courier">opendmljpeg</font>, <font face="Courier">bmp</font>, <font face="Courier">winraw</font>, <font face="Courier">vector</font>, <font face="Courier">qd</font>, <font face="Courier">h261</font>, <font face="Courier">h263</font>, <font face="Courier">dvntsc</font>, <font face="Courier">dvpal</font>, <font face="Courier">dvprontsc</font>, <font face="Courier">dvpropal</font>, <font face="Courier">flc</font>, <font face="Courier">targa</font>, <font face="Courier">png</font>, <font face="Courier">tiff</font>, <font face="Courier">componentvideosigned</font>, <font face="Courier">componentvideounsigned</font>, <font face="Courier">cmyk</font>, <font face="Courier">microsoft</font>, <font face="Courier">sorenson</font>, <font face="Courier">indeo4</font>, <font face="Courier">argb64</font>, <font face="Courier">rgb48</font>, <font face="Courier">alphagrey32</font>, <font face="Courier">grey16</font>, <font face="Courier">mpegyuv420</font>, <font face="Courier">yuv420</font>, and <font face="Courier">sorensonyuv9</font>.</p>
| |
| − | <p>
| |
| − | The default <i>quality</i> is <font face="Courier">max</font>. Supported quality settings are
| |
| − | <font face="Courier">lossless</font>, <font face="Courier">max</font>, <font face="Courier">min</font>, <font face="Courier">low</font>, <font face="Courier">normal</font>, and <font face="Courier">high</font>.<br>
| |
| − | <br>
| |
| − | Note that minimum quality is, in many cases, the codec's default quality. Use "low" quality for consistent results.</p>
| |
| − | <p>
| |
| − | The default <i>timescale</i> is 600 units per second.</p>
| |
| − | </font>
| |
| − | </td>
| |
| − | </tr>
| |
| − | </table>
| |
| − | <br>
| |
| − | <table border=0 width=100%>
| |
| − | <tr>
| |
| − | <td bgcolor=#CCCCCC><font face="Courier">float </font></td>
| |
| − | </tr>
| |
| − | <tr>
| |
| − | <td bgcolor=#D7D7CC>
| |
| − | <font face="Times">
| |
| − | | |
| − | Equivalent to the <font face="Courier">setall [float]</font> message, followed by the <font face="Courier">bang</font> message.
| |
| − | </font>
| |
| − | </td>
| |
| − | </tr>
| |
| − | </table>
| |
| − | <br>
| |
| − | <table border=0 width=100%>
| |
| − | <tr>
| |
| − | <td bgcolor=#CCCCCC><font face="Courier">getcell </font><font face="Times"> [<i>position</i> (<font face="Courier">list</font>)]</font></td>
| |
| − | </tr>
| |
| − | <tr>
| |
| − | <td bgcolor=#D7D7CC>
| |
| − | <font face="Times">
| |
| − | | |
| − | Sends the value(s) in the cell specified by <font face="Courier">position</font>
| |
| − | out the right outlet of the object as a
| |
| − | list in the form
| |
| − | <font face="Courier">cell</font> <i>cell-position0</i> ... <i>cell-positionN</i> <font face="Courier">val</font> <i>plane0-value</i> ... <i>planeN-value</i>.
| |
| − | </font>
| |
| − | </td>
| |
| − | </tr>
| |
| − | </table>
| |
| − | <br>
| |
| − | <table border=0 width=100%>
| |
| − | <tr>
| |
| − | <td bgcolor=#CCCCCC><font face="Courier">importmovie </font><font face="Times"> { [<i>filename</i> (<font face="Courier">symbol</font>)] } [<i>time-offset</i> (<font face="Courier">int</font>)]</font></td>
| |
| − | </tr>
| |
| − | <tr>
| |
| − | <td bgcolor=#D7D7CC>
| |
| − | <font face="Times">
| |
| − | | |
| − | Imports a QuickTime movie into the matrix. If no filename is specified, a file dialog will open to let you choose a file.
| |
| − | The <i>time-offset</i> argument may be used to set a time offset for the QuickTime movie being imported (the default is 0).
| |
| − | </font>
| |
| − | </td>
| |
| − | </tr>
| |
| − | </table>
| |
| − | <br>
| |
| − | <table border=0 width=100%>
| |
| − | <tr>
| |
| − | <td bgcolor=#CCCCCC><font face="Courier">int </font></td>
| |
| − | </tr>
| |
| − | <tr>
| |
| − | <td bgcolor=#D7D7CC>
| |
| − | <font face="Times">
| |
| − | | |
| − | Equivalent to the <font face="Courier">setall [int]</font> message, followed by the <font face="Courier">bang</font> message.
| |
| − | </font>
| |
| − | </td>
| |
| − | </tr>
| |
| − | </table>
| |
| − | <br>
| |
| − | <table border=0 width=100%>
| |
| − | <tr>
| |
| − | <td bgcolor=#CCCCCC><font face="Courier">exprfill </font><font face="Times"> [<i>expression</i> (<font face="Courier">symbol</font>)]</font></td>
| |
| − | </tr>
| |
| − | <tr>
| |
| − | <td bgcolor=#D7D7CC>
| |
| − | <font face="Times">
| |
| − | | |
| − | Evlauates <i>expression</i> to fill the matrix. See <b><a href="jit.expr.html">jit.expr</a></b> for more information on expressions.
| |
| − | </font>
| |
| − | </td>
| |
| − | </tr>
| |
| − | </table>
| |
| − | <br>
| |
| − | <table border=0 width=100%>
| |
| − | <tr>
| |
| − | <td bgcolor=#CCCCCC><font face="Courier">jit_gl_texture </font><font face="Times"> [<i>texture-name</i> (<font face="Courier">symbol</font>)]</font></td>
| |
| − | </tr>
| |
| − | <tr>
| |
| − | <td bgcolor=#D7D7CC>
| |
| − | <font face="Times">
| |
| − | | |
| − | Copies the texture specified by <i>texture-name</i> to the matrix.
| |
| − | </font>
| |
| − | </td>
| |
| − | </tr>
| |
| − | </table>
| |
| − | <br>
| |
| − | <table border=0 width=100%>
| |
| − | <tr>
| |
| − | <td bgcolor=#CCCCCC><font face="Courier">jit_matrix </font><font face="Times"> [<i>matrix-name</i> (<font face="Courier">symbol</font>)]</font></td>
| |
| − | </tr>
| |
| − | <tr>
| |
| − | <td bgcolor=#D7D7CC>
| |
| − | <font face="Times">
| |
| − | | |
| − | Copies a matrix specified by <i>matrix-name</i> to the matrix.
| |
| − | </font>
| |
| − | </td>
| |
| − | </tr>
| |
| − | </table>
| |
| − | <br>
| |
| − | <table border=0 width=100%>
| |
| − | <tr>
| |
| − | <td bgcolor=#CCCCCC><font face="Courier">list </font></td>
| |
| − | </tr>
| |
| − | <tr>
| |
| − | <td bgcolor=#D7D7CC>
| |
| − | <font face="Times">
| |
| − | | |
| − | Equivalent to the <font face="Courier">setall [list]</font> message, followed by the <font face="Courier">bang</font> message.
| |
| − | </font>
| |
| − | </td>
| |
| − | </tr>
| |
| − | </table>
| |
| − | <br>
| |
| − | <table border=0 width=100%>
| |
| − | <tr>
| |
| − | <td bgcolor=#CCCCCC><font face="Courier">outputmatrix </font></td>
| |
| − | </tr>
| |
| − | <tr>
| |
| − | <td bgcolor=#D7D7CC>
| |
| − | <font face="Times">
| |
| − | | |
| − | Sends the matrix out the left outlet.
| |
| − | </font>
| |
| − | </td>
| |
| − | </tr>
| |
| − | </table>
| |
| − | <br>
| |
| − | <table border=0 width=100%>
| |
| − | <tr>
| |
| − | <td bgcolor=#CCCCCC><font face="Courier">read </font><font face="Times"> { [<i>filename</i> (<font face="Courier">symbol</font>)] }</font></td>
| |
| − | </tr>
| |
| − | <tr>
| |
| − | <td bgcolor=#D7D7CC>
| |
| − | <font face="Times">
| |
| − | | |
| − | Reads Jitter binary data files (.jxf) into a matrix set.
| |
| − | If no filename is specified, a file dialog will open to let you choose a file.
| |
| − | </font>
| |
| − | </td>
| |
| − | </tr>
| |
| − | </table>
| |
| − | <br>
| |
| − | <table border=0 width=100%>
| |
| − | <tr>
| |
| − | <td bgcolor=#CCCCCC><font face="Courier">register </font></td>
| |
| − | </tr>
| |
| − | <tr>
| |
| − | <td bgcolor=#D7D7CC>
| |
| − | <font face="Times">
| |
| − | | |
| − | Equivalent to setting the <font face="Courier">name</font> attribute.
| |
| − | </font>
| |
| − | </td>
| |
| − | </tr>
| |
| − | </table>
| |
| − | <br>
| |
| − | <table border=0 width=100%>
| |
| − | <tr>
| |
| − | <td bgcolor=#CCCCCC><font face="Courier">setall </font><font face="Times"> [<i>value</i> (<font face="Courier">list</font>)]</font></td>
| |
| − | </tr>
| |
| − | <tr>
| |
| − | <td bgcolor=#D7D7CC>
| |
| − | <font face="Times">
| |
| − | | |
| − | Sets all cells to the value specified by <font face="Courier">value(s)</font>. Position is specified of a list whose length is equal to the number of dimensions (<font face="Courier">dimcount</font>).
| |
| − | </font>
| |
| − | </td>
| |
| − | </tr>
| |
| − | </table>
| |
| − | <br>
| |
| − | <table border=0 width=100%>
| |
| − | <tr>
| |
| − | <td bgcolor=#CCCCCC><font face="Courier">setcell </font><font face="Times"> [<i>position</i> (<font face="Courier">list</font>)] { <font face="Courier">plane</font> [<i>plane-number</i> (<font face="Courier">int</font>)] } <font face="Courier">val</font> [<i>value</i> (<font face="Courier">list</font>)]</font></td>
| |
| − | </tr>
| |
| − | <tr>
| |
| − | <td bgcolor=#D7D7CC>
| |
| − | <font face="Times">
| |
| | | | |
| − | Sets the cell specified by <font face="Courier">position</font> to the value specified by <font face="Courier">value</font>.
| + | == Examples == |
| − | Position is specified of a list whose length is equal to the number of dimensions (<font face="Courier">dimcount</font>). The
| |
| − | optional arguments <font face="Courier">plane</font> <i>plane-number</i> can be used to specify a plane. If a plane is specified, <i>value</i> should be a single number, otherwise it should be a list of numbers of size <font face="Courier">planecount - 1</font>.
| |
| − | </font>
| |
| − | </td>
| |
| − | </tr>
| |
| − | </table>
| |
| − | <br>
| |
| − | <table border=0 width=100%>
| |
| − | <tr>
| |
| − | <td bgcolor=#CCCCCC><font face="Courier">val </font><font face="Times"> [<i>value</i> (<font face="Courier">list</font>)]</font></td>
| |
| − | </tr>
| |
| − | <tr>
| |
| − | <td bgcolor=#D7D7CC>
| |
| − | <font face="Times">
| |
| | | | |
| − | Equivalent to the <font face="Courier">setall</font> message, followed by the <font face="Courier">bang</font> message.
| + | In the version [http://recherche.ircam.fr/equipes/temps-reel/gesturefollower/gesture_follower_v0.3.zip v0.3], the following examples can be found: |
| − | </font>
| + | * writing |
| − | </td>
| + | * Wii |
| − | </tr>
| + | * audio parameters (pitch, periodicity, energy) |
| − | </table>
| + | * voice (mfcc) |
| − | <br>
| |
| − | <table border=0 width=100%>
| |
| − | <tr>
| |
| − | <td bgcolor=#CCCCCC><font face="Courier">write </font><font face="Times"> { [<i>filename</i> (<font face="Courier">symbol</font>)] }</font></td>
| |
| − | </tr>
| |
| − | <tr>
| |
| − | <td bgcolor=#D7D7CC>
| |
| − | <font face="Times">
| |
| | | | |
| − | Writes matrix set as a Jitter binary data file (.jxf).
| + | == Links == |
| − | If no filename is specified, a file dialog will open to let you choose a file.
| |
| − | </font>
| |
| − | </td>
| |
| − | </tr>
| |
| − | </table>
| |
| − | <br>
| |
| − | <p>
| |
| − | <table border=0 width=100%>
| |
| − | <tr>
| |
| − | <td bgcolor=#CCCCDD><font size=+1 face="Times"><b>Example:</b></a></td>
| |
| − | </tr>
| |
| − | </table>
| |
| | | | |
| − | <p>
| + | == Credits and Aknowledgements== |
| − | <center>
| + | Real Time Musical Interaction - Ircam - CNRS STMS |
| − | <img src="images/jit.matrix.gif" border=0></center>
| |
| − | <p>
| |
| − | <table border=0 width=100%>
| |
| − | <tr>
| |
| − | <td bgcolor=#CCCCDD><font size=+1 face="Times"><b>See Also:</b></a></td>
| |
| − | </tr>
| |
| − | <tr>
| |
| − | <td bgcolor=#CCCCCC>
| |
| − | <a href="jit.coerce.html"><b>jit.coerce</b></a>,
| |
| − | <a href="jit.fill.html"><b>jit.fill</b></a>,
| |
| − | <a href="jit.matrixset.html"><b>jit.matrixset</b></a>,
| |
| − | <a href="jit.matrixinfo.html"><b>jit.matrixinfo</b></a>,
| |
| − | <a href="jit.peek~.html"><b>jit.peek~</b></a>,
| |
| − | <a href="jit.poke~.html"><b>jit.poke~</b></a>,
| |
| − | <a href="jit.spill.html"><b>jit.spill</b></a>,
| |
| − | <a href="jit.submatrix.html"><b>jit.submatrix</b></a></td>
| |
| − | </tr>
| |
| − | </table>
| |
| | | | |
| | | | |
| − | </body>
| + | Frédéric Bevilacqua, Rémy Muller, Norbert Schnell, Fabrice Guédy, Jean-Philippe Lambert, Aymeric Devergié, Anthony Sypniewski, Bruno Zamborlin, Donald Glowinski (thanks the scree captures!) |
| − | </html>
| |
There is no technical limit for the dimension of the gesture data (or number of sensor channel) other than what your computer can afford as a CPU load (for example 20 is generally no problem).
This software is intended for artistic work and/or scientific research. Any commercial use is reserved. Copyrights 2004-2007 IRCAM - Centre Pompidou.
If appropriate please cite the Real Time Interaction Team, IRCAM or reference the following article:
F. Bevilacqua, F. Guédy, N. Schnell, E. Fléty, N. Leroy, Wireless sensor interface and gesture-follower for music pedagogy, Proceedings of the International Conference of New Interfaces for Musical Expression (NIME 07), New York,NY, USA, pp 124-12, 2007.
Any type of feedback, problem, bug report, feature request are welcome and we will try our best to help you.
Please post any message/questions directly to the FTM list.
Nevertheless, ...this is work in progress!!! Use this software at your own risk. We do not assume any reponsability for possible problems caused by the use of this software.
Pay attention to the curves below. They represent the velocity in X and Y axis of the mouse trajectories. That give a useful temporal information on how you realize your drawing.
EyesWeb XMI, the open platform for real-time analysis of multimodal interaction, can be connected to Max/Msp throughout the OSC protocol (Open Sound Control). OSC is open, message-based protocol which was originally developed for communication between computers and sythesizers (cf. wiki).
