Face Tracking

Click here for an updated version of this with support for the HTC Face Tracker

To utilize face tracking sensors SightLab will support any devices that utilize the OpenXR standard for either Meta or HTC headsets.

Here are some of the built in examples/templates included in SightLab for Face Tracking

You can add collecting and saving of face tracking data to any script by adding the lines from utils import face_tracker_data . This will save all the facial tracking data to the utils/data folder
See the Mirror Demo to get a template on how to map the expressions to an animated avatar (can also just build from this code to create a script that maps the expressions to an avatar, remove the mirror code if not needed)
See the FaceTracker_Sliders to see sliders that move with your facial tracker data (this can also be pulled up in the Mirror demo)
facial_expressions_over_time will show a matplotlib chart of the facial expression data over time. This can load data from the mirror demo or other demos using face tracking

Here is some sample code on importing the face_tracker_data module and collecting and saving that data (which will save in the utils/data folder:

import viz,vizfx, viztaskfrom utils import sightlab, face_tracker_datafrom settings import *
sightlab.is_GUI = 1
# Set up with custom valuesface_tracker_data.setup()
def sightLabExperiment(): while True: yield viztask.waitKeyDown(' ') # Start the update process vizact.ontimer(0, face_tracker_data.UpdateAvatarFace) viz.sendEvent(TRIAL_START_EVENT)
yield viztask.waitKeyDown(' ') viz.sendEvent(TRIAL_END_EVENT) viztask.schedule(sightlab.experiment)viztask.schedule(sightLabExperiment)

Here are the parameters available for Face Tracking Expression:

For Facebook (Meta)

EXPRESSION_BROW_LOWERER_L = 0

EXPRESSION_BROW_LOWERER_R = 1

EXPRESSION_CHEEK_PUFF_L = 2

EXPRESSION_CHEEK_PUFF_R = 3

EXPRESSION_CHEEK_RAISER_L = 4

EXPRESSION_CHEEK_RAISER_R = 5

EXPRESSION_CHEEK_SUCK_L = 6

EXPRESSION_CHEEK_SUCK_R = 7

EXPRESSION_CHIN_RAISER_B = 8

EXPRESSION_CHIN_RAISER_T = 9

EXPRESSION_DIMPLER_L = 10

EXPRESSION_DIMPLER_R = 11

EXPRESSION_EYES_CLOSED_L = 12

EXPRESSION_EYES_CLOSED_R = 13

EXPRESSION_EYES_LOOK_DOWN_L = 14

EXPRESSION_EYES_LOOK_DOWN_R = 15

EXPRESSION_EYES_LOOK_LEFT_L = 16

EXPRESSION_EYES_LOOK_LEFT_R = 17

EXPRESSION_EYES_LOOK_RIGHT_L = 18

EXPRESSION_EYES_LOOK_RIGHT_R = 19

EXPRESSION_EYES_LOOK_UP_L = 20

EXPRESSION_EYES_LOOK_UP_R = 21

EXPRESSION_INNER_BROW_RAISER_L = 22

EXPRESSION_INNER_BROW_RAISER_R = 23

EXPRESSION_JAW_DROP = 24

EXPRESSION_JAW_SIDEWAYS_LEFT = 25

EXPRESSION_JAW_SIDEWAYS_RIGHT = 26

EXPRESSION_JAW_THRUST = 27

EXPRESSION_LID_TIGHTENER_L = 28

EXPRESSION_LID_TIGHTENER_R = 29

EXPRESSION_LIP_CORNER_DEPRESSOR_L = 30

EXPRESSION_LIP_CORNER_DEPRESSOR_R = 31

EXPRESSION_LIP_CORNER_PULLER_L = 32

EXPRESSION_LIP_CORNER_PULLER_R = 33

EXPRESSION_LIP_FUNNELER_LB = 34

EXPRESSION_LIP_FUNNELER_LT = 35

EXPRESSION_LIP_FUNNELER_RB = 36

EXPRESSION_LIP_FUNNELER_RT = 37

EXPRESSION_LIP_PRESSOR_L = 38

EXPRESSION_LIP_PRESSOR_R = 39

EXPRESSION_LIP_PUCKER_L = 40

EXPRESSION_LIP_PUCKER_R = 41

EXPRESSION_LIP_STRETCHER_L = 42

EXPRESSION_LIP_STRETCHER_R = 43

EXPRESSION_LIP_SUCK_LB = 44

EXPRESSION_LIP_SUCK_LT = 45

EXPRESSION_LIP_SUCK_RB = 46

EXPRESSION_LIP_SUCK_RT = 47

EXPRESSION_LIP_TIGHTENER_L = 48

EXPRESSION_LIP_TIGHTENER_R = 49

EXPRESSION_LIPS_TOWARD = 50

EXPRESSION_LOWER_LIP_DEPRESSOR_L = 51

EXPRESSION_LOWER_LIP_DEPRESSOR_R = 52

EXPRESSION_MOUTH_LEFT = 53

EXPRESSION_MOUTH_RIGHT = 54

EXPRESSION_NOSE_WRINKLER_L = 55

EXPRESSION_NOSE_WRINKLER_R = 56

EXPRESSION_OUTER_BROW_RAISER_L = 57

EXPRESSION_OUTER_BROW_RAISER_R = 58

EXPRESSION_UPPER_LID_RAISER_L = 59

EXPRESSION_UPPER_LID_RAISER_R = 60

EXPRESSION_UPPER_LIP_RAISER_L = 61

EXPRESSION_UPPER_LIP_RAISER_R = 62

CONFIDENCE_LOWER_FACE = 0

CONFIDENCE_UPPER_FACE = 1

For HTC:

EYE_LEFT_BLINK = 0

EYE_LEFT_WIDE = 1

EYE_RIGHT_BLINK = 2

EYE_RIGHT_WIDE = 3

EYE_LEFT_SQUEEZE = 4

EYE_RIGHT_SQUEEZE = 5

EYE_LEFT_DOWN = 6

EYE_RIGHT_DOWN = 7

EYE_LEFT_OUT = 8

EYE_RIGHT_IN = 9

EYE_LEFT_IN = 10

EYE_RIGHT_OUT = 11

EYE_LEFT_UP = 12

EYE_RIGHT_UP = 13

LIP_JAW_RIGHT = 0

LIP_JAW_LEFT = 1

LIP_JAW_FORWARD = 2

LIP_JAW_OPEN = 3

LIP_MOUTH_APE_SHAPE = 4

LIP_MOUTH_UPPER_RIGHT = 5

LIP_MOUTH_UPPER_LEFT = 6

LIP_MOUTH_LOWER_RIGHT = 7

LIP_MOUTH_LOWER_LEFT = 8

LIP_MOUTH_UPPER_OVERTURN = 9

LIP_MOUTH_LOWER_OVERTURN = 10

LIP_MOUTH_POUT = 11

LIP_MOUTH_SMILE_RIGHT = 12

LIP_MOUTH_SMILE_LEFT = 13

LIP_MOUTH_SAD_RIGHT = 14

LIP_MOUTH_SAD_LEFT = 15

LIP_CHEEK_PUFF_RIGHT = 16

LIP_CHEEK_PUFF_LEFT = 17

LIP_CHEEK_SUCK = 18

LIP_MOUTH_UPPER_UPRIGHT = 19

LIP_MOUTH_UPPER_UPLEFT = 20

LIP_MOUTH_LOWER_DOWNRIGHT = 21

LIP_MOUTH_LOWER_DOWNLEFT = 22

LIP_MOUTH_UPPER_INSIDE = 23

LIP_MOUTH_LOWER_INSIDE = 24

LIP_MOUTH_LOWER_OVERLAY = 25

LIP_TONGUE_LONGSTEP1 = 26

LIP_TONGUE_LEFT = 27

LIP_TONGUE_RIGHT = 28

LIP_TONGUE_UP = 29

LIP_TONGUE_DOWN = 30

LIP_TONGUE_ROLL = 31

LIP_TONGUE_LONGSTEP2 = 32

LIP_TONGUE_UPRIGHT_MORPH = 33

LIP_TONGUE_UPLEFT_MORPH = 34

LIP_TONGUE_DOWNRIGHT_MORPH = 35

LIP_TONGUE_DOWNLEFT_MORPH = 36