Symmetry of Expressions

Bitbox analyzes the asymmetry of facial expressions and body actions (coming soon). It measures the disparity between the left and right sides by first mirroring the object (e.g., eye, brow, half of the mouth) across the perpendicular-bisector plane between the left and right objects. A full plane reflection is used since there is no guarantee that the midsagittal plane is exactly the global 0 plane. Once the right side reflected to the left side, Euclidean distance between the coordinates of landmarks is computed for each frame.

This function only accepts facial landmarks as input, allowing either 2D or 3D canonicalized landmarks. We highly recommend using 3D canonicalized landmarks, particularly if the face is not strictly frontal, as pose variations can greatly and differentially influence 2D landmark coordinates.

Use 2D landmarks only if there is a compelling reason and the face is perfectly frontal in all frames.

from bitbox.expressions import asymmetry

# detect 2D landmarks
lands2D = processor.detect_landmarks()

# detect 3D landmarks
exp_global, pose, lands3D = processor.fit()

# compute per-frame asymmetry scores
reliable_asymmetry_scores = asymmetry(lands3D)
not_so_much_reliable_asymmetry_scores = asymmetry(lands2D)

The output is a Pandas DataFrame including five asymmetry scores per frame: eyes, brows, nose, mouth, and overall.

          eye	           brow	          nose	          mouth	         overall
0	0.292884	0.265071	0.041300	0.075939	0.211353
1	0.210291	0.260469	0.045354	0.097847	0.195869
2	0.283825	0.156247	0.000000	0.000000	0.114181
3	0.327324	0.119029	0.094368	0.040651	0.182186
4	0.196593	0.167545	0.196875	0.051681	0.171618
...	...	...	...	...	...

Output Ranges

Bitbox performs a straightforward normalization on the asymmetry scores, by default. For 3D landmarks, the values typically fall between 0 (guaranteed) and 1 (not guaranteed). For 2D landmarks, the values are guaranteed to start from 0, but the upper limit is unspecified.

When interpreting normalized scores for 3D landmarks, use these as rough, human-visible thresholds:

• Eyes: > 0.20 ≈ noticeable; > 0.40 ≈ clearly asymmetric

• Brows: > 0.35 ≈ noticeable; > 0.40 ≈ clearly asymmetric

• Nose: > 0.10 ≈ noticeable; > 0.40 ≈ clearly asymmetric

• Mouth: > 0.10 ≈ noticeable; > 0.40 ≈ clearly asymmetric

These are recommendations, not rules. Adjust them to your data and use case.

With 2D landmarks, no universal thresholds exist as scale and camera effects vary. Calibrate on your dataset.

Score values may change based on the backend processor used to generate landmark coordinates. For example, 3D landmarks generated by 3DI-Lite, on average, tend to yield lower asymmetry score. Thresholds listed above are for 3D landmarks generated by 3DI.

Disable normalization to compute scores in the same units as the landmarks.

# disable normalization
asymmetry_scores = asymmetry(lands3D, normalize=False)