Mixed Reality

Mixed Reality Examples/Templates

In the "Example Scripts" folder you will find some examples of using mixed reality (in your SightLab folder under "ExampleScripts- Mixed Reality"). Mixed Reality will work through OpenXR support of Mixed Reality (Meta Quest 3, Meta Quest Pro, Varjo XR-4) currently.  You can also use an XR virtual screen under ExampleScripts- 2D screen/Video_Player.py

Viewing Passthrough on Mirrored Desktop View:

To see the overlay of the real world with the mixed reality objects on the mirrored view for Meta headsets, you will need to create a developer account and download the  Meta Quest Developer Hub for Windows. Once that is installed, login, connect the Meta Quest Pro and click "cast". You may also need to change the ADB path by selecting the one that comes from the dropdown and clicking "cancel". See this video for more information.

For Varjo, you can use the Varjo Base to see the overlay of the real world and virtual objects in the mirrored view.
https://varjo.com/use-center/get-to-know-your-headset/mixed-reality/ 

Interacting with the environment

If you want to have objects interact with real world objects you can place a 3D shape (such as one of the "regions of interest" boxes) over the real world object and then set its alpha to "0" in inspector (see "adding regions of interest"). Then you can turn on physics (see this page for how to enable physics).  For instance you can add an object on top of the real world ground. 

Mixed Reality with Multi-User:

In order to see other user's avatars while using Mixed Reality- see the example ExampleScripts-SightLabVR_Client_mixedReality.py and run this for any clients using mixed reality. 

For the server script you will need to choose "empty" for the environment. 

Meta Pro Grabbing XR

Example that shows interacting with physics and grabbing objects with your hands in mixed reality (uses OpenXR hand tracking)

Mixed Reality Eye Tracking:

See here  to see the set up steps for collecting eye tracking data on real world objects

Mixed Realit y (Simple Example)

To add Mixed Reality to any script just add this code: