FOVE SUPPORT CENTER

How to Access Gaze Convergence Data with Unreal Engine?

Answered

Comments

14 comments

  • Official comment
    Avatar
    Jeff

    The latest version of the plug in (0.12.1) now has FoveHMD.h in the public folder, and a C++ helper wrapper for convergence:

    bool GetGazeConvergence(bool bRelativeToHMD, FVector* outRayOrigin, FVector* outRayDirection, float* outDistance, float* outAccuracy) const;

    This means you can, from C++, include "FoveHMD.h" and then invoke:

    FVector rayOrigin;
    FVector rayDirection;
    if (FoveHMD::Get()->GetGazeConvergence(true, &rayOrigin, outRayDirection, ....)
    {
        // Use the ray for whatever you like. Perhaps use it for a raycast into the world and use the distance to disambiguate multiple hits.
    }

    However, the convergence distance is still fairly noisy, so you can use the averagng technique that Tamas pointed out above if you like.

    Convergence is also available in Blueprints too.

    Comment actions Permalink
  • Avatar
    Tamas Keresztes

    Considering this is not supported in 0.12, its probably not exposed. Currently, the convergence is a vector with an accuracy of 0 and no distance. Read a post somewhere here that this feature is a prototype at best at the moment.

    1
    Comment actions Permalink
  • Avatar
    pandaman

    @Tamas

    Is this the post you referred?

    https://support.getfove.com/hc/en-us/community/posts/115000860847/comments/115001116928

    It seems to say distance have some meaningful data.

    0
    Comment actions Permalink
  • Avatar
    Tamas Keresztes

    Yes, it does have. My experience with 0.12.0 matches that comment - distance is always 0, accuracy zero, and its unreliable at best. Often gets stuck to the stronger eye, and varies from calibration to calibration. I have used the SDK, which is probably what Unreal relies on too.

    Convergence is in infant prototype stage, based on what I read so far.

     

    0
    Comment actions Permalink
  • Avatar
    Jeff

    In 0.12.0, GetGazeConvergence doesn’t give reliable convergence data yet. We are working on this still.

    There is nonetheless one benefit to using it. It will always give you one of the two eye vectors. In the event that one eye closes, or otherwise can’t be read, you will automatically get the other eye, should it have a valid vector.

    Exposing convergence (among some other things) within the Unreal plugin is on my to-do list.

    In the meantime, from the C++ side, you can fetch the IFVRHeadset object (from our C++ SDK) directly via FoveHMD::Get()->GetHeadset(). That then provides the convergence function. You will need to convert coordinates, as FOVE uses a y-up coordinate system and Unreal uses a z-up coordinate system. There are some examples of this in FoveHMD.cpp, though we should probably make helpers for it.

    Hope that helps,

    - Jeff

    0
    Comment actions Permalink
  • Avatar
    pandaman

    Thank you for your reply, Jeff.

    I tried to #include "FoveHMD.h" in my Unreal Engine project. However the compiler complains that it cannot find the header file.

    It is probably bacause the header file lies in the Private folder of the FoveHMD plugin.

    What is the best solution to utilize the functionality of the C++ SDK with Unreal Engine?

    0
    Comment actions Permalink
  • Avatar
    Jeff

    You should move it to the public folder so it can be found. This is an oversight on my part (since I usually test things from within the plugin), I'll get that fixed up in the next release!

     

    0
    Comment actions Permalink
  • Avatar
    Tamas Keresztes

    I found that averaging (running) the left & right eye directions over a few frames gives a fairly stable convergence vector.

    0
    Comment actions Permalink
  • Avatar
    pandaman

    Thanks! I'll try that.

    0
    Comment actions Permalink
  • Avatar
    Philip Weiss

    Could someone post some example code for this? It would be very helpful. Thanks.

    0
    Comment actions Permalink
  • Avatar
    Philip Weiss

    Thanks for the info, Jeff. But now I'm getting the errot:

     

    1>D:\My Documents\Unreal Engine Projects\foveTest\Source\foveTest\foveTestCharacter.h(6): fatal error C1083: Cannot open include file: 'FoveVR.h': No such file or directory

     

    What am I doing wrong? Or is it possible there's a bug in the plugin code? Thanks.

    0
    Comment actions Permalink
  • Avatar
    Jeff

    Hi Philip,

    FoveVR.h isn't the right header.

    FoveHMD.h contains our Unreal plugin API (which you can use to get gaze convergence).

    FoveTypes.h / IFVRHeadset.h / IFVRCompositor.h are the Fove C++ API (not Unreal specific), which should also be accessible, though generally you only need them if something isn't wrapped in FoveHMD.h.

    0
    Comment actions Permalink
  • Avatar
    Philip Weiss

    Oh, oops, that's the wrong error message. I was trying some things to fix it, but here's the initial error, which happens right after #include "FoveHMD.h":

     

    2>D:\My Documents\Unreal Engine Projects\foveTest\Plugins\FoveVR\Source\FoveHMD\Public\FoveHMD.h(2): fatal error C1083: Cannot open include file: 'FoveTypes.h': No such file or directory

    0
    Comment actions Permalink
  • Avatar
    Jeff

    For now you'll need to add FoveVR to your module's PublicDependencyModuleNames, in addition to FoveHMD.

    PublicDependencyModuleNames.AddRange(new string[] { "Core", ..., "FoveHMD", "FoveVR" });

    I'll see if I can simplify that in a future version.

    0
    Comment actions Permalink

Please sign in to leave a comment.