This is a great question with no single answer, but we hope this article will give you significantly more clarity on the subject.
Degrees of Success
Calibration failures are possible if there are major issues with detecting eyes, or if the user does not follow the calibration point at all. Nonetheless, the FOVE Eye Tracker always tries to achieve a high success rate, even if the calibration process went poorly or the conditions were less than ideal.
In this way you at least get some data, rather than none at all, in the event of a poor calibration.
However, since not all successes are equal, we've introduced three different success codes to our API:
|Successful_HighQuality||The calibration is successful and of high quality.|
|Successful_MediumQuality||The calibration is successful and of medium quality.|
|Successful_LowQuality||The calibration is successful but of low quality.|
The thresholds that determine the various levels are internal to the tracker, and arbitrary in nature, but they are designed to give you a general sense of how well the calibration went overall.
In cases where high quality data is of critical importance, and you have the time to recalibrate, it's perfectly ok to retry to get a better result.
But remember that the eye tracker will function with any of the above 3 success codes. MediumQuality is perfectly suitable, and is the most common result. Medium quality just means that there's some risk of less-than-best-possible accuracy, not that necessarily it's worse.
Criteria of Success
There are currently three factors which participate in the estimation of calibration quality:
- How many eye tracker failures happened during the smooth pursuit
- An eye tracker failure is simply a frame where the eye tracker was unable to parse a frame from the camera image. The most common cause is blinking, but occasionally there are difficulties in image processing such as being unable to locate features of the eye due to extra reflections from glasses, sensor noise, etc.
- The more such frames occurred during calibration, the lower the score.
- How many glints were observed during eye smooth pursuit
- The infrared LEDs around the lens produce small reflections called glints. These glints are critical to calibration and to eye tracking after calibration.
- The glints must reflect off of the cornea, as opposed to the eyeball, as part of their function is to allow us to compute the size of the cornea.
- Not every glint reflects off of the cornea, or even is visible every frame simply due to the geometry of the eye and headset. This is the main reason that the smooth pursuit spiral is there to make the user look in different directions over the calibration process.
- Over the course of the smooth pursuit, we try to identify all the glints we can find on the image. The less we find, the lower the score.
- How well the gaze moved with the calibration point
- While it is strongly recommended to look at the calibration the entire time, it is not completely necessary. Temporary shifts of gaze attention happen, and the algorithm is tolerant to that.
- However, this is taken into account for the scoring. If for large portions of the spiral, the user was not looking in the right direction, then the score will be reduced.
Improving the Odds
Everyone's eyes are a little different, and every calibration is slightly unique. So there's no way to guarantee a good calibration, however, these tips will help you increase your calibration quality. Listed in order of importance.
- Ensure that the eyes are well positioned during the calibration.
- This is the most common source of trouble because the headset mounts slightly differently on every user.
- The ideal position for the eye is to be exactly centered in the lens horizontally, and slightly below center vertically, as seen from the eye tracking camera.
- It is common, when the eye rotates, for the pupil/iris region to move a lot in this image. For this reason it's important to keep things centered so there's some buffer on every side for the pupil to move around without leaving the lens area.
- Ensure that the eyes are opened wide enough.
- Glints and other features are often obscured by the eyelid itself.
- Ensure that you are tracking the calibration moving point for all the time from the beginning to end.
- Prefer contacts to glasses if possible.
- While we support both contacts and glasses, our benchmarks show better performance from contacts than glasses. This is due to increased distortion and additional reflections from the glasses lenses.
- Also make sure to remove any other obstructions like hair that may be in the image.
Running Your Own Check
We benchmark extensively, but more is better, so we always encourage others to benchmark as well if they can.
In most cases, you wouldn't want to put every user through a long benchmarking process before getting to the actual content, but it's not uncommon to have a couple "confirmation points" where, after calibration, you intentionally have the user gaze at a few locations on the screen. You can compare the gaze estimation against the known location of these points to validate that the eye tracker is working suitably to your needs.