README.md 8.82 KB
Newer Older
Thiago Santini's avatar
Thiago Santini committed
1 2
# EyeRecToo

Thiago Santini's avatar
Thiago Santini committed
3 4 5 6 7
EyeRecToo is a second-generation hardware-agnostic open-source software for
head-mounted eye trackers.  Its main raison d'être is to provide an open
platform to replace the data acquisition functionality from eye-tracker
vendors' software, which typically are expensive, closed-source, and geared
toward their own devices.
Thiago Santini's avatar
Thiago Santini committed
8 9 10 11 12 13 14 15 16 17 18 19 20 21 22
It deprecates [EyeRec](https://www-ti.informatik.uni-tuebingen.de/santini/EyeRec).

For data analysis functionality replacement, we recommend
[Eyetrace](http://www.ti.uni-tuebingen.de/Eyetrace.1751.0.html).

>*Why the name?*
>
>Originally, EyeRec was a phonetical play on words and the
>eye-related functionality as in **I rec**[ord].
>EyeRecToo adds on the word play: It is the version **two** and, similar to EyeRec, records **too**.
>All bad puns are intended :-)

## Supported Platforms

- Windows 64 bits
Thiago Santini's avatar
Thiago Santini committed
23 24
>This is the platform that gets the most testing and recommended.

Thiago Santini's avatar
Thiago Santini committed
25 26
- Initiall support for Ubuntu 16.04 64 bits has been added; no binaries available at the moment though.
>Only supports camera access using the uvcengine, including the Pupil Labs eye tracker. It has been tested for
Thiago Santini's avatar
Thiago Santini committed
27 28 29
pupil detection, gaze estimation, and data recording. This is **alpha**.

- Notes
Thiago Santini's avatar
Thiago Santini committed
30 31

>It's also possible to get EyeRecToo running on 32 bits platforms, but no Pupil
Thiago Santini's avatar
Thiago Santini committed
32
>eye tracker support is available.
Thiago Santini's avatar
Thiago Santini committed
33 34 35 36 37 38

## Supported Devices

Theoretically, cameras using DirectShow (on Windows) and v4l2 (on Linux)
should work out of the box.

Thiago Santini's avatar
Thiago Santini committed
39 40 41 42
UVC-compliant cameras are suuported through the [UVC Engine](https://atreus.informatik.uni-tuebingen.de/santini/uvcengine).
On Windows, it is typically necessary to change camera drivers to be able to
use the uvcengine; see the **Running** section for details.

Thiago Santini's avatar
Thiago Santini committed
43 44 45 46
*Tested Eye Trackers:*
- Dikablis Essential / Pro (see [Ergoneers](http://www.ergoneers.com))
- Pupil DIY (see [Pupil Labs](https://pupil-labs.com/))
- Eivazi's microscope add-on (see [Eivazi, S et al.  2016](http://ieeexplore.ieee.org/document/7329925/))
Thiago Santini's avatar
Thiago Santini committed
47 48
- Pupil Eye Tracker, both Cam1 and Cam2 models (see [Pupil Labs](https://pupil-labs.com/store/))
- The inconspicuous modular eye trackers from [Eivazi, S et al 2018](http://www.ti.uni-tuebingen.de/typo3conf/ext/timitarbeiter/pi4/show_bibtex.php?uid=887&tid=1)
Thiago Santini's avatar
Thiago Santini committed
49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64

*Tested webcams:*
- [Playstation Eye](https://en.wikipedia.org/wiki/PlayStation_Eye)
- [Microsoft LifeCam HD-3000](https://www.microsoft.com/accessories/en-us/products/webcams/lifecam-hd-3000/t3h-00011)
- [Microsoft LifeCam HD-5000](https://www.microsoft.com/accessories/en-us/d/lifecam-hd-5000)
- [Microsoft LifeCam HD-6000](https://www.microsoft.com/accessories/en-us/d/lifecam-hd-6000-for-notebooks)
- [Microsoft LifeCam VX-1000](https://www.microsoft.com/accessories/en-us/d/lifecam-vx-1000)
- [Logitech C920](http://www.logitech.com/en-us/product/hd-pro-webcam-c920)
- [Logitech C510](http://support.logitech.com/en_us/product/hd-webcam-c510)
- [Logitech C525](http://www.logitech.com/en-us/product/hd-webcam-c525)
- [Logitech QuickCam PRO 9000](http://support.logitech.com/en_us/product/quickcam-pro-9000)
- [Logitech QuickCam Express](http://support.logitech.com/en_us/product/quickcam-express)


## Running

Thiago Santini's avatar
Thiago Santini committed
65
Binaries can be downloaded [here](https://www.hci.uni-tuebingen.de/research/eyerectoo.html).
Thiago Santini's avatar
Thiago Santini committed
66 67 68 69 70

**NOTE:** Additionally, you may need to install drivers to access your eye tracker
cameras:

1. Dikablis eye trackers require the
Thiago Santini's avatar
Thiago Santini committed
71
[VRMagic drivers](https://www.vrmagic-imaging.com/service/downloads/usb-platform-downloads) (make sure you
Thiago Santini's avatar
Thiago Santini committed
72 73
have the 64 bits version)
2. For Pupil eye trackers, check the [Pupil Labs' guide](https://github.com/pupil-labs/pupil/wiki/Windows-Driver-Setup).
Thiago Santini's avatar
Thiago Santini committed
74 75 76 77 78 79
3. (Windows only) for better interfacing with UVC-compliant cameras, we recommend to change the camera drivers to enable the uvcengine.
This can be easily achieved through [Zadig](https://zadig.akeo.ie/):
a) Check *Options->"list all devices"*,
b) uncheck *Options->"Ignore Hubs or Composite Parents"* ,
c) change your camera's *interface* driver to *libusbK*,
d) change your camera's *composite parent* to *libusbK*
Thiago Santini's avatar
Thiago Santini committed
80 81 82

## Developing

Thiago Santini's avatar
Thiago Santini committed
83
The official EyeRecToo repository is located at:
Thiago Santini's avatar
Thiago Santini committed
84 85
[https://atreus.informatik.uni-tuebingen.de/santini/EyeRecToo](https://atreus.informatik.uni-tuebingen.de/santini/EyeRecToo)

Thiago Santini's avatar
Thiago Santini committed
86 87
**Windows**

Thiago Santini's avatar
Thiago Santini committed
88 89
The particulars of the test build system we use are:
- QtCreator 4.0.3
Thiago Santini's avatar
Thiago Santini committed
90
- Qt >= 5.7.0 (MSVC 2015, 64 bit)
Thiago Santini's avatar
Thiago Santini committed
91 92 93 94 95 96 97 98
- Visual Studio 2015
- Microsoft Windows 8.1

All libraries/includes (for Windows 64 bits) should be in the deps directory already; to build:
1. Install Visual Studio 2015
2. Install Qt 5.7.0 (and Qt Creator)
3. Open the project
4. Run qmake and build
Thiago Santini's avatar
Thiago Santini committed
99
5. **Before running, add deps/runtime/Release or deps/runtime/Debug to your PATH accordingly to your build type.**
Thiago Santini's avatar
Thiago Santini committed
100

Thiago Santini's avatar
Thiago Santini committed
101
*Both Release and Debug versions are functional although the latter may run at much lower sampling rates.*
Thiago Santini's avatar
Thiago Santini committed
102

Thiago Santini's avatar
Thiago Santini committed
103 104 105 106
**Linux**

The particulars of the test build system we use are:
- QtCreator 4.4.1
Thiago Santini's avatar
Thiago Santini committed
107
- Qt 5.9.2
Thiago Santini's avatar
Thiago Santini committed
108 109 110 111 112 113 114 115 116 117
- GCC 5.4.0
- Ubuntu 16.04.3

Requires the installation of some packages (see README.LINUX); to build:
1. Install Qt 5.9.2 (and Qt Creator)
2. Run ./linux-setup.sh (or do the steps manually)
3. Open the project
4. Run qmake and build
5. Should be good to go :-)

Thiago Santini's avatar
Thiago Santini committed
118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191
## Data Format

Text data files (*.tsv*) use the [tsv format](https://en.wikipedia.org/wiki/Tab-separated_values) -- i.e., **tab** delimited files.

Video data files (*.mp4*) are uncompressed MJPEG files using MPEG-4 Part 14 containers.
**Note: timing information from these files is not reliable. You can find the timestamp for each frame in its respective *\*Data.tsv* counterpart.**

---

- *meta.csv*
>Contains meta data about the program, recording, and host machine.


- *\<Camera Widget\>Data.tsv* and *\<Camera Widget\>.mp4*
> Frames and respective processing tuples, e.g., detected pupil, markers, etc; these are FieldData and EyeData in the code. The number of entries in the *.tsv* **must** match the amount of frames in the video file. For each camera widget (e.g., eyes, field), one of these pairs is generated with the appropriate name -- e.g., RightEye.tsv, RightEye.mp4.

- *JournalData.tsv*
> Synchronized data file. This file includes entries containing only synchronized tuples (DataTuple in the code). **Note: despite gaze estimation being part of FieldData (since it's the correct reference frame), it is available only after synchronization and, thus, in this file, not on FieldData.tsv).**

- *[0-9]\*-calibration.tup*
> Data tuples used for calibration. The number prefix indicates the timestamp of the calibration.

---

For convenience, we also provide the *ERTViewer.exe* application, which allows one to visualize, do some basic annotation, and record gaze-overlayed videos from an EyeRecToo recording.

**Note:**
This is just a temporary application until we start integrating a data viewer/analyzer for video data in [*Eyetrace*](http://www.ti.uni-tuebingen.de/Eyetrace.1751.0.html). There are some bugs (e.g., the overlayed pupil can be slightly unsynchronized), and there is no focus on usability nor documentation.

## Markers

Default ArUco markers can be found under the *utils/all-markers* directory.

Default collection marker for *CalibMe* calibrations is provided at
*utils/CalibMeCollectionMarker.png*.

Camera calibration patterns are *utils/asymmetric_circle_grid_4x11_20mm.pdf* (default)
and *utils/chessboard_9x6_25mm.pdf*.

These markers can also be found only
[here](https://atreus.informatik.uni-tuebingen.de/santini/EyeRecToo/tree/master/EyeRecToo/utils).

## Video Tutorials

An introduction to EyeRecToo is available on YouTube. It is a bit outdated since
its from 2017, but the principles remain the same.

<iframe width="640" height="480" src="https://www.youtube.com/embed/vbrfjpfYkvg"
frameborder="0" allow="autoplay; encrypted-media" allowfullscreen></iframe>

If you want to learn more about *CalibMe* and unassisted calibration, here is a talk about it at CHI2017.

<iframe width="640" height="480"
src="https://youtube.com/embed/gYtF3j2Hdl4" frameborder="0"
allow="autoplay; encrypted-media" allowfullscreen></iframe>

## Configuration Files

All configuration files are found in the *cfg* directory. These are
[*.ini](https://en.wikipedia.org/wiki/INI_file) files that can be easily edited
if needed.

## Keyboard Shortcuts

These shortcuts can be changed through *cfg/CommandManager.ini*. Key codes can
be found [here](http://doc.qt.io/qt-5/qt.html#Key-enum).

Exposed functionality at the moment includes:

- remoteCalibrationToggleKey = Qt::Key_PageDown:
> Toggles calibration (including *CalibMe* marker collection).

- calibrationToggleKey = Qt::Key_S:
> Toggles calibration.
Thiago Santini's avatar
Thiago Santini committed
192

Thiago Santini's avatar
Thiago Santini committed
193 194
- collectionToggleKey = Qt::Key_C:
> Toggles *CalibMe* marker collection.
Thiago Santini's avatar
Thiago Santini committed
195

Thiago Santini's avatar
Thiago Santini committed
196 197 198 199 200 201 202 203 204
- recordingToggleKey = Qt::Key_R:
> Toggles recording.

- remoteRecordingToggleKey = Qt::Key_PageUp:
> If subject is not set, sets subject to "remote". Toggles recording.

- previewToggleKey = Qt::Key_B:
> Freezes previews; useful, e.g., during development to check gaze estimation / pupil detection, or for didactic purposes.