Eye tracking and virtual reality in the rehabilitation of mobility of hemianopia patients: a user experience study

Publications

Share / Export Citation / Email / Print / Text size:

Vision Rehabilitation International

Guide Dogs NSW/ACT

Subject: Health Care Sciences & Services , Medicine , Rehabilitation

GET ALERTS

eISSN: 2652-3647

DESCRIPTION

32
Reader(s)
83
Visit(s)
0
Comment(s)
0
Share(s)

SEARCH WITHIN CONTENT

FIND ARTICLE

Volume / Issue / page

Archive
Volume 11 (2020)
Volume 10 (2019)
Volume 9 (2017-2018)
Volume 8 (2016)
Volume 7 (2015)
Volume 6 (2013-2014)
Volume 5 (2012)
Volume 4 (2011)
Volume 3 (2010)
Volume 2 (2009)
Volume 1 (2008)
Related articles

VOLUME 11 , ISSUE 1 (Jun 2020) > List of articles

Eye tracking and virtual reality in the rehabilitation of mobility of hemianopia patients: a user experience study

Birte Gestefeld / Jan Koopman / Anne Vrijling / Frans W. Cornelissen / Gera de Haan *

Keywords : Compensatory scanning training, Eye tracking, Virtual reality, Hemianopia, Visual rehabilitation

Citation Information : Vision Rehabilitation International. Volume 11, Issue 1, Pages 7-19, DOI: https://doi.org/10.21307/vri-2020-002

License : (CC-BY-NC-ND 4.0)

Received Date : 14-February-2020 / Published Online: 01-July-2020

ARTICLE

ABSTRACT

Purpose: To test the usability of eye tracking and virtual reality during vision rehabilitation training of hemianopia patients.

Methods: Individuals with hemianopia (n = 13) and normal-sighted controls (n = 4) performed various exercises that are commonly used in vision rehabilitation for mobility, while wearing a head-mounted eye tracker or a head-mounted virtual reality (VR) display. Occupational therapists (n = 4) guided them through the exercises. All participants (including therapists) filled out a questionnaire, assessing their experience with the used device. Individuals with hemianopia were split into three groups according to their stage in vision rehabilitation therapy and performed 1 (beginner), 2 (intermediate) or 3 (advanced) different exercises.

Results: Individuals with hemianopia rated the mobile eye tracker with a score of 3.97 ± 0.5 points (beginner), 3.8 ± 0.5 points (intermediate) and 4 ± 0 points (advanced) the corresponding occupational therapists with a score of 3.6 ± 0.6, 3.4 ± 0.9 and 3.87 ± 0.6 points (out of a maximum of 4 points). The VR headset was rated with 3.9 ± 0.5 points by individuals with hemianopia, 3.8 ± 0.5 points by normal-sighted controls and 2.5 ± 1.4 points by the occupational therapist in a virtual hallway scenario. In a street-crossing scenario, it was rated with 3.7 ± 0.5 points by individuals with hemianopia, 3.7 ± 0.8 points by controls and 2.8 ± 1.2 by occupational therapists. In a walking along a pavement scenario the individual with hemianopia gave 4 ± 0 points and the controls 3.8 ± 0.4 points on average.

Conclusions: Both devices were seen as useful additions to vision rehabilitation therapy, as they enable better feedback to patients and the opportunity to do different exercises at different levels of difficulty.

Graphical ABSTRACT

Introduction

Patients with hemianopia are blind in one half of their visual field, due to damage to the visual system posterior to the optic chiasm. This large visual field defect may lead to difficulties in different daily life situations, such as mobility related activities. There is however a large variability in how severely different individuals are impacted by their visual field loss. A crucial factor that determines if an individual can successfully adapt to visual field loss is the development of compensatory strategies, such as certain eye movement patterns (Howard and Rowe, 2018). While some hemianopia patients are able to spontaneously develop compensatory strategies for their visual field loss, others retain difficulties, in particular with detecting objects in the periphery in situations when they are moving, e.g. during walking (Iorizzo et al., 2011). These patients can enter vision rehabilitation to learn compensatory eye movement strategies (for an overview see Pollock et al., 2019). In one type of compensatory scanning training, InSight-Hemianopia Compensatory Scanning Training (IH-CST), these patients learn to make horizontal scanning eye-movements (De Haan et al., 2015) in order to compensate for the field defect during mobility-related activities. In IH-CST training, hemianopia patients learn to repetitively perform a triad of saccades including an initial large saccade made towards the blind hemifield. Training hemianopia patients in making this type of scanning patterns improves their detection of peripheral stimuli and their avoidance of obstacles while walking (De Haan et al., 2015).

IH-CST consists of the following exercises:

  • 1. Creating awareness of the extent of visual field loss. The therapist presents objects in different parts of the visual field (while patients keep their gaze fixated on a single location).

  • 2. Learning the scanning pattern. The patient is taught to systematically make the eye movement pattern, being seated and fixating objects in the desired gaze locations at different distances. The occupational therapist observes the eye movements of the patient.

  • 3. Further practice to automate the scanning pattern in a static condition. The patient has to call out numbers presented on a screen. If numbers are named correctly and quickly, it is assumed that the scanning pattern is also performed correctly.

  • 4. Applying the scanning pattern in indoor mobility conditions. Again, the therapist observes the patient’s eye movements.

  • 5. Applying the scanning pattern in outdoor mobility conditions. The scanning pattern is practiced in different traffic situations outside the therapy centre with increasing complexity. Patients walk, cycle or drive a mobility scooter, depending on their needs.

Studies investigating the effect of eye movement compensatory strategies, so far, have only compared the performance of participants in certain tasks, such as visual scanning or detecting obstacles while walking, before and after compensatory scanning training (Nelles et al., 2001; De Haan et al., 2015). They did not evaluate whether participants had correctly performed the scanning pattern during the training exercises. In the practical application of IH-CST at Royal Dutch Visio, there is also no objective measure of the eye movements during rehabilitation training. Instead, therapists attempt to gauge the approximate gaze direction of the patient or judge whether the scanning pattern was correctly applied based on the behaviour of the patient. As this procedure is rather subjective and error-prone, adding eye tracking in the IH-CST training would provide the patient and the therapist with better insights into the actual scanning patterns made during the different exercises. These insights will then be the basis for better feedback to the patients, which should in turn improve their progress in visual rehabilitation.

Secondly, the change in difficulty when transitioning from one step to the next in the training procedure can be too large for some people, as it is difficult to transfer learned perceptual skills to a new task (Ellison and Walsh, 1998). At present, the training environment transitions from a rather simple, static and predictable situation (step 3) to a dynamic environment (step 4 and 5). Therefore, having intermediate levels of difficulty, complexity and predictability would ease the transition between these steps. A possible way to achieve this is by using virtual reality (VR). In VR it is possible to create environments with increasing complexity, which can be controlled by the occupational therapist and are safe by nature (Rizzo, 2005).

An important aspect for successful integration of eye tracking and VR is that individuals with hemianopia and occupational therapists need to be willing and able to use the devices without external help.

We therefore want to assess the user experience of individuals with hemianopia, normal-sighted controls and the occupational therapists with a mobile eye tracker (study 1) and a VR headset (study 2) during different exercises for mobility training. The devices were tested in exercises that were part of the current IH-CST protocol. We established a list of requirements, which should be fulfilled by the tested devices, based on the expertise of occupational therapists, technical staff of Royal Dutch Visio and the authors of the study. Given this list of requirements we designed a questionnaire to assess the usability of the devices (Appendix A and B). The goals of this study are to find out whether the overall user experience of participants is sufficiently positive to continue working with these devices. In addition, we want to establish which criteria still need improvement based on the feedback of our participants to make the integration into IH-CST training possible.

Methods

Questionnaire for both studies

The first part of the questionnaire (part A) consisted of questions for individuals with hemianopia and normal-sighted control subjects. The second part (part B) consisted of the questions for occupational therapists. Each question could be answered by ticking one of four boxes, which stood for the options: strongly disagree, somewhat disagree, somewhat agree and strongly agree. For the analysis, the answers were then converted to a score. This ranged from 1 to 4 if a positive answer to the question meant that the requirement was fulfilled. The score ranged from 4 to 1, if a negative answer to the question meant that the requirement was fulfilled. In addition, participants could comment on each question in an additional box giving them the opportunity to explain their answer and provide details about problems that occurred.

Study 1) Eye tracking

Participants

Seven individuals with hemianopia (mean age: 53 years, SD 17), who were enrolled in the vision rehabilitation program at Royal Dutch Visio in Amsterdam, and three occupational therapists tested the head mounted eye tracker. We did not include the normal-sighted controls in the first study, as the device is intended for usage with individuals with hemianopia. As the exercises that were performed in this study wer already established as part of IH-CST training, we knew the effect of exposing our participants to these exercises.

Detailed information about the individuals with hemianopia who participated in study 1 is provided in Table 1. All occupational therapists had received the standard education for their profession in the Netherlands had several years of professional experience in visual rehabilitation and they were trained and highly experienced in providing the IH-CST.

Table 1.

Information on participants (individuals with hemianopia) in study 1.

10.21307_vri-2020-002-tbl1.jpg

Materials

We used a head mounted eye tracker (Pupil Labs: Pupil Core; Kassner, 2014), which field of view horizontally and two eye cameras recording the eye movements at a sampling rate of 200 Hz. The eye tracker was connected to a laptop, which had the Pupil Labs software installed (Pupil Labs, Berlin; https://github.com/pupil-labs/pupil). The Pupil Labs software provides an interface through which the eye tracker can be calibrated, the data can be recorded, and the recordings can be replayed on top of the video recordings from the world camera.

Procedure

Study 1 was performed at Royal Dutch Visio in Amsterdam. The study was approved by the Ethical Committee of the Psychology department of the University of Groningen and all participants provided informed consent. Each participant completed one, two or three exercises from the IH-CST protocol while their eye movements were tracked with a head mounted eye tracker. Some exercises were repeated several times during a training session and eye movements were recorded each time. Participants were instructed to perform the scan pattern that they had already learned in previous steps of IH-CST training. As participants with hemianopia were clients of Royal Dutch Visio, who were at different stages of their rehabilitation, they only performed exercises that were suitable for their current training stage. They could be split into three groups: beginners (2 participants), intermediate (3 participants) and advanced (2 participants). Beginners only performed exercise 1, intermediates performed exercise 1 and 2, and advanced participants performed exercise 1, 2 and 3. Table 1 shows which participant performed which exercises. The eye movement datasheet and the movie of the eye movements projected onto the visual field of the patients were saved. Individuals with hemianopia and their occupational therapist watched the video recordings of the eye movements directly after each exercise, with one exception where the video recording was discussed about a week later. The occupational therapists assessed whether the required scanning pattern had been performed correctly based on the video recordings and gave feedback to the individuals accordingly. Individuals with hemianopia filled out part A and the occupational therapist, who accompanied them, filled out part B of the questionnaire after a test session. This means that beginners rated the eye tracker based on their experience in exercise 1, intermediates based on their experience in exercise 1 and 2, and advanced participants based on their experience in all exercises. This is the same for the therapists who accompanied them. We determined the average score over one questionnaire of each participant, as well as the average score over all questionnaires of each participant group (beginners, intermediates, advanced and their corresponding occupational therapists). A score of above 2 out of 4 points indicates an overall positive user experience. We also collected the comments that were made by all participants about a certain criterion to illustrate which specific aspects need to be changed to improve the usability of the mobile eye tracker.

Exercise 1: Applying the scanning pattern in a static condition

Participants had to call out 30 or 60 numbers presented on a large screen (2x2 m, viewing distance: 85 height of the numbers: 3cm). They initially fixate a cross in the centre. Then numbers were displayed at different locations on the screen. The patient needed to use the learned scan pattern (without head movements) to see all numbers. Each exercise took on about 1-3 minutes, the exact time depending on the reaction times of the participant. Participants wore the head mounted eye tracker while performing exercise 1, seated in a chair with the laptop, to which the eye tracker was connected, outside their field of view.

Exercise 2: Applying the scanning pattern in a simple indoor environment with static obstacles.

Individuals with hemianopia walked along a hallway of 17 m length and 1.5 to 3 m widths in the therapy centre, while performing the learned scanning pattern. This exercise can be varied, by introducing obstacles and targets on the walls, that the participant needs to see. In this study participants performed the exercise with coloured cards pinned to the walls, of which they have to report the colour to the therapist. Individuals with hemianopia performed this exercise wearing the head mounted eye tracker, while the occupational therapists accompanied them carrying the laptop, which was connected to the eye tracker. Individuals with hemianopia were instructed to perform the scanning pattern while walking, as they would usually do this exercise during the IH-CST. This exercise takes approximately 3 minutes, in which the participant walks along the hallway in both direction.

Exercise 3: Applying the scanning pattern in outdoor mobility situations.

Individuals with hemianopia and occupational therapists crossed a road and walked down a footpath, while the individuals with hemianopia were wearing the head mounted eye tracker and a backpack with the laptop was connected to. The individual with hemianopia was instructed to perform the learned scanning pattern while walking. All participants crossed the same road and walked along the same footpaths. As this exercise was performed in public space, the traffic conditions were not the same for all participants. Crossing a road once took about 1 min and walking along a footpath was recorded for 1:40 minutes per recording.

Study 2) Virtual reality

Participants

Six individuals with hemianopia, four normal-sighted controls (3 male, 1 female, mean age: 48 years, SD: 22 years) tested the VR headset. One occupational therapist guided them while performing the exercises. The therapist explained the virtual environment, which task to perform and guaranteed the safety of the person wearing the headset, while they moved within the setup. Detailed information on the individuals with hemianopia can be found in Table 2. Normal-Sighted controls were included to test the effect of a visual field defect on the experience of virtual environments. This way we could control for the possibility that individuals with hemianopia had a negative experience with VR due to the presence of their visual field defect. The occupational therapist had received the standard education for their profession in the Netherlands, had several years of professional experience in visual rehabilitation, and was trained and highly experienced in providing the IH-CST. Individuals with hemianopia were enrolled in the vision rehabilitation program at Royal Dutch Visio in Haren. They had completed different stages of IH-CST training at the time of the experiment. All participants were naïve to using a VR headset. The occupational therapist was involved in planning the tests and therefore had about 4 months time to familiarize with the technology.

Table 2.

Information for participants with hemianopia and normal-sighted controls in Study 2

10.21307_vri-2020-002-tbl2.jpg

Materials

The exercises in virtual reality were presented with a HTC Vive (Manufacturer, City; horizontal visual field size: 110 degrees, refresh rate: 90 Hz, resolution: 1080 x 1200 pixels per eye). Two motion sensors (called Vive base stations) tracked the position of the headset in space by creating infrared pulses that are detected by the headset. The range of motion, which that can be tracked, is determined by the distance between the two sensors. They are therefore placed at opposite ends of the setup.

We used two different setups of the hardware. The first setup was used while developing the virtual environments. The second one was used in the final experimental session with our participants.

Setup 1: The VR headset was connected to a laptop that was placed on a table, with a cable of 4 m length. The maximum range of movement of the person wearing the headset walking in a straight line was about 4 m. due to the size of the room in which the experiment was performed. The motion sensors were placed at either end of the setup, meaning that they were approximately 4 m apart.

Setup 2: The VR headset was connected to a PC that was placed on a cart. The motion sensors were placed about 5.5 m apart. The therapist moved the cart with the PC so that the range of movement for the participant was 5.5 m. Virtual environments were developed by the ‘The Virtual Dutch Men’ using Unity (https://thevirtualdutchmen.com).

Procedure

Study 2 was performed at Royal Dutch Visio in Haren. The study was approved by the Ethical Committee of the Psychology department of the University of Groningen and all participants provided informed consent. We first met with our VR developers for two development sessions, where they presented a first version of the requested virtual environments for three different exercises. These first versions were tested by one individual with hemianopia, an age matched control and authors of the study. Two of these exercises were further developed into their final versions and tested in an experimental session by the participants mentioned in the ‘participants’ section above. For completeness we will describe all three exercises, including the first version of exercise 3, which was performed by one individual with hemianopia (70 years) and 3 controls (mean age 43). Individuals with hemianopia and normal-sighted controls filled out a questionnaire after each exercise. The occupational therapist filled out one questionnaire for each exercise at the end of the experimental session. We computed the average score that was given for exercises 1 and 2 of each participant group (individuals with hemianopia, normal-sighted controls and the occupational therapist) in their final versions. We also computed the score that was given by the individual with hemianopia and each of the controls given to exercise 3. In addition, we collected the comments that were made by each participant about each exercise to evaluate how we can improve the exercises in the future.

Exercise 1: Walking along a virtual hallway

Normal-sighted controls and individuals with hemianopia started walking at one end of the virtual hallway and had to walk to the other end. The view for the participant when starting this exercise is shown in Figure 1A. Figure 1B shows the entire hallway from above to get an overview over the full trajectory (not shown to the participant). The position of the headset was tracked in space so that participants perceived motion in VR in synchrony with their actual motion in the setup. As the range of motion, that our final setup allowed, was too short to complete a walk along the entire trajectory of the hallway, participants turned around after 5.5 m. While turning around, a black screen was shown and the virtual environment was turned 180 degrees, so that the participants continued to walk along a new part of the hallway.

Figure 1:

(A) Initial view of the participant in exercise 1 (walking along a hallway) of version 5 of the exercise showing different obstacles, as well as doors, windows and avatars. (B) View from above in exercise 1 (walking along a hallway) of version of the exercise. This figure shows the whole trajectory the participants have to walk along, while having to turn around every 5 m within the setup. (C) View from above in exercise 2 (crossing a street) of version 1 of this exercise with traffic, showing an overview over this environment. (D) Initial view of the participant in exercise 3 (walking along a pavement) showing the pavement with construction site and surrounding buildings.

10.21307_vri-2020-002-f001.jpg

Three different versions and six scenarios (differing in the amount of obstacles in the hallway) of a virtual hallway scenario were developed. Details about these versions and scenarios can be found in Table 3. Participants tested several scenarios of this exercise, depending on how long completing a certain scenario took. The number of scenarios tested differed between participants.

Table 3.

Versions, setups and scenarios per exercise used in virtual reality.

10.21307_vri-2020-002-tbl3.jpg

Exercise 2: Crossing a virtual street

The exercise started with the participant standing on a pavement facing a two-lane road. The virtual environment was scaled in such a way that the participant walked the same distance in the virtual scenario as in the real-world setup. This means that they stood on one end of the setup and walked in a straight line to the other end of the setup to cross the virtual street, completely.

Two scenarios of this exercise were implemented. Crossing the street with or without traffic (see Table 3). Figure 1C shows the scenario with traffic from above to give an impression of the virtual environment and the relative sizes of the objects in it.

Exercise 3: Walking along a pavement

Participants were placed on a pavement, which was surrounded by houses. The pavement was partly blocked by a construction site. Participants were instructed to walk past the construction site. Figure 1D shows this environment from the initial perspective of the participants.

Results

Study 1) Eye tracking

All exercises

The overall feedback for the head mounted eye tracker was very positive with a mean score of 3.7 ± 0.5 given by the beginners, a mean score of 3.8 ± 0.5 given by the intermediates and a mean score of 4 ± 0 given by the advanced individuals with hemianopia. The corresponding therapist questionnaires gave an average score of 3.6 ± 0.6, 3.4 ± 0.9 and 3.8 ± 0.6. Figure 2 shows the average score and standard deviations of each participant.

Figure 2:

Average score given to the head-mounted eye tracker by each participant and their corresponding occupational therapist. The error bars represent the standard deviation.

10.21307_vri-2020-002-f002.jpg

We identified four main criteria that were most important for the user experience, based on the comments mentioning them:

  • 1. Usability in mobile situations and outside

  • 2. Usability with glasses

  • 3. Improvement of feedback

  • 4. Being easy to use (no disruption of regular procedure of therapy session)

Table 4 provides an overview of the comments made by individuals with hemianopia and occupational therapists.

Table 4.

Score and comments for the head mounted eye tracker for each criterion evaluated by patients and therapists.

10.21307_vri-2020-002-tbl4.jpg

Study 2) Virtual reality

Exercise 1: Walking along a virtual hallway

The feedback of individuals with hemianopia and normal-sighted controls was more positive than the feedback from the occupational therapist for this exercise.

Individuals with hemianopia rated this exercise with an average score of 3.9 ± 0.5, controls with an average score of 3.8 ± 0.5 and the occupational therapist with an average score of 2.5 ± 1.4. Figure 3 shows the score given by each participant.

Figure 3:

Average score given to the virtual hallway scenario by individuals with hemianopia, normal sighted controls and the occupational therapist. The error bars represent the standard deviation.

10.21307_vri-2020-002-f003.jpg

The most important challenge in this exercise was to simulate the movement of the participant over a distance of several meters in the virtual environment, which is much longer than the range of movement in exercise 2. The current setup of the hardware did not allow for a large enough range of movement to walk all the way to the other end of the hallway, although using setup 2 improved it. Our approach of letting the participants turn around before continuing along the next section of the hallway, was not a good solution as it led to a feeling of insecurity by participants, when they had to turn around without visual input. This comment was made in several questionnaires. An overview over the comments, which were specifically relevant for this exercise, is shown in Table 5.

Table 5.

Score and comments on criteria that are specifically important to use the VR headset for walking along a virtual hallway.

10.21307_vri-2020-002-tbl5.jpg

Exercise 2: Crossing the street

This exercise was rated with 3.9 ± 0.5 points by individuals with hemianopia and 3.7 ± 0.8 points by normal-sighted controls. The occupational therapist gave this exercise a mean score of 2.8 ± 1.2 points. Figure 4 shows the score and standard deviation of each participant. An overview over the comments made about this exercise are shown in Table 6.

Figure 4:

Average score given to the virtual street-crossing scenario by individuals with hemianopia, normal sighted controls and the occupational therapist. The error bars represent the standard deviation.

10.21307_vri-2020-002-f004.jpg
Table 6.

Comments on criteria specifically to use the VR headset for crossing a street. The occupational therapist who tested this exercise did not make any comments.

10.21307_vri-2020-002-tbl6.jpg

The range of motion was sufficient to cross the road entirely. Several participants (individuals with hemianopia and normal-sighted controls) commented that they did not like the aesthetics of the virtual environment. The occupational therapist rated the range of motion as not sufficient in this exercise.

Exercise 3: Walking along a pavement

Table 7 shows the comments that were made about this exercise. Participants had a positive opinion about this virtual environment. The individual with hemianopia rated it with a score of 4 ± 0 points and the normal sighted controls with a score of 3.8 ± 0.4 points.

Table 7.

Comments on criteria that are specifically important to use the VR headset for walking along a pavement.

10.21307_vri-2020-002-tbl7.jpg

Discussion

The main finding of this study is that the user experience of individuals with hemianopia and normal-sighted controls was generally positive for both tested devices. While the occupational therapists, who evaluated the head mounted eye tracker, also reported a positive user experience the occupational therapist, who evaluated the virtual reality setup, reported a more negative experience. Below, we describe our findings, conclusions and these experiences in more detail.

Providing feedback by showing replay of eye movements improves the user experience of individuals with hemianopia and therapists in mobility training.

A positive aspect, noted by individuals with hemianopia and therapists, of using the head mounted eye tracker during the therapy sessions was that it offers the possibility to replay the eye movements to give feedback to the participant. This was seen as a way to improve therapy progress, as individuals with hemianopia are able to reflect on their own viewing behaviour. Moreover, it helps the therapists to explain the (need for) compensatory strategies. All participants and therapists saw the recording of the eye movements during the four exercises as a beneficial addition to the training The device fulfilled most of the requirements of individuals and therapists.

A problematic aspect was the usability with glasses, as the eye tracker was less precise in tracking the pupil when participants were wearing their glasses, especially under varying luminance conditions. This is a common problem for eye trackers, as glasses and contact lenses and changes in luminance distort the image of the eye that is captured by the eye tracker (Dahlberg, 2010; Fuhl et al., 2016). The Pupil Labs eye tracker provides a confidence interval of the recorded data, which can be used to judge the reliability of the recorded eye tracking data.

VR Scenarios

All participants (individuals with hemianopia and normal-sighted controls) reported a preference for ‘lively’ and ‘realistic’ scenarios in virtual reality. Most participants thought that adding exercises in virtual reality would benefit vision rehabilitation therapy. They also think that if we can add the possibility to replay a recording of the exercise from the point of view of the participant, which also shows the eye movement behaviour, this will improve feedback.

These findings are in line with a study by Ishoel and Kanstad (2017), who tested different head mounted VR displays and concluded that VR is a promising method for vision rehabilitation therapy. While Ishoel and Kanstad developed a mini game, which is supposed to support therapy progress, we attempted to make the current virtual reality scenario resemble the real-world exercise as much as possible by letting our participants walk within the setup. This approach makes it easy for participants to understand the task and, especially for individuals with hemianopia, to understand the objective of the exercise and transfer learned skills. It has previously been shown that using VR to train patients with unilateral spatial neglect can improve their visual – spatial performance, as well as their ability to cross a real street (Katz et al., 2005). However, the setup described by Katz et al. was not immersive and did not require the participants to move. In a study by Iorizzo et al. (2011) individuals with hemianopia and normal-sighted controls were presented with an immersive virtual environment in which they had to detect basketballs that were presented in the periphery. This task was performed seated as well as while walking in an L-shaped path. They found that individuals with hemianopia had more difficulty detecting targets in the periphery when walking than the control group. This finding emphasizes that it is necessary for individuals with hemianopia to practice eye movements while walking. Under ideal circumstances, the immersive VR environment that we created can be used while a large space is available in which the participats can safely move the required distances. To walk through the complete virtual hallway at once, the available real walking distance should be long enough, for example the same length as the actual hallway, in which the exercise is performed. Preferably, it would be even larger to allow for directional errors. At present, this was technically not feasible as it would also require that the sensors of the VR headset had to be placed much further than 5.5 meters apart, which would have made tracking of the headset impossible. In the first version of this exercise, we tested two options to compensate for this. The first idea was that one step in reality corresponded to several steps in the virtual world. The second idea was to let the participant continue along a new part of the hall virtual hallway when they turn around at the end of the setup. The participant could continue to walk along the virtual hallway by walking back to the other side of the room. As the first caused dizziness and insecurity when walking, the second idea was implemented in the following sessions. Our solution to let participants turn around after a few meters however interrupted the flow of the task and this was experienced as irritating by the participants, both the normal-sighted controls and the individuals with hemianopia. Another problem of our current setup 2 was that the cart with the computer had to be dragged along next to the participant. In our setup, the power cable for the computer was often in the way causing problems to move the cart. Therefore the occupational therapist was assisted by two people to move the cart, which is an undesirable solution in an actual therapy sessions. When turning around at the end of the setup, the cable of the VR headset was also often in the way and participants had to be instructed such that they would only turn around in one direction.

Therefore, to be able to perform these exercises within a regular training session, the setup will need to change fundamentally. Despite these difficulties, the participants and the therapist were of the opinion that adding VR exercises to the regular training sessions will be beneficial. Instead of using a larger space, which can be costly or simply not available, the space issue may also be solved by adding an alldirectional treadmill to the VR setup that will enable walking longer distances in all directions (e.g. https://www.virtuix.com). A further way to improve future setups is to add eye tracking to the VR exercises, which is technically feasible by now (e.g. https://pupillabs.com/products/vr-ar/).

Limitations

In this study, not all of our participants completed and evaluated the same exercises. This has some practical reasons. In study 1 the eye tracker was tested during regular training sessions of the participants. Therefore they only performed exercises that were suitable for their current training stage. VR exercise 3 was not included in the final test session, due to time constraints, meaning that there was too little time for the developers to continue developing this exercise and there would also not have been enough time on the day of testing to let all participants perform another exercise. We still decided to report the results of the questionnaire for this exercise here, as the participants liked the virtual environment and we therefore plan to continue its development in the future. This study does not evaluate the benefit in terms of training progress of adding eye tracking and VR to the current protocol. This will be the topic of a follow up study. The results of this study represent the subjective experiences of our participants. In our opinion a positive user experience is an important aspect for the successful integration of these devices in the IH-CST training.

Conclusion

This study shows that integrating eye tracking and VR technologies into vision rehabilitation therapy may positively impact training outcomes. Individuals with hemianopia are able to use these technologies and they consider these helpful additions to the existing training exercises. Therapists welcome the additional objective information provided but indicate that VR setups first need to allow for a larger range of mobility. Moreover, integrating eye tracking into the setup could further increase the value of VR. relevant for this exercise, is shown in Table 5.

References


  1. Dahlberg, J. (2010) ‘Eye Tracking With Eye Glasses’, Umea University, Master The.
  2. Ellison, A. and Walsh, V. (1998) ‘Perceptual learning in visual search: Some evidence of specificities’, Vision Research, 38(3), pp. 333–345. doi: 10.1016/S0042-6989(97)00195-8.
    [PUBMED]
  3. Fuhl, W. et al. (2016) ‘Pupil detection for head-mounted eye tracking in the wild : an evaluation of the state of the art’, Machine Vision and Applications. Springer Berlin Heidelberg, 27(8), pp. 1275–1288. doi: 10.1007/s00138-016-0776-4.
  4. Haan, G. A. De et al. (2015) ‘The Effects of Compensatory Scanning Training on Mobility in Patients with Homonymous Visual Field Defects: A Randomized Controlled Trial’, pp. 1–29. doi: 10.1371/journal.pone.0134459.
  5. Howard, C. and Rowe, F. J. (2018) ‘Adaptation to poststroke visual field loss: A systematic review’, Brain and Behavior, 8(8), pp. 1–21. doi: 10.1002/brb3.1041.
  6. Iorizzo, D. B. et al. (2011) ‘Differential impact of partial cortical blindness on gaze strategies when sitting and walking – An immersive virtual reality study’, Vision Research. Elsevier Ltd, 51(10), pp. 1173–1184. doi: 10.1016/j.visres.2011.03.006.
  7. Ishoel, M. and Kanstad, I. M. (2017) ‘Martinus Ishoel Iselin Molnes Kanstad’, (June).
  8. Katz, N. et al. (2005) ‘Interactive virtual environment training for safe street crossing of right hemisphere stroke patients with Unilateral Spatial Neglect’, Disability and Rehabilitation, 27(20), pp. 1235–1243. doi: 10.1080/09638280500076079.
  9. Kassner, M., Patera, W., & Bulling, A. (2014). Pupil: An Open Source Platform for Pervasive Eye Tracking and Mobile Gaze-Based Interaction. Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing: Adjunct Publication, 1151–1160. https://doi.org/10.1-145/2638728.2641695
  10. Nelles, G. et al. (2001) ‘Compensatory visual field training for patients with hemianopia after stroke’, Neuroscience Letters, 306(3), pp. 189–192. doi: 10.1016/S0304-3940(01)01907-3.
    [PUBMED]
  11. Pollock A, Hazelton C, Rowe FJ, Jonuscheit S, Kernohan A, Angilley J, Henderson CA, Langhorne P, Campbell P. Interventions for visual field defects in people with stroke. Cochrane Database of Systematic Reviews 2019, Issue 5. Art. No.: CD008388. doi: 10.1002/14651858.CD008388.pub3.
  12. Rizzo, A. S. (2005) ‘A SWOT Analysis of the Field of Virtual Reality Rehabilitation’, Presence, 14(2), pp. 119–146.
XML PDF Share

FIGURES & TABLES

Figure 1:

(A) Initial view of the participant in exercise 1 (walking along a hallway) of version 5 of the exercise showing different obstacles, as well as doors, windows and avatars. (B) View from above in exercise 1 (walking along a hallway) of version of the exercise. This figure shows the whole trajectory the participants have to walk along, while having to turn around every 5 m within the setup. (C) View from above in exercise 2 (crossing a street) of version 1 of this exercise with traffic, showing an overview over this environment. (D) Initial view of the participant in exercise 3 (walking along a pavement) showing the pavement with construction site and surrounding buildings.

Full Size   |   Slide (.pptx)

Figure 2:

Average score given to the head-mounted eye tracker by each participant and their corresponding occupational therapist. The error bars represent the standard deviation.

Full Size   |   Slide (.pptx)

Figure 3:

Average score given to the virtual hallway scenario by individuals with hemianopia, normal sighted controls and the occupational therapist. The error bars represent the standard deviation.

Full Size   |   Slide (.pptx)

Figure 4:

Average score given to the virtual street-crossing scenario by individuals with hemianopia, normal sighted controls and the occupational therapist. The error bars represent the standard deviation.

Full Size   |   Slide (.pptx)

REFERENCES

  1. Dahlberg, J. (2010) ‘Eye Tracking With Eye Glasses’, Umea University, Master The.
  2. Ellison, A. and Walsh, V. (1998) ‘Perceptual learning in visual search: Some evidence of specificities’, Vision Research, 38(3), pp. 333–345. doi: 10.1016/S0042-6989(97)00195-8.
    [PUBMED]
  3. Fuhl, W. et al. (2016) ‘Pupil detection for head-mounted eye tracking in the wild : an evaluation of the state of the art’, Machine Vision and Applications. Springer Berlin Heidelberg, 27(8), pp. 1275–1288. doi: 10.1007/s00138-016-0776-4.
  4. Haan, G. A. De et al. (2015) ‘The Effects of Compensatory Scanning Training on Mobility in Patients with Homonymous Visual Field Defects: A Randomized Controlled Trial’, pp. 1–29. doi: 10.1371/journal.pone.0134459.
  5. Howard, C. and Rowe, F. J. (2018) ‘Adaptation to poststroke visual field loss: A systematic review’, Brain and Behavior, 8(8), pp. 1–21. doi: 10.1002/brb3.1041.
  6. Iorizzo, D. B. et al. (2011) ‘Differential impact of partial cortical blindness on gaze strategies when sitting and walking – An immersive virtual reality study’, Vision Research. Elsevier Ltd, 51(10), pp. 1173–1184. doi: 10.1016/j.visres.2011.03.006.
  7. Ishoel, M. and Kanstad, I. M. (2017) ‘Martinus Ishoel Iselin Molnes Kanstad’, (June).
  8. Katz, N. et al. (2005) ‘Interactive virtual environment training for safe street crossing of right hemisphere stroke patients with Unilateral Spatial Neglect’, Disability and Rehabilitation, 27(20), pp. 1235–1243. doi: 10.1080/09638280500076079.
  9. Kassner, M., Patera, W., & Bulling, A. (2014). Pupil: An Open Source Platform for Pervasive Eye Tracking and Mobile Gaze-Based Interaction. Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing: Adjunct Publication, 1151–1160. https://doi.org/10.1-145/2638728.2641695
  10. Nelles, G. et al. (2001) ‘Compensatory visual field training for patients with hemianopia after stroke’, Neuroscience Letters, 306(3), pp. 189–192. doi: 10.1016/S0304-3940(01)01907-3.
    [PUBMED]
  11. Pollock A, Hazelton C, Rowe FJ, Jonuscheit S, Kernohan A, Angilley J, Henderson CA, Langhorne P, Campbell P. Interventions for visual field defects in people with stroke. Cochrane Database of Systematic Reviews 2019, Issue 5. Art. No.: CD008388. doi: 10.1002/14651858.CD008388.pub3.
  12. Rizzo, A. S. (2005) ‘A SWOT Analysis of the Field of Virtual Reality Rehabilitation’, Presence, 14(2), pp. 119–146.

EXTRA FILES

COMMENTS