Imprint: The content of this blog entry is based on VR experiments in the context of projects carried out in the scope of the authors masters degree studies at FHNW University of Applied Sciences and Arts Northwestern Switzerland under the supervision of Prof. Dr. Doris Agotai.
Because VR is still a rather new technology and there are not many established interaction patterns yet, it is important to continuously get user feedback during the development process and apply the learnings immediately to the next iteration of the prototype. This frequent lean feedback loop happens in informal test settings as opposed to periodical more formal user and usability testing procedure, which may include a variety of aspects such as comprehensibility, effectiveness, efficiency, satisfaction, learnability, retainability, ease of use, speed of performance, rate of errors and user comfort.
Testing procedure
In our testing scenario the context, overarching idea and features of the application were briefly explained to the probands prior to launching the test user into the VR experience where they had the opportunity to explore the virtual data space applying the thinking aloud method. Then they were given specific tasks to solve, for instance changing the perspective, retrieving detail information about a collection item, applying filters to the dataset, or sharing their interpretation of the visualization concept. Following the test the probands were surveyed about their experience by the means of a questionnaire asking questions on the usability of specific features of the application, how they estimate their own performance in the given test tasks as well as their overall user experience.
Test subjects and summary of results
In total a sample of 8 users participated in this more comprehensive testing procedure. The probands were 75% male, 25% female, aged from 22 to 35 years with diverse occupational backgrounds. Most of them had little experience with VR stating that they had only tried it once or twice, or a few times before, yet some were more experienced with computer games.
More than the majority found the experience easy or rather easy to use and most had no problems with locomotion in the virtual space. The operation of the perspective controller was a little bit more challenging. The application of filters, obtaining detail information about collection items and comprehensibility of the data visualisation were perceived differently by the users resulting in mixed feedback. Nevertheless all users found the given task easy to solve. 7 out of 8 considered the experience to be very enjoyable or enjoyable, however there were mixed reactions concerning the insight provided, yet all reported that it offered them a fresh perspective on the dataset.
Insights from Testing
The testing observations resulted in many valuable findings. The most important learnings are summarized below:
- Regarding the perspective controller, many test users commented that plus and minus icons (as in the previous version of the prototype) would be more intuitive for them than the arrow icons for up and down. In addition, seamless zooming instead of stepwise adjustment of zoom levels for shifting the perspective between overview and detail was suggested. And the user experience of this feature might be further improved if the the user would be moved along the line of sight instead of the vertical axis. It was also mentioned that it is difficult to keep orientation when on ground level. Stronger orientation support could probably be provided by the control center or additional orientation aid such as a compass.
- Concerning filter interactions the consensus was that filters should be applied instantly. An additional option to select all or deselect all would be desired by some. And the default setting has to be a filter combination that provides an interesting selection of items.
- Most users intuitively discovered how supplementary information about collection objects can be retrieved on demand and for those who did not discover this feature on their own, it was easy to apply the corresponding interaction pattern once a hint was given. Due to the orientation challenges on ground level perspective some users suggested that it would be convenient to find out more about a collection item by the means of magnification of the selected item, triggered by simply pointing at it instead of having to approach it for this purpose. However, we hold the view that the spatial movement intensifies the exploration process.
- No attention was given to the control center. The reason for this is that it was almost invisible for most users, for some it was out of the field of view or illegible due to too small font size and others just suppressed it because the information on display was not considered to be relevant. A better solution might be to display the information only on demand, though larger in size and more prominent in position.
- The abstract visualisation style of the application was well received by the majority of the test users, yet a visual differentiation between countries and surrounding might increase clarity. Only one person expressed the desire for a higher level of detail of geographical features such as rivers, mountains and so on. Additional landmarks might offer help for orientation in the virtual world. The higher-level data visualization by the means of the choropleth map was not obvious for the majority of test user, yet after being pointed out the feature was well received and comprehensible for all.
- What may constitute legitimate reasons for the mixed reactions concerning the insight provided by the experience could be the fact that on one hand side the tested prototype only included a small portion of the original data set because the data owner hasn’t provided full access to the archive database yet and on the other hand side the intended filtering options have not yet fully been implemented. It is assumed that more data will lead to more connections and correlations between individual data points and more interaction possibilities will result in a more interesting exploration space. And the combination will eventually lead to the discovery of more insight.
- Last but not least, most users of the target group are inexperienced with VR and therefore need clear instructions about the use of the hardware (the controllers in particular). Explanation on how to navigation in the virtual space as well as how to make use of the features provided by the application are required to take full advantage of the proposed data exploration experience. The best way to learn this would be by the means of an interactive tutorial in the way it is often done in games. To compensate for the lack of such an introduction, clear instructions need to be given to the user and the operative rules have to be explained by the host. In addition it can be useful to guide the user through a specific usecase to become familiar with the application.
A way to demonstrate usage and interaction principles in one specific use case is by applying a narrative pattern called Martini glass structure, which begins with a single-path author driven approach following a tight narrative path (representing the stem of the glass). Once the intended narrative is complete, the experience is opening up to a user driven explorative stage where the user is free to interactively explore the data (representing the body of the glass). [1]
Testing has provided many valuable insights that will feed into the further development of the prototype. Although pointing out some minor issues it has revealed that this form of spatial visualization and interaction with data provides benefits for the data exploration task, that the application is engaging and enjoyable and that the experience indeed offers a fresh perspective on the given dataset.
[1] Edward Segel and Jeffrey Heer (2010): “Narrative Visualization: Telling Stories with Data” in IEEE Transactions on Visualization & Computer Graphics.