Opening Keynote:

Tabs, Pads, Boards and Brains: Mental Imagery as a Pervasive Display

Sarah Clinch, University of Manchester, UK

Abstract

For the last few decades, the PerDis community has realised Mark Weiser’s vision for pervasive displays that are seamlessly embedded into our environments. More recently, head-mounted displays (HMD) have become a part of that vision, making displays ever more pervasive. These displays give rise to a plethora of applications, changing the way we interact with space as well as the ways in which we think and communicate. My research has explored how pervasive displays can act as a form of distributed or extended cognition, most notably as an aid to human memory. This allows us to rehearse and enhance our memories, whilst introducing new risks for cognitive manipulation. However, many of us have access to a display that’s more pervasive than any of these devices, our own “mind’s eye”. On this mental display, we recollect and reimagine our pasts, situate ourselves in familiar places for route planning and navigation, take ourselves out of a busy moment into one of stillness and relaxation, and lose ourselves in worlds created in response to music, literature, art and film. However, the ability to create these mental sensory experiences is not universal – estimates suggest that around ~4% of people are unable to create these experiences (a condition referred to as aphantasia). This talk tracks my research journey from pervasive physical displays to visual mental imagery. I’ll outline some key research contributions at the intersection of cognition and digital technologies, and a vision for pervasive digital displays to facilitate visual thought in those with atypical mental imagery.

Bio

Sarah Clinch leads the Human-Computer Systems group at the University of Manchester. Her research spans a variety of human-focused applications of mobile and ubiquitous computing, with particular focus on the role of technology in understanding and shaping cognition (with particular interests in aphantasia and human memory). She is passionate about interdisciplinary research that creates robust systems or software that can stand up to long-term, real-world deployment, and follows best practices for open data and open science. Clinch has a PhD in Computer Science from Lancaster University (2014), was a post-doctoral researcher in the School of Computing and Communications, Lancaster University (UK), and has held visiting positions at the University of Cambridge (2016); TU Dresden (2015); and Carnegie Mellon University (2009).

Closing Keynote:

Beyond Attentive Displays: From Attention-Aware to Surprise-Aware Design

Roel Vertegaal, Radboud University, The Netherlands

Abstract

Over the past 25 years, attention-aware displays have gone from a niche research idea to wide-spread adoption. Originally invented to combat information overload in a future world of many devices, attention-aware displays are more relevant than ever in a world of fragmented attention spans. Attentive UX is now everywhere: in cars, smartphones, ambient displays, virtual reality headsets, and smart homes. In this talk, I revisit what attention actually is, why eye tracking does not describe it sufficiently, and how surprise forms a new design commodity for pervasive displays. Surprise is what results from the user’s brain predicting the world – a discrepancy between prediction and observation, in bits. It builds in the brain as a potential energy that is released as action, learning or emotion. Within this neuromorphic design framework, user performance and error can be analyzed through a single formalism. The talk closes with some examples of how surprise-aware displays might be a solution for keeping humans in the loop with AI agents, and an agenda for the design and evaluation of surprise-aware interaction using methods borrowed directly from statistical mechanics.

Bio

Professor Dr. Roel Vertegaal is Chair of Human-Computer Interaction (HCI) at Radboud University in Nijmegen, the Netherlands, where he studies, models, and builds user experiences of tomorrow. He is the founding director of the Human Media Lab, known for pioneering attentive user interfaces — including the attention-aware techniques now common in most smartphones — as well as organic user interfaces such as the foldable phone. His work spans a broad range of empirical, theoretical and design contributions to HCI for which he was elected a member of ACM SIGCHI Academy. His current research focuses on neuromorphic design, with particular interest in (inter)active inference.