Faculty and students from the Department of Human Centered Design presented research, posters and prototypes at the Association for Computing Machinery (ACM) CHI conference, April 23-28, in Hamburg, Germany. It is the oldest and largest conference in the field of human-computer interaction (HCI), bringing together researchers and practitioners from around the globe who have an overarching goal to make the world a better place with interactive digital technologies.
“This conference is enormous,” said Armin Mostafavi, a doctoral student in human behavior and design and a conference presenter. “There were nine or 10 interesting topics offered at each time slot; it was hard to choose which sessions to attend.”
The cross-disciplinary research from students and faculty in Human Centered Design featured artificial intelligence (AI), extended reality (XR), robotics and wearable technology, all with a focus on improving human access through collaborative design.
SkinPaper: Exploring Opportunities for Woven Paper as a Wearable Material for On-Skin Interactions
Paper presenter: Jingwen Zhu, doctoral student, human behavior and design
Faculty co-author: Cindy (Hsin-Liu) Kao, assistant professor of human centered design, director Hybrid Body Lab
SkinPaper is an accessible and wearable material for on-skin computer interactions. Inspired by origami, kirigami and other classic forms of paper crafting, SkinPaper uses woven strips of washi paper and technology to create a flexible on-skin material.
“Because the material is being used on-skin, we wanted to create a safe, accessible, hydro-phobic surface. We found that using an off-the-shelf silicone-spray was an effective way to waterproof the paper,” said Jingwen Zhu, a doctoral student and researcher in the Hybrid Body Lab, which works at the intersection of HCI and fashion design and body art.
After optimizing the weaving process, the lab partnered with local weavers to explore prototypes. Designs included a serpentine weave that fits the space between the hairline and the back of the ear and contains sensors that can be rubbed to communicate anxiety or concern to loved ones and an open weave that conforms to the shoulder or knee joint and uses shape-memory (SMA) wire to transfer heat to the joint.
Nadine El Nesr ’23, a fashion design management major who works in the Hybrid Body Lab, is a paper co-author: “As a designer it’s been interesting to work with SkinPaper. The weave holds shape in a way that fabric couldn’t due to its drape,” said El Nesr.
KnitDema: Robotic Textile as Personalized Edema Mobilization Device
Paper presenter: Jin Hee (Heather) Kim, doctoral student, human behavior and design
Faculty co-author: Cindy (Hsin-Liu) Kao
KnitDema is a robotic textile for patients suffering from hand edema, a painful condition caused by fluid trapped in the tissue. Presented by Heather Kim, a doctoral student in human behavior and design, the project is a collaboration between Cornell’s Hybrid Body Lab, Weill Cornell Medical Center and therapists at Cayuga Medical Center in Ithaca. Patients suffering from lymphedema must use bulky pumps or work with licensed therapists for relief. KnitDema is a personalized rehabilitation device that compresses in sequence to move fluid out of the fingers. The glove is knitted with hollow channels that are inserted with SMA wire that contract when heated. Using a knitted garment as the foundation provides natural elasticity and makes it easier and more affordable to customize the device to individual patients.
“We first built a silicone finger to mock the flow of the lymphedema,” said Kim. “We then worked on a pilot study with patients living with the condition.”
“The team at Cayuga Medical Center were great partners,” said Kao. “We really value the human-centered approach—whether working with patients or makers—to design products that provide broad access.”
Harnessing Biomedical Literature to Calibrate Clinicians’ Trust in AI Decision Support Systems
Paper presenter: Yuexing Hao, doctoral student, human behavior and design
Faculty co-author: Qian Yang, assistant professor of information science in the Cornell Ann S. Bowers College of Computing and Information Science
Doctoral student Yuexing Hao, together with a research team from information science, is using artificial intelligence (AI) to improve doctor decision making. The decision support tool (DST), powered by AI, works more like a clinician dialogue with colleagues than a search tool. The DST lists patient information, medical history and lab test results on one side, with the AI’s personalized diagnosis or treatment suggestion on the other, followed by relevant biomedical studies.
"We interviewed a diverse group of clinicians and clinician librarians to help develop the new tool,” said Hao.
Hao’s previous experience in computer science and engineering focused on theoretical problem solving. After the illness of a family member, she chose to focus her design education on a human-centered approach, enabling families to have access to the best possible information and to have a better dialogue with clinicians.
“I want to integrate the human-centered mindset with my engineering skillset to improve health,” said Hao.
Poster presenter: Qi Yang, doctoral student, human behavior and design
Faculty co-author: Saleh Kalantari, assistant professor, human centered design and director Design + Augmented Intelligence Lab
Can AI enhance design? Yes, according to Qi Yang. Yang, a doctoral student with a background in architecture, is using AI to improve architectural design. Working with Saleh Kalantari, director of the Design and Augmented Intelligence Lab (DAIL), Yang presented a CHI poster session, “Co-Design with Myself,” about a brain-computer interaction virtual reality (VR) tool that enhances metacognitive monitoring, or the awareness of one's own thoughts, during the design process.
“This tool is meant to enhance creativity, allowing users to design better,” said Yang. “Modulating one’s feelings of confidence toward design options and metacognitive monitoring are important components of design intuition.”
The AI companion learns from the user’s own brain activity using electroencephalogram, or EEG. By measuring users’ EEG to panoramic images in VR, researchers tried to decode users’ emotional responses to different environmental designs. VR is then used to provide more immersive design scenarios.
“The design process happens very fast,” said Yang. “The goal of this project is to augment that process by providing a novel interaction to the designer: to design in VR, and see real-time feedback from their own emotional reactions predicted by the AI companion.”
Designing Virtual Environments for Social Engagement in Older Adults: A Qualitative Multi-site Study
Paper presenter: Armin Mostafavi, doctoral student, human behavior and design
Faculty co-author: Saleh Kalanatari
Bill (Tong) Xu and Armin Mostafavi, both doctoral students in human behavior and design, co-authored a paper using extended reality (XR) research to design virtual environments for social engagement in older adults with Kalantari and partners at Weill Cornell and Florida State University (FSU).
VR is increasingly used as a platform for social interaction. However, it is still underused with older adults, and few empirical studies have been conducted. The CHI presentation followed previous studies with a VR garden that demonstrated that VR can be a good tool for engaging older adults with and without cognitive impairments. The new research focused on designing for social interaction by pairing two individuals from different parts of the country in a VR environment.
During the multi-step study, users first became acquainted with one another, then planned a virtual trip together, and finally put together a virtual collage of trip memories. The results indicated a strong link between the VR tool and social connections and conversations, indicating that design guidelines could improve social VR for older adults. “
Our collaboration with Weill and FSU on the VR social interaction study underscores the transformative potential of design in enhancing lives,” said Kalantari. “By meticulously examining human-environment interactions within virtual realms, our user-centered approach enables us to craft immersive experiences that nurture meaningful connections.”
SocialStools: a socio-spatial interface for fostering togetherness between strangers.
Paper presenter: Ge (Serena) Guo, doctoral student, information science
Faculty co-author: Keith Evan Green, Jean and Douglas McClean Professor in Human Centered Design and Mechanical Engineering, director Architectural Robotics Lab
Doctoral student Serena Guo, advised by Keith Evan Green, director of the Architectural Robotics Lab, presented research and prototypes that encourage human connection. Ge (Serena) Guo presented a paper co-authored by Green about furniture that fosters togetherness between strangers. The paper, which received a conference honorable mention award, focuses on SocialStools—responsive stools on casters that generate sound and imagery in the environment. The study transformed three strangers, each perched on a SocialStool, into a trio of sound-and-image makers who interact through creative play, indicating that spatial interfaces can help break down barriers to interpersonal connection.
“The research intended to bring the lost connections of strangers back, which is especially important in the time after the Covid-19 quarantine, ” said Guo. “By emphasizing in-person interactions, SocialStools help strangers cultivate a sense of togetherness by augmenting their proxemic behaviors (the amount of spatial separation individuals naturally maintain in various social and interpersonal situations) in physical space”.
Student researcher: Debbie Jung ’23, design and environmental analysis
Faculty mentor: Keith Evan Green
Debbie Jung ’23, a design and environmental analysis major, was a student finalist in the conference’s student research competition with her prototype, Coco: Compost Companion. The wearable pet supports composting habits via an interactive design. Jung originally created the prototype for a design solution assignment in Green’s course, DEA 2730: Human Centered Design.
“Sometimes we lack the empathy needed to make climate-based lifestyle changes if we do not live in an area that feels the effects of climate change directly,” said Jung.
With Coco, users wear a cute green pin with GPS tracking to help them find the closest compost facility. The pet wags its tail when the user composts waste, fostering empathy and engagement with other users.
Top image: SkinPaper: this woven application is a sensing slider with the pass of a thumb