Virtutello: Balancing Physical and Digital Interaction in a Mixed Reality Sculpting Tool

Abstract

Sculpture is an art form dating back over millennia. More recently, 3D digital sculptures are used in video games, movies, and manufacturing. Traditionally, sculptors have used their hands and tools to shape wood, clay, stone or metal into an art piece. Currently, digital sculpting is accomplished using two dimensional flat screens, mice and keyboards. In this thesis, our goal is to understand how best to combine traditional and digital sculpting interaction techniques. We developed a mixed reality sculpting tool, Virtutello, which allows us to explore this question. Virtutello allows the user to use their hands to point and sculpt within a mixed virtual digital environment. As well, the user can physically move around the sculpture to see it from different perspectives. In an observational study of 20 participants, we found that users could be trained within thirty minutes and had a moderate degree of success in completing basic sculpting tasks. We focused our investigation on three main research questions. First, a core of sculpting is the ability to touch and select areas of a sculpture. We investigate the success of two pointing techniques, the novel open-hand pointing and the traditional finger pointing. We found there is an advantage using open hand pointing while sculpting in a mixed reality application. Participants completed tasks faster and with a higher degree of success when using open-hand pointing. Participants also expressed they felt more in control and had less of a challenge while completing tasks using open-hand pointing. Second, operating in a virtual world allows the user to physically move around the sculpture. We investigate whether there is a cost to sculpting while standing versus sitting, finding there is no cost to accuracy and task completion time. Third, we investigated user’s behavior when given this freedom to physically move around the sculpture. We found that participants preferred to remain stationary while sculpting, regardless of whether they were sitting or standing. Instead of moving around to change their perspective, participants preferred to rotate the sculpture digitally in front of them.