Projects

Projects

Embodied Cognition and Music Interaction Design

In domains such as music, technical understanding of the parameters, processes and interactions involved in defining and analysing the structure of artifacts is often restricted to domain experts. Consequently, music software is often difficult to use for those without such specialised knowledge. The present work explores how the latter problem might be addressed by drawing explicitly on domain–specific conceptual metaphors in the design of user interfaces. Using Embodied Cognition to improve Music Interaction Design
(Katie Wilkie, Simon Holland, Paul Mulholland)

Song Walker Harmony Space

Song Walker Harmony Space exists as a tangent from the Whole Body Harmony Space project. It is a relatively compact setup, yet still makes full use of bodily movement and gesture.

The system is suitable for use by 2-4 players simultaneously, with roles and functions being distributed among the players.
(Simon Holland, Anders Bouwer, Mat Dalgleish)

Whole Body Harmony Space

Novice improvisers typically get stuck on ‘noodling’ around individual chords, partly because they are oblivious to larger scale harmonic pathways. Harmony Space enables harmonic relationships to be represented as 2D spatial phenomena such as trajectories. When users move through the space they hear the sounds corresponding to the different locations.

Our working hypothesis is that providing tactile guidance around Harmony Space will improve musicians’ knowledge of larger scale harmonic elements such as chord progressions and thereby improve their improvisation skills.

An open research question we are investigating is whether full-body exploration of Harmony Space, for example moving around a large floor projection, brings any benefits over using a mouse and keyboard to interact with Harmony Space on a computer monitor.
(Simon Holland)

Exploring computational models of rhythm perception

We are testing computational rhythm perception models (including Ed Large’s non-linear neural resonance model) with rhythms containing two different pulses—such as three against four rhythms found in African-derived music. This allows us to test the performance of the model against human empirical data - e.g. tapping to polyrhythms. In order to better test the model, we have carried out new empirical tests of human behaviour. Exploring computational models of rhythm perception
(Vassilis Angelis, Simon Holland, Martin Clayton, Paul Upton)

Neurophony

Resulting from collaboration with the Eduardo Miranda at the University of Plymouth and visiting intern Fabien Leon, Neurophony is a brain interface for music, incorporating the Harmony Space software.
(Fabien Leon, Eduardo Miranda, Simon Holland)

The Haptic Drum Kit

The Haptic Drum Kit consists of four computer-controlled vibrotactile devices, one attached to each wrist and ankle, that generate pulses to help a drummer play rhythmic patterns.

One aim is to foster rhythm skills and multi-limb coordination. Another aim is to systematically develop skills in recognizing, identifying, memorizing, analyzing, reproducing and composing monophonic and polyphonic rhythms.

A pilot study compared the efficacy of audio and haptic guidance for learning to play polyphonic drum patterns of varying complexity. Although novice drummers can learn intricate drum patterns using haptic guidance alone, the study found that most subjects prefer a combination of audio and haptic signals. The Haptic Drum kit appears to have possible applications for sufferers from Parkinsons and patients in need of multi-limb rehabilitation. We are exploring possible research partnerships to explore such applications.The Haptic Drumkit research, including work on the Haptic Ipod is being carried out in collaboration with the Music Computing Lab, expert musicologists, drummers and music educators.
(Simon Holland, Anders Bouwer, Mat Dalgleish)

The Haptic Bracelets

The Haptic Bracelets are prototype self-contained wireless bracelets for wrists and ankles, with built-in accelerometers, vibrotactiles and processors.

They can work by themselves, in networked synchronised groups, or with smart phones, tablets or computers. The Haptic Bracelets are designed to be worn four at once, one on each wrist and one on each ankle, although any number from one to four may be usefully worn. A pair on ankles or wrists has a rich range of uses. The Haptic Bracelets are interactive devices for both input and output, enabling communication, processing, analysis and logging. They can be used for local or remote tactile communication between two or more wearers. Musical applications include

Understanding rhythm
Drumming and rhythm Tuition
- teacher-learner show and feel
- feel explicitly what each limb of drummer does or feel abstracted version of rhyhm
- performance recording and playback
- mental haptic mixer
- slow motion playback
- passive learning
Compositional understanding
Inter-Musician Co-ordination
African Polyrhythm training
Drumming for the Deaf

While originally designed for musical purposes, we are collaborating with researchers, practitioners, varied user groups and individuals to develop a range of new applications for: everyday use, navigation, musicians, stroke victims, Parkinson’s sufferers, dancers, the movement impaired, the hearing impaired and others.
(Simon Holland, Mat Dalgleish, Anders Bouwer, Maxime Canelli, Oliver Hoedl)

Direct Combination

As mobile and pervasive computing resources proliferate, there is an increasing need for end users to create one-off services by causing two or more devices or resources to interoperate together, often in ad-hoc circumstances. In general, users find this kind of process hard to manage. At the same time, existing UI architectures are not well suited to supporting such activities. It is proposed that a good basis for addressing these and related problems in a principled, scaleable way is the principle of Direct Combination (DC). Both analytical arguments and preliminary empirical evidence suggests that DC can reduce the amount of search required by the user and can offer interactions which are faster, less frustrating, and impose less mental load on the user. Direct Combination makes it easy for individuals and organisations to publish and adapt new services, and to create viewpoints to facilitate different ecologies of activities.

Completed Projects

The E-sense Project

The E-sense Project was funded by an AHRC speculative research grant. In this interdisciplinary research, philosophy and art combined with various flavours of computing: ubiquitous; wearable; and physical.The project involved a range of sensory augmentation devices including the Music Jacket, The Haptic Drum Kit and Song Walker Harmony Space.

New User Interfaces for Musical Timbre Project

The user interface of most synthesisers tends to be expressed in system (i.e. engineering) terminology, obliging the user to become fluent with the synthesis method employed. This causes problems for many users.The New User Interfaces for Musical Timbre Design Project involves the design, implementation , and evaluation of a prototype of a new kind of user interface for controlling musical timbre that addresses this problem in a principled way. (Allan Seago, Simon Holland, Paul Mulholland)

Aspect-Orientated Music Representation

This project devised, and now continues to refine and investigate, ways in which Aspect Oriented Programming (AOP) and Multi-Dimensional Separation of Concerns (MDSOC) approaches can be applied to the organization of musical materials for the purposes of music composition and musical analysis. (Patrick Hill, Simon Holland, Robin Laney)

Spatial Audio Navigation

The Music Computing Lab has a strong interest in Audio Computing, particularly Spatial Audio. This project was was the first paper to apply spatial audio to navigation for freely mobile sighted users, using a minimal attention approach Spatial Audio Navigation

PlanC: A constraint–based musical planner

PlanC was an early musical constraint system, but remains uniquely well-suited for manipulating some aspects of tonal harmony. While most musical constraint systems work with descriptions of musical materials such as individual notes and chords; PlanC draws power from a representation called Harmony Space that encodes high level information about tonal harmony in a three dimensional spatial form as shapes, paths, home locations, forbidden areas and trajectories. PlanC: A constraint–based musical planner

Completed Projects Overview