Visitors who came to the Herbert F. Johnson Museum of Art to watch monks creating a sand mandala for the recent visit of the Dalai Lama were themselves part of a new art form translating movement into sound.

As they clustered around to view the mandala or moved in or out of the room, cameras in the ceiling tracked their movements. The behavior of the crowd triggered subtle changes in the background sounds playing in the room.

The sounds were created by professional sound designer Ron Riddle, who combined monastery chants, Tibetan bowls, bells, chimes, piano and guitar, and created different tracks to reflect different audience behaviors. In some cases, as when people entered, a heartbeat sound was overlaid.

The soundtrack was adjusted to reflect the "energy, density and flow" of museum visitors, according to Kirsten Boehner, a postdoctoral researcher working with Phoebe Sengers, professor of information science and leader of the Culturally Embedded Computer Group. The project was a collaboration with the Wireless Intelligent Systems Laboratory directed by Stephen Wicker, professor of electrical and computer engineering, and the Human-Computer Interaction Group led by Geri Gay, the Kenneth J. Bissett '89 Senior Professor of Communication.

The goal, Boehner said, was to make people reflect on the fact that their presence in the museum changes the way the art is perceived. Boehner and colleagues observed and interviewed museum visitors, noting that some were completely unaware of the music ("A good soundtrack you don't notice," Riddle had told them), some bobbed their heads in time to the heartbeat, some stopped and looked at the speakers.

While all that may seem a bit ethereal, the technology that makes it possible could, paradoxically, increase opportunities for video surveillance in public places while at the same time enhancing the privacy of those observed.

To both simplify processing and protect privacy, software from Wicker's group performs "background subtraction." It stores an image of the floor and sends any pixels that don't look like floor as white, so people become white blobs. The computer finds the center of each blob and tracks the movement of that single point. (Even so, signs were posted advising people that they were being observed and saying that the system could be turned off if they asked.) Other technology can locate faces in a scene and blur them, or transmit to an observer only the track followed by an object, without any image at all.

Forthcoming is a project in which cameras will be mounted on the ceiling of the Japanese exhibit room. When people approach a particular exhibit, a device next to the exhibit will emit a sound. The purpose is mainly to test the ability to track people's movements, but "we want to see how people will react," said Sameer Pai, a graduate student working with Wicker. Pai also is participating in a study of public reaction to video surveillance in San Francisco, including an experimental system that will turn on street cameras and start recording when gunshots are detected.

Pai sees these experiments as foreshadowing much more sophisticated surveillance in which, for example, computers might try to discern a person's intentions from facial expressions or gait. Mindful of privacy concerns, Wicker says visual processing would be done at the camera and information sent to a central observer only if some significant image appears, such as a person carrying a gun.

For now, the research is as much about human reactions as technology. "We want to make people aware of what we can track and see how that influences their behavior," said Gay.