Faculty, students blend art and science for "visual" music experience

ABOUT THE NURTURING INNOVATION CAMPAIGN

A new award bestowed at Fredonia will bring music to life visually in ways never seen before, and underscore the synchronicity of the arts and sciences.

The Costello Interplay Award, established by Dennis R., ’72, and Kathryn L. Costello through the Fredonia College Foundation, provides grants to faculty and students for projects which explore the intersections between the visual and performing arts and the natural and mathematical sciences. “The idea for the award was based on the unique position of Fredonia as both an outstanding undergraduate school in the arts and a very solid school in the sciences,” Dennis said. “I have always been intrigued by the under-appreciated relationship between science and art.”

The fund will encourage collaborative exploration, research, and creativity resulting in new work that is enriched by both fields. Faculty and students will gain a deeper understanding of one another’s areas of practice and inquiry, enhancing their future work as artists and scientists.

"We in academia often complain about silos, yet we continually reinforce them by discussing our various areas of study as though they are separate from each other,” said Dean of the College of Liberal Arts and Sciences J. Andy Karafa. “The Costello Interplay Award provides an impetus for getting away from such segmentation and embracing, not only the overlap, but the interactions between areas.”

The first Costello Interplay Award is funding a research project entitled “Making Sound You Can See: Visualization of Audio Source Separation and Instrument Detection for Augmenting Live Concert Experience.” Department of Computer and Information Sciences Assistant Professor Gang Hu and School of Music Associate Professor Ray Stewart are teaming with Fredonia students Daniel Luong, Corbin Dzimian, and Samuel McCagg to make the sound you hear at a concert “visual” in new and exciting ways.

“(Concert) audiences can close their eyes and enjoy the sounds or their surroundings,” the team wrote in their proposal. “However, for many untrained ears, distinguishing the different instrument timbres and color can be difficult in heavily orchestrated sections. For example, there may be a single instrument on a counter melody but the part is not heard in the back of the hall. Therefore, there is an a rising demand: can we utilize smart technology to build a whole new concert experience for audiences?”

The goal of this project is to extract and visually present smart contents to concert audiences. Specifically, by developing state-of-the-art technologies, additional information during the concert playing can be extracted, such as separated signals from individual instruments, visual representations of a music piece, gestures of the main players or conductor. Perceiving the visualized music is not meant to defy the traditional concert experience, but to offer the audience enhanced and engaging experiences beyond the traditional understanding.”

“It was difficult for the selection committee to choose among several great proposals,” said Dean of the College of Visual and Performing Arts Ralph J. Blasting. “This one has a strong balance between Music and Computer Science, and will result in ‘music you can see.’ The research will enrich students from both departments, and I hope it's the start of a beautiful relationship between these two fields at Fredonia.”

“This is exciting and a good example of how the arts and science are so closely related,” Dennis said. “Exploring these intersections can lead to deeper understandings in both fields and discoveries that are unexpected. To see this happen at Fredonia is rewarding as well as inspirational.”