Broods used the Microsoft Band to create music-video magic

Microsoft is no stranger to collaborating with musicians to show off creative uses for its technology. With the Music x Technology project, the company has worked with acts like Big Grams, Neon Indian, KEXP, Phantogram and others to use its Kinect to enhance the musical experience for fans. I got a first-hand look at "Realiti: Inside the Music of Grimes" back at Moogfest in May, an exhibit that let fans remix parts of a song by interacting with a mesh surface. With the help of creative agency Listen, Microsoft teamed up with electropop duo Broods to produce a music video using the Microsoft Band.

1 / 10

In the past, the Music x Technology initiative has done a lot of motion tracking using the Kinect camera's depth sensors. This time around, Microsoft's wearable is the key to the project, tracking Georgia Nott's movements during the video shoot for the song "Heartlines." In fact, this is the first time the project has used the device while working with an artist. After gathering heart-rate and body-temperature data during the filming process, the team created geometric visuals based on the stats that were overlaid on top of actual footage. Why do this? Hint: It's about more than just showing off what the Microsoft Band can do.

"The goal is trying to communicate on another emotional level, not just through audio and facial expressions," Caleb Nott, Georgia's brother and the other half of Broods, told Engadget. "There are all sorts of ways that emotion can translate into something else. If you can communicate your emotions through your temperature and heartbeat, that's a more sensual experience."

With that in mind, Broods wanted a subtler way to capture the heart-rate data, because their previous video had a more industrial feel. Using a device like the Microsoft Band that's worn on the wrist like a watch did just that. "A piece of medical equipment would have been more invasive, so I think this was a really cool way of doing it," Nott said. "It's really awesome to get biodata and visualize it in an interesting way."

A still from the "Heartlines" video with the Microsoft Band-generated graphics added.

At certain points in the video, you'll notice the pink-hued geometric shapes that look like they're pulsing from the torsos of the musicians. Given the title of the song, the choice to track heart rate seems obvious, but the way it's used in the video isn't too on the nose. The added graphics aren't present in every shot in which one of the Broods appears; they show up only to emphasize words and action at specific spots in the video. "I want to feel your heartlines, I want to feel your heart," the song's lyrics plead, and the pulsing shapes allow the viewer do just that as the graphics rotate around the performers.

The process of tracking vitals during the video shoot wasn't without its challenges. Nott said one entertaining part of the process was watching the poor production person who had to stay within range of Georgia's wrist while also being careful not to step into the frame. "There was a guy running around with a laptop dodging and darting trying to stay out of the shot," he said. "He was so stressed out, the poor guy."

Caleb Nott also said this was his first time trying a fitness tracker; prior to the project with Microsoft, he hadn't worn one. His sentiments about having the Band on his wrist echoed what I've heard from wearable owners before: It's hard getting used to the gadgets for long periods of time if you don't wear a watch on the regular. "I don't wear watches or any kind of jewelry, so it was pretty different," he said.

Explaining how the Microsoft Band is used live during the LA event.

In addition to the video, Broods will use the Microsoft Band to generate visuals during performances on their upcoming tour. Starting next week, the duo's heart rate and temperature will be tracked and translated into on-stage lighting and visuals in real time. Steve Milton, of creative agency Listen, said what concert-goers can expect to see during the tour will be different from what's in the new video, but the wearable will still be keeping tabs on Georgia's stats as she moves around the stage.

"On tour, the band has a really great lighting display, so we've worked with the tour production team to use the Microsoft Band to capture Georgia's biodata during her performance and effect the lighting display on stage," Milton said. As a preview of how the Microsoft Band will be used live, Broods did a small performance at The Microsoft Lounge in Los Angeles this week.

What's next for the Microsoft Band in terms of the Music x Technology project? Milton says the team of collaborators has yet to find the limits of the wearable device when it comes to more creative uses. "The Microsoft Band can track a number of things: heart rate, calorie burn, sleep quality and exercise," he explained. "We focused on heart rate given the creative direction of the project, but we have really just started scratching the surface here. We're excited to see where we can take this and how we can move beyond visuals to music and sound."

Speaking of controlling music and sound, Caleb Nott already has his eye on using the Band to manipulate a drum machine. In some behind-the-scenes footage detailing the making of the music video, he said that connecting the gadget so that heart rate controls the kick and body temperature can manipulate a snare or hi-hat isn't beyond the realm of possibility.

"It's something that I'd like to do, but I think first on the list is integrating it into our live performance," Nott said. "We have to see where that takes us, because [using the Band to make music] could be quite different depending on how you're feeling."

Who knows -- maybe on the duo's next tour the siblings will employ a Microsoft Band to create a new version of a song, live on stage? One thing's for sure: We haven't seen the last of the company using its wearable in a musical setting.

All images courtesy of Microsoft Music x Technology. Video shoot photos by Nils Erik Vogth-Eriksen and LA event photo by Mekael Dawson.