Clik here to view.
Imagine a storm whipping across the Martian desert, blasting everything in its path. When the wind dies down, a robot slowly ventures out of a habitat station and into the dusty aftermath, picking its way around rocks and debris. It assembles a solar panel and adjusts an antenna, providing energy to the station and strengthening its connection to Earth. Then it heads back inside where it detects a leak and patches it, making the room safe for human inhabitants.
The scenario may sound like science fiction, but humans are getting closer to visiting the red planet. Robots, however, may arrive before them. That’s why NASA’s recent Space Robotics Challenge asked programmers to develop the best method for controlling one of NASA’s R5 Valkyrie robots from 140 million miles away. A humanoid robot designed to function in environments that would be deadly to humans, the R5 Valkyrie is sleek, white and armored in plastic.
While the stakes of NASA’s contest weren’t quite life and death, they were still high: The winners would receive $1 million in prizes, the chance to work with actual Valkyrie robots and some fairly stellar bragging rights.
Clik here to view.
Top: NASA’s R5 Valkyrie is a humanoid robot designed to function in environments where Earthlings can’t tread. Above: GE Global Research engineers Shiraj Sen (left) and Steven Gray worked nights and weekends to design a method for controlling one of the bots. Their efforts paid off: Their team, called Walk Softly, won second place in NASA’s recent Space Robotics Challenge. Images credit: Walk Softly
The competition attracted 405 teams from 55 countries, including groups from some of the world’s top engineering schools. Some teams had dozens of members and thousands of dollars to spend, but — in the end — the strongest competitors were some of the leanest. The winner, Coordinated Robotics, was a one-man team from Newbury Park, California. Second place went to Walk Softly, a pair of GE engineers from upstate New York.
Walk Softly’s team leader was Steven Gray, a lead robotics engineer at GE Global Research in Niskayuna, New York. The name is a play on Teddy Roosevelt’s famous exhortation to “Speak softly and carry a big stick.” In this case, the “big stick” was the team’s years of experience working with robots. As for speaking softly, the team, which included Gray’s coworker Shiraj Sen, kept a low profile on the competition’s forums, choosing not to boast about their approach.
But Walk Softly’s silence concealed a furious determination. Before moving to GE in May 2016, Gray competed in the high-profile DARPA Robotics Challenge, where competitors tried to create robots that could function in environments that would be deadly to humans. His team didn’t place. “I wasn’t thrilled with that,” he recalls, noting that, when the NASA contest was announced, he felt like it was a chance to try again.
In August 2016, Sen and Gray began working on their submission. They started out with little more than their own skills and knowledge and a lot of coffee, taking over a room in Gray’s house.
The first challenge was to design software that would allow a robot to autonomously push a button to open a door, walk through, and interpret lights on a control panel. In late January 2017, NASA selected Sen and Gray as one of 20 finalists, which meant that they had four more months to come up with a final method for programming and controlling a Valkyrie to complete its tasks in a Martian environment. “We pretty much gave up our weekends,” Sen says.
Clik here to view.
The challenge’s goal was to create an automation or teleoperation system to help the NASA R5 Valkyrie Robot accomplish tasks at a simulated Mars outpost. These tasks included adjusting a satellite dish, setting up a solar array and fixing an air leak in a habitat. Image credit: Walk Softly
Gray and Sen’s day jobs helped them with the NASA challenge. The pair work together on robotics at GE, programming robots to do maintenance on refineries and other installations where it would be expensive or dangerous to send humans. “Our work experience helped us avoid pitfalls,” Sen says, “but we were working full time on multiple projects, so finding the time to work on this was a challenge.”
In some ways, that lack of time may have helped the pair. Rather than trying to completely automate the robot, they trained it to do some things — like turn a valve or adjust its center of gravity — autonomously. For more complex actions, a human took over operations.
That mix of autonomy and control came out of Gray and Sen’s programming experience. Fully autonomous solutions, in which the robot completely operates on its own, are “brittle,” Gray says, and can quickly deteriorate when something unexpected happens. For example, if a robot is programmed to follow a path that turns to the left, or pick up a briefcase from a table, it can become disoriented if the path turns to the right or the briefcase is missing. This was a crucial part of the NASA challenge, which randomly changed details like topography or the position of key items with every run.
Clik here to view.
NASA designed its challenge to include variables — an unexpected turn or a missing object, for example — that can confound fully autonomous robot systems. Image credit: Walk Softly
And changing details was only one of many ways that NASA designed the competition to ensure that things often went wrong. To begin with, NASA instituted a communications delay of up to 20 seconds, which meant that there was always a distracting lag between when the team sent the robot a command and when it was carried out. There was also a wide shift in the level of bandwidth available to the competitors. Sometimes, they were able to send information to their robots at 16 kilobytes per second, but at other times it dropped to an anemic 64 bytes per second. This meant that there were times when the robot was almost out of their control. However, its ability to conduct semi-autonomous operations meant that Gray and Sen’s robot could continue to function, even when it was out of contact.
Team Walk Softly’s second-place standing means that they will be able to spend two weeks working with an actual Valkyrie robot. They’re hoping to bring along a team from GE Global Research’s robotics and artificial intelligence team to help them test out some of their telerobotics innovations. “We’re hoping that this will lead to more partnerships with NASA,” Sen says.
Clik here to view.
To make matters even trickier, NASA added a communications delay of up to 20 seconds between the time the team gave a command and the time the robot acted on it. Image credit: Walk Softly
The pair were also awarded $100,000 for coming in second place. “That part’s going to be split evenly between me, Shiraj and the IRS,” Gray laughs. Ultimately, though, a big part of the reward was the chance to connect with other robotics enthusiasts, both fellow competitors and those organizing the competition. “It’s fun to see all the different ways teams approached the same problems. There are so many right answers,” he says.