The room was abuzz with pint-sized autonomous vehicles, mind-numbing calculations and geeks in shoe socks.
Julia Harvey had heard about the curious Lockheed Martin Space Systems event from a school newsletter and thought, “Oh, that looks really cool!”
So, the Colorado School of Mines computer science major spent Halloween morning training her AWS DeepRacer (on loan from Amazon) to steer left or right, to earn rewards for “good” behavior and, importantly, to stay on the road, a painted race track on a plastic mat where only machine-learning AI racers and stocking-footed humans were allowed.
“I thought it was going to go crashing into walls, especially my first game where I’m like, ‘I don’t know anything about deep learning. I’m gonna keep my expectations low,’” said Harvey, a sophomore taking the Mines’ robotics and intelligent systems track. “But it made it (around the track) in about 20 seconds.”
Not bad for an amateur. The winning team, made up of Lockheed engineers (also, cough, cough, amateurs) made their lap in 10.25 seconds. (Globally, 7.17 seconds is the score to beat.) But under the guise of a fun competition with access to cool toys and hobnobbing with aerospace brainiacs at the ultimate STEM event (space, people!), Lockheed Martin Space had its future in mind.
“Whether it’s machine learning or artificial intelligence, a lot of our missions are going to require more and more autonomous behavior,” said Dan Driscoll, acting chief information officer at the Lockheed space division in Littleton. “Building some of these rudimentary understandings of AI and machine learning and reinforcement learning, which is what this little project here does, is very foundational to cultivating that knowledge, whether it’s in a high school student or college student, that we hope will be here one day as our own employees.”
If you didn’t already know, Colorado is and has been a major hub of aerospace activity. The state is second only to California, as the Colorado Space Coalition likes to tout (and first based on private aerospace jobs per capita). We’ve got a space port in Watkins. The Pentagon’s Space Command’s temporary headquarters is in Colorado Springs.
And a number of companies are working on high-profile missions: Lockheed’s developing a habitat for astronauts to live while orbiting the moon, as is Sierra Nevada Corporation’s Space Systems division in Louisville; Maxar Technologies in Westminster just delivered a robotic arm to NASA that’s headed to Mars next year; and Sierra Nevada’s Dream Chaser takes off in 2021, boosted by a rocket from Centennial-based ULA.
The number of private aerospace jobs in the state had a five-year growth rate of 7.9%, besting the national average of -2%, according to the Coalition. Colorado’s aerospace industry is having to figure out how to attract and retain young workers as competition for science, tech, engineering and math (or STEM) talent has increased, said Vicky Lea, director of aerospace and aviation for the coalition, which is part of Metro Denver Economic Development Corp.
“You have time frames that are much longer than other industry projects if you’re talking about a mission that takes 10 years to develop and execute,” Lea said. “So when you have younger people coming up through the pipelines who are perhaps used to a job that is going to get more immediate results, 10 years can be a long time to work on something.”
Beyond fellow aerospace companies, there’s also competition from the high-tech community especially in the Denver area, she said. Companies including Google, Facebook and Amazon have opened large professional offices in the metro area in the past five years.
An answer is to get kids interested early. That’s what Colorado AeroLab does for all students with after-school STEM-related classes in northwestern Colorado, said Elaine Menardi, AeroLab’s chief innovator. She says it’s hugely important for professionals and local companies in STEM fields to get involved in mentoring locally.
“Firstly, because (schools) need the workers and if they have a hand in training the specific skills they are seeking, all the better,” Menardi said in an email. “Secondly, schools just don’t have the capacity, or more importantly, the expertise to teach those skills.”
Organizations like the Colorado Space Business Roundtable tap their members to help support the pipeline of future space employees. It offers two-week space internships (applications are now being taken) to high school and college students, with a priority given to kids from rural areas or who are underrepresented in the industry, said Alires J. Almon, chair of the Roundtable. The organization also sponsors the Colorado Aerospace STEM magazine, which is distributed to 3,000 schools and teachers in Colorado.
This week, Lockheed had 752 job openings within 50 miles of Littleton. And beyond finding enough talent, Lockheed also must deal with security clearances, though not all jobs need one. The company has 10,400 employees in Colorado and 18,000 in its space division companywide.
In the past, Lockheed would hire students out of school and sponsor them through the security clearance process. But that could take up to 18 months and newbies could leave or get recruited to go elsewhere.
About two years ago, the company created ENACT, short for Enabling Entrepreneurial Action Teams, Driscoll said. Now, as new hires wait for security clearance, they are split into teams that work on projects related to a broad objective.
One team was simply told to do something with drones. They used off-the-shelf flying drones to simulate satellites and the communication between them. Project HiveStar is now moving forward, partly thanks to customer interest.
“These were recent college graduates so they weren’t specifically trained,” Driscoll said. “They had a general base of knowledge. We’re able to partner them with some of the more experienced employees who could mentor them on space-specific things. It’s just one way that we’re addressing the talent shortages.”
Retaining talent is why Lockheed Space recently opened the Pulsar Accelerator at its scenic Littleton headquarters, where the spread out campus often attracts wild turkeys and elk. The playroom is open 24/7 for employees to “cultivate their imagination” using tools from 3D printing to augmented and virtual reality tech and immersive displays.
“It’s actually changing the culture within the company,” said Christopher Pettigrew, a Lockheed spokesman. “Traditionally, Lockheed’s not very different from a lot of companies in that there’s a hierarchy that is top down. You look at startups, they’re very collaborative. Stuff like this is helping change that a little bit. … Obviously 10,000 employees in Colorado, that’s a lot of minds to change. But it’s actually a good thing. And we are trying as a company to change because we have to.”
Where sci fi starts to get real
Every second it races on the track, DeepRacer takes a picture and pumps the image through the deep-learning algorithms users created. It sees the dotted yellow line in the middle of the black asphalt. It sees the road’s white shoulders and the mass of green “grassy” paint indicating off the road.
The trick to getting the vehicle around the track as fast and as accurately as possible is by giving it rewards. Stay close to the center line? Get points. Go off road? Subtract points. Like training a puppy.
That’s reinforcement learning and the program uses AI to start making predictions to achieve the most rewards, said Stephen Gerali, chief architect for Lockheed Martin Space. (He’s trained his racer to get a top speed of 13.3 seconds.)
“What you end up doing is you run (virtual) simulations on this over and over and over again,” Gerali said. “The computer itself is trying lots of things. And really all it’s trying to do is have a maximization function on what the rewards are going to be.”
Users start with a virtual vehicle and then for the race, move to the real track. But that’s where it gets tricky, said Ethan Wich, a senior at Pine Creek High School in Colorado Springs who interned at Lockheed over the summer.
“The fastest we got was like 18 seconds in the simulation. When going into the real world, we got like 22 seconds. The simulation to the real world is not really one to one,” said Wich, representing team E&E with high school friend Eugin Pahk. “The car would just act in ways we hadn’t seen in the simulation, like it would go right when we programmed it to reward going left. In the simulation, it would go left.”
He had some theories why the car didn’t do what it was programmed to do.
“It could be something physical with the car, like maybe the tires are trimmed one way or the other. It could be just how the track is. It might be the camera picking up different images or friction,” he said. “I don’t know.”
The sensitivity of the camera is why participants who jump onto the mat to put cars back on the track must wear shoe covers. In between each race, a volunteer also Swiffers the entire mat.
While it may take a smart dog a few dozen treats before it learns to pee outdoors instead of next to the couch, DeepRacer usually takes four to eight hours, Gerali said (and anyone at home can start testing out a virtual racer of their own on Amazon’s site). And then a trainer must coach it along by fine-tuning parameters to see what works best.
“The more that you train your model, the better it gets at the track. But you can over fit that model as well, in which case, it can’t be used across any other track,” Gerali said. “So if you’re just in this competition, it’s just one track that you’re going for. But ultimately the greater goal is you could build in a general purpose capability for any track.”
Ahh. Like a real autonomous vehicle.
In the end, Lockheed employees swept the top eight spots with Team Gopnik coming in at 10:25 seconds.
Harvey landed in 11th place at 17 seconds. Of course, winning wasn’t necessarily the goal for the School of Mines student. She learned from other players.
“I said, ‘Mine is wobbling back and forth a lot and your’s seem pretty stable. Can I see your reward function?’ and he said, ‘Oh yeah,’ and he showed me that,” she said. “And then he said, ‘OK, mine’s not getting as quick progress in reaching the end as yours, how are you doing that?’ And I said, ‘I’m just giving it a reward based on if it reaches the end.’”
Wich said he got really excited after talking to some Lockheed employees because “they said they just hired interns and college grads straight out of school and they’re already working on designing a new satellite.”
Wich and Pahk, the youngest participants at 17 and 15 years respectively, felt good about what they’d learned and glad they drove up for the opportunity — even though they weren’t finalists.
“Uhh… we placed,” Wich said. “Yeah. So we’re not last.”