In early 2018, Pandora was in a position many tech companies are facing today. The business was growing quickly, but we couldn’t hire quickly enough to fill all of our open technical roles. We had a robust university recruiting pipeline and had successfully converted a large percentage of summer interns into full-time employees, but even that high-caliber talent pool was too limited to meet all of our hiring needs for entry-level engineers.

While apprenticeships have a long history in the U.S. in industries like construction, plumbing, and carpentry, they have also recently gained national attention as a potential solution to the tech skills gap. Apprenticeships offer both on-the-job training and project-based, real-world experience while helping employers fill their hiring pipelines with skilled talent. In an effort to build new, stronger talent pipelines, my manager asked me to focus on creating an apprenticeship program of our own.

In a little more than a year since we kicked off the project, we implemented a program with a 100% full-time conversion rate and boosted employee morale in the process. Apprenticeships are still rare in our industry, but I’m confident they can become a major force in the future of tech hiring. Here are the five lessons we learned along the way that can help you make apprenticeships work for your tech company.

1. Identify your untapped talent pools

Upon reviewing our engineering title progression, I realized the earliest entry-level title we offered was aimed at the small number of highly-sought graduates from top universities. We were guilty of a hiring paradox we’ve heard millennials gripe about for years: Candidates need significant experience to get an entry-level job, but they need an entry-level job to get that experience in the first place. But it wasn’t always this way.

When I was first starting my engineering career in the early 1980s, computer science was still an emerging discipline. Graduates were relatively scarce and it was much more common for companies to hire self-taught coders without degrees. At some point in the decades that followed, the industry shifted significantly and today has a heavy bias in favor of software engineers with four-year computer science degrees. Yet demand for computer science graduates now outpaces the number of degrees awarded 10-to-1.

Pandora is also very active in our community, so many of our engineers volunteer their time to teach software development skills to community members. We had direct exposure to so much raw talent, but because many of these community members were unable to access formal technical training, they weren’t making it through our hiring pipelines. This was clearly our untapped talent pool.

2. Find the right partner for your needs

When I was tasked with developing this apprenticeship program, I started out by researching other companies who had tried to implement apprenticeships on their own. I found that starting from scratch would take significant work and that it was even more difficult to scale a program like this. That’s when I knew we needed a partner.

To find the right fit, I looked into direct partnerships with coding academies, local community groups, and online training companies. There are hundreds of coding bootcamps out there, including Hack Reactor, Flatiron School, and Hackbright Academy as well as other online programs such as Treehouse and Udemy, so creating multiple direct partnerships takes a lot of effort, and choosing only one limits your talent pool.

Eventually, I came across Onramp, a new company focused on helping tech companies like ours reconnect with a rich talent pool the industry had once valued but was now missing. These applicants were career changers, coding bootcamp graduates, and self-taught developers with high potential and a strong motivation to grow their skills, but they were still being shut out of traditional recruiting pipelines.

3. Customize your cohorts by the technical demands of each role

While there have been a few companies that recently dropped their degree requirements, candidates from non-traditional backgrounds often need additional last-mile training to be ready for the workplace.

Although some code academies had partnership programs you could sign up for, it was much more difficult for them to build out a curriculum tailored to our needs. With this in mind, we discussed what we needed from the candidates, the gap between what they already knew and what they needed to learn, and how we would select and onboard them for the final apprenticeship. From there, we reverse-engineered a training program for our first cohort focused on Java backend services with Python elements.

After the success of our first cohort, we decided that creating additional small cohorts with curricula tailored to specific technology areas would allow us to scale the program more effectively than creating a single larger cohort. For example, applicants for the first cohort were mainly code academy alumni with fullstack Javascript web development skills. Because few code academies teach Java, that language was the main focus for the last-mile training. By contrast, our second cohort is focused on Android development, with more of our applicants coming from self-taught backgrounds. Their training will be more focused on how to work well within a team setting and how to use the specific tools required for the role.

4. Ask for feedback and make adjustments as you go

During your pilot program, ask for feedback early and often. The honest feedback we received from our apprentices and mentors led to significant improvements in the overall program for our second cohort. In the first cohort, mentors received a document describing their responsibilities, and how best to support non-traditional talent. Based on feedback from the first cohort of apprentices, we expanded that to a live mentor training session that includes specific recommendations based on their experience.

We’ve also made adjustments to our interviewing process to limit implicit bias. For example, we developed a standard rubric to help interviewers evaluate candidates in a more consistent way, focusing on growth potential rather than previous professional accomplishments. We also did away with on-the-spot whiteboarding exercises and other brain teasers, and instead have candidates discuss a project they completed during their training. We have found this allows candidates to present a more accurate picture of their capabilities, without the anxiety that some experience when faced with the need to write code on the spot. This anxiety is compounded by the natural fear that they already start at a disadvantage to traditional candidates.

5. Soak in the unexpected benefits

The main goal of our apprenticeship program was to find motivated talent and give them the skills they’d need to fill our job openings. What we didn’t know when we started the program was just how much it would benefit our company beyond that goal.

The enthusiasm, tenacity, and inquisitiveness the apprentices bring invigorates their teams. Some of our current engineers who don’t have four-year degrees are speaking more freely about their career journeys with a sense that a stigma has been lifted.

Our apprenticeship program is also helping us bridge the opportunity gap for people from underrepresented groups. Though they represent a small sample size, our first two cohorts included 50 percent people of color, and 87 percent women. Those demographics immediately stood out, especially in an industry where just 20-25 percent of employees are women and even fewer are people of color.

Separately, while we initially worried the apprenticeship program would burden employee mentors, they instead saw the role as a leadership growth opportunity. It’s still too early to tell what the long-term benefits will be for mentors who participate in the program, but we hope it will help accelerate their professional growth and possibly increase retention.

Final thoughts

If your engineering team is feeling the talent crunch, take a step back and ask what stones you’re leaving unturned when looking for new talent. While apprenticeships are still a work in progress for our industry, any tech companies that are not looking at candidates from non-traditional backgrounds are missing out on a very strong talent pool. We still have a long way to go before tech apprenticeships are widespread, but they hold a lot of potential.

Dave Edwards is Director of Engineering Programs at Pandora.