It’s not often that you read a book that solves a mystery that has baffled you for years. But that's what happened a couple of weeks ago when I read Why Good People Can't Get Jobs: The Skills Gap and What Companies Can Do About It. Written by Peter Cappelli, a professor of management and director of the Wharton School’s Center for Human Resources at the University of Pennsylvania, this tiny book provides a remarkably enlightening, strikingly original, and extremely important explanation of what’s wrong with America’s skilled labor market and how to fix it. Should this little book get the attention it deserves, it could, like Rachel Carson's Silent Spring or Michael Harrington's The Other America , change the way Americans think about a pressing issue.

That's because the book resolves the vexing conundrum of how two conflicting narratives about high-skilled employment have coexisted in our national conversation. On the one hand, countless unemployed or underemployed workers with perfectly good skills, education, and experience are struggling through a severe job drought, many sending out hundreds of applications and resumes to no avail. On the other hand, employers (especially in technical fields) complain of great difficulty finding workers, citing serious gaps between the requirements of available jobs and the skills of the workforce. One company that Cappelli mentions didn’t find a single worker that it considered qualified among 25,000 applicants for a fairly ordinary engineering job. Employers and their organizations fault an inadequate school system that fails to prepare Americans and restrictive immigration laws that prevent employers from importing the skilled workers they need from abroad.

Is one side hallucinating? Are American workers really not up to the challenge of today’s workplace? Are employers, as some critics argue, lying to hold down wages? Is something else entirely going on?

A broken system

Something is indeed broken, Cappelli compellingly argues, but it’s not America’s schools or the skills of its workers. He finds the workforce “largely competent and able” and clearly up to the jobs on offer. Instead, Cappelli writes, “the hiring process by which supply and demand are brought together is an absolute mess.” The needed workers are available, but employers don’t know it because they use ineffective, self-defeating methods to evaluate applicants’ qualifications. They also eschew obvious and time-tested steps that could mold those abilities to meet their exact needs.

Fixing “the present, debilitating disconnect between job supply and job demand” would immeasurably aid countless companies and workers, saving the former the huge but generally unrecognized costs of persistent vacancies and the latter the very obvious costs of unemployment (and underemployment). Fixing this problem—which isn't that hard—could unleash a new age of invention and prosperity in the United States.

The main reason that companies aren’t finding the workers they seek in an ocean of available ability, Cappelli believes, is that in recent decades, for reasons he explains, those companies have allowed their traditional human resources (HR) departments and training programs to atrophy. Another reason is that some complaining companies simply offer too little money to attract the people they want.

The current lack of adequately staffed HR departments, and companies’ refusal to teach workers on the job, have combined to produce what the book terms “a Home Depot view of the hiring process, in which filling a job vacancy is seen as akin to replacing a part in a washing machine. … Like a replacement part, job requirements have very precise specifications. Job candidates must fit them perfectly or the job won’t be filled.”

The problem, Cappelli writes, is that “no perfect fit exists between applicants and job requirements.” In a great many cases, people with various combinations of credentials and experience can do a given job well, and work can be structured in many different ways. In decades past, companies routinely hired people with capabilities related to the work and then, if necessary, trained them to do specific jobs; those jobs needed doing, after all, and there didn't seem to be any viable staffing alternatives. But then Silicon Valley “invented the ‘free agent’ model of hiring for new skills rather than training and then letting workers go once those skills aren’t needed.” That model has spread to other industries.

Rather than investing in workers to cultivate the skills that companies need, many employers now think—erroneously, Cappelli persuasively argues—that it is cheaper to limit hiring to people who can do a job from day one. The best way to ensure that applicants have the desired skills, those companies believe, is to find people who’ve done the job before.

We have often mentioned in this space that many nonacademic employers complain that early-career scientists don't understand business culture and such practices as budgeting and project management. Following Cappelli’s logic, companies could solve that problem by hiring deeply educated, very able people and then providing on-the-job opportunities—as part of a probationary period, perhaps—to pick up the nonscientific knowledge they lack. Instead, too many employers refuse to consider applicants who do not already meet exact requirements. Jobs stay needlessly unfilled. Workers stay unemployed.

A lack of judgment

If this weren’t bad enough, another supposed cost saver has tremendously complicated the hiring process. Instead of using humans—who, admittedly, expect salaries, paid leave, and health insurance—to evaluate applications and decide which candidates possess or can quickly learn needed skills, companies entrust these tasks to computer algorithms, which are cheaper in the short term but lack any ability to judge what resumes and applications indicate about applicants' abilities. Instead of weighing various combinations of education and experience against a job’s demands, they search for keywords and reject as unqualified all applications that do not use exact, predetermined phraseology. “Once it’s in the software,” Cappelli writes, “each requirement, critical or trivial, essentially becomes something like a hurdle that applicants have to clear to become a qualified candidate.”

Computers cost more in the long run because they discard applications from people capable of excelling at a job but who failed to divine the right wording or demonstrate the specified qualifications exactly, Cappelli states. The book recounts the reductio ad absurdum of this system: a temporary employee doing very well at a job but disqualified from permanent hiring because she doesn’t match the job’s computerized criteria. “Apparently,” Cappelli marvels, “doing the job well wasn’t enough of a qualification.” Not every company is so foolish: He also describes companies that build capable staffs by helping smart, motivated people with relevant abilities learn how to do those jobs.

“I think all the [people who sell application-screening software] know that the clients would find candidates if they would only hire some recruiters, some humans, and put them to work on this,” Cappelli tells Science Careers in an interview. “But the vendors don’t make any money on that. [Instead,] they say to clients, … ‘Here’s a system that automates the process so you can get rid of the HR department.’ It’s not in their interest as vendors to say, ‘You ought to staff up your company and do a lot of this stuff yourself.’ ”

Employers thus have it in their power to eliminate the skills gap. Currently, he writes, the United States is “the only country in the world where the notion that employers are simply the consumers of skills is seriously considered.”

Money talks

After showing how the “skills gap” originates in company policies, Why Good People Can't Get Jobs explains why “the skills gap argument has gotten so much traction, and the actual causes of the supply-demand job mismatch are so poorly understood.” It’s because “the associations and organizations that serve employers dominate the discussion.” Those associations and organizations are “reluctant to tell employers that they should do something the employers don’t necessarily want to do, such as provide more training.”

In addition, top executives who perpetuate the narrative are often ignorant of the true situation because they do so little analysis of human resources issues. “If this were in their supply chain, if they were buying widgets, they’d be all over these problems,” Cappelli says in the interview. That doesn't happen because “it’s human resources and they’ve gutted the human resources function, and they’ve decided long ago that this is not a place they were going to apply analytics.” Often, there are people within these companies who know that the skills gap narrative is false, but they’re generally “pretty far down in the organization. They’re not inclined to raise their hands and say, ‘Excuse me boss, what you said here is stupid.’ ”

Mismanagement of the HR function is not the whole story. A desire for more low-cost foreign workers motivates some complainers, as critics contend. “There are people in the IT world who really do want to see more H-1B people,” Cappelli says. “They’re cheaper. They’re easier to manage. There’s no doubt about that.”

Anyone who wants to understand what's going on in employment will find Why Good People Can’t Get Jobs a swift and persuasive read. It's really a long article designed to be read on mobile devices rather than a traditional book, Cappelli tells me. Company executives absolutely need to read it, but so do many others including government and educational policymakers, university career advisers, recruiters, job seekers, and journalists who help perpetuate the skill-shortage myth. If enough people were to take the message to heart, this little book could change America by drastically reducing the frustration—and, in millions of cases, the misery—of people on both sides of the hiring equation.

The hiring conundrum has stymied the nation’s economic and technological progress long enough. It’s time for Cappelli’s lucid and compelling call to action to get the attention it deserves.

Beryl Lieff Benderly writes from Washington, D.C.

10.1126/science.caredit.a1200112