Last Saturday, on the first morning of the fourth annual “Sabermetrics, Scouting, and the Science of Baseball” seminar in Boston (“Saber Seminar” for short), Astros general manager Jeff Luhnow strode onto the auditorium stage at BU’s School of Management. A few hours later, Red Sox GM Ben Cherington would stand in the same spot sporting designer jeans, but Luhnow, a former management consultant who dresses down so rarely that it was surprising to see him accept the Ice Bucket Challenge in something other than a suit, looked like he’d just stepped off the set of Up in the Air and had come to compare rewards cards.

Luhnow, who broke into the industry with the Cardinals in 2003 and jumped to Houston in late 2011, lacks the baseball background and the leathery, sun-toughened skin of, say, Diamondbacks GM Kevin Towers, who pitched in the minors for eight seasons and scouted for several more. Luhnow is, or once was, an outsider, living the life on the inside that some in the Saber Seminar crowd hope to gain. Yet even in front of a roomful of Internet quants and analytics enthusiasts — an audience more philosophically attuned than most to the Astros’ recent commitment to intelligent losing — Luhnow deviated from the traditional sabermetric script.

“I’m not going to talk about the Astros’ analytical approach, because that has been well-documented,” Luhnow said. “I thought what would be insightful to talk about, or for you all to hear about, is a little bit more on the softer side.”

The softer side was the star of this year’s Saber Seminar, a two-day charitable event¹ that resembled a scaled-down, baseball-specific version of the MIT Sloan Sports Analytics Conference. Admittedly, the seminar included a cross section of acronyms and equations that looked just as complex on PowerPoint slides as they would have in a spreadsheet or a SQL query. The schedule featured a panel on player projections; two presentations on the effect of pitch framing; other talks about interpreting college stats, applying motion-tracking technology, and the physics of the ball-bat collision; and abstracts on the optimal place to locate a fastball, real-time TV ratings, and whether umpires call the strike zone differently in the daytime and outdoors than they do at night and in domes.² Speakers cited not just that old econ class standby, multiple linear regression, but also symbolic regressions and generalized additive models. Even without counting the representatives of the 14 major league clubs in attendance, there was enough brainpower present to fill a few front offices.

Even so, several of the talks focused less on the analytics themselves and more on what Luhnow described as “the impact that the analytics have, and the impact that all of this rigorous analysis and thinking has, on an actual baseball team, on the human beings that are involved.” Saber Seminar is the closest the public comes to getting a glimpse behind the wall of NDAs and proprietary metrics that separate us from the insiders.³ So why is the cutting edge suddenly sounding so soft?

Luhnow acknowledged that the exponential increase in information available to team analysts has led to certain discoveries that could confer a fleeting advantage. However, that information is available to all 30 teams, and even the late adopters who haven’t mined it as thoroughly get some of the benefit from reading public research.⁴ As a result, having innovative ideas and doing the research to support them isn’t always enough to differentiate one front office from another. “The biggest challenge that I think the 30 clubs face in digesting all this information and utilizing it is, ‘How do you implement change in an organization where the industry has essentially been a little bit resistant to change?’” Luhnow said.

Luhnow recounted a meeting early in his Cardinals career when he and sabermetrician Mitchel Lichtman (who was then consulting for St. Louis) tried to explain the relationship between leverage and reliever usage to Tony La Russa, who responded with a litany of objections (relievers are conditioned to pitch at predetermined times; closers know the market pays for saves) that Luhnow and Lichtman weren’t prepared to address. Years later, when Luhnow and other analysts began to research the shift, he knew he needn’t bother bringing it up with the Cardinals’ coaches. Circumstances changed when he left St. Louis. “Once I got to Houston, I thought, ‘Well, OK, I’m the general manager now, I can do what I want,’” Luhnow said, semi-seriously.

However, he soon realized that a promotion wouldn’t make the task any easier. “Our staff wasn’t really configured to [implement the shift],” Luhnow said. “There was not a lot of acceptance. We didn’t have the tools built to really sell it well, and so after a few conversations with the major league staff, and expressing a desire to go down this path, we really didn’t do a whole lot.” According to Baseball Info Solutions, the Astros shifted only 138 times in 2012, Luhnow’s first season with the team.

Last year, the Astros hired a new bench coach, Eduardo Perez, who was receptive to front-office recommendations. With Perez (who’s now an ESPN analyst) on hand to help corral rebellious players, the Astros shifted 96 times in April. Then, Luhnow said, “The complaints started to come from the pitchers, from some of the infielders, from the media, from basically anybody out there, and sure enough, as the season wore on, we found more and more reasons not to do the shift.” The Astros’ shift totals fell to 33 in May and 32 in June (though their rate rebounded later in the year, particularly in September).

As a result, Luhnow said, the Astros realized that in order to make an experiment stick, “You’ve got to market it to the people that are involved.”⁵ This spring, the Astros spent an hour explaining the thought process behind the shift to the team’s pitching staff, building a tool to display evidence that would answer any questions the players might ask. “I think they weren’t completely satisfied, but I think they felt like we had at least given them a lot more information, and this year we haven’t had anywhere near the pushback from our pitching staff that we did last year,” Luhnow said. Houston also started shifting in the minor leagues to ease the adjustment to the majors. This year, the Astros are leading the majors in shifts by a wide margin, and their monthly totals tell a more consistent story: 272, 263, 208, and 216, with a pace that would put them close to 300 in August.

Admittedly, Luhnow’s chosen subject was self-serving. Even before bungling the Brady Aiken situation, at least from a public-relations perspective, the Astros had inspired industry sniping about a perceived tendency to overlook the messy problem of players’ personalities. What better way to do damage control than by portraying the team as touchy-feely?

But Luhnow wasn’t the only speaker who discussed sabermetrics’ sensitive side. Later on Saturday, Vince Gennaro, the president of the Society for American Baseball Research and the author of Diamond Dollars: The Economics of Winning in Baseball, gave a talk on the same topic. Echoing comments Luhnow had made, Gennaro emphasized the importance of organizational buy-in to analytics-driven innovations, warning that top-down implementation without the proper finesse can lead to resentment, subtle sabotage, and lost opportunities.

Cherington seems to have come closer than most GMs to solving the implementation problem by hiring progressive field staffers like third-base/infield coach/shift evangelist Brian Butterfield and manager John Farrell. (Farrell, a broad-shouldered, square-jawed former major leaguer who looks like he’s a hat change and casting call away from commanding the Last Ship, also spoke at Saber Seminar, citing his early exposure to statistical analysis as the Indians’ director of player development.) However, while Cherington is relatively free of fears about recalcitrant coaches, Boston’s last-place season has spawned other concerns, chief among them the number of Red Sox players who’ve underperformed their projections. “Maybe it’s just that 2014 is the year that that’s happening, and random things happen in baseball,” Cherington said. “That’s possible, I suppose.⁶But because losing games is so painful, it’s hard for us to just stop and be comfortable with that as the answer. We have to ask more questions and try to figure out what else could be going on.”

The most important of those questions probably isn’t, “How can we develop projections that are marginally more accurate than our rivals’?” As Cherington observed, every underperforming player’s season falls somewhere within the range of possibilities that a projection system spits out. The crucial question, as he framed it, is, “How do you get more of your players operating at the higher end of the range of possibilities year in and year out?”⁷

Because record revenues and higher stakes have made front offices so smart, the available edges have grown smaller and more slippery. “It sure felt like in ’02, ’03, ’04, we could more easily create a talent gap between the best teams and the worst teams, and you could more easily count on a bunch of wins before the season ever started,” Cherington said. “That feels harder to do now. It feels like talent is more evenly distributed. If that’s the case, then finding ways to optimize player performance and get guys into the higher range of possibilities is more and more important.”

Cherington claimed not to have the answer(s) to the question he posed, and Boston’s struggles this season suggest he was telling the truth. One possible answer was supplied by UC Riverside’s Dr. Aaron Seitz, an expert in perceptual learning who presented evidence that players’ already elite visual acuity (and by extension, their play) can be improved further through vision training. Cherington mentioned conditioning and mental health as rich veins of research, and when an audience member asked him about Eagles coach Chip Kelly’s sports science innovations, he vaguely suggested that the Red Sox are doing similar work.⁸ Cherington closed with a call to arms. “There’s so much collective intelligence poured into analytics and finding player value, it seems to me that we could take advantage of some of that intelligence and find some solutions to these other issues, ” he said.

Eric Kilby

Luhnow urged the spectators to stretch their areas of inquiry not only beyond the box score, but beyond the database. “As you do the academic work, I would urge you to take it a step further and talk to some people who are actually in the industry … and find out what the objections are going to be,” he said.

On Sunday, Baseball Prospectus author Russell Carleton, who has previously written about soft factors like sleep and nutrition, showed the Saber Seminar crowd what the new frontier for sabermetrics looks like. Carleton, a clinical psychologist by training, reminded the audience that 18-year-olds are “replacement level adults” and noted that most adolescents learn crucial life skills during the period when top prospects are expected to spend their time perfecting their on-field performance. Without those skills, though, the prospects’ play might suffer, sapping some or all of the advantage a team could gain by improving its ability to spot and sign amateur talent. Sensing an opportunity for his expertise to add value, Carleton surveyed player development and front office officials to glean details about programs they’ve put in place to educate and counsel their players, drawing on his background to recommend smarter structures. Numerical analysis of “granular, atomic-level detail is just peachy,” Carleton concluded, “but it’s not the only way to do real research with real baseball value.”

While number-crunching efforts in other sports whose structures present more obstacles to analysis are still in the midst of an exciting summer, with landmark studies and hirings coming fast and furious (at least until the league turns the data tap off), the pace of purely statistical discoveries in baseball seems to be slowing as sabermetrics matures. Many discoveries remain to be made, but the discoveries might be smaller, the advantages more difficult to maintain.⁹ We’ve reached the point where the biggest edge might come not from being better than other teams at acquiring players and dissecting stats, but from making sure the players already on the roster make the most of their talent, and that little is lost in translation between the front office and the field.¹⁰ That doesn’t mean it’s time to call off baseball’s “search for objective knowledge about baseball.” But for sabermetrics to stay relevant, the search radius has to expand.

Photos courtesy of Eric Kilby.

1. One hundred percent of the proceeds, which topped $30,000 this year, go to support the Jimmy Fund.

2. Evidently, the zone is shifted down and to the right at night and indoors, though not by so much that you’d notice without a database full of a few million pitches.

3. One of whom tweeted (gloated?) from Saber Seminar that public analysts lag four to five years behind the most advanced internal ones.

4. “Work that’s been done at major websites has narrowed the gap between the teams that are making this investment and those that aren’t,” Red Sox senior baseball analyst Tom Tippett said Sunday. Tippett, who counts the Red Sox as one of the “teams that are,” told the assembled PITCHf/x analysts that he wishes PITCHf/x info weren’t publicly available, the least crowd-pleasing statement since Madonna told a New Orleans audience to vote Democrat.

5. A process I once dubbed “baseball’s next big inefficiency.”

6. Some amount of randomness is inevitable. Last year, unpredictable events helped the Sox; this year, they’ve hurt them.

7. Particularly, as Cherington suggested, in an era when the pressure on prospects and young players is more intense than ever, and older players can’t rely on chemical means to turn back the clock.

8. According to catcher David Ross, the Sox are among the best teams at doing “little things” to keep their players happy, which could improve performance (or at least retention rates).

9. “There’s so much turnover in our industry that if you’re doing something that you feel is truly unique and potentially could create a competitive advantage for your organization, [it’s] going to be in all 30 organizations very quickly,” Luhnow said.

10. Something Baseball Prospectus founder Gary Huckabay was hinting at when he pronounced baseball analysis dead in 2007.