Why Moneyball Matters

Apr
30
2013
Why Moneyball Matters
William H. Panning –

This terrific story also has crucial implications for business executives, especially in insurance

Billy Beane, general manager of the Oakland A’s, had a problem: recruiting a pennant-winning team despite having one of the smallest payrolls in the major leagues. How he solved that problem is expertly told in the movie Moneyball and the book with the same title, by Michael Lewis. But often overlooked are the important strategic implications of this story, particularly for insurers.

From one point of view, Moneyball is a dramatic story of how four determined individuals overcame a crisis. Oakland’s new owners had precipitated the crisis by dramatically reducing Oakland’s once substantial budget for player salaries, making it one of the lowest in the major leagues. The heroes of this story — Bill James, Sandy Alderson, Billy Beane, and Paul Podesta (given an alias in the movie version) – found an innovative way for the A’s to nonetheless recruit a team that outperformed better funded rivals.

From a strategic point of view, Beane’s accomplishment was demonstrating the possibility, practicality, and superiority of an innovative business model for major league baseball. Each of the four heroes played a key role in this process. Bill James independently pioneered sabermetrics, the use of rigorous statistical methods to analyze baseball performance. Sandy Alderson, Beane’s predecessor as general manager, recognized the potential value of adopting a more scientific approach to identifying talented players. Alderson helped Beane understand sabermetrics and the potential competitive advantages of embracing it. Beane championed and implemented the new business model and hired Paul Podesta, a statistician, to identify promising recruits that could be hired cheaply.

For businesses other than baseball, Moneyball matters most because it illustrates the creative use of fundamental technological innovations that are accelerating and profoundly disrupting businesses of all kinds, especially insurance and other financial services. Here’s how Oakland’s experience can help insurers recognize the specific opportunities that technological change makes possible, and overcome a crucial obstacle to translating these possibilities into actual competitive advantages.

First, Moneyball illustrates the potential value of applying rigorous analytics to business decisions. The term “rigorous” is crucial. When Bill James published his 1977 Baseball Abstract, baseball statistics consisted solely of box scores, invented in 1845 and last improved by Henry Chadwick in 1859. Although the defects of measures like runs batted in (RBI) had been recognized for decades, it and other flawed measures had not been supplanted. James developed new performance measures, conclusively demonstrated their superiority in predicting a team’s success in winning games, and consequently challenged or amended much conventional wisdom.

The insurance industry likewise has flawed measures on which firms and regulators rely. For example, an increase in a firm’s premium to surplus ratio is commonly thought to indicate that the firm’s risk has increased. This is sometimes true, but often not, as when prices increase in response to catastrophes or a turn in the underwriting cycle. Yet the industry has shown little inclination to developed improved measures, since current ones are enshrined by tradition and regulatory reporting requirements. The problem, as in baseball, is that using flawed measures leads to false conclusions and wrongheaded decisions.

Second, James’ development and Oakland’s application of sabermetrics depended crucially on technological innovations — the increased availability of pertinent data, and enormous improvements in low-cost but powerful computers — that can be creatively exploited by many other industries as well. The merits and defects of RBI had been debated for years, but James was able to use new data and powerful computers to find and demonstrate the merits of superior performance measures. For example, he convincingly showed that on-base percentage is much superior to RBI, which depends on the accidental circumstances of how many players are already on base. He also persuasively demonstrated that being walked matters to a team’s final score, and that some players are significantly better than others at getting walked. In short, James was able to use newly-available data and computing power to settle debates that had raged for decades.

Third, Oakland’s new business model was innovative in another crucial way: it replaced cognitive labor – white collar work – with data, computers, and statistical algorithms. The new business model was particularly galling to scouts, because relying on statistical data analysis undermined their typical intuitive reliance on experience and on stereotypes of potentially outstanding performers. Fortunately for Oakland, Billy Beane was the ideal manager to champion this innovation because, as he and the scouts both recognized, he conformed perfectly to their image of the ideal player but had never succeeded in meeting the expectations that had created. It helped enormously, of course, that the actual performance of the Oakland A’s new recruits, and of the team as a whole, ultimately exceeded skeptics’ expectations. Yes, it took some time to get the lineup right, but in the end the A’s had great success against teams with much larger player payrolls.

There are two important risks of relying on intuition and experience when hard data are available. The first is perceptual insensitivity to small but important differences. The difference between a .300 hitter and a .275 hitter is roughly one hit every two weeks, a difference not readily visible to a scout but readily discernible in statistical analysis. The second is that stereotypes may be too crude or inaccurate to be effective. Improved analytical measures provided a direct challenge to entrenched stereotypes supported by intuition rather than rigorous analysis of abundant data. Are the players who score highest on improved analytic measures the same ones recommended by scouts? Not always, because some of the better-scoring players really don’t look like what scouts expect. Is the best strategy to hire players with the highest on-base percentages? No; to maximize the effectiveness of a now shrunken payroll, Oakland fared best by hiring players with high on-base percentages who ranked low in scouting reports, and so were valued less highly by competitors who relied on scouting. Can statistics help identify valuable trades? Sure: trade highly paid players for others with similar on-base percentages but lower pay. This process of using data and statistics to answer crucial strategic questions that were once answered intuitively was what James developed, Sandy Alderson had recognized as possible, and Billy Beane came to understand and implement based on calculations by Paul Podesta.

In insurance as in baseball, when pertinent and detailed data is unavailable or ignored, the use of potentially misleading stereotypes remains prevalent. Some years ago the chief personal lines actuary at a major insurer told me that he was “cleaning up” his firm’s book of business. As a newcomer to the business, I asked him what that meant, and he explained that it meant getting rid of all the policies that his company’s top executives considered unprofitable. Unfortunately, he explained, that meant getting rid of business that was actually highly profitable but conflicted with underwriters’ firm conviction that motorcyclists were high risk high loss customers that should be eliminated from their book of business. For example, he noted, motorcycle commuters over age 35 had high premiums, since they were rejected by underwriters at most firms, but low losses, and so were highly profitable. But the underwriters were unyielding and, in this case, had the final decision.

None of this means that intuition and experience are obsolete and irrelevant to business decisions. It simply means that when it comes to determining facts, intuition and experience can be misleading and imprecise when superior methods are available. Intuition and experience nonetheless remain crucial in other ways.

Fourth, the fundamental innovations on which Oakland’s new business model relied jeopardize many business models that rely heavily on fact-based cognitive labor – white collar work – by creating opportunities to revise or replace them with more sophisticated and effective ones. The displacement of cognitive labor is becoming pervasive throughout our economy, and stands in marked contrast to earlier and distinct technological innovations that supplanted or supplemented physical labor and skills. Just as factories now rely heavily on machinery and robots to assemble their products, companies everywhere now rely on computers and software, rather than rooms of accountants, to produce their financial reports. Similarly, law firms increasingly rely on sophisticated software, rather than on large teams of paralegal assistants, to screen data files and emails during the discovery process at firms being sued. Banks make personal loans only after detailed analysis of credit scores calculated from huge amounts of data, and retailers focus their marketing efforts based on analysis of past purchases by actual and prospective customers. In insurance, underwriting and pricing decisions increasingly rely less on intuition and personal experience than on sophisticated analysis of vast and increasing data resources.

Fifth, as was initially the case in baseball, in insurance there are huge gaps between firms that lead their industry in exploiting these innovations and the laggards that do not. The (freely accessible) January 2013 issue of Best’s Review contains the results of its second annual Innovation Showcase, in which insurance-related firms are invited to submit descriptions of innovations they have developed and applied. Serving as one of the reviewers in both years, I was surprised by a number of entries from firms who proudly described as “innovative” some routine improvements that many of their competitors had implemented years ago. More generally, my regular review of innovations reported in the business press reveal few that exhibit the imagination and effectiveness of Oakland’s new business model. A prevalent view among the laggards may be the one expressed by a CEO who vigorously rejected the adoption of innovative analytics: “Show me a dozen firms who have dramatically increased their profitability by using these ideas,” he demanded, “and I will potentially consider using them here.”

A crucial reason for this huge distance between leaders and laggards is a growing and pervasive gap between quants and managers, groups that differ significantly in their culture, experience, and skills. This gap is partly a consequence of differences in age and experience. Senior executives are typically older, were educated in the early stages of this current technological revolution, and often have little training in or experience with the analytic tools now available. By contrast, the much younger quants, who create or apply these new tools, are familiar and comfortable with technology and analytics. In practice, many executives fail to comprehend how to effectively manage quants and utilize their work. And quants often lack the organizational understanding and skills needed to make their work as useful as possible to senior executives. Because of these and other differences, the two groups often fail to understand each other.

Bridging this gap is one of the central challenges of our industry for firms who aspire to become or remain leaders. There is no simple fix or permanent cure. The gap will not be bridged by sending a few senior managers to a typical executive education seminar or program, nor by their taking courses in statistics or some other analytical discipline, nor by having quants take courses in organizational psychology and behavior. These options may be relevant in some theoretical way, but are typically too far removed from the day to day activities most affected by the gap between quants and managers. Much more effective would be the kind of continuing and goal-specific advisory relationship that developed between Beane and Paul Podesta, the quant he hired to help him maximize the effectiveness of his payroll.

At several firms I personally experienced an analogous relationship with managers who wanted to utilize my quantitative skills to solve some complex and serious strategic problems. The success of our collaboration depended crucially on establishing a common understanding of each other and our respective ways of thinking about the problems at hand. Bridging the gap between quants and managers is necessarily a two-way street that requires a knack for effective communication, a willingness to learn, and a commitment to successful cooperation.

For managers who wish to implement and effectively utilize sophisticated quantitative tools, two tasks are crucial. The first is entertaining a broad panorama of possibilities rather than focusing on a narrow area of specific expertise. The second is asking crucial questions that enable them to better understand both the potential benefits of new analytic methods and their important limitations. Quants who want to maximize the positive impact of their work likewise have two crucial tasks. First, they need to learn how executives think about, prioritize, and utilize the information they receive in the course of making strategic decisions. Then they need to use this understanding to communicate far more effectively with their potential internal customers, corporate managers. Too many quants unconsciously fall into the trap of writing and speaking to an implicit audience of fellow quants rather than the managers they profess to be addressing.

Firms that aspire to industry leadership might well give serious thought to emulating Oakland’s success in exploiting the opportunities presented by technological change. But success may be only temporary unless there is a commitment to continuing innovation. In business as in baseball, successful innovation provides competitive advantages that may be significant but are ultimately imitated by competitors. In major league baseball, the business model pioneered by Billy Beane is now pervasive, so that the market for new recruits is much more efficient than before. But as the technological revolution continues there will be still more opportunities for business model innovation or improvement. Firms that recognize the need for continued innovation will likely capture and retain leadership positions, while those that fail to embrace these opportunities will gradually become also-rans or even extinct. Being too late to the party may mean missing it altogether.

William H. Panning is Founder and Principal of ERMetrics, LLC. His email is bill@ERMetrics.com.

Copyright © 2019 | Property Casualty Insider