Nate Silver is the founder of Fiverthirtyeight.com, creator of the PECOTA baseball forecasting system used by Baseball Prospectus, and a renowned political forecaster. In his book, The Signal and the Noise, Nate outlines the creation of the PECOTA system and lessons learned from Bill James (founder of Sabermetrics), along with taking a look at other forecasting
problems opportunities. Silver’s PECOTA system relies on a metric resembling the similarity index proposed by Bill James in his 1986 Baseball Abstract. James developed the similarity index as a tool for comparing any two major league players. In James system the index starts with a 1000 points and detects points based on a set of guidelines. Highly similar players will have indexes as high as 950 or 975. Similarly the PECOTA system uses an index to evaluate a player against a multitude of former major and minor leaguers to project a players performance.
For a young metallurgist whose livelihood depends on projecting the results of varying parameters of an assortment of metallurgical processes to achieve a desired result, how could the lessons of a Sabermetrician help? The opportunity presented itself with the need to develop a high strength product in Alloy 825, an austenitic iron-nickel-chromium alloy commonly used in environments where enhanced corrosion performance is required. The product was to be cold-worked (i.e. deformed at room temperature) to a desired size and strength level. The challenge is none of this data was readily available!
After performing a simple Google search, data for other austenitic alloys such as Alloy 625 (a Ni based alloy) and 316 stainless steel (Fe based) could readily be obtained from sources like ATI and Special Metals. Thus, a simple curve could be fitted to the results for these two alloys. Following Silver’s first principle, Think Probabilistically, a Monte Carlo simulation was developed using several distributions fed into the model to generate a distribution of results at each cold working level. The Monte Carlo simulation was formulated feeding a similarity index varying uniformly (0.5-0.9), a normal distribution of fully annealed Alloy 825 yield strengths, and a normal distribution of residuals from the fitted cold working curves for Alloy 625 and 316. An outline of the model is provided in the figure below.
The Monte Carlo simulation results are provided in the graph below with the blue line representing the mean result with respect to degree of deformation (i.e. percent cold work / area reduction), the redline representing the 99% probability and the bottom line representing the 1% probability. The customer upper and lower specification limits (USL & LSL) are also plotted for reference. The work hardening curve below shows that at a cold working percent of about 30 the product is nearly assured to meet the tensile strength requirements. These results were subsequently validated with actual experiments with a percent error of less than 3%. Eureka!