How did the Adequate Precision become a favored measure of model usefulness?

Original question from a Research Geneticist:
“Stat-Ease software provides an array of standard fit-statistics, e.g., adjusted and predicted R²; that provide very helpful insights on the quality of models produced by designed experiments. The ‘bottom line’ is a statistic called “Adequate Precision” (AP), for which your programs describe as a measure of the ratio of signal to noise. They advise that models generating an AP greater than 4 can be used to “navigate the design space”. I find this statistic, which seems unique to your software, quite valuable. Where did it come from?”

Answer from Stat-Ease Consultant Joe Carriere:
“George Box and John Wetz developed the essence of this statistic in 1973 as a ‘Criteria for judging adequacy of estimation by an approximating response function’ (University of Wisconsin Statistics Department Technical Report No. 9). In their book on Empirical Model-Building and Response Surfaces (1987) Box and Draper provide an ‘alternative and, in spirit, equivalent procedure’ to the Box-Wetz criterion that compares the range of fitted responses with their average standard error, a la our measure of Adequate Precision.”

PS Kudos to Joe for digging this up. Being an engineer, I love ‘rules-of-thumb’—they are simple and very comforting. One that I heard about many years ago (possibly in a short course taught by Box) is a rule that the critical value for the F statistic should be multiplied by a factor of 4 when using a regression model for prediction. In their book on Applied Regression Analysis (3^rd Edition), Draper and Smith spell this rule out in the summary for their Chapter 11 "On Worthwhile Regression, Big F’s, and R²". In any case, as Yogi Berra said, “It’s tough to make predictions, especially about the future.”

PPS By the way, Joe says that in equation 11.1.2 Draper and Smith detail mathematically the ‘adequate precision’ statistic. They say that “provided this is sufficiently large (a rough rule of thumb of about four is suggested as minimal) and provided no other defect is seen in the fit, a worthwhile regression interpretation is likely to be possible."

(Learn more about model-fitting by enrolling in the workshop(s) Modern DOE for Process Optimization and/or Mixture Design for Optimal Formulations.)

Stat-Ease now on Instagram

Click here to see Stat-Ease (@stateaseinc) on Instagram.  We hope you will follow us.  Also stay in touch with us via:

Suggestion on numeric optimization from power user of Stat-Ease Software

Regarding my May/June 2022 FAQ “Overall desirability for optimization not very high” (seen here), Andrew (“Drew”) Ekstrom suggests that “with the desirability function, sometimes relaxing a requirement/constraint or redefining an objective function in a constraint within a range, can help immensely; for example, if you set out to reduce variability and maximize output, you can get some poor, unusable outcomes; but, if you make maximizing output the goal and set a range for variability, you can get some highly desirable outcomes.” Good advice, Drew!

We invite you to join us for the Stat-Ease 2022 Online DOE Summit on October 4-6—the world’s premier conference on practical applications for design of experiments. Our Summit is presented virtually, free of charge! The agenda is shaping up with fun and informative talks, such as:

• “The Magic of Multifactor Testing” (my contribution!),
• “Case Cracked: Setting Process Windows with Design-Expert” and
• “Saving the Mega Blaster from Disaster”.

These are just a few of the planned presentations that cover a wide range of topics and industries. Get ready for many entertaining and educational talks on making the most from industrial experiments.

See the entire lineup and register now at our DOE Summit site.

You can do no better for quickly advancing your DOE skills than attending a Stat-Ease workshop. Our expert instructors provide you with a lively and extremely informative series of lectures interspersed by valuable hands-on exercises. Enroll early to ensure your spot!

• Mixture Design for Optimal Formulations
  • September 26-29, 4 half-days online, $995
• DOE Crash Course for Experimenters
  • October 19, 10 AM CST, 1-hour free webinar
• Modern DOE for Process Optimization
  • October 24-28, 5 half-days online, $995

See this web page for the complete schedule of upcoming Stat-Ease courses. To enroll in the workshop that suits you best, click Register on that webpage, or click here to contact us.
 

“In 1955, a young man named Norman Draper worked with me at ICI as a summer student.  He recalls riding his motorbike to the job, which paid the paltry sum of five pounds a week.  He expected that the job would entail endless data entry. Instead I sent him all over to talk to scientists, get answers to questions, and discuss problems.”
 
—George Box, An Accidental Statistician, p46 on setting a path for Norman Draper that led to him developing many great tools for industrial experimenters. (For more about Norman Draper and his life, see his Legacy.com obituary.)

Circulation: Over 4000 worldwide