Issue: Volume 5, Number 8
Date: August 2005
From: Mark J. Anderson, Stat-Ease, Inc. (

Dear Experimenter,

Here's another set of frequently asked questions (FAQs) about doing design of experiments (DOE), plus alerts to timely information and free software updates. If you missed the previous DOE FAQ Alert, please click on the links at the bottom of this page. If you have a question that needs answering, click the Search tab and enter the key words. This finds not only answers from previous Alerts, but also other documents posted to the Stat-Ease web site.

Feel free to forward this newsletter to your colleagues. They can subscribe by going to If this newsletter prompts you to ask your own questions about DOE, please address them via mail

Here's an appetizer to get this Alert off to a good start: I found this article by Tom Geoghegan of BBC News Magazine intriguing because some statistics instructors fall far short of the charisma needed to make this dry subject invigorating to their students. The article defines a charismatic person by three attributes:
- feels emotions quite strongly
- induces them in others
- remains impervious to the influences of other charismatics
(Regarding this last point: I wonder what happens when charismatics gather together?)
PS. Check out the comments, pro and con, on this article, including "...if anybody makes a habit of trying to touch me on the upper arm (as recommended in the article) it drives me insane." Does this bother you too?

Here's what I cover in the body text of this DOE FAQ Alert (topics that delve into statistical detail are designated "Expert"):

1. FAQ: How many samples should you take per experimental run
2. FAQ: Duplication versus true replication and its impact on statistical lack of fit
3. Help wanted: Stat-Ease seeks an enthusiastic DOE professional
4. Events alert: Stat-Ease is exhibiting at the Joint Statistical Meetings (JSM) next week in Minneapolis
5. Reader response: An unsung pioneer for DOE—Kirstine Smith
6. Workshop alert: See when and where to learn about DOE; Also, note the short course on process analytical chemistry

PS. Quote for the month: A definition of enthusiasm (ties in to my appetizer on charisma).


1. FAQ: How many samples should you take per experimental run

-----Original Question-----
From: Salt Lake City

"In our design of experiments (DOE) for every run we collect some samples. It is always better to have more samples to reduce error, but how many? Normally we do measurements on 10 samples. However, my current optimization design on 5 factors requires 38 runs which would generate 380 samples—far more than we can afford. What is the optimum sample size in this situation?"

It would be best to work out the 'optimum' sample size using power calculation based on the signal you wish to detect and estimates of the components of variance.* For now, I will only provide some general ideas that may save you a lot of unnecessary work. Consider that you will get tremendous averaging due to running a designed experiment—probably far more than the 10 samples normally gathered per for measurement. You haven't told me what sort of design you plan, but for example, in a simple two-level with 32 runs each effect is based on averages of 16 high vs 16 low. Therefore, I suspect you might get by with only one sample per run! However, since the mind-set is now 10, I'd think that everyone would be happy even if you cut down to three samples per DOE run—even two might be plenty.

I went through a similar discussion with a client doing a DOE on their sterilization process. Normally they take three samples and test for pathogens in triplicate. They realized that a 30-run DOE would generate several thousand tests! I convinced them to take only one sample and do one test per pathogen. It generated significant results that satisfied FDA for validation of process changes, but with only 1/9th the tests they first thought necessary.

This really illustrates the power of the averaging built into two-level factorial DOE.

*(Learn more about sample size by attending the two-day computer- intensive workshop "Statistics for Technical Professionals." See for a course description. Link from this page to the course outline and schedule. Then, if you like, enroll online.)


2. FAQ: Duplication versus true replication and its impact on statistical lack of fit

-----Original Question-----
From: Minnesota

"I've attached data from a bio-assay for insulin. Each treatment combination was run 6 times. I've looked at this data a number of ways and I always get significant lack of fit. My question is: what are the contributors to lack of fit? Does it have anything to do with the design or is it the data?"

This may be our number 1 FAQ*—treating duplicate samples and/or repeated measures as true "replicates" of experimental runs, that is, re-setting the process at random intervals at the same factor levels and performing a re-run. Sometimes, most of the overall variation will be captured by doing a number (n) of consecutive duplicates/measures. In such cases, if entered as individual runs, the pure error symbolized with green triangles on the half- normal plot may line up fairly well with insignificant effects (the ones nearest zero). Then lack of fit would be insignificant. However, in your case, I believe that the error has been underestimated by the duplication (n=6) versus true replication—a re-run from start to finish done after a complete re-set of factor levels. Thus, your lack of fit (LOF) test has been biased toward the side of significance (p-value < 0.1). Similarly, the p-values shown on the analysis of variance (ANOVA) are also biased low. Therefore, in my opinion, you should ignore the LOF and be skeptical of all but the very largest effects uncovered by the half-normal plot. In your case, my guess is that only the one main effect of factor B really affected your process significantly.

Next time around you ought to fully replicate (no short cuts!) the 2^2 design in a randomized run plan. Then enter the average results from each set of duplicates, which will provide much more stable results (by Central Limit Theorem—variance of sample-to-sample plus test-to-test will be reduced by n).

*For more details, including illustrations, see "FAQ—Interpreting Lack of Fit" by Stat-Ease Consultant Shari Kraber on page 2 of the May 2004 "Stat-Teaser" newsletter posted at

(Learn more about lack of fit by attending the three-day computer-intensive workshop "Experiment Design Made Easy." See for a complete description. Link from this page to the course outline and schedule. Then, if you like, enroll online.)


3. Help Wanted: Stat-Ease seeks an enthusiastic DOE professional

We at Stat-Ease, Inc., a Minneapolis-based statistical software company, offer an opportunity for an energetic person to join our team. The full-time job opening encompasses a combination of teaching and technical development. Job responsibilities include:
- teaching workshops
- providing statistical support
- defining software test cases
- researching statistical methods
- writing technical materials
- other statistically-related duties based on experience.
Expect to be traveling out of our Minneapolis headquarters about 30% of the time.

Minimum qualifications for this DOE professional are:
- Master’s degree in statistics
- hands-on DOE experience
- excellent verbal and written communication skills
- strong ability for applied mathematics.

We desire experience in:
- teaching statistics to non-statisticians
- manufacturing or other industrial functions (two years of work—internships acceptable)
- mixture design for optimal formulation
- computer programming (C++ desirable)
- developing online course materials.

E-mail resumes to Shari Kraber at by August 10th to be considered for this position as a DOE professional. Watch for our job notice at the Joint Statistical Meetings in Minneapolis, MN, August 8–11.


4. Events alert: Stat-Ease is exhibiting at the Joint Statistical Meetings (JSM) next week in Minneapolis

Please visit us at Booth #402 at the exhibit hall for the Joint Statistical Meetings in our home town of Minneapolis, Minnesota, next week, August 8-11.

Click on for a list of appearances by Stat-Ease professionals. We hope to see you sometime in the near future!


5. Reader response: A unsung pioneer for DOE—Kirstine Smith

-----Original Comment-----
From: Selden B. Crary, President, Crary Group, Palo Alto, CA

"I just read your article "Trimming the FAT out of Experimental Methods" ( It starts with a bit of history, emphasizing Fisher's 2^k factorial designs, and ends by encouraging RSM for the 21st century. You might be interested to learn that the true pioneer in DOE/RSM was Kirstine Smith, a graduate student from Denmark, who studied with Karl Pearson in London during WWI. In 1918 she published a wonderful paper, "On the 'Best' Distribution of Observations" in Biometrika that laid out the foundation of what is now known as statistical optimal design of experiments. Her example designs, which were all calculated by pencil and paper, are in the class of optimality now known as global optimality or G-optimality. You may learn a bit more about the history of Ms. Dr. Lecturer Smith from the biographical information on the WebDOE site at URL:

Yes, I first heard of Dr. Smith from Stephen M Stigler, author of a wonderful book titled "The History of Statistics" (link to this at Amazon via, who lectured at the University of Minnesota not too long ago. I'd bought his book years before, but learned a lot of interesting things such as the role of women like Smith in the development of statistical methods early in the last century.


6. Workshop alerts: See when and where to learn about DOE; Also, note the short course on process analytical chemistry

See for schedule and site information on all Stat-Ease workshops open to the public. To enroll, click the "register online" link on our web site or call Stat-Ease at 1.612.378.9449. If spots remain available, bring along several colleagues and take advantage of quantity discounts in tuition, or consider bringing in an expert from Stat-Ease to teach a private class at your site. Call us to get a quote.

Also, as a favor to a Dr. Ann M. Brearley, with whom I shared duties as speaker for a recent symposium on chemical engineering, I am passing along this note from her: "If you have any clients who want to improve their chemical measurements, here is a short course that they might be interested in. Katherine Bakeev (GlaxoSmithKline—GSK) and I will be teaching a one-day short course on process analytical chemistry at the Eastern Analytical Symposium (Somerset, New Jersey) this coming November. The course is intended for chemists or engineers involved in process improvement projects who need better, faster chemical measurement tools such as NIR or in-line analyzers. Further information is available at the EAS web site,, under Short Courses.* Feel free to pass this along to anyone interested.


Ann M. Brearley, Ph.D.
Consultant, Process Analytical Chemistry
2435 Shadyview Lane N., Plymouth, MN 55447
p: 763.238.7200"

*I notice several workshops at the Eastern Analytical Symposium on statistics and experiment design.— Mark :)


I hope you learned something from this issue. Address your general questions and comments to me at:



Mark J. Anderson, PE, CQE
Principal, Stat-Ease, Inc. (
2021 East Hennepin Avenue, Suite 480
Minneapolis, Minnesota 55413 USA

PS. Quote for the month: A definition of enthusiasm (ties in to my appetizer on charisma):

"Enthusiasm is a kind of faith that has been set on fire."
- George Matthew Adams

Trademarks: Design-Ease, Design-Expert and Stat-Ease are registered trademarks of Stat-Ease, Inc.

Acknowledgements to contributors:
— Students of Stat-Ease training and users of Stat-Ease software
— Fellow Stat-Ease consultants Pat Whitcomb and Shari Kraber (see for resumes)
— Statistical advisor to Stat-Ease: Dr. Gary Oehlert (
— Stat-Ease programmers, especially Tryg Helseth (
— Heidi Hansel, Stat-Ease marketing director, and all the remaining staff


Interested in previous FAQ DOE Alert e-mail newsletters?
To view a past issue, choose it below.

#1 Mar 01
, #2 Apr 01, #3 May 01, #4 Jun 01, #5 Jul 01 , #6 Aug 01, #7 Sep 01, #8 Oct 01, #9 Nov 01, #10 Dec 01, #2-1 Jan 02, #2-2 Feb 02, #2-3 Mar 02, #2-4 Apr 02, #2-5 May 02, #2-6 Jun 02, #2-7 Jul 02, #2-8 Aug 02, #2-9 Sep 02, #2-10 Oct 02, #2-11 Nov 02, #2-12 Dec 02, #3-1 Jan 03, #3-2 Feb 03, #3-3 Mar 03, #3-4 Apr 03, #3-5 May 03, #3-6 Jun 03, #3-7 Jul 03, #3-8 Aug 03, #3-9 Sep 03 #3-10 Oct 03, #3-11 Nov 03, #3-12 Dec 03, #4-1 Jan 04, #4-2 Feb 04, #4-3 Mar 04, #4-4 Apr 04, #4-5 May 04, #4-6 Jun 04, #4-7 Jul 04, #4-8 Aug 04, #4-9 Sep 04, #4-10 Oct 04, #4-11 Nov 04, #4-12 Dec 04, #5-1 Jan 05, #5-2 Feb 05, #5-3 Mar 05, #5-4 Apr 05, #5-5 May 05, #5-6 June 05, #5-7 July 05, #5-8 August 05 (see above)

Click here to add your name to the FAQ DOE Alert newsletter list server.

Statistics Made Easy™

DOE FAQ Alert ©2005 Stat-Ease, Inc.
All rights reserved.


Software      Training      Consulting      Publications      Order Online      Contact Us       Search

Stat-Ease, Inc.
2021 E. Hennepin Avenue, Ste 480
Minneapolis, MN 55413-2726
p: 612.378.9449, f: 612.378.2152