Issue: Volume 10, Number 6 (Circulation: Over 5500 worldwide)
Date: June 2010
From: Mark J. Anderson, Stat-Ease, Inc., Statistics Made Easy® Blog

Dear Experimenter,

Here's another set of frequently asked questions (FAQs) about doing design of experiments (DOE), plus alerts to timely information and free software updates. If you missed the previous DOE FAQ Alert, see below.

==> Tip: Get immediate answers to questions about DOE via the Search feature on the main menu of the Stat-Ease® web site. This not only pores over previous alerts, but also the wealth of technical publications posted throughout the site.

Feel free to forward this newsletter to your colleagues. They can subscribe by going to If this newsletter prompts you to ask your own questions about DOE, please address them via mail to:

Also, Stat-Ease offers an interactive web site — the Support Forum for Experiment Design at Anyone (after gaining approval for registration) can post questions and answers to the Forum, which is open for all to see (with moderation). Furthermore, the Forum provides program help for Design-Ease® and Design-Expert® software. Check it out and search for answers. If you come up empty, do not be shy: Ask your question! Also, this being a forum, we encourage you to weigh in!

For an assortment of appetizers to get this Alert off to a good start, follow this link,* (-> new web site!), and see a number of new blogs (listed below, beginning with the most recent one):

—PB&J please, but hold the jelly (and margarine) and put it on toast — a mixture design combined with a categorical factor
—Stat-Ease Corporation celebrates 25 years in business
—Two-level factorial experimentation might make music for my ears

Also see the new comments on post #520 (3/28/10): "Misuse of statistics calls into question the credibility of science."

*Need a feed or e-mail updates from StatsMadeEasy? Go to It's easy!

"Your StatsMadeEasy blogs brighten up a dreary workday!"
—Applied Statistician, Florida

Topics in the body text of this DOE FAQ Alert are headlined below (the "Expert" ones, if any, delve into statistical details).

1. FAQ: Is it necessary to introduce replicate runs for factorial designs?
2. Info Alert: Practical considerations for the implementation of design of experiments in quality by design (QBD)
3. Book Giveaway: Winners of 1 copy each of "DOE Simplified, 2nd Ed" (signed) and "Response Surface Methodology, 3rd Ed"
4. Webinar Alert (1st): DOE Made Easy and More Powerful via Design-Expert Software, Part 2 — Response Surface Methods (RSM) for Process Optimization
5. Events Alert: Where and when to attend talks by experts from Stat-Ease and see its software exhibited
6. Workshop Alert: Unique opportunity to learn factorial DOE and RSM in back-to-back workshops — a powerful one-two punch

P.S. Quote for the month: Soundness of observations on "obvious things" versus the "fancy" of the brain. (Wisdom from the first Curator of Experiments.)

(Page down to the end of this ezine to enjoy the actual quote.)


1. FAQ: Is it necessary to introduce replicate runs for factorial designs?

-----Original Question-----
Fabrication Engineer
"I have two questions:
—First: I would like to know how calculate the effect of each factor with no replicates for a general factorial design on 4 factors with 4 levels each. My Design-Expert software somehow determines the significance of each factor without the presence of any replicates in the design of experiment. How can this be? How does it calculates the variance? Should I introduce replicates?
—Second: What is the 'lack of fit'? It appears when I add replicates. If there is not any replicate, this parameter disappears from the ANOVA (analysis of variance)."

Answer (from Stat-Ease Consultant Wayne Adams):
"Effects in a factorial design can be estimated without any true replicates because there is hidden replication. The average outcome of a given treatment (level) can always be estimated and the difference between treatment means (or grand mean in the case of categoric factors) can be calculated. This produces an estimated effect and will eventually lead to an estimated coefficient if included in the fitted model.

If all the effects are estimated and included in the model then you are correct, there is not a valid test for significance. What if some of the treatment means were only slightly different? Would this small difference be attributed to a true factor effect or considered random noise in the system? We have a choice when fitting the model. We can choose to say the variation in the treatment means is mostly coming from random noise, or mostly coming from a true effect. If it is determined to be mostly noise (small effects) we pool all the noise effects together (pooled error). This pool of errors is compared to the larger effects. If the larger effects are enough larger (F-ratio), then they "test" significant.

Replicates show us variation that we KNOW belongs in the error pool. Nothing changed in the factors, yet the response measurements are somewhat different. This is called "pure error" in Design-Expert. The pure error is literally added to the pool of errors when the effects are tested. Because we know that some of the error comes from replicates we can also test the pooled error from the "too small" effects noted above against the "pure error". This is also a ratio of variances (F-ratio) and generates a p-value. A significant lack-of-fit indicates the pool of errors is larger than the pure error, which is usually interpreted to indicate the wrong model has been fitted.

Take care with the interpretation of lack-of-fit tests. If a process is extremely stable, the replicates will have very little variation. The tiny pure error this produces will create a significant lack-of-fit test. In these cases take a look at the adjusted and predicted R-squareds. If they are high and within 0.2 of each other then the model is representing the data and true response surface well. If there are no replicates, then there is no "pure error" estimate, therefore no lack-of-fit test.

Do you always need to replicate? No. Do replicates improve the analysis? Yes. Whether or not replicates are needed in a particular experiment depends on many things. For example, if the initial design falls short on power then we advise you add runs. However, if the initial design is a fraction of all combinations, then building a larger fraction is better than replicating the existing one, at least so far as power is concerned. Adding a number of replicated runs (we advise at least 4 — otherwise do not bother to do any) is useful for enabling the lack-of-fit test."

This answer by Wayne covers many bases. One that I'd like to reinforce is that when replicates are added to a design, be very wary of their agreement with each other being too good to be true, that is, not really reflecting the true error overall. I've seen cases where every replicate agreed exactly! Obviously someone simply copied over the responses rather than actually replicating the run. Other common mistakes are to simply re-sample or re-test the materials made during a given run, which underestimates the error by not going through a compleat process re-set. Finally, even replicates done fair and square, re-done from start to finish, might produce results that are too far off for the experimenter to accept, so they get discarded and another run is done that agrees better. That is not fair! — Mark

PS. Version 8 of Stat-Ease software offers a wonderful new tool for general factorials such as this 4x4x4x4 —the ability to plot effects on a half-normal plot. This makes it really easy to sort out at a glance which, if any, effects stand out from the rest (the vital few). Simply click these into your provisional model (subject to verification via ANOVA and residual diagnostics) and leave the remainder of effects, those near zero that are lined up (the trivial many, normally distributed) for the error pool. This is really handy for all factorials, but especially ones that are unreplicated. For details on Design-Expert V8 and a link to the free trial download (at the bottom of the long list of new features), go to

(Learn more about factorials by attending the two-day computer-intensive workshop "Experiment Design Made Easy." See for a description of this class and link from this page to the course outline and schedule. Then, if you like, enroll online.)


2. Info Alert: Practical considerations for the implementation of design of experiments in quality by design (QBD)

The June issue of Bioprocess International features an article on "Practical Considerations for the Implementation of Design of Experiments in Quality by Design" by Mahesh Shivhare, PhD, statistician and Graham McCreath, PhD, head of process design for Avecia Biologics, Billingham, UK. Here's the abstract: "Quality by Design (QbD) is a systematic approach to bioprocess development of pharmaceutical products involving the use of science-based understanding, quality risk management, and statistically designed experiments. By defining target product quality profiles and applying QbD approaches, product critical quality attributes (CQA) can be identified and manufacturing processes can be developed that maximize control and minimize variability. Statistical design of experiment (DoE) has a major role to play in QbD by allowing the relationships between critical process parameters (CPP), material attributes, and CQAs to be identified and understood. However, to extract the most relevant information from DoE studies and to avoid potential problems of misidentification of important process parameters, a careful consideration of the experimental objectives, design, and interpretation has to be made. The authors present examples of best practice in DoE to avoid such problems that should ultimately contribute to correct identification of a process 'Design Space.'"

The latest version 8 of Design-Expert software, posted* at for free trial evaluation, provides a feature of particular interest to biopharmaceutical experimenters aiming for Quality by Design (QbD): Graphical optimization now frames the design space where all modeled responses fall within confidence, prediction or tolerance intervals (user choice). Check it out!

* This web site also provides free patches to update older licensed versions of 8.0.

(Learn how to apply response surface method (RSM) tools for mapping out design space for QbD by attending the two-day computer-intensive workshop "Designed Experiments for Life Sciences." For a complete course description, see Link from this page to the course outline and schedule. Then, if you like, enroll online.)


3. Book Giveaway: Winners of 1 copy each of "DOE Simplified, 2nd Ed" (signed) and "Response Surface Methodology, 3rd Ed"

These lucky readers were selected at random from over nearly 50 entrants for our latest book giveaway.

—Bob Schiffmann, R.F. Schiffmann Associates, Inc., NY, NY (Mister Popcorn!) won the "DOE Simplified, Practical Tools for Effective Experimentation, 2nd edition,"* by Mark J. Anderson and Patrick J. Whitcomb, 2007. (I signed this copy.)
—George Shahin, Sr. Technology Specialist, Atotech USA Inc., Rock Hill, SC got drawn for the copy of "Response Surface Methodology, Process and Product Optimization Using Designed Experiments, 3rd edition," by Raymond H. Myers, Douglas C. Montgomery and Christine Anderson-Cook, 2009.

Congratulations to these two winners and condolences to the others who entered into this drawing. Keep watching for more great books to be given away in the future.

*(This economical and informative how-to soft cover primer on factorial design, and its companion "RSM Simplified" are available via e-commerce at


4. Webinar Alert: DOE Made Easy and More Powerful via Design-Expert Software, Part 2 — Response Surface Methods (RSM) for Process Optimization

Keeping it simple and making it fun, Stat-Ease is introducing an array of statistical methods for design of experiments (DOE) made easy and more powerful via version 8 of Design-Expert software:

—Two-level factorials for process screening, characterization and verification
—Response surface methods (RSM) for process optimization
—Multicomponent mixture design for optimal formulation.

I will present the second of this series of free webinars by working through case studies on RSM on Wednesday, July 14 at 2 PM USA Central Time* (CT). I will repeat this presentation on Thursday, July 15 at 8 AM.

Stat-Ease webinars vary somewhat in length depending on the presenter and the particular session — mainly due to breaks for questions: Plan for 45 minutes to 1.5 hours, with 1 hour being the target median.

When developing these one-hour educational sessions, our presenters often draw valuable material from Stat-Ease DOE workshops. Attendance may be limited, so sign up soon by contacting our Communications Specialist, Karen, via If you can be accommodated, she will provide immediate confirmation and, in timely fashion, the link with instructions for our new web-conferencing vendor: GotoWebinar (see

*(To determine the time in your zone of the world, try using this link: Note that we are based in Minneapolis, which appears on the city list that you must manipulate to calculate the time correctly. It seems that figuring out the clock on international communications is even more complicated than statistics! Good luck!)


5. Events Alert: Where and when to attend talks by experts from Stat-Ease and see its software exhibited

Click for a list of upcoming appearances by Stat-Ease professionals. We hope to see you sometime in the near future!

PS. Do you need a speaker on DOE for a learning session within your company or technical society at regional, national, or even international levels? If so, contact me. It may not cost you anything if Stat-Ease has a consultant close by, or if a web conference will be suitable. However, for presentations involving travel, we appreciate reimbursement for travel expenses. In any case, it never hurts to ask Stat-Ease for a speaker on this topic.


6. Workshop Alert: Unique opportunity to learn factorial DOE and RSM in back-to-back workshops — a powerful one-two punch

Seats are filling fast for the following DOE classes. If possible, enroll at least 4 weeks prior to the date so your place
can be assured. However, do not hesitate to ask whether seats remain on classes that are fast approaching! Also, take advantage of a $395 discount when you take two complementary workshops that are offered on consecutive days.

—> Experiment Design Made Easy (EDME)
> June 15-16 (Minneapolis, MN)
> July 27-28 (Minneapolis)

—> Response Surface Methods for Process Optimization (RSM)
> July 29-30 (Minneapolis)

(Take both EDME and RSM to earn $395 off on combined tuition.)

—> Designed Experiments for Life Sciences (DELS)
> September 29-30 (Minneapolis)

—> Mixture Design for Optimal Formulations (MIX)
> August 17-18 (Minneapolis)
> October 26-27 (Minneapolis)

—> Advanced Formulations: Combining Mixture & Process Variables
(MIX2) (
> October 28-29 (Minneapolis)

(Take both MIX and MIX2 to earn $395 off on combined tuition.)

—> Designed Experiments for Life Sciences (DELS)
> September 29-30, 2010 (Minneapolis)

See for complete schedule and site information on all Stat-Ease workshops open to the public. To enroll, click the "register online" link on our web site or call Elicia at 612.746.2038. If spots remain available, bring along several colleagues and take advantage of quantity discounts in tuition. Or, consider bringing in an expert from Stat-Ease to teach a private class at your site.*

*Once you achieve a critical mass of about 6 students, it becomes very economical to sponsor a private workshop, which is most convenient and effective for your staff. For a quote, e-mail


I hope you learned something from this issue. Address your general questions and comments to me at:




Mark J. Anderson, PE, CQE
Principal, Stat-Ease, Inc. (
2021 East Hennepin Avenue, Suite 480
Minneapolis, Minnesota 55413 USA

PS. Quote for the month—Soundness of observations on "obvious things" versus the "fancy" of the brain.

"The science of nature has been already too long made only a work of the brain and the fancy: It is now high time that it should return to the plainness and soundness of observations on materials and obvious things."

—Robert Hooke, first Curator of Experiments for the Royal Society of England, founded 1662. (Source: "The Discoverers" by Daniel J. Boorstin, Random House, New York, 1983.)

Trademarks: Stat-Ease, Design-Ease, Design-Expert and Statistics Made Easy are registered trademarks of Stat-Ease, Inc.

Acknowledgements to contributors:
—Students of Stat-Ease training and users of Stat-Ease software
—Stat-Ease consultants Pat Whitcomb, Shari Kraber, Wayne Adams and Brooks Henderson (see for resumes)
—Statistical advisor to Stat-Ease: Dr. Gary Oehlert (
—Stat-Ease programmers led by Neal Vaughn and Tryg Helseth (
—Heidi Hansel Wolfe, Stat-Ease marketing director, Karen Dulski, and all the remaining staff that provide such supreme support!


Interested in previous FAQ DOE Alert e-mail newsletters?
To view a past issue, choose it below.

#1 Mar 01, #2 Apr 01, #3 May 01, #4 Jun 01, #5 Jul 01 , #6 Aug 01, #7 Sep 01, #8 Oct 01, #9 Nov 01, #10 Dec 01, #2-1 Jan 02, #2-2 Feb 02, #2-3 Mar 02, #2-4 Apr 02, #2-5 May 02, #2-6 Jun 02, #2-7 Jul 02, #2-8 Aug 02, #2-9 Sep 02, #2-10 Oct 02, #2-11 Nov 02, #2-12 Dec 02, #3-1 Jan 03, #3-2 Feb 03, #3-3 Mar 03, #3-4 Apr 03, #3-5 May 03, #3-6 Jun 03, #3-7 Jul 03, #3-8 Aug 03, #3-9 Sep 03 #3-10 Oct 03, #3-11 Nov 03, #3-12 Dec 03, #4-1 Jan 04, #4-2 Feb 04, #4-3 Mar 04, #4-4 Apr 04, #4-5 May 04, #4-6 Jun 04, #4-7 Jul 04, #4-8 Aug 04, #4-9 Sep 04, #4-10 Oct 04, #4-11 Nov 04, #4-12 Dec 04, #5-1 Jan 05, #5-2 Feb 05, #5-3 Mar 05, #5-4 Apr 05, #5-5 May 05, #5-6 Jun 05, #5-7 Jul 05, #5-8 Aug 05, #5-9 Sep 05, #5-10 Oct 05, #5-11 Nov 05, #5-12 Dec 05, #6-01 Jan 06, #6-02 Feb 06, #6-03 Mar 06, #6-4 Apr 06, #6-5 May 06, #6-6 Jun 06, #6-7 Jul 06, #6-8 Aug 06, #6-9 Sep 06, #6-10 Oct 06, #6-11 Nov 06, #6-12 Dec 06, #7-1 Jan 07, #7-2 Feb 07, #7-3 Mar 07, #7-4 Apr 07, #7-5 May 07, #7-6 Jun 07, #7-7 Jul 07, #7-8 Aug 07, #7-9 Sep 07, #7-10 Oct 07, #7-11 Nov 07, #7-12 Dec 07, #8-1 Jan 08, #8-2 Feb 08, #8-3 Mar 08, #8-4 Apr 08, #8-5 May 08, #8-6 June 08, #8-7 July 08, #8-8 Aug 08, #8-9 Sep 08, #8-10 Oct 08, #8-11 Nov 08, #8-12 Dec 08, #9-01 Jan 09, #9-02 Feb 09, #9-03 Mar 09, #9-04 Apr 09, #9-05 May 09, #9-06 June 09, #9-07 July 09, #9-08 Aug 09, #9-09 Sep 09, #9-10 Oct 09, #9-11 Nov 09, #9-12 Dec 09, #10-1 Jan 10, #10-2 Feb 10, #10-3 Mar 10, #10-4 April 10, #10-5 May 10, #10-6 June 10 (see above)

Click here to add your name to the DOE FAQ Alert newsletter list server.

Statistics Made Easy®

DOE FAQ Alert ©2010 Stat-Ease, Inc.
All rights reserved.

* Feel free to forward this newsletter to your colleagues. They can subscribe at

Software      Training      Consulting      Publications      Order Online      Support      Contact Us       Search

Stat-Ease, Inc.
2021 E. Hennepin Avenue, Suite 480
Minneapolis, MN 55413-2726
p: 612.378.9449, f: 612.746.2069