Issue: Volume 9, Number 6
Date: June 2009
From: Mark J. Anderson, Stat-Ease, Inc., Statistics Made Easy® Blog

Dear Experimenter,

Here's another set of frequently asked questions (FAQs) about doing design of experiments (DOE), plus alerts to timely information and free software updates. If you missed the previous DOE FAQ Alert, see below.

==> Tip: Get immediate answers to questions about DOE via the Search feature on the main menu of the Stat-Ease® web site. This not only pores over previous alerts, but also the wealth of technical publications posted throughout the site.

Feel free to forward this newsletter to your colleagues. They can subscribe by going to If this newsletter prompts you to ask your own questions about DOE, please address them via mail

Normally this is the place in my monthly ezine where I provide a heads up to the StatsMadeEasy blogs I've written over the past several weeks. However, Google disabled our blog... or I should say, their robots did. They left a ray of hope via this admission: "Since you're an actual person reading this, your blog is probably not a spam blog. Automated spam detection is inherently fuzzy, and we sincerely apologize for this false positive." Also I am supposed to feel better by this condolence: "On behalf of the robots, we apologize for locking your non-spam blog." Evidently Google's robots have caught up with humankind's aversion to statistics. ;) If Google cannot tolerate my sidebars on statistical deviations, we will seek another blogging host. Stay tuned!

Also, Stat-Ease offers an interactive web site — its Support Forum for Experiment Design at Whereas this monthly ezine — the DOE FAQ Alert — shares one-on-one communications with Stat-Ease StatHelp, anyone (after gaining approval for registration) can post questions and answers to the Forum, which is open for all to see (with moderation). Check it out and weigh in!

Here's a synopsis the most recent question posted (found in the DOE "Analysis" section):
>From: A Registered Member
"I'm a simple grad student chemist with VERY little statistics knowledge but have been tasked by my advisor to analyze some data for a project. Using the EXCELLENT (in my humble opinion) tutorials, I got the data inputted and the graphs made. The only problem is I don't know enough about the program to know what correlation test it uses? Is it a T-test?"

See the answer (from Stat-Ease consultant Wayne Adams) and feel free to contribute to this discussion thread by getting registered as a user of the Support Forum for Experiment Design (if you're not already).

Topics in the body text of this DOE FAQ Alert are headlined below (the "Expert" ones, if any, delve into statistical details).

1. FAQ: Coding of categorical factors for predictive model
2. FAQ: How to approach confirmation experiments
3. Expert-FAQ: Ranking the effectiveness of components in a mixture design for screening purposes only
4. Info Alert : DOE for Non-Manufacturing Processes
5. Events Alert: When and where to see Stat-Ease
6. Workshop Alert: Do not miss DOE for Life Sciences in Boston

P.S. Quote for the month: A cynical view of how precisely engineers report experimental results.


1. FAQ: Coding of categorical factors for predictive model

-----Original Question-----
Senior Project Engineer
"You helped me on an experiment when I was at a previous employer. Now I have what I hope is a simple question. I'm looking at the "Final Equation in Terms of Coded Factors" part of my analysis and it has A[1] and A[2] as coefficients. I'm not sure what that means. My "A" factor was categorical at 3 levels, so that might be part of it. Can you help me?

PS. Just in case you wondered — I've downloaded the trial version of Design-Expert® software because I'm hoping to convince my management to buy it for me. I really hate the general-purpose statistical software they use here - in my humble opinion, it's a nightmare."

The nominal coding provided by Design-Expert or Design-Ease® software for factors with three or more categoric treatments compares the average at each level (categoric treatment) of the factor to the overall mean.* What you're seeing are the two coefficients needed to code three categorical levels of a given factor (treatment). This makes sense if you account for the fact that the average is already calculated. It’s like the classic math problem of knowing a basketball team’s average height and 4 of the 5 players’ heights — you then can infer the 5th player’s height with some algebra. Statisticians refer to this as degrees of freedom — one being lost when the average is computed at the outset.

*Program Help provides this further detail:
>For multi-level discrete categorical factors the first coefficient "A[1]" is the difference of level 1 from the overall average, the second coefficient "A[2]" is the difference of level 2 from the overall average and so on. The last coefficient is not independent and is equal to the overall average minus the sum of the other coefficients.<

(Learn more about factorial coding by attending the three-day computer-intensive workshop "Experiment Design Made Easy." See for a description of this class and link from this page to the course outline and schedule. Then, if you like, enroll online.)


2. FAQ: How to approach confirmation experiments

-----Original Question-----
DOE Trainer
"It is very common to see advice (included, for example, in textbooks by Douglas Montgomery, et al) that one should conduct 'confirmation' experiments at the end of designed experiments. However, there appears to be little specific advice on how to design, analyze, and interpret the outcome of such experiments. It seems that experimenters are advised to carry out a 'few' additional experiments at the desired experimental levels and then to make a 'judgment' on whether the outcome of these additional experiments confirms that the most appropriate levels of the factors have been correctly identified in the original experiment. Do you good folk at Stat-Ease have any more specific advice than this on how to approach confirmation experiments?"

Answer (from Stat-Ease Consultant Wayne Adams):
"To start, we need to quantify your requirements and establish how much confidence is needed before a decision can be made. However, here are some general answers to frequently asked questions on this vital issue.

A. How many additional runs should one undertake at the confirmation levels?

As many as you can afford. That answer is more serious than it sounds. The more confirmation work you do the more trust you will have that you have the right solution to the problem. Repeatable, safe, consistent.

B. Do these additional runs need to be true replicates (i.e. reset up for each run) or would it be OK to just run off some repetitions, and, say, calculate the mean?

It depends. Replicates will give the most sound statistical test, but producing a batch can provide more information about how the process will actually be used.

C. Should a statistical test be conducted to compare the outcome of the confirmation runs with the predicted outcome from the original experiment (perhaps a one-sample t-test with the predicted outcome from the original experiment used as the hypothesized mean?), or is it just a matter of making an informed judgment?

Informed judgments are always better than statistical tests. A statistical test backed by informed judgment is even better.

D. What should one do if the outcome of the confirmation runs appears not to confirm the conclusion from the original experiment — does the original experiment or some part of it need to be rerun?

I appreciate that the confirmation runs would need to be conducted, as far as possible, under the same conditions as the original experiment, but given that this was achieved, how should one react to a large difference? Assuming the experiment, analysis, and model fit were done correctly, a widely divergent confirmation test usually means there was a missed factor (lurking variable). Something has changed between the experiment and the confirmation. If these assumptions are not valid, then your guess is as good as mine as to what to do next. It's back to the drawing board if the subject matter knowledge cannot provide some insight.

E. Is there any specific advice in the Design-Expert manuals or Help on performing confirmation experiments that I could use as a reference for my students (I couldn't identify any)?

There is no one right way to do confirmation experiments. We would be doing a disservice if we tried to impose a single method for doing confirmation runs. Our advice is simply to always confirm the findings before making conclusions. The confirmation runs you choose to do are up to you.

Here's a recap of some confirmation choices that an experimenter might pursue:
— Trust the model — no confirmation runs are necessary.
— Run the optimal setup once.
— Run the optimal setup a few times (replicates or batch depending on the process).
— Run the optimal setup a few times, plus a few other test conditions to test the validity of the model everywhere.
— Run an experiment around the optimal setting to verify the process is stable (keep the ranges small)."

This is sound advice from Wayne, I think. However, if any of you can provide further specific advice on what constitutes a "confirmation" of a predictive model, please weigh in!
— Mark


3. Expert-FAQ: Ranking the effectiveness of components in a mixture design for screening purposes only

-----Original Question-----
Industrial statistician
"I'm setting up an 8-component screening mixture experiment. In your workshop on mixture design for optimal formulation, you say that when doing screening the main objective is to simply rank the effectiveness of components — statistical tests of significance are a secondary consideration. By "effectiveness" do you mean those components with the steepest slope in the trace plots?"

The trace plot* allows formulators to quickly identify the vital components — those creating a large change by their rise (slope) and/or run (constraint range), that is, the net effect. Naturally, the highly-potent ingredients, such as a catalyst, are restricted the most, so that makes the task of screening far more tricky. After playing with the trace plot (look in both the Cox and Piepel directions), go back to the ANOVA output and check out the gradients and component effects for purposes of screening quantitatively. To avoid the syndrome of paralysis by analysis, we recommend that, when formulators embark on screening designs, they remain bound-and-determined to reduce the field of candidates. Of course, this strategy is effective only by committing to a second, more in-depth, experiment to follow up on the candidate
components. For more details on mixture screening see Cornell’s "Experiments with Mixtures, 2nd Ed," section 5.7 (p.244).

*See, for example, Figure 6 at this posted chapter on mixture design by the Turner-Fairbank Highway Research Center (US Department of Transportation - Federal Highway Administration):

(Learn more about mixture design by attending the three-day computer-intensive workshop "Mixture Design for Optimal Formulations." For a complete description of this class, see Link from this page tothe course outline and schedule. Then, if you like, enroll online.)


4. Info Alert: DOE for Non-Manufacturing Processes

See an updated compilation of case studies on "Achieving Breakthroughs in Non-Manufacturing Processes via Design of Experiments" at This offers inspiration for Six Sigma practitioners, statisticians and any others who see the opportunity for applying DOE to:

— Advertising, sales and marketing
— Billing
— Business (in general)
— Human performance
— Information technology (IT) and services (IS)
— Medical
— Productivity and Quality
— Service
— Transactional processes.

I'd like very much to hear about any other documented case studies that do not fall in the traditional arena of DOE for product development and manufacturing improvement. However, I am particularly interested in transactional processes, such as hospital administration and the like. Let me know if you can share any of your own experiences along these lines, or you have a lead on a published study that I can pursue.

— Mark


5. Events Alert: When and where to see Stat-Ease

Click for a list of upcoming appearances by Stat-Ease professionals. We hope to see you sometime in the near future!

PS. Do you need a speaker on DOE for a learning session within your company or technical society at regional, national, or even international levels? If so, contact me. It may not cost you anything if Stat-Ease has a consultant close by, or if a web conference will be suitable. However, for presentations involving travel, we appreciate reimbursements for airfare, hotel and meals — expenses only. In any case, it never hurts to ask Stat-Ease for a speaker on this topic.


6. Workshop alert: Do not miss DOE for Life Sciences in Boston

Seats are filling fast for the following DOE classes. If possible, enroll at least 4 weeks prior to the date so your place can be assured. However, do not hesitate to ask whether seats remain on classes that are fast approaching!

—> Designed Experiments for Life Sciences (DELS)
> June 8-9, 2009 (Cambridge, MA)

—> Experiment Design Made Easy (EDME)
> June 9-11 (Minneapolis, MN)
> August 18-20 (Minneapolis)

—> Response Surface Methods for Process Optimization (RSM)
> July 7-9 (Minneapolis)

—> Mixture Design for Optimal Formulations (MIX)
> August 11-13 (Minneapolis)

—> DOE for DFSS: Variation by Design (DDFSS)
> November 10-11 (Minneapolis)

See for complete schedule and site information on all Stat-Ease workshops open to the public. To enroll, click the "register online" link on our web site or call Elicia at 612.746.2038. If spots remain available, bring along several colleagues and take advantage of quantity
discounts in tuition. Or, consider bringing in an expert from Stat-Ease to teach a private class at your site.*

*Once you achieve a critical mass of about 6 students, it becomes very economical to sponsor a private workshop, which is most convenient and effective for your staff. For a quote, e-mail


I hope you learned something from this issue. Address your general questions and comments to me at:




Mark J. Anderson, PE, CQE
Principal, Stat-Ease, Inc. (
2021 East Hennepin Avenue, Suite 480
Minneapolis, Minnesota 55413 USA

PS. Quote for the month —A cynical view of how precisely engineers report experimental results:

"The weaker the data available upon which to base one's conclusion, the greater the precision which should be quoted in order to give the data authenticity."

—Norman R. Augustine, aeronautical engineer, former CEO of Lockheed Martin.

Trademarks: Stat-Ease, Design-Ease, Design-Expert and Statistics Made Easy are registered trademarks of Stat-Ease, Inc.

Acknowledgements to contributors:
—Students of Stat-Ease training and users of Stat-Ease software
—Stat-Ease consultants Pat Whitcomb, Shari Kraber and Wayne Adams (see for resumes)
—Statistical advisor to Stat-Ease: Dr. Gary Oehlert (
—Stat-Ease programmers, led by Neal Vaughn and Tryg Helseth (
—Heidi Hansel Wolfe, Stat-Ease sales and marketing director, and all the remaining staff that provide such supreme support!


Interested in previous FAQ DOE Alert e-mail newsletters?
To view a past issue, choose it below.

#1 Mar 01, #2 Apr 01, #3 May 01, #4 Jun 01, #5 Jul 01 , #6 Aug 01, #7 Sep 01, #8 Oct 01, #9 Nov 01, #10 Dec 01, #2-1 Jan 02, #2-2 Feb 02, #2-3 Mar 02, #2-4 Apr 02, #2-5 May 02, #2-6 Jun 02, #2-7 Jul 02, #2-8 Aug 02, #2-9 Sep 02, #2-10 Oct 02, #2-11 Nov 02, #2-12 Dec 02, #3-1 Jan 03, #3-2 Feb 03, #3-3 Mar 03, #3-4 Apr 03, #3-5 May 03, #3-6 Jun 03, #3-7 Jul 03, #3-8 Aug 03, #3-9 Sep 03 #3-10 Oct 03, #3-11 Nov 03, #3-12 Dec 03, #4-1 Jan 04, #4-2 Feb 04, #4-3 Mar 04, #4-4 Apr 04, #4-5 May 04, #4-6 Jun 04, #4-7 Jul 04, #4-8 Aug 04, #4-9 Sep 04, #4-10 Oct 04, #4-11 Nov 04, #4-12 Dec 04, #5-1 Jan 05, #5-2 Feb 05, #5-3 Mar 05, #5-4 Apr 05, #5-5 May 05, #5-6 Jun 05, #5-7 Jul 05, #5-8 Aug 05, #5-9 Sep 05, #5-10 Oct 05, #5-11 Nov 05, #5-12 Dec 05, #6-01 Jan 06, #6-02 Feb 06, #6-03 Mar 06, #6-4 Apr 06, #6-5 May 06, #6-6 Jun 06, #6-7 Jul 06, #6-8 Aug 06, #6-9 Sep 06, #6-10 Oct 06, #6-11 Nov 06, #6-12 Dec 06, #7-1 Jan 07, #7-2 Feb 07, #7-3 Mar 07, #7-4 Apr 07, #7-5 May 07, #7-6 Jun 07, #7-7 Jul 07, #7-8 Aug 07, #7-9 Sep 07, #7-10 Oct 07, #7-11 Nov 07, #7-12 Dec 07, #8-1 Jan 08, #8-2 Feb 08, #8-3 Mar 08, #8-4 Apr 08, #8-5 May 08, #8-6 June 08, #8-7 July 08, #8-8 Aug 08, #8-9 Sep 08, #8-10 Oct 08, #8-11 Nov 08, #8-12 Dec 08, #9-01 Jan 09, #9-02 Feb 09, #9-03 Mar 09, #9-04 Apr 09, #9-05 May 09, #9-06 June 09 (see above)

Click here to add your name to the DOE FAQ Alert newsletter list server.

Statistics Made Easy®

DOE FAQ Alert ©2009 Stat-Ease, Inc.
All rights reserved.

* Feel free to forward this newsletter to your colleagues. They can subscribe at

Software      Training      Consulting      Publications      Order Online      Support      Contact Us       Search

Stat-Ease, Inc.
2021 E. Hennepin Avenue, Suite 480
Minneapolis, MN 55413-2726
p: 612.378.9449, f: 612.746.2069