Issue: Volume 4, Number 9
Date: September 2004
From: Mark J. Anderson, Stat-Ease, Inc. (

Dear Experimenter,

Here's another set of frequently asked questions (FAQs) about doing design of experiments (DOE), plus alerts to timely information and free software updates. If you missed previous DOE FAQ Alerts, please click on the links at the bottom of this page. If you have a question that needs answering, click the Search tab and enter the key words. This finds not only answers from previous Alerts, but also other documents posted to the Stat-Ease web site.

Feel free to forward this newsletter to your colleagues. They can subscribe by going to If this newsletter prompts you to ask your own questions about DOE, please address them via mail to:

Here's an appetizer to get this Alert off to a good start: See to learn how a compact disk works at the Molecular Expressions web site. Explore their acclaimed photo galleries featuring the fascinating world of optical microscopy.

Here's what I cover in the body text of this DOE FAQ Alert (topics that delve into statistical detail are designated "Expert"):

1. FAQ: Seven steps for doing DOE successfully
2. Expert-FAQ: Pass/fail response to marketing Internet services
3. Free books (US/Canada only): Enter the drawing for Box and Drapers' "Empirical Model-Building and Response Surfaces" text
4. Info alert: Wanted—case study authors
5. Events alert: Link to a schedule of appearances by Stat-Ease
6. Workshop alert: See when and where to learn about DOE

PS. Quote for the month: Inflammatory comments about statistics by Ernest Rutherford (1908 Nobel prize-winner for chemistry)

PPS. Employment wanted: Chemist with a strong interest in DOE looking for work in central California.


1. FAQ: Seven steps for doing DOE successfully

From: Texas
"I am just getting started with design of experiments. Do you have a 'script' for doing DOE successfully?"

From: Stat-Ease consultant Shari Kraber

"By investing in design of experiments you develop profound process knowledge that enlightens paths to improved quality and decreased waste. The key to successful DOE is planning. Short-cutting this preparatory stage leads to more failed experiments than errors in execution. Here are seven steps to successful DOE:

1. Define the objective of the experiment: What specifically are you trying to achieve?

2. Define the responses (quality characteristics) that will be measured and their criteria. For example, if you are primarily concerned with visual defects, a reliable system of measuring them must be defined before the DOE is run. Prior to doing the experiment, try collecting information on any new measurements over some length of time—say a week. This will provide information on variability which helps determine how many samples to take and the number of experimental runs needed to find significant factor effects.

3. Decide which factors will be studied in the DOE. Get fellow technical professionals together for a free-wheeling brainstorming session, which will likely lead to a very large number of potential variables. However, the best DOEs focus only on 5 to 8 factors at a time, so you must prioritize the variables. For example, raw materials are likely to be identified as a source of problems in product quality. However, it may be best initially to hold raw materials constant (use one batch) to gain better understanding about
the process variables.

4. Select a design that will give the desired amount of information in a reasonable number of runs. In your initial design, try to identify all main effects and as many interactions as you can within the restrictions on time, material, cost and other resources.

5. Once the design is chosen, look for combinations of conditions that cannot be met. Also, think carefully about the logistics, such as the people needed to do the experiment and availability of testing equipment. An essential element of good DOE is randomization of the run order to reduce potential bias from lurking variables, such as machine warm-up or changes in ambient conditions. However, you must review the randomized plan and consider the practicality of arbitrarily changing factor levels. Are there restrictions that must be taken into account? These things must be identified up-front so that the design can be modified, ideally with the help of a statistician who's competent on the tool of DOE.

6. At this point, the DOE can be run. During its execution expect a mishap or two. This is when it pays to have an expert statistician available on short notice, because then you can obtain timely advice on fixing flawed designs on the fly, so that good information will still be obtained.

7. After the data is collected, the analysis begins. More questions will arise, but hopefully a path to improvement will begin to emerge.

It is likely to take two to three of these seven-step DOE cycles before you reach optimum processing conditions.

Stat-Ease offers expert consulting on DOE.* If you hire one of us (or whomever), here's how I recommend you proceed:

A. Have us come on-site for one day to plan the details of the experiment. Some up-front work can be done by your engineers before we get there, like brain-storming on the variables and thinking about how the responses can be measured.

B. Give the engineers time to plan the execution of the DOE so that it can be carried out well.

C. We can return for the actual run to oversee the experiment. If data are immediately available, we can analyze it on-the-spot and provide a preliminary report.

*(See for details on consulting services offered by Stat-Ease, Inc.)


2. Expert-FAQ: Pass/fail response to marketing Internet services

From: San Francisco

"I have just joined a new company that isn't very statistically oriented. It offers online Internet services for which I believe DOE can be put to great use. I am keen to demonstrate the efficacy of DOE to the company so that it may be used consistently in the future. Therefore, I set up a seven-factor design with multiple categories per level using the free, trial-version of Design-Expert® software posted at your web site.*The experiment will track factors relating to profitability—including price, trial period, type of incentive, etc. For the response, I take the percentage of users who cancel their subscription before the first payment period."

"My question is how can I tell how many replicates I will need to run in order to be able to distinguish between the main effects. Also, how should I go about entering those replicates? I am likely to have 70-200 distinct (paid/did not pay) data points per test cell. The only way I can currently see to do this is to replicate the entire row multiple times, but this is not practical for the number of replicates I am considering."

First off, our software comes up a bit short of ideal for analyzing binary data (in your case: cancel versus continue subscription) when the sample size varies. The proper tool for this is binary logistic regression. However, for the sake of discussion, assume that the sample size per combination will be roughly equal. In this case, you need not replicate each row >70 times. Rather, for each unique combination of offers, enter the response as proportion reject ("p"—the fraction on a 0-1 scale), rather than percent (0-100 scale), as the primary response for statistical analysis. Analyze this with the arc-sin square root transformation.

How large a sample will be needed depends on:

- "p"—the expected proportion reject
- your desired level of risk (typically 5%) for saying an effect exists when's it's really due to chance, and
- power (recommend this be at least 80% to see a real difference)

*(The free trial of Design-Expert can be downloaded from


3. Free books (US/Canada only): Enter the drawing for Box and Drapers'
"Empirical Model-Building and Response Surfaces"

Sorry, due to the high cost of shipping, this offer applies only to residents of the United States and Canada: Send me an e-mail if you'd like a free copy of Box and Drapers' classic statistical text "Empirical Model-Building and Response Surfaces." Originally published in 1987 (John Wiley and Sons, New York), this book is a classic in the field of response surface methods (RSM) for process optimization. However, for workshops on this topic,* Stat-Ease now uses Myers and Montgomerys' "Response Surface Methods" (see this book and others listed for purchase via ecommerce at

*(Learn more about RSM by attending the three-day computer-intensive workshop "Response Surface Methods for Process Optimization." See for a complete description. Link from this page to the course outline and schedule. Then, if you like, enroll online.)


4. Info alert: Wanted—case study authors

Gain technical recognition in your company and scientific field.

Share a DOE with our writer at:

Rename proprietary factors if needed to maintain confidentiality.
A thirty-minute interview, some e-mails, a review, and it's done!


5. Events alert: Link to a schedule of appearances by Stat-Ease

Click for a list of appearances by Stat-Ease professionals. We hope to see you sometime in the near future!


6. Workshop alert: See when and where to learn about DOE

See for schedule and site information on all Stat-Ease workshops open to the public. To enroll, click the "register online" link on our web site or call Stat-Ease at 1.612.378.9449. If spots remain available, bring along several colleagues and take advantage of quantity discounts in tuition, or consider bringing in an expert from Stat-Ease to teach a private class at your site. Call us to get a quote.


I hope you learned something from this issue. Address your general questions and comments to me



Mark J. Anderson, PE, CQE
Principal, Stat-Ease, Inc. (
2021 East Hennepin Avenue, Suite 480
Minneapolis, Minnesota 55413 USA

PS. Quote for the month: Inflammatory comments about statistics:

"If your experiment needs statistics, then you ought to have done a better experiment."
—Ernest Rutherford (1908 Nobel prize-winner for chemistry)

PPS. Employment wanted: A chemist with a strong interest in DOE is looking for work in central California.

"I am a polymer chemist and DOE enthusiast looking to apply skills Stat-Ease taught me. I have recently relocated to the Sacramento, CA area and am interested in a job nearby that would benefit from an experienced DOE person. I really enjoy designing and analyzing experiments, and want to focus my career search in that direction."

Charles Smith
6420 Gray Rock Road,
Somerset, CA 95684

Trademarks: Design-Ease, Design-Expert and Stat-Ease are registered trademarks of Stat-Ease, Inc.

Acknowledgements to contributors:
—Students of Stat-Ease training and users of Stat-Ease software
—Fellow Stat-Ease consultants Pat Whitcomb and Shari Kraber (see for resumes)
—Statistical advisor to Stat-Ease: Dr. Gary Oehlert (
—Stat-Ease programmers, especially Tryg Helseth (
—Heidi Hansel, Stat-Ease marketing director, and all the remaining staff

DOE FAQ Alert—Copyright 2004
Stat-Ease, Inc.
All rights reserved.


Interested in previous FAQ DOE Alert e-mail newsletters?
To view a past issue, choose it below.

#1 Mar 01
, #2 Apr 01, #3 May 01, #4 Jun 01, #5 Jul 01 , #6 Aug 01, #7 Sep 01, #8 Oct 01, #9 Nov 01, #10 Dec 01, #2-1 Jan 02, #2-2 Feb 02, #2-3 Mar 02, #2-4 Apr 02, #2-5 May 02, #2-6 Jun 02, #2-7 Jul 02, #2-8 Aug 02, #2-9 Sep 02, #2-10 Oct 02, #2-11 Nov 02, #2-12 Dec 02, #3-1 Jan 03, #3-2 Feb 03, #3-3 Mar 03, #3-4 Apr 03, #3-5 May 03, #3-6 Jun 03
, #3-7 Jul 03, #3-8 Aug 03, #3-9 Sep 03 #3-10 Oct 03, #3-11 Nov 03, #3-12 Dec 03, #4-1 Jan 04, #4-2 Feb 04, #4-3 Mar 04, #4-4 Apr 04, #4-5 May 04, #4-6 Jun 04, #4-7 Jul 04, #4-8 Aug 04, #4-9 Sep 04 (see above)

Click here to add your name to the FAQ DOE Alert newsletter list server.

Statistics Made Easy™

DOE FAQ Alert ©2004 Stat-Ease, Inc.
All rights reserved.


Software      Training      Consulting      Publications      Order Online      Contact Us       Search

Stat-Ease, Inc.
2021 E. Hennepin Avenue, Ste 480
Minneapolis, MN 55413-2726
p: 612.378.9449, f: 612.378.2152