DOE FAQ Alert Electronic Newsletter

Issue: Volume 2, Number 3
March 2002
Mark J. Anderson, Stat-Ease, Inc.

Here's another set of frequently asked questions (FAQs) about doing design of experiments (DOE), plus alerts to timely information and free software updates. If you missed previous DOE FAQ Alerts, go to the links below.
Feel free to forward this newsletter to your colleagues. They can subscribe by going to

I offer the following link as an appetizer: which tells the story of Brian Walker - a man following his dream to build a rocket ship that will propel him into outer space. For more details on Brian's visions of blue sky and beyond, see his web site at

Here's what I cover in the body text of this DOE FAQ Alert (topics that delve into statistical detail are rated "X"):

1. FAQ: DOE for non-manufacturing areas such as marketing
2. X-FAQ: Setting up and evaluating a Plackett-Burman (P-B) design as specified by standards for ruggedness testing
a. How Design-Expert handles P-B designs
b. Problems with aliasing in P-B are not resolved via replication

3. Info alert: Ruggedness DOE done on machine-made bread (link to an article published in "Today's Chemist at Work")
4. Workshop alert: "Robust Design, DOE Tools for Reducing Variation" and other upcoming classes. "Statistics for Technical Professionals workshop now computer-intensive"
5. Book alert: Free copies (seven only!) of Cornell's "How to Apply Response Surface Methodology" pamphlet

PS. Quote for the month - math versus creativity

1 - FAQ: DOE for non-manufacturing areas such as marketing

-----Original Question-----

From: Kansas

"I was recently asked to assist our sales and marketing staff with implementing DOE principles within their department. I have only been working with DOE for about a year and it's been with chemical formulations so I don't have a really good grasp of business practices in relation to sales and marketing. Would you have any information, reports or recommendations as to how I might approach this experiment? Ideally I thought about using customer satisfaction (qualitative) or market share (quantitative) as a response variable, am I on the right track? Has this type of experiment, in this environment, been performed before?"


DOE requires controllable inputs and measurable outputs. That's all! Therefore it can be applied to non-manufacturing processes given the right circumstances. For example, it's common in marketing to test variables such as price, promotion, place and product configuration. The response can be sales, consumer preference, etc. These issues are near and dear to me, because long ago, after completing my MBA, I branched out from my chemical engineering career into product management, marketing and other business-related functions. Unfortunately, as a practical matter, it's not all that easy to apply DOE in these fields.

Several years ago I researched this issue and came up with a list of case studies that you can view at There's a story behind this. In March of 1996 I suddenly got a spate of calls asking if we did "MVT" because according to "Forbes" magazine issue 3/11/96* "A Minneapolis software firm, Stat-Ease, sells most of the software these MVT types use." Not having seen this issue of Forbes, my first question was "What's MVT?" It turns out that this meant "multivariable testing," which is an apt description of what DOE does. The article went on to say "If you haven't yet applied multivariable testing to your business, get moving. Whether you run a factory, a mail-order house or a hospital, it will probably improve your performance." I agree!

I see that you're from Kansas - the great state from which Dorothy embarked on her wonderful adventure to the land of Oz. Remember that classic scene when Toto the dog pulls back the curtain and we all see the machinations of the Wizard of Oz? I hope DOE can also pull back the curtain and help you see what really affects your marketing process. Good luck!

*("The new mantra: MVT" by Rita Koselka, pages 114-118. For a synopsis and discussion on this article see archives only go back to 2005 )

PS. Stat-Ease will be leading a session on "The Use of DOE in Non-Manufacturing Environments" at the Spring Research Conference of the Quality and Productivity (Q&P) section of the American Statistical Association (ASA) on June 5-7 in Tempe, Arizona (near Phoenix). It will be a great opportunity to hear how DOE and other statistical tools can be used to enhance productivity and improve the quality of products and services. For more details, see (this link is no longer available).

2 - X-FAQ: Setting up and evaluating a Plackett-Burman (P-B) as design specified by standards for ruggedness testing
a. How Design-Expert handles P-B designs
b. Problems with aliasing in P-B not resolved via replication

-----Original Question-----

From: Connecticut (paraphrased from face-to-face consult)

a. "I don't see the eight-run option for Plackett-Burman design in Design-Expert software. Why not? How do I enter this?
b. "The design was fully replicated per protocol dictated by the standards I must meet for validation of my process. Does this improve its resolution?"


a. Design-Ease® and Design-Expert® software offer Plackett-Burman (P-B) designs only for 12, 20, 24, 28 and 32 runs, which fill in gaps in options offered by our standard two-level catalog (2^(k-p). For designs with eight runs, choose the seven-factor 2^(k-p) option rather than the P-B. Then on the next screen click the "Make generators editable" option and enter:

[D] = -AC
[E] = ABC
[F] = -AB
[G] = -BC

These generators reproduce the design in Plackett and Burman's original 1946 paper. Be aware that many standards and other references list alternative versions. If you find that the patterns don't match your version of the P-B, simply type over the design layout of factor levels and save it for future use as a template.

(My partner Pat Whitcomb provided the generators shown above. Check out the option to input design generators on your copy of Stat-Ease software. If you don't own a license, download a fully-functional trial version of Design-Expert at

b. The alias structure for the P-B noted above is:

[A] = A - BF - CD - EG + BCE + BDG + CFG + DEF
[B] = B - AF - CG - DE + ACE + ADG + CDF + EFG
[C] = C - AD - BG - EF + ABE + AFG + BDF + DEG
[D] = D - AC - BE - FG + ABG + AEF + BCF + CEG
[E] = E - AG - BD - CF + ABC + ADF + BFG + CDG
[F] = F - AB - CE - DG + ACG + ADE + BCD + BEG
[G] = G - AE - BC - DF + ABD + ACF + BEF + CDE

Notice that main effects are confounded with two-factor interactions, making this design Resolution III. Equivalent P-B's or 2^(k-p) options for seven factors in eight runs will also be Resolution III. Replicating such a design will improve the power, which obviously is the intention of ASTM, but it will not increase the resolution. If analysis of variance (ANOVA) shows significant effect(s), thus indicating ruggedness failure, you will need resolve the troublesome aliases of main effects by performing further runs that involve new combinations of factors. Typically
for P-B's this is done via a procedure called "foldover," which, in this case, would require eight more runs with levels opposite those performed in the first block.

If you're not constrained by protocol dictated by any particular standard, consider running a higher Resolution IV design for your initial ruggedness test. See the next Alert (#3) for an example and explanation on why I recommend this.

(Learn more about Plackett-Burman designs and aliasing of two-level factorial designs by attending the 3-day computer-intensive workshop "Experiment Design Made Easy" (EDME). For a description, see Link from this page to the course outline and schedule. We've got classes coming up in:

- Minneapolis, April 9-11 and June 4-6, and
- San Jose on May 7-9.)

3 - Info alert: Ruggedness DOE done on machine-made bread (link to an article published in "Today's Chemist at Work")

Go to, page down and click the link labeled "The Knead for Speed" to see how I did a ruggedness DOE to evaluate alternative raw materials for making machine-made bread. I encourage all of you kitchen chemists and engineers to try this at home!

Here's a 'sidebar' on the terminology of "Ruggedness" versus "Robustness" that I uncovered while doing the research for my bread DOE. Ruggedness is a term that's been used for several decades, primarily for application to analytical method development, for testing possible sources of variation. I discovered that more recent (within last 10 years) handbooks and articles on assay validation assign the term "ruggedness" to "external" conditions such as when and where the assay gets done, ambient temperature and humidity, alternative sources of raw materials and lot-by-lot changes. On the other hand, the term "robustness" now seems to be in favor for those variables that are "internal" to the assay (or process or product). For example, the USP (U.S. Pharmacopeia) and ICH (International Conference on Harmonization of Technical Requirements for Registration of Pharmaceuticals for Human Use) define robustness as "a measure of its [an analytical method] capacity to remain unaffected by small but deliberate variations in method parameters." Doing this, typically with the aid of DOE, "provides an indication of its reliability during normal use." Method parameters include changes in input factors, such as processing time and temperature,
composition of reagents, etc. Both types of variables, rugged versus robust, must be included in a proper validation test, so it really becomes just a matter of semantics.

4 - Workshop alert: "Robust Design, DOE Tools for Reducing Variation" and other classes coming up. "Statistics for Technical Professionals workshop now computer-intensive"

If you are proficient with the tools of response surface methods (RSM) and desire knowledge on advanced tools for Six Sigma, attend "Robust Design, DOE Tools for Reducing Variability" (RDRV). See for course content. We've got plenty of room in our class next week in Minneapolis. Call now to sign up for this session or the next one, also in Minneapolis, on June 11-13. Do not enroll in these classes unless you know RSM. If you need this prerequisite, attend our "Response Surface Methods for Process Optimization" workshop in Minneapolis on April 16-18 (see for details).

Do you seek more elementary tools for quality improvement? If so, come to our "Statistics for Technical Professionals" (STP) workshop on March 3/26-27 in Minneapolis. We've tossed out the calculators and now solve all of the problems with the aid of Microsoft Excel, which will be made available in class for use by students on classroom computers. This is a great opportunity to gain a working knowledge of confidence versus tolerance intervals and much, much more (for details, see ).

See for schedule and site information on all Stat-Ease workshops open to the public. To enroll, click the "register online" link at our web site or call Stat-Ease at 1.612.378.9449. If spots remain available, bring along several colleagues and take advantage of quantity discounts in tuition, or consider bringing in an expert from Stat-Ease to teach a private class at your site. Call us to get a quote.

5 - Book alert: Free copies (seven only!) of "How to Apply Response Surface Methodology" pamphlet

A few months ago we cleaned house at Stat-Ease and found a number of old, but still relevant, statistical texts that we no longer use in our workshops. This month I am giving away seven paperback copies of John Cornell's "How to Apply Response Surface Methodology." The pamphlet is Volume 8 in a series published by the Statistics Division of what was then called the American Society of Quality Control (they lost "Control" a few years ago!). I will send these RSM pamphlets (four first editions from 1984 plus three revised in 1990) to the first seven people who e-mail me. Based on my experience giving away other books the last few months, this will happen within 1 hour of my DOE FAQ Alert
broadcast, so don't bother asking after that. I want to spread the wealth of information, so those of you who already received a book won't be eligible this time. Please note the shipping address in your e-mail request. If you want the Cornell pamphlet, but don't reply soon enough, go to the ASQ publications web site at where you can buy the revised edition for $32.00 (less if you're a member).

Last month I mentioned that I had just received new editions of two books which I highly recommend, but are not free: - "Response Surface Methodology," 2nd Edition, by Myers and Montgomery, John Wiley & Sons, 2002 - "Experiments with Mixtures," 3rd Edition, by Cornell, John Wiley & Sons, 2002. Purchase these books at

In addition to the new editions noted above, we recently added two new books to our e-commerce site:

- "Engineering Statistics," 2nd Edition, by Montgomery, Runger and Hubele, John Wiley & Sons, 2001
- "Statistical Intervals: A Guide for Practitioners," by Hahn and Meeker, John Wiley & Sons, 1991

We recommend these texts for our new "Statistics for Technical Professionals" workshop (

I hope you learned something from this issue. Address your questions and comments to me at:

Mark J. Anderson, PE, CQE
Principal, Stat-Ease, Inc. (
Minneapolis, Minnesota USA

PS. Quote for the month - math versus creativity.

"A lot of engineers are so into the math that they can't be creative. After four or five years of equations you start to wonder if you can think creatively any more."

- Robert Ripley, mechanical engineer and friend of Rocket Guy (see the "appetizer" at the beginning of this Alert)

Trademarks: Design-Ease, Design-Expert and Stat-Ease are registered trademarks of Stat-Ease, Inc.

Acknowledgements to contributors:

- Students of Stat-Ease training and users of Stat-Ease software
- Fellow Stat-Ease consultants Pat Whitcomb and Shari Kraber (see for resumes)
- Statistical advisor to Stat-Ease: Dr. Gary Oehlert (
- Stat-Ease programmers, especially Tryg Helseth (
- Heidi Hansel, Stat-Ease marketing director, and all the remaining staff.

Interested in previous FAQ DOE Alert e-mail newsletters? To view a past issue, choose it below.

#1 - Mar 01, #2 - Apr 01, #3 - May 01, #4 - Jun 01, #5 - Jul 01 , #6 - Aug 01, #7 - Sep 01, #8 - Oct 01, #9 - Nov 01, #10 - Dec 01, #2-1 Jan 02, #2-2 Feb 02, #2-3 Mar 02 (see above)

Click here to add your name to the FAQ DOE Alert newsletter list server.

Statistics Made Easy

©2002 Stat-Ease, Inc. All rights reserved.