Issue: Volume 4, Number 7
Date: July 2004
From: Mark J. Anderson, Stat-Ease, Inc. (

Dear Experimenter,

Here's another set of frequently asked questions (FAQs) about doing design of experiments (DOE), plus alerts to timely information and free software updates. If you missed previous DOE FAQ Alerts, please click on the links at the bottom of this page. If you have a question that needs answering, click the Search tab and enter the key words. This finds not only answers from previous Alerts, but also other documents posted to the Stat-Ease web site.

Feel free to forward this newsletter to your colleagues. They can subscribe by going to If this newsletter prompts you ask to your own questions about DOE, please address them to

This month's 'appetizer' relates to tools I taught at a session of "Statistics for Technical Professionals"* presented last week on-site for a pharmaceutical client of Stat-Ease. Among other things, we discussed how to construct confidence and tolerance intervals for binomial responses, such as the proportion of products that fail to pass manufacturing specifications. In 1873, Francis Galton devised an ingenious machine he called the "Quicunx" that demonstrates how the binomial distribution becomes approximately normal. For a virtual reconstruction of Galton's Quincunx, see an applet by John Carroll University posted at Warning: you may be mesmerized by the balls falling through the pins, which in Galton's original machine were laid out in patterns of five, hence the name "Quicunx."

*(See for a course description of "Statistics for Technical Professionals," a two-day computer-intensive workshop.)

Here's what I cover in the body text of this DOE FAQ Alert (topics that delve into statistical detail are designated "Expert"):

1. FAQ: The use of factorial design for market research
2. Expert-FAQ: Why power statistics look so poor for mixtures
3. Info alert: Stat-Ease is featured on the cover of "Advance"—a publication for medical laboratory professionals
4. Book alert: Recommended summertime reading on statistics
5. Reader reply: Coke® versus Pepsi® test in the Stat-Teaser
6. Events alert: A talk in Toronto at the Joint Statistical Meetings; also questions about a previous presentation on screening
7. Workshop alert: "Experiment Design Made Easy" is coming to Philadelphia

PS. Quote for the month
—Francis Galton on why the normal distribution is normal


1. FAQ: The use of factorial design for market research

From: Ohio

"We own a copy of Design-Expert® software, which we want to use for market research. Our goal is to evaluate customer needs for a new product. A classical tool for this purpose is "conjoint" or "trade-off" analysis. I'd appreciate your advice on how to perform such analysis with your software."


This can be done via Design-Expert, but you won't see it labeled as "conjoint analysis." It's been a while since I've heard this term, but I recall studying this type of analysis for my MBA (University of Minnesota, 1980). I found the example below via Internet.* It can be easily set up in Design-Expert (or Design-Ease(R) software) via the two-level factorial design feature.

"Conjoint analysis presents choice alternatives between products/services defined by sets of attributes. This is illustrated by the following choice: Would you prefer a flight with regular seats, that costs $400 and takes 5 hours, or a flight which costs $700, has extra-wide seats and takes 3 hours? Extending this, we see that if seat comfort, price and duration are the only relevant attributes, there are potentially eight flight choices:

Choice: Seat-Comfort, Price, Duration
1 extra-wide, $700, 5 hours
2 extra-wide, $700, 3 hours
3 extra-wide, $400, 5 hours
4 extra-wide, $400, 3 hours
5 regular, $700, 5 hours
6 regular, $700, 3 hours
7 regular, $400, 5 hours
8 regular, $400, 3 hours"

Unfortunately no responses are given for this 'thought' experiment, but my choice would be to get a free, frequent-flyer upgrade to the extra-wide seat in first class!



2. Expert-FAQ: Why power statistics look so poor for mixtures

From: Missouri

"I used your Design-Expert software to set up a D-optimal mixture design, but when I performed an evaluation I found that it had no power. Could you tell me what is wrong with the design?"

From: Stat-Ease consultant Pat Whitcomb

"Power in a two-level factorial makes perfect sense, i.e., we design to estimate individual effects and test their significance. Since a factorial coefficient is exactly equal to one half the factor effect, power directly measures our ability to do this. However when using response surface methods (process or mixture) we design to estimate a given order polynomial. The emphasis is now on the ability of the design to approximate certain types of behavior (linear, quadratic, etc.) and we are not generally interested in the individual model coefficients. Therefore power is no longer a direct measure of what we are designing for.

For mixtures power calculations are further clouded by collinearity among the coefficients. Recall that in a factorial design the coefficients are orthogonal, i.e., the size of each coefficient is the same regardless of what other coefficients are present in the model. In a mixture this is far from the case, depending on the constraints there is usually extreme dependence. Therefore the power to resolve individual effects is very low, while the ability of the design to adequately model the type of behavior (linear, quadratic, etc.) designed for is high. Unfortunately no one knows how to measure this ability directly.

Thus, power for mixture designs is at best a relative measure for comparing designs over the same mixture space. Even for these direct comparisons the standard error of the coefficients, leverages, determinate and trace are more useful than power."

Pat is working on a technical article that will detail the inherent deficiencies in power for mixture designs. I took the liberty of excerpting key portions of Pat's pending paper and posting it at:

(Learn more about power by attending the three-day computer-intensive workshop "Experiment Design Made Easy." See for a course description. Link from this page to the course outline and schedule. Then, if you like, enroll online.)


3. Info alert: Stat-Ease featured in cover of "Advance"—a publication for medical laboratory professionals

See the cover of the June 28 issue of "Advance" magazine at: It features an article titled "How Experimental Design Optimizes Assay Automation," co-authored by Stat-Ease consultant Shari Kraber, which describes how "an innovative blend of hardware, software and the right training in statistical know-how supercharges research automation" provided by Beckman Coulter. An unedited version of this article is posted at


4. Book alert: Recommended summertime reading on statistics

Here's some good summertime reading recommended by Bill Risk—a student of mine at the recent "Experiment Design Made Easy" workshop in San Jose (what a great name for someone interested in statistics for experimenters!): "The Lady Tasting Tea, How Statistics Revolutionized Science in the Twentieth Century." This soft-cover book by David Salsburg is available for purchase at I already had the title story, which was originally told by Sir Ronald Fisher.* As he told it a Lady claims to have the ability to tell which went into her cup first—the tea or the milk. Fisher devised a test whereupon the Lady is presented eight cups in random order, four of which are made one way (tea first) and four the other (milk first). He calculates the odds of correct identification as 1 right way out of 70 possible selections, which falls below the standard 5% probability value generally accepted for statistical significance. Salsburg reveals on good authority (H. Fairfield Smith--a colleague of Fisher) that the Lady identified all eight cups correctly!

Here's another interesting anecdote from Salsburg:
"When I first began to work in the drug to...uncertainty [as] 'error.' One of the senior executives refused to send such a report to the U.S. Food and Drug Administration [FDA]. 'How can we admit to having error in our data?' he asked [and]...insisted I find some other way to describe it...I contacted H.F. Smith [who] suggested that I call the line 'residual'...I mentioned this to other statisticians...and they began to use it...It seems that no one [in the FDA, at least]...will admit to having error."

*("Mathematics of a Lady Tasting Tea" in Newman’s "The World of Mathematics," Vol. 3, Dover Press, available from in paperback.)

PS. A book I recommended last summer, "A Short History of Nearly Everything" by Bill Bryson, just won the Aventis Prize for best science book. The 2004 winner for children's science is a book called "Really Rotten Experiments." That sounds fun! The press releases on these two prize winners and others that won honorable mention are posted at


5. Reader reply: Coke versus Pepsi test in the Stat-Teaser

Original Message
From: Steve Aprahamian

"I just read the May 2004 Stat-Teaser with your daughter's experiment on Coke and Pepsi [posted at] and I thought I would share an experience I had with the real test done in a mall.

To preface my story, I want to say that, personally, I prefer my soft drinks cold and over ice. I drink them fairly rapidly to keep dilution to a minimum. I also prefer Coke to Pepsi, though I will drink whatever is on sale, my preference for cash overriding the minor preference for taste difference (it is not fine wine we are discussing).

I can tell the difference between Coke and Pepsi, and can also tell "other brands" though I am not always "accurate" in estimation of what the other brand is (RC Cola, Sam's Choice, etc). My kids have taken me up and done blind tests and they haven't stumped me yet on telling the difference between Coke and Pepsi. ("New Coke", really an other, was miscalled as Pepsi on occasion, though I never considered New Coke to be Coke).

I was at the mall a number of years back, when they were doing the "Pepsi Challenge", and I was going to do it and put in my vote for Coke (which is indeed my preference). They put out the two identical cups and I tasted and I noticed that the Coke was (at least to me) noticeably warmer than the Pepsi. I asked: "why is the Coke warmer than the Pepsi?" (My hypothesis being that perhaps this biases the testing, most people I know preferring colder beverages), but they would not answer, they only wanted a response as to what I preferred.

In actuality, I did prefer the cold(er) Pepsi to the cold (yet warmer) Coke (though I would have preferred a cold Coke over a cold Pepsi), but I lied and still chose the Coke over the Pepsi.

I don't know if it was part of the design of the challenge (I would not put it past the designers of the test, since they were trying to get people to "choose Pepsi"), but I did notice it and thought that you might be interested. I could see how this could be a "built-in confounding" to help push a preference to the
"desired response".

I thank you for the sharing of your excellent DOE "experiences". I really like to see the examples of more "everyday" testing that is done as a way of educating. Keep up the good work."

Thanks for relating another great example of badly executed design (?) of experiments that aliases multiple factors, in your case temperature with cola type.

PS. I am biased to RC Cola because my grandmother always gave it to me and my cousins (our mothers sisters—forbade any soda pop in our respective homes, but they were over-ruled by their mother—our grandmother when on these visits).


6. Events alert: A talk in Toronto at the Joint Statistical Meetings; also questions about a previous presentation on screening

If you make it to the 2004 Joint Statistical Meetings (JSM) in Toronto, Canada in August, stop and visit Stat-Ease at booth #405. Also, sign up for the JSM RoundTable Discussion on Wednesday, August 11th on "Design of Experiments Trials and Tribulations." Stat-Ease consultant Shari Kraber will be the moderator. She says "Participate in a lively discussion of the trials and tribulations of planning and running designed experiments. Plan to share your experiences and learn from the experiences of others. Discuss the most common pitfalls that experimenters encounter and learn how to avoid them. Explore problems with fractional factorials, mixtures, missing data, pass/fail data, etc."

I received this question on my talk titled "Screening Process Factors In The Presence of Interactions," which I presented recently for the Annual Quality Congress

From: Jim Alloway

"I was distracted at the start of your session and missed the explanation that you gave as to why one should ignore the known factors when starting a screening design (it is the first figure in your paper). That might make a nice "question" in your DOE FAQ Alert."


Yes, this is a bit unsettling, but the idea is to set these 'known' factors aside while screening others that may yield a vital few unknown ones. At a later stage the previously-known factors are combined with the newly discovered ones in a second DOE that's done at a higher resolution to reveal two-factor interactions. Aside from this being more manageable in runs per iterative experiment, it also keeps large (known) factors from overwhelming others (making them appear insignificant) that could be important nonetheless—both in their own right (main effect) and possibly as an interactor with a known factor (to be revealed in the followup DOE(s)).

(Click for a complete list of appearances by Stat-Ease professionals. We hope to see you sometime in the near future!)


7. Workshop alert: "Experiment Design Made Easy" is coming to

Stat-Ease will present "Experiment Design Made Easy" on August 17-19 in Philadelphia. See for schedule and site information on this and all other Stat-Ease
workshops open to the public. To enroll, click the "register online" link on our web site or call Stat-Ease at 1.612.378.9449. If spots remain available, bring along several colleagues and take advantage of quantity discounts in tuition, or consider bringing in an expert from Stat-Ease to teach a private class at your site. Call us to get a quote.


I hope you learned something from this issue. Address your general questions and comments to me at:



Mark J. Anderson, PE, CQE
Principal, Stat-Ease, Inc. (
Minneapolis, Minnesota USA

PS. Quote for the month—why the normal distribution is normal:

"Whenever a large sample of chaotic elements are taken in hand... an unsuspected and most beautiful form of regularity proves to have been latent all along."

—Francis Galton

Trademarks: Design-Ease, Design-Expert and Stat-Ease are registered trademarks of Stat-Ease, Inc.

Acknowledgements to contributors:

- Students of Stat-Ease training and users of Stat-Ease software
- Fellow Stat-Ease consultants Pat Whitcomb and Shari Kraber (see for resumes)
- Statistical advisor to Stat-Ease: Dr. Gary Oehlert (
- Stat-Ease programmers, especially Tryg Helseth (
- Heidi Hansel, Stat-Ease marketing director, and all the remaining staff

DOE FAQ Alert—Copyright 2004
Stat-Ease, Inc.
All rights reserved.


Interested in previous FAQ DOE Alert e-mail newsletters?
To view a past issue, choose it below.

#1 Mar 01
, #2 Apr 01, #3 May 01, #4 Jun 01, #5 Jul 01 , #6 Aug 01, #7 Sep 01, #8 Oct 01, #9 Nov 01, #10 Dec 01, #2-1 Jan 02, #2-2 Feb 02, #2-3 Mar 02, #2-4 Apr 02, #2-5 May 02, #2-6 Jun 02, #2-7 Jul 02, #2-8 Aug 02, #2-9 Sep 02, #2-10 Oct 02, #2-11 Nov 02, #2-12 Dec 02, #3-1 Jan 03, #3-2 Feb 03, #3-3 Mar 03, #3-4 Apr 03, #3-5 May 03, #3-6 Jun 03
, #3-7 Jul 03, #3-8 Aug 03, #3-9 Sep 03 #3-10 Oct 03, #3-11 Nov 03, #3-12 Dec 03, #4-1 Jan 04, #4-2 Feb 04, #4-3 Mar 04, #4-4 Apr 04, #4-5 May 04, #4-6 June 04, #4-7 July 04 (see above)

Click here to add your name to the FAQ DOE Alert newsletter list server.

Statistics Made Easy™

DOE FAQ Alert ©2004 Stat-Ease, Inc.
All rights reserved.


Software      Training      Consulting      Publications      Order Online      Contact Us       Search

Stat-Ease, Inc.
2021 E. Hennepin Avenue, Ste 480
Minneapolis, MN 55413-2726
p: 612.378.9449, f: 612.378.2152