#1 - A nonprofit organization concerned with a social issue

[Administrator]

Please post below with any questions!

[actionjackson]

What kind of flaw is present in this question? In breaking down the stimulus I was left with:
Premises: An NPO sent a fundraising letter along with a survey to 5,000 people. 300 responded, and 283 of those 300 agreed with the NPO.
Conclusion: Most of the 5,000 people to whom the letter was sent agreed with the position.
So as soon as I finished reading the stimulus I thought, "most of 5,000 is 2,501, isn't it?"
I chose C, not the correct answer choice A. I think I get why C is wrong though, and would like some confirmation that my thinking is accurate here. Flaw questions, are a member of the must be true family and are subject to the fact test, which C fails? We don't know anything about the conditions under which respondents did their responding. All we know is that these surveys were mailed to the people along with a fundraising letter. Where did I go wrong?

[Ricky_Hutchens]

Hi actionjackson,

You did a lot of thinks right. First, I think your assessment of C is right. We don't know anything about the survey itself, so we have no reason to think the responses do not reflect the opinions of the respondents. Now, let's break this down.

NPO sends survey to 5,000 people
300 people respond
283 agree
17 disagree
2,501 is most of 5,000

The conclusion is that at least 2,501 of the 5,000 people agree with the NPO. But there is a problem with that, because we only know the opinion of 300 of that 5,000. Whenever you only know something about a small subset of a larger population, you should always ask yourself, is that subset representative of the larger population. In this case, there is a reason to believe it is not.

Think about it, if someone sends you a survey asking you about an issue you don't care about, you are less likely to respond than someone who does care. Because they received such a percentage of surveys back, it is likely that a large number of people are simply indifferent as to the NPO's position. If 2,501 or more of the people agreed with the position, wouldn't you expect more than 300 replies? For that matter, if 2,501 or more disagreed with the position, wouldn't you expect more than 300 replies?

In short, it seems likely that the sample in this case is unrepresentative of the population.

[erust2]

The “quite likely” through me off in answer (A). Why are they “quite likely” to be different? Is it because it’s a fund-raising letter? It seems that they “may be” different, and it also seems hard to support that they are “quite likely” to be different.

[Dave Killoran]

The “quite likely” through me off in answer (A). Why are they “quite likely” to be different? Is it because it’s a fund-raising letter? It seems that they “may be” different, and it also seems hard to support that they are “quite likely” to be different.

Hi Erust,

You ask a good question! In this case, what LSAC is driving at is that the subgroup is mostly (283 out of 300) composed of people who agreed with the organization's position strongly enough to reply to the survey. So that group largely has the specific characteristic of agreeing with the organization. Compare that to the overall group, which was largely indifferent or perhaps even against the organizations position. In either case, you are suddenly comparing two groups that are looking to be different on this key point(the two groups being the 5000 people in the overall pool vs the 300 respondents). That is what they mean when they say the 300 respondents are "quite likely to be unrepresentative of that population in certain relevant respects."

Please let me know if that helps. Thanks!

[Khodi7531]

What' wrong with B?

I was stuck between A and B but really couldn't differentiate. Chose A at first the changed to B because it's assuming that people in the group would respond differently on "different occasions"?

At first I thought that's what makes it right; Diff occasions as in when it isn't surveyed by letter. But is this the reason it's incorrect because it doesn't assume this about the group on different occasions...

[Robert Carroll]

Khodi,

Answer choice (B) isn't always going to be flawed reasoning at all. Is it a flaw to assume that people's opinions are consistent? It depends, really. If something has happened that could change people's opinions, relying on an outdated statement of their opinions would be a mistake. So, let's say that people are polled about a president and give him favorable ratings. Then that president is implicated in a serious scandal. Saying "Well, he was popular in this poll, so the scandal hasn't affected public perception of him" would be assuming that people haven't changed opinions. And in that case, there's a good reason to think people HAVE changed opinions, so the old poll is almost worthless.

Is that what's happening here? I don't see it. First, the conclusion is about what people agreed with. It doesn't even appear to be saying they STILL agree, just that they DID agree when the survey happened. Further, is there any special reason to think that opinions would change, so that the survey results would be different if it were conducted again? I see no special reason to think so. It could only be flawed to expect consistency of opinion if there's some reason to think it might not be consistent. That's just not present here.

All that having been said, what's the real problem with the argument? It's that the opinions of 300 people don't necessarily reflect the opinions of 5000. And that's what answer choice (A) says.

Robert Carroll