Bradley Wright/ July 21, 2015
Do Christian churches in the United States actually welcome people from different racial and ethnic groups?
To answer this question, another sociological researcher and I conducted a nationwide field experiment to see how churches respond to emails from potential newcomers. More than 3,000 congregations received an email ostensibly from someone moving to their community and looking for a new church. We measured whether the churches replied to this email and, if so, what they said. But there was a catch: We varied the names attached to the emails so that they conveyed different racial and ethnic identities. Would the names alone change how churches replied?
Yes, they did—but not for the churches that we expected.
These graphs are pretty misleading. Compare Catholics and evangelical Lutherans. The graph shows Catholics to be worse, but by the percentages, Catholics are much better. Look–our 60% for African Americans gets a lower bar than the 40% bar right next to it. …What’s up with that? :shrug:
The graphs are a mess. The article has another graph, with Evangelical responses, but there’s no consistency in ratios between the different graphs. Whoever did the graphic art for the article screwed up.
The article itself is a different story—it explains the methodology of the study, and says that both Evangelical and Catholic churches’ responses where similar, with no statistically significant differences in results which would be attributable to preceptions of the race of the person sending the email.
You really have to read the article. The graphs are misleading----the ratios on the Catholic graph are inexplicably as if the artist went from 50% at the bottom to 100% at the top, not 0% at the bottom to 100% at the top.:shrug:
The values are not supposed to add up to 100%. These percentages are representing the fraction of churches that sent responses after receiving a particular email from a particular “ethnic name”. What matters most here is the differences between each of the bars since each bar represents a different ethnic group. As long as the response ratios for each group within a denomination were roughly equal to each other, the church was considered to lack an implicit racial bias. If the bars on the graph of a particular denomination are roughly equal, but all very low that just means the denomination is awful at evangelizing anyone, though they do so equally.
In the first graph, which triumphguy posted, if you take the space between the bottom graph line to the lighter line right under “Percent of emails responded to by Mainline Protestant denominations”, the bars are proportioned to mark out 0% to 100% between those two horizontal lines. The Evangelical graph is taller overall, proportionally—so no common rule of proportion was used between the graphs for the three groups, when a common rule should have been kept. I’m a fine artist, not a graphic artist, but whoever was the graphic artist here made a confusing mess out everything.
Oh I understand what you’re getting at now, thanks for the clarification. I disagree though that this is actually bad form in representing the data. I’m no social scientist so I have no idea if they labor under different SOPs than physical and life scientists do, but we frequently try to make the most use out of a graph’s space. This includes altering how much of a scale is shown (in this case it’s altering what percentage will be shown as the y-axis maximum). This isn’t in any way duplicitous, rather its a technique used to show clear differences between data sets.
Take the chart comparing Mainline Protestants to Evangelicals. The maxim is at about 70% because the largest datum in either set is 67%. Keeping the scale to 100% would render a lot of blank white space in the chart and would make an immediate observation of the graph difficult to compare the relative racial bias differences between the two sets (Mainline vs. Evangelical). The result is the same: If you were to plot a trendline over both sets of data the Mainline data would have a diagonal line (showing bias) while the Evangelical data would have a nearly perfect horizontal line (zero slope, showing nearly zero bias). Switching the y-axis maximum to 100% would minimize the slopes of both lines making the difference between the two more difficult to ascertain, and making the Evangelical trendline appear to have a slope even more closer to zero.
What I wish more pop articles with graphs would start doing is showing the error bars!!! Doing so would clearly show which differences are statistically significant and which aren’t. Take the same chart I was just mentioning: There’s an observed difference between Evangelical response to White emails versus Asian emails (59% vs. 56%, respectively). An error bar accompanying the data bars would tell us immediately if the measurement uncertainty was ± 2%, or ± 4%. If the former, then the difference is significant. If the latter, then there is no statistical difference between the two data.
Okay, that’s interesting. I wasn’t aware of any of that. I was just looking at the graphs as an artist and being aesthetically annoyed by how the varying proportions made no sense to me. I thought the different hues of blue were a nice touch, though.
In the article, the researcher admits that they undersampled Catholic Churches and that the results for them weren’t statistically significant. This may or may not be something for parishes to be concerned about.
I don’t think there was any bias or deliberate misrepresentation of data. The article shows none; the numbers on the graphs are clearly in place for all to see; and well, folks, it’s not a great practice to just skim over the bars of a graph and ignore the article which is the real substance of the topic, unless you intend to have merely the shallowest understanding of a topic.
I think we’re simply looking at plain old artist error, not some anti-Catholic conspiracy on the part of the CT graphics department.