The bare fact that this survey was produced at all is perhaps the most important thing about it. It suggests a greater degree of seriousness about confronting the many challenges we face as teachers in the liberal arts, and in religious studies more specifically. We should all be grateful for that. But we have to be honest: the second most interesting thing about this survey is how little it tells us, and how much more we might want to know.
1. Two Problems: Curious Demographics and Lack of Context
Begin with a couple sampling problems that the survey seems to face. First, the population studied strikes me as remarkably white, with 89.5% of the participants at least partially identifying as white (and only 2.6% of them (at most) identified another ethnicity or race). Also, 55.8% were female. The latter seems to me more statistically similar with my department’s experience, but the former seems a bit off. Does this skew the findings of the survey? It’s a genuine question.
Second, we don’t really have any way to measure these findings against other majors, and so we don’t know how much of this is common to higher education graduates in general, and how much is specific to religious studies majors in particular. There’s nothing to do about this, except to note it, and recognize that this means any judgment we make is very obscure. In the future, it could be useful for the AAR to coordinate with other scholarly associations to produce a larger survey that procures findings for majors across a number of disciplines. That would help us with this problem, at least a bit.
2. A Concern: Subjectivity of assessment of religious studies major overall
Alongside those obstacles, it is also quite difficult to discern what exactly is demonstrated by the questions about how the interviewees think they improved, apart from their own current subjective self-assessment. But what does that tell us about their improvements? It is hard to know what these questions measure apart from interviewees’ current conception of their past, and they are hardly without incentive to tell a certain narrative. A skeptic might suggest that these questions demonstrate simply our ability, as teachers, to convince the students of a narrative. But we don’t need to be very cynical to admit that they do not speak directly to any objective assessment of actual learning.