Cross posted at Huffington Post's Off the Bus pages (but this DebateScoop has new info unavailable earlier).
Jack Muse reported that Monday's first YouTube debate "has been widely praised in all corners of the media" because of the one respect in which it really differed from previous debates, the use of citizen generated video questions.
But what is a good question? Which of the questions in Monday's debate were better and which worse? Our students and faculty at the Ben Franklin Transatlantic Fellows Initiative and the Wake Forest University Debate program set out to answer just that question and we have some interesting, albeit preliminary, answers.
The next step was to devise a simple rubric for assessing questions. We wanted something that did not require young students, many from Europe, to have intimate familiarity with the campaign to date. That's both a feature and a bug -- there are qualities of questions that are independent of their content. What's makes a question a good one regardless of whether it's on Iraq, education or health care?
During the debate our students used a Likert scale to evaluate each question on a 1-5 scale (5 being "strongly agree) in four categories:
*Clarity. The question made sense.
*Interesting. The question made me especially interested to hear the answer.
*Demanding. The question demanded that the candidates explain or justify their answer.
*Audio/visual. The video was visually powerful with good sound quality.
You can read more about the method and some preliminary analysis of what it means(including its limitations) but this is, to my knowledge, the first report on judging the "winners and losers" among the real stars of the debate, the questioners.
So what "won"? People love to jump straight to the results and you can click-and-go to the question ranking page. For the stats inclined you can view scatter plots of the individual variables vs. total score or contact me for the raw data.
We hope some of you scholars and students reading DebateScoop can pick up this research thread and weave it into something more powerful. Here are a few teasers from Gordon Mitchell's our staff's analysis:
* Policy questions had a definitive advantage in the rankings.
* Iraq questions scored high. Three of the top eight questions centered on the ongoing conflict in Iraq.
* Audio-visual production values were not particularly significant. While many YouTube videos feature slick production elements, the variations among the videos shown in the debates were not significant enough to create a marked difference between the videos.
* Neither foreign policy nor domestic policy questions had a clear edge.
* Questions that "box" a candidate in (based on the "demanding" criteria) were more interesting to participants. The top tier of questions tended to push students by referencing ongoing political debates and then prompt a candidate response, rather than asking "What do you think..." or "What would you do..." without much of a setup.
Even without statistically certain results, is it not clear that the real winners include our students whose YouTube submissions have already had over 10,000 views? And the students who are using what we learned from the first debate to submit questions for the GOP debate in September? Can Mitt Romney really contemplate skipping that debate if he understands this story? (More coming soon on rumors that the GOP candidates will all bail on the event).
The educational effect of the YouTube debate is of inestimable value. The younger generations are not only viewing the debates in record numbers, they are becoming empowered. A note a student left on my Facebook wall should inspire us all: "I learned how much power a kid and a laptop possessed."
Finally, for the patient who have not already clicked off to the full results page, a drum roll and the best and worst questions according to our study (go ahead to the results page to see the embedded video with the precise scores for all 41 questions) :
Best: CNN's question 9:
Is it OK to cite religion as a reason to deny gay rights?
Worst: CNN's question 18:
Who was your favorite teacher?
{UPDATE] Updated to change incorrect attribution of the "preliminary analysis." Ron Von Berg, Damien Pfister, and Kurt Zemlicka were the co-authors of that.
