Tuesday, March 4, 2008

Spec changes: question order

I want to change the way questions are ordered on a survey. In the past, departments and instructors were allowed to set the order of the questions they added. The questions would appear in blocks, first campus questions, then department, followed by instructor. Each block of questions would appear in the order in which the user set them.

There is a fundamental flaw in this manner of ordering the questions. The issue is that standard survey practice is to group questions together into logical chunks, for example all student development questions should be grouped together, and all instructor related questions should appear together. Our original method of handling this resulted in the survey bouncing around between logical groups. For example the student would see the department questions on student development, followed by the department's course evaluation questions, and these would be followed by more student development questions as set by the instructor. This problem is further compounded when secretaries or instructors not familiar with survey theory order questions randomly without regard to category.

Research indicated that when surveys bounce around like this, survey takers are more likely to ignore the subsequent sets of questions from a subject for which they already answered questions. The change I'm proposing is that organizations (departments, etc) and instructors are no longer able to order their questions. Instead, they just select the questions they want asked, and the questions will appear in the order they appeared on the selection page. The questions appear on the selection sheet grouped into their logical categories. See http://www.cafe.hawaii.edu/cafe_catalog.asp to see the categories and order.

For example, lets say the history department selects questions 1, 5, and 10, and one of their instructors selects questions 3, 7, and 12. When the student views the survey, the questions will be ordered as followed: 1, 3, 5, 7, 10, and 12.

2 comments:

hae said...

This makes sense - However, in terms of ease of use from the department or faculty - while they are selecting, would it be in the original blocks, but then will be reordered when presented to the students?

Ghenghy said...

For faculty and staff, the questions would appear in the original blocks. Look at this image for a screenshot of what it will look like for them. Notice the questions are already grouped into their respective categories.

The students will see the questions exactly in the order in which they appeared to the faculty/staff. So in the example image, the staff member selected questions 1, 3, and 4. If the faculty member selects 2, 6, and 162 the student will see the questions in order: 1, 2, 3, 4, 6, 162.