Darlena Jones, Ph.D., Director of Assessment and Research - Association for Institutional Research
September 28, 2018
The Director needs information about a new program that she’s considering implementing. She asks you to survey the right population, analyze the data, and write a report before next month’s meeting with the President. Where do you start? Do data already exist? If not, how do you write a survey?
Survey research uses surveys to collect data. Those data are reported and used by decision makers to improve the student experience. As its name implies, survey research uses a survey as a data collection tool. All surveys start with one common mission: collect data to help answer an important question that available data can’t address. For example, an analyst in enrollment management might survey non-returning students to determine why they didn’t re-enroll; the director of academic advising might survey first-year students to understand the impact of advising on student success; the director of residence life might survey sophomores for feedback on ways to increase satisfaction; senior leaders might survey to gather information to support or understand policy decisions; and academic units might survey to understand teaching effectiveness or as evidence for accreditation.
When should you conduct a survey? Before conducting a survey, answer these questions:
If you answered “yes” to any of these questions, then reconsider conducting a survey. But, if you can’t address the issue using existing data, reports, or other data collection methods and you’re confident that action will be taken based on the survey results, then survey research is appropriate.
In other words, survey research is the last resort in data collection; not the first because it utilizes institutional resources (i.e., time, effort, energy) and can contribute to survey fatigue of participants.
Who is the audience for this work? We conduct survey research to answer important questions. There must be an impetus that starts this work. Who asked for this research to be conducted? Besides that person(s), who else could use this information to improve the student experience? Once you have identified your audience, keep them in your mind; ultimately our work must answer their questions.
Is there a published/vendor instrument I could use? Sometimes there may be a published or vendor instrument you could use for a fee. Survey authoring isn’t easy and can take many days to draft; it might be cost effective to use a vendor survey. Before drafting a survey, check out some vendors first.
Do you need IRB approval? Some institutions may consider all surveys subject to the state and federal laws for Human Research Protection Program (HRPP) and may require them to be reviewed by an Institutional Review Board (IRB) while other institutions may regard some surveys as not meeting the standards for IRB review. Check with your institution’s IRB to understand their guidelines for survey research.
What online survey software will you use? Commercial software that makes it possible to design your own online survey, collect data and produce results is ubiquitous (e.g., SurveyMonkey, Zoomerang, eSurveysPro, Qualtrics). Before exploring these products, ask your institutional research/effectiveness office if your institution has a contract established with a vendor that allows researchers to use their software as part of the existing institution package.
How will you ensure motivated responses? The data you collect are only as good as the honest responses your respondents provide. Inspiring motivated responses is a challenge for every researcher. Some simple things to consider:
How will your survey research project impact other units? None of us work in a bubble but sometimes it might feel like that. Our work will always directly, or indirectly, impact other units and being aware of that impact will be important to your survey research work. Offices like institutional research (IR), institutional effectiveness (IE), and/or institutional assessment often conduct surveys. You’ll need to communicate and coordinate with these offices to ensure that your survey isn’t launched at the same time as theirs.
Are there institutional data that can be merged with the survey data? Ideally your survey won’t ask questions in which those data are already known. Consider working with your IR/IE office or the registrar to identify data that can be merged with your survey. Data that is commonly stored and might be useful in survey research includes:
When requesting institutional data, ask for a unique student identifier (e.g. a student ID, student email) that can be used to link those data with your survey.
Why is someone from the Association for Institutional Research (AIR) writing a blog post for NASPA about survey research? AIR is a membership organization for higher education professionals around the world who are involved in institutional research, assessment, effectiveness, and planning; nearly one-third of members are assessment professionals!
To improve higher education’s survey research skills, AIR offers several resources:
If you’re interested in learning more about AIR or accessing our resources, please visit our website, www.airweb.org.
Opinions expressed in this blog are those of the author(s) and do not necessarily reflect the opinions of NASPA. If you agree or disagree with the content of this post, we encourage you to dialogue in the comment section below. NASPA reserves the right to remove any blog that is inaccurate or offensive.