There’s a wealth of data available to help academic leaders lead their units, but making sense of it can be overwhelming. Institutional research staff can help leaders ask the right questions and get at the most relevant data for a particular issue. In an interview with Academic Leader, Ricky Tompkins, assistant vice president for research and planning at NorthWest Arkansas Community College, offered his insights on how to work with institutional research staff to use data effectively to make decisions.

AL: What types of data are available, and how are academic leaders using it?

Tompkins: There’s a lot more data available than what we use, and sometimes what we have isn’t in a format that the users—deans, directors, and those types—can take back to their people and be able to implement procedures and processes and make decisions. Data is there, of course. It’s required for the state and federal governments and IPEDS. Institutions collect data through their learning management systems. So there’s information there, but at the same time I think we have a problem with asking the right questions when we start looking at the data. Some of the things you should be looking at as a dean and may or may not be getting are things such as retention from class to class. We look at retention primarily from fall to fall or fall to spring. You can also break it down from class to class. You can break it down all the way to particular students. You can break it down by majors. It’s those types of things—drilling down into the data that we know we have and getting it into a format based on solid research questions—that allow people to look at the overall mix of the institution and base decisions on that.

We’ll meet with [academic leaders] and they’ll say, “We want to make sure that our majors are completing their degrees in a timely manner.” And we’ll say, “We can look at their overall success rate. Let’s look at their fall-to-fall retention and their fall-to-spring retention. How is that going? Do you have a series of courses? How are they doing in those? How long is it taking your students to get through a program? How are they doing after they graduate?” Basically, we’re just helping them frame those questions that will yield the information that they actually want and need to make the decisions that can benefit their programs.

I think [academic leaders] are surprised that we can go all the way down to the course level and say, “This is the retention rate for someone who is taking developmental reading.” We can consider questions such as how long does it take them to progress to the next level of development? How long does it take them to progress into Comp 1? How long does it take them to progress into Comp 2 and beyond? We can track that by student. Most folks don’t know that’s available, but it is. Most learning management systems can track that by student. When you start looking at institutional data—we can track expenditures by department or division. We can track a number of sections very quickly by department or division all the way down to the faculty member and do it on a longitudinal scale. That’s something that I think is probably most surprising to them.

AL: How do you help leaders ask the right questions?

Tompkins: I sit down with deans and department chairs and faculty and of course student services administrators, and the first question I generally ask is, “What is it that you’re trying to find? What is it that you’re actually looking for?” Normally it’s the 30,000-foot view of an issue. If I’m speaking with someone from student services or admissions, I’ll ask, “Are you looking at retention from a certain time frame? Are you looking at retention based on a demographic group? Are you looking at retention among traditional or nontraditional students? What are you looking for?” Basically it’s a series of questions, almost an interview-type format to try to get people to understand that in order to make decisions based on data you have to ask the right questions, and the question is very rarely at that high, all-encompassing level. You have to dig deeper than that.

AL: What do you find to be the biggest challenge for leaders in making sense of the data or framing the questions?

Tompkins: I think honestly what we deal with the most every day is the idea of anecdotes. “I’ve heard this. …” or “In my experience, we used to do this. …” It’s this idea of anecdotal information leading decisions. [My job] is to get them to understand that anecdotes can drive the discussion, anecdotes can start the discussion, but anecdotes are not adequate data to make decisions. You have to look at both qualitative data from focus groups and interviews and quantitative data to make sure that you are seeing the picture the way it actually is. Both types of data are needed to do that.

AL: What trends do you see in terms of questions being asked?

Tompkins: For the longest time the questions revolved around the admissions side—getting folks in, getting them admitted into a program, getting them on the path, and [retaining them]. Now that has shifted. We’re still focusing on retention, but now we’re looking more at the retention/completion and then either transferring to a four-year college or going into the workforce. The completion agenda is driving that. Faculty understand that, particularly with performance-based funding. And so we see that shift from admissions and retention to retention/graduation/transfer or workforce. That is the shift that I’ve seen very recently, particularly when you’re looking at the idea of higher education taking more responsibility for making sure the student gets in the workforce.

AL: What suggestions do you have for making this data clear to stakeholders?

Tompkins: I think it’s all about presentation and understanding your audience. You can have all the data in the world, but if you’re presenting it in a way that people don’t understand, it’s pretty much worthless. Most of the time that means you’re presenting it in a way that’s very practical and applicable to what they’re going to be doing. And most individuals that we deal with—faculty, the administration, and even outside stakeholders—are not looking at in-depth statistical analysis of data. They’re really looking for trends. Sometimes we present it as one-page fact sheets. We try to keep the information understandable, but we’re not talking down to anyone. We simply want to make sure it’s in a format that they understand. And most of the time a complex SPSS report or spreadsheet or whatever just won’t meet their needs. So we try to keep it basic and provide recommendations on what we think the data and information are telling us.

AL: What advice do you have to avoid being overwhelmed by all the data that’s available?

Tompkins: Understand what it is you’re actually looking for and ask questions. And understand that you can actually go back multiple times. Because what happens is you start out saying, “This is what we’re looking for” and get the information, and you kind of digest that, and then you say, “OK, this is great, but let’s take another step. What about this?” Then go back to your data people and say, “What about this?” You’re getting it in chunks, not all together. By the time you get to the end of your process you have a complete story. If you’re given the complete story and data all at the same time you’ll be completely overwhelmed. Take it in chunks. I’ve seen that work very well across the spectrum of leadership.

We have some core indicators that we consistently publish. I frequently get asked, “Will these core indicators change? Will these consistent data reports change?” The answer is yes, they will, and they change based on the institution and the needs of its constituents. So from an institutional research perspective, you always have to be looking at the trends and seeing what changes are coming about, because the data that we looked at 30 years ago, though still helpful, may not be applicable simply because of the changing dynamics in higher education. For example, 30 years ago where were we in distance education? We were doing compressed video more than anything else. Now you look at online learning and it’s a totally changed landscape. You’re going to have to continue to evolve when you’re looking at data, when you’re making decisions because yes, you are making decisions that are going to affect today, but you also have to understand that the decision you make today also is going to make an impact one year, two years, five years, 10 years from now. So the more information you have available, [the greater your ability] to be able to do some predictive things, to be able to do some modeling.

AL: Does having data help soften the blow of unpopular decisions?

Tompkins: It does. I think it goes back to the idea of transparency—having a process you use and data to support the decision you made. Most of the time [the data] doesn’t necessarily change people’s minds, but it provides understanding as to why you made the decision you did. The most difficult thing for a leader to do and probably the least effective is to make a decision and simply say, “The justification is because I said so.” You can’t support that. Having the data, looking at it, being able to sit down with folks and say, “This is why we did this. This is why we have to look at our staffing in student services, because this is where we see the data taking us” provides a different level of understanding. It’s very helpful when you’re looking at tough decisions. It probably helps the leader sleep at night, because tough decisions are difficult on everyone.

<@SM>