Grand Plans for Data Collection

At the School Board meeting on June 7, we heard a presentation on the school district’s implementation of the REAL grant.  This is a $5.3 million, five-year federal Small Learning Communities grant the district received in 2008 to support changes in our high schools designed to increase student achievement, enhance student relationships with other students and with adults in the schools, and improve post-secondary outcomes for all students.

The presentation lasted more than an hour and included a torrent of information.  Anyone interested can view it here and review the 114 pages of information we received here. It does seem as if quite a bit is going on.

One of the topics touched upon in passing was the district’s efforts to track the progress of our graduates once they move on to college or wherever their destiny takes them. I have a particular interest in this.  Last year, I put together (with the help of Jeff Henriques) an informal survey of our recent graduates who have gone on to college.  The goal was to gauge our graduates’ opinions of how well they feel our schools prepared them for college work and to invite feedback on what they think our schools do well and what they do not so well.  My effort was written up in the Cap Times a while ago. What I learned is that the graduates who responded had a lot to say and appreciated the opportunity to share their opinions.  Also, their views were pretty positive, though we certainly didn’t have a statistically valid sample and there may have been some sampling bias.

Included among the hefty package of REAL grant material we received for last month’s Board meeting was a one-page summary of plans for improved data collection regarding our high school graduates and their post-high school progress.   The summary described an ambitious plan.

The scope of the ambition comes into a focus a bit by comparing what we’ve got now to what is proposed.  As I understand it, right now our high school seniors are asked to fill out a survey in May of each year.  The survey is a DPI requirement and includes questions that DPI has developed.  The type of questions included can be found here.

This current survey does not currently seem to be a point of emphasis for our schools.  Apparently, seniors, or at least some of them, fill out the forms, the results are shipped to DPI, and that’s about the end of it.  As the informational sheet we received for our meeting put it, “an insufficient number of surveys were completed over time to provide reliable data that could be analyzed.”  The responses are apparently stored on the DPI’s WINSS system – which, by the way, is a great resource – but, unlike almost all the other data on WINSS, access to the responses is restricted; they’re not generally available to whomever happens to visit the webpage.  So, right now the survey doesn’t seem to be doing anything for us.

What is planned is something far more grand.  It is apparently the intent that the DPI senior survey will be taken more seriously, presumably to the point where we get some useful information from it.  In addition, the district will begin to participate in the Student/Tracker for High Schools program of the National Student Clearinghouse , an organization founded about 20 years ago by the secondary school community to serve as a repository of data on college students. The program relies on nationwide data to enable school districts to track their graduates’ enrollment status in colleges. An example of the kind of report that the system makes possible, this one for St. Paul public schools, can be found here.

We’re not ahead of the curve on this, by the way. According to its website, among the school districts in the country that already participate in the Student/Tracker for High Schools program are 58 districts in Wisconsin, including De Forest, Middleton-Cross Plans, Oregon, Stoughton and Sun Prairie.  Also, I’m told that DPI has worked out an arrangement such that all Wisconsin school districts can join the program for free.

Our new and improved data collection system is also planned to include information on the number of our students “who needed remediation in college.”  This is information that Board members, particularly Lucy Mathiak, have been requesting for some time, but the description of the program we received does not say where the information will come from.  It’s not the sort of thing that the Student/Tracker system includes.  I’ve been told that we’ve asked the UW for this type of information for our graduates, and the UW has agreed to provide it but hasn’t done so yet.  I once asked about getting this kind of information from Madison College (nee MATC) but was given to understand that it wasn’t available.  I’m not aware of any other source of this type of information.

The intent seems to be to have the data assembled in such a way that we can collate the following information about our graduates: demographic information; performance in our schools, including classes taken, grades received, and WKCE scores; responses to the senior survey; remediation information (if applicable); and the type of college enrollment information that we’ll obtain from the National Student Clearinghouse.

Once we’ve assembled all the data, then the plan is to undertake the following kinds of analyses (the descriptions of which are all taken verbatim from the information sheet we received):

  • Determine the percentage of each high school’s graduates who enrolled in college within 15 months after graduation.
  • Determine the percentage of last year’s graduates from each high school who needed remediation in college, and how this percentage varied by student income and ethnicity;
  • Determine the percentage of students who met the proficiency standard on the state high school test (WKCE) and still needed remediation in the same subject in college;
  • Examine how students’ ability to stay in and complete college is related to their high school courses, grades, and test scores;
  • Identify key information on graduates’ family backgrounds, high school activities and methods of college preparation that cannot be determined from our current school records; and
  • Share the results annually with MMSD staff, local post-secondary educators, and local business leaders to improve educational practices for future cohorts of high school students.

Whew, that’d all be something, wouldn’t it?  I would love to have this sort of information available.  I confess to being just the tiniest bit skeptical, however.  This seems to me to be a fairly massive undertaking and far from our current level of data analysis, which is, to put it mildly, not a strength of our district at the moment. The district received a DOE Smaller Learning Communities Supplemental Grant Award of about $28,000 to carry out this work, but that seems like a relative drop in the bucket, given the scope of what is planned.

I very much hope to find in the not-too-distant future that my skepticism was unfounded as the district undergoes a sea change towards increasing depth and sophistication of its data analyses, including the kind of multi-dimensional analyses of our graduates’ outcomes that are proposed as part of the REAL grant projects.  I know that many of our top administrators would love to see that kind of change come about and are working hard to nudge us closer in that direction.

But it’s a long (and expensive) way from here to there.  In the meantime, I don’t think I’m going to abandon my own humble, hand-made survey of our high school graduates.  I think I’ll gear up to try to get another one going sometime next spring.

Advertisements
This entry was posted in Uncategorized. Bookmark the permalink.

2 Responses to Grand Plans for Data Collection

  1. Chan Stroman says:

    I’d be interested in seeing how the REAL grant implementation will be coordinated with the USDOE Statewide Longitudinal Data System grant for preK-college tracking awarded to the state in May ($13.8M); lots of overlap, lots of opportunity for synergy/efficiency.

  2. Chan –

    Thanks for commenting. I’m not familiar with the DOE grant but the relevant folks at the District are. Here’s a brief excerpt from the Program Evaluation Protocol (http://boeweb.madison.k12.wi.us/files/boe/Appx%201-8.pdf ) we approved last night: “StateWide Longitudinal Data System (LDS) – As part of a federal grant obtained by the Wisconsin Department of Public Instruction (DPI), a state wide data warehouse has been created. That data could be used for collaborative research and evaluation projects across the districts in Wisconsin. MMSD has been a strong advocate for greater use of the LDS for these purposes and will continue to push for more projects.”

    Ed

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s