Behind the Numbers: What I Found Hidden in the HiSET Manual

By Elizabeth Alber

Have you ever stumbled on a question while looking for the answer to a different question?  That happened to me recently.  I submitted a question and was given a simple answer and then directed to a specific page for further explanation.  So I turned to said page and before I got to the paragraph answering question #1 I stumbled onto questions #2.  It stood out to me because the numbers didn’t seem to match the numbers on my wall. My first thought it was a typo or maybe old information that when the handbook was updated just got missed. Before I jumped too far to a conclusion I checked with my test proctor and he confirmed that my information matched his answer keys which means there was  conflict with what was printed.  So, I did exactly what we tell our students to do- if you have a question, ask!

I submitted the following question to HiSET.

One month ago at the IAACE conference I asked you a question from page 56 of the HiSET Technical manual. The sentence in question reads “The HiSET Mathematics, Science, Social Studies and Writing subtests  each contain 50 items, while the Reading contains 40 items.”

The answer keys however have Writing 60 questions plus 1 essay (not 50 plus 1 essay),  Reading 50 (not 40), Math 55 (not 50), Science 60 (not 50) and Soc. Studies 60 (not 50). So why the discrepancy?

If indeed they are taking 60 but only being scored on 50 what happens to those extra answers and any points that might go with those answers?

Shortly thereafter I got a response.

Our psychometrician has been kind enough to help us with this response.

The extra 10 questions (5 for math) are pretest (or trial) questions. These questions are not scored. They are embedded in the operational test forms so that we can collect data on test taker performance to determine whether these items are suitable for future use, or whether they need to be revised or removed. This helps us ensure that the scored items presented to the test takers are valid.

My first thought when looking at their response was frustration, so I called my colleague  Anne, who’s been doing Adult Education for many years.  She assured me that this was to be expected, an industry standard practiced in lots of kinds of tests, for example,  GED, ILEARN, ACT, SAT, and the like. She shared that she assumed it was two or three questions per subject.  When I told her it was 10 questions in four subjects and 5 questions in another her response was, “What!?”.  I also shared this with another colleague, Abbie,who is the director of an Adult Ed. program and she shared that she was unaware of the practice.  That got me thinking, who else doesn’t know, and why did HiSET not  make this clear?   So I worked on a response to their answer.

Thank you for your response to my question. I have concerns in two areas.

First of all, I understand the need for some research questions within the test, but I have concerns regarding how many are used. I expected two or three questions per test for a maximum of fifteen questions across five tests but instead there are forty-five questions across five tests. This means 20% of the Reading (10/50 or 1/5th of the test);  9% of Math (5/55 or 1/11th of the test); and 16% each of Writing, Science, and Social Studies (10/60 questions or 1/6th) of the test are for research purposes. Forty-five questions that students have to take time to answer are for your benefit and not the students’ scores. I’m thankful that incorrect answers don’t count against them, but neither do correct answers count for them.  So if my students get those research questions correct but then miss the same number of regular questions there is no benefit to my students only to you. Are these percentages consistent with industry standards?

I understand the value of those research questions, but could the percentage of those questions be reduced to something under 10% per test? There are already three different forms of tests per subject. If each set has forty-five research questions then you are asking one hundred thirty-five unscored questions across the three versions of the test. This seems like a lot of research questions, or are those same forty-five questions being asked in each different form?

Secondly, I also have concerns that this research process is not explained anywhere in your handbooks. I happened to stumble upon this information while looking into a different question. Would it be possible to make this information more transparent? My first thought when I read the information in the handbook was that it was old or incorrect information, possibly a typo,  because it did not match answer sheets students are given. If this is industry standard, a small clear explanation would have alleviated the confusion while also building rapport with those of us who utilize HiSET for their student’s success.

I like HiSET. I think that HiSET is the right kind of hard: a test that proves you’ve earned something, a score that businesses can respect. I was surprised to find that in addition to  testing them, you are using them as test subjects for your future success, but not necessarily for theirs. While this doesn’t change my opinion of HiSET’s rigor or validity, it did surprise me how the rigor and validity were achieved.

I really wasn’t sure what to do next. I knew this information, as did Abbie and Anne but did this need to be told far and wide?  Were we the only ones who didn’t already know this?   Am I calling you to action? Maybe! I found what I saw as a problem and sought out an answer. When I had concerns with the answer I sent a response.  If this information is new to you and you have some concerns, maybe you need to send a response as well.  If the lack of transparency caught you off guard, you might consider your own response. Mine is one voice which HiSET might  choose to ignore, but if more voices are raised, the issue could be at least better explained and not hidden away in a technical manuel.  Should you respond to HiSET or investigate if GED does the same practice? Should we read our manuals a little more carefully to be sure nothing else is hidden there?

 Knowledge is power and now you know, what you do next is up to you.

Guest Blogger