Tag: measurement

Posted on

Data Sheets for IEP Goals Must be Individualized, Too

09/25/2011 – UPDATE! Instead of resorting to paper-based data tracking sheets, consider using Goalbook instead. Now in beta – sign up for your free account today at http://goalbookapp.com.

One of the most common search queries that puts people on our web site is for data sheets to use to measure a child’s progress towards his/her IEP goals. The fact that people are looking for pre-written data sheets speaks to the larger issue of anything in IEPs being canned or pre-written.

The term “IEP” stands for Individualized Education Plan. Individualized. As in, tailored to the individual. You don’t tailor an IEP by using pre-written, canned content.

That isn’t to say that you can’t use a basic skeleton of pre-written material as your starting point, but just as goals have to be tailored to the individual needs of the child, the data sheets that measure progress towards that goal must also be tailored to suit the goal. It depends on how the goal is formatted as to how the data sheet should be devised.

For example, if you had a goal that read: “When given a worksheet of 10 single digit subtraction problems per trial, [Student] will accurately calculate the correct answers with at least 80% accuracy in 4 of 5 consecutive trials within a 2-week period as measured by work samples,” then you don’t need a data sheet, per se. You’re using work samples to measure progress towards the goals.

That said, the scores on each assigned worksheet could be conveniently tracked on a single data sheet just to keep the outcomes all in one place. If the worksheets are already required of the student as part of his/her math curriculum, then presumably the grades on each will be recorded by the teacher in a grade book or a computer-based grade tracking system, as well.

But, if you have a “stranger danger” goal that reads: “Following the pre-teaching of one social story per trial, where each social story pertains to a unique situation that calls for [Student] to determine when he should say something to another person, [Student] will appropriately role play speaking or refraining from speaking to the other person as appropriate and what he should say when speaking in 4 of 5 consecutive trials within a three-week period as measured by data collection,” you’re going to have to have a data sheet.

A data sheet could include a table that has a column for the social story titles, a column for the date that each social story was presented and role played, a column indicating how many role-plays it took for the student to correctly give an appropriate response, a column indicating how many prompts were necessary for the student to correctly give an appropriate response, and a column for teacher comments. It might look like this:

Title Date # Role Plays # Prompts Comments
Stranger Wants Money for Beer 09/12/09 2 3 verbal, 1 gestural The first time through, [Student] needed prompting, but after we discussed how he might have handled the situation differently following 1st role play, he was able to complete 2nd role play w/o prompting.
Clown is Giving Out Balloons at the Mall 09/15/09 4 5 verbal [Student] had a hard time grasping that, at age 18, getting a balloon from a clown in the toddler play area at the mall is not age-appropriate. He understood what the role-play required, but he really wanted the balloon and it took 4 role-plays before he demonstrated the desired response. I’m not sure he would respond appropriately in the actual situation.

Just as goal-writing can be likened to science experiments, so can data sheets. Think about every science fair experiment you did as a kid in school. What was required? You had to delineate the steps of each experimental trial and collect data on your outcomes. This is no different. Your data sheets should collect data on what you’re trying to measure, which means you’re going to have to tailor them to each goal for which they are written.

Click here to download a podcast version of this blog article.

Was this article helpful? Please donate to help cover the costs of our blog.
.

.

KPS4Parents is a 501(c)(3) non-profit organization, tax ID 65-1195513.
All donations are tax-deductible.
Posted on

Writing Measurable Annual Goals – Part 2

In our posting, Writing Measurable Annual Goals?- Part 1, we talked about what goals are, the purpose they serve, and how they relate to present levels of performance. In today’s posting, we’re going to talk about the federal requirement that goals be measurable and how measurable goals can be written.

I cannot emphasize enough how critically important it is that goals are clearly and succinctly written in objective, quantifiable terms. In order for anything to be measurable, it must deal in absolutes and the language must mean the same thing to anyone who reads it.

First I’m going to give examples of unmeasurable goals and then I’m going to give you examples of measurable goals. I want you to contrast and compare them against each other.

Examples of Unmeasurable Goals:

[Student] will demonstrate understanding of language concepts by naming items within a given category with 80% accuracy.

Now pretend you have to implement this goal. First you’ll notice that this sentence contains two verbs: “demonstrate”  and “naming”. This is confusing. The piece that reads “will demonstrate understanding of language concepts” is superfluous and misleading. Including this language implies that something greater is going on here than that for which the language of the goal really provides.

There is another place on the goal page where the type of goal can be indicated without cluttering up the language of the goal itself. The language of the goal should describe what exact task the student is supposed to perform and nothing more.

If you eliminate this unnecessary bit to render the goal down to only what outcome the student needs to demonstrate, you’re left with “[Student] will name items within a given category with 80% accuracy.”  This is incredibly vague.

Does the student only have to do this once during the entire year that the IEP is in effect in order to have met the goal? If so, has he really mastered any new skill ? If not, how many trials must he perform at 80% accuracy in order to determine that the targeted skill has been mastered? How is the 80% calculated? How is any of this supposed to be measured? How many items must the student name within a given category? From how many categories will he have to name items? I’d also prefer to see the words “at least” immediately precede “80%” so that the student is expected to achieve “at least 80% accuracy,” which is different that saying flat-out the target is 80% and no more.

[Student] will write 15/26 letter sounds when dictated orally.

Again, this is vague. While we have some numbers in here, the goal overall is not measurable. How many trials must the student perform in order for it to be said that he mastered the targeted skill? And which 15 of the 26 letter sounds must he write? The way the goal is written, he’d only have to perform as described once during the entire year the IEP is in effect and the goal could be said to have been met. That doesn’t mean that knowledge was gained or a skill was acquired. It could be a totally random fluke.

I’d also like to see “at least” precede “15/26” for the same reasons indicated above under the previous example, along with clarification of which 15 letter sounds he’s supposed to learn. It shouldn’t include the ones he already knew when the baseline data was taken for the present levels statement.

In the classroom setting, [Student] will follow an individually designed visual schedule of his daily activities with minimal verbal prompts as measured by observation record achieving 80% accuracy.

Here, this sounds pretty okay except for the measurability. At least you have a decent idea of the spirit of the goal, but how many is “minimal”? Is the 80% accuracy meant to be averaged over the course of the year or just within the final stage of the goal? If it’s just the final stage, how long of a period of time is that? How is the 80% calculated? Is it a flat-out 80% or at least 80%?

In my many years as a special education advocate, I’ve come to realize that “achieving 80% accuracy” has become the arbitrary language that gets plugged in by default because it sounds measurable to parents, most of whom really don’t understand the measurability requirement or how it can actually be satisfied. It implies that some kind of calculation must be taking place or you wouldn’t be able to arrive at a percentage and 80% sounds like a high enough number that most parents will think the goal is reasonably ambitious.

But, when you start picking apart the language of the goal to turn it into a math word problem, you realize that there are too many unidentified variables to do any kind of calculation that can result in any kind of a percentage. Far too often, the percentage isn’t really the result of measurement; it’s a “guesstimation.”? Teaching staff will say, “Oh, I’d say he was about 75% accurate.”? There is no measurement in a situation like this at all.

To arrive at 80%, you need 4 out of 5, 8 out of 10, or 16 out of 20, etc. things done a certain way for the math to work out. This is where the number of trials and the number of presentations per trial becomes important.

When you get into reading and writing goals, this becomes even more complex. You need to specify the grade level of the passages to be read or written or use other terminology that describes the complexity of the language that the student is expected to read or write. You cannot use vague language like “at his instructional level” without identifying what the student’s instructional level is in the present levels of performance statement.

Really, that kind of language should be avoided altogether because the purpose of a goal is to move the student forward. If the goal is expecting the student to be performing at the same instructional level at the end of the goal as he was performing at the beginning, this generally means that he?isn’t actually expected?to progress.

The only real way to use this kind of language is to make sure there is also a goal in the IEP targeting the increase of the student’s instructional reading level and collecting data on progress towards that goal throughout the same annual IEP time frame so that the instructional reading level is known while any other goals that require presentation of text at the student’s instructional level can be appropriately adjusted as the year progresses.

Sometimes goals are poorly written because the school members of the IEP team really don’t know how to write them. Other times, they are deliberately keeping the language vague so that there is little that parents can actually hold them to. If the student fails to progress, it’s hard to point to the goal and say he didn’t meet it if it was vaguely written. Some school team members deliberately write weak goals so that they aren’t accountable to much.

Examples of Measurable Goals:

When presented with a worksheet containing 20 numbers, [Student] will correctly identify the place value of each number independently (tens, hundreds, thousands, ten thousands) in at least 4 of 5 trials over 2 consecutive weeks as measured by work samples.

This is nicely written. There is no way that two different people could pick this up and walk away with different understandings of what needs to be done.

Given a picture of interest, [Student] will be able to independently write 2 sentences (no dictation or adult prompting) using correct punctuation and capitalization in at least 3 out of 4 trials in a two week period as measured by work samples.

Again, this is really succinct clear language that describes an outcome that can be easily measured. It’s evident what the student has to do to demonstrate mastery of the targeted skill.

When given a reading passage at the end of first grade level about a topic of interest, [Student] will orally answer factual questions correctly by giving a sentence of at least 3 words in at least 3 out of 5 opportunities per trial for 4 trials within a 2 week period as measured by teacher made test.

This is a little convoluted, but you still arrive at the same place no matter how you look at it. Goal-writing isn’t meant to be Shakespeare. You could rearrange the words so they flow more smoothly, but the targeted outcome is still explained even as it is.

When you compare these last three examples against the first three, you can see the stark differences. The first three are vague and don’t really tell you what you’re supposed to do. The last three describe specific outcomes that can be measured.

The first two examples of the unmeasurable goals don’t even explain how they will be measured. The third unmeasurable goal example uses “observation record” as its method of measurement, but you have to be really careful with this.

Just like an arbitrary percentage without any indication of how it is actually supposed to be calculated is really just an estimate rather than a measurement, the term “observation” is often used as an arbitrary indication of how progress will be measured. In and of itself, “observation” isn’t any kind of measurement at all. “Observation” is simply looking at something.

An “observation record” is only as good as the kind of data it’s meant to collect and if that data isn’t described in the goal, whatever is recorded is likely to be unmeasurable.? I prefer that, instead of observation records or logs, there are data sheets and the method of measurement is indicated as “specific data collection,” but I’ll go along with “observation record” if the language of the goal is clear on what will be recorded or a copy of the observation record to be used is attached to the IEP as one of its pages, so there’s nothing left to anyone’s imagination as to what is supposed to happen.

If the goal is written in measurable terms, then the data sheets can be fairly simple check-off lists or tally sheets. Comments and anecdotal observations should supplement the data to provide context, but they shouldn’t replace the data.

We hope this makes goal writing make more sense. Please post your questions and comments. We realize this can be complicated and want to make sure you understand. As an added resource, you can visit http://www.calstat.org/iep/.

Was this article helpful? Please donate to help cover the costs of our blog.
.

.
KPS4Parents is a 501(c)(3) non-profit organization, tax ID 65-1195513.
All donations are tax-deductible.

Posted on

Present Levels of Performance – Where They Come From and Why We Need Them

We’ve discussed and provided a basic definition of present levels of performance in previous postings, but I really want to focus in on them in today’s posting because they are so critically important and so often over-looked. I’ve encountered IEPs that didn’t have them at all. I’ve also encountered IEPs that had such vague and non-specific present levels statements that they served no useful purpose whatsoever.

34 CFR ? 300.320 requires statements of present levels of performance as well as measurable annual goals. This is one of those things about the law that requires some reverse engineering and common sense. While the federal regulations do not spell out what all the elements of properly written present levels statements are, because they describe what a child can and cannot do at the time an IEP is written and because the goals describe what the child should be able to do one year from the date the IEP is written after receiving services, you can deduce that the present levels statements and the goals have to directly relate to each other.

In essence, your present levels are your “befores” and the goals are the “afters” that you’re aiming for. Measurability is required of goals so that you can tell if the student has made any progress or not, but that also requires that you knew where he/she was as of the start-date of the IEP as a point of reference. The progress a child is making over the course of the annual period covered by the goals has to be compared back against the present levels that were written at the time the IEP was developed.

For example, let’s take something easily measured like reading fluency. Reading fluency is basically how fast someone can either recognize on sight or decode a word while reading?- in essence, how fast can the person read (which doesn’t necessarily imply that the person understood what was read). Fluency is purely a measure of how fast a person can read off the text on the page.

Let’s say a child starts out at the beginning of an IEP with a fluency of 80 words per minute with first grade level text. The present levels of performance statement would read something like, “When provided with five consecutive passages of 150-200 words at the first grade level within a two-week period, [Student] demonstrated a reading fluency rate of 80 words per minute.” That’s pretty straightforward. The goal might read something like, “[Student] will read a passage of 200-250 words at the second grade reading level per trial with a fluency rate of at least 120 words per minute in 3 out of 5 consecutive trials within a two-week period as measured by teacher-recorded data.”

I don’t want to delve too deeply into the science of goal-writing right now; that’s an upcoming posting. But, because goals directly relate to present levels statements, I have to give an example here simply to make the point that without solid present levels, you have no idea whether a student’s performance towards a particular goal represents growth or not.

If you didn’t already know that the child read first grade level text at 80 wpm, then you wouldn’t know that second grade material at 120 wpm was an improvement. Where would you be?- where would the child be?- if you wrote a goal targeting 120 wpm and it turned out the child could already read 120 wpm? That’s not progress. That’s stagnation. What if the child actually read at only 10 wpm at the time the goal was written? Is it realistic to expect a fluency rate of 120 wpm after one year’s worth of intervention in a situation like that?

Because goals must be measurable, and because they refer back to the present levels of performance, the present levels themselves must be measurable. This really shouldn’t be that hard to accomplish if the last body of assessments were properly conducted and reported and all the present levels and goals from the time the assessments were conducted forward were properly written. But those, unfortunately, are big “ifs.”

I took the following example from the IEP of a student whose case we helped take to due process?a few years ago: “[Student] can copy anything. She is writing her first and last name on her own with few errors. She voluntarily writes ‘Daddy.’ She loves to write on the white boards.”

This is one of my favorite examples of how not to write a present levels statement. It was written for a seven-year-old with Down’s Syndrome and very serious holes in her knowledge due to poorly designed programming over a period of years. When I first read this child’s IEP and came across this language, I said to her father (rather sarcastically, I’ll admit), “She can copy anything? Like, the Mona Lisa? Wow! That’s amazing!”

Here are the major failings of this present levels statement: the word “anything” is wholly inappropriate. Additionally, there is no way to know what the author of this present levels statement meant by “few” errors. How many is that? What kind of errors? Could she write her name with or without prompting? With or without a model? Plus, the language that she could write her first and last name with few errors was, verbatim, the same language in her present levels statement of her writing goal written one year prior, which suggests that she failed to make any progress over that one year’s time.

Her whole IEP was written like this. I can’t fathom why the District didn’t settle the case; if I’d been the District’s director of special education, I would have been mortified for this case to go before a Judge.

I distinctly recall sitting in the hearing and watching the Judge shake the IEP in the air at the Program Coordinator from the District who was testifying at the time, demanding, “You just testified that it’s your job to make sure IEPs are written properly by your staff.? How do you explain yourself?” She started to cry. He’d had a box of tissue brought in right before she began her testimony and shoved it in her direction as he threw the IEP back down on his table in disgust. The parent and I certainly felt vindicated. We’d made that same argument at the IEP level and it hadn’t gotten us anywhere.

Conversely, here is a properly written present levels statement from a real IEP: [Student] has difficulty recognizing and explaining how words are related, as demonstrated on the CELF-4. His responses tend to be vague and do not identify the most important elements. Word Classes Total: Percentile Rank 1, WC Receptive PR=2, WC Expressive Subtest PR=4. Verbal analogies and quantity vocabulary are areas of particular need for [Student]. He also confuses words that are phonologically similar (e.g., cricket, crooked). He needs to learn to hear the differences in the sounds of the words and recognize salient information.”

Granted, it took a lot to arrive at an IEP with great language like this in it, but once it was all said in done, this child’s IEP was truly a document worth enforcing and we were able to avoid litigation altogether. He’s being doing great ever since.

That was from a couple of years ago. Here’s another good example of sound present levels statements taken from a recent IEP for a student that I attended earlier this month: “[Student] has difficulty comprehending his own reading and that of others. He is often unable to answer surface questions about the story he or others are reading aloud. When a simple passage is read to [Student] and he is asked to answer 10 comprehension questions [Student] answers 4 out of 10 correct. When [Student] has read the passage and asked to answer 10 comprehension questions in writing [Student] correctly answered 6 out or 10 questions.?An SRA reading/comprehension assessment found [Student] answering 4 out of 40 comprehension questions.”

You will note that the present levels statements I’ve cited as being relatively good are much longer than the one I cited as being bad. But, just because a present levels statement has a lot of words in it, that doesn’t mean it really says anything of value. It’s easy to say a lot of nothing with a lot of words.

You will notice that these good present levels statements include numbers. It’s important to appreciate the logic of basing IEP goals on empirical data. Measurability, which is required of annual goals, means that you have to count something. IEPs have to be reasonably calculated to render educational benefit. You can’t count or calculate anything without numbers.

Imagine hiring a contractor to build a privacy wall along the side of your property. If the contractor came out to your house and “eyeballed it” rather than taking measurements and diagramming out his work in advance, and failed to mark out with stakes and string where to dig for the foundation of the wall according to his measurements and diagrams, what would be the likelihood of you actually letting this guy tear up your yard, pour concrete, and stack bricks along your property line? What do you think the finished wall would look like if the contractor were to just “eyeball it” along the way rather than have taken measurements and worked off of them throughout the project?

If you wouldn’t dream of letting a contractor “eyeball it” on a wall in your yard, why on Earth would you trust anything less empirical with your children’s or your students’ educations? There is no room for vague, wishy-washy language when it comes to describing what a child can and cannot do at the time an IEP is written. That is the foundation upon which everything else is built.

I hope this information helps you better understand present levels of performance. Please do comment and let us know if you have questions about anything discussed in today’s posting or have an example of your own to share.