Thanks a lot for your very informative report!
Eugene
-----Original Message-----
From: Bill Penuel <bpenuel who-is-at unix.sri.com>
To: ematusov who-is-at UDel.Edu <ematusov@UDel.Edu>
Date: Monday, March 01, 1999 11:07 AM
Subject: RE: assessment and evaluation
>Hi Eugene and everybody--
>
>Yes, I am still in th program evaluation business, but now with SRI. My new
>position also signifies a different relation to stakeholders,
>accountability, and the like, that may enable me to answer your questions a
>with another set of perspectives.
>
>You ask:
> Anyway, I wonder if in negotiation with evaluation
>>stakeholders you use existing types of program assessment or you have to
>>develop new or modify existing ones to find a compromise in diverse and,
>>probably, often conflicting goals of the stakeholders?
>
>In the school program evaluation business, you have much less flexibility
>in this regard, I think, at the district level, though the short answer
>there is that it depends. One function of school-system based evaluators
>is accountability. In San Francisco, for example, all programs funded by
>the district are "taxed" a 5% evaluation fee if their aim is to change
>teaching or learning for the better (interestingly, most professional
>development initiatives were exempt, however). This tax then goes into an
>evaluation that in some fashion must measure student learning in terms of
>standardized achievement scores, which are then used to determine continued
>funding for programs by the Superintendent.
>
>There are considerable differences in program type, though, and we had some
>flexibility with regard to either inventing some scales of measurement in
>partnership with programs for assessing growth or using another standard
>measure (e.g. attendance) as the primary benchmark.
>
>But our position in the end makes for some odd compromises--we can
>participate in the development of these measures with one group of
>stakeholders (usually program managers) but they will then be held
>accountable for these measures by another group (usually the
>Superintendent).
>
>
>Working for a private non-profit, you often have fewer "accountability"
>strings to pull as an evaluator. Rather, there are other stakeholder
>conflicts, perhaps notably between the client (who is often both paying for
>your services and running the program) and the research community, with its
>standards of objectivity, neutrality, etc. Of course, these standards are
>shifting to reflect the realities of constructing evaluative research....
>
>Also, have you
>>noticed that stakeholders change their positions during the process of
>>deciding of what are the goals and how to assess the program?
>
>Yes--I think this happens a lot, and I know that's something you've been
>interested in in the past. When you have several years of an evaluation as
>we do at SRI on many projects, we have the advantage of being able to
>change and adapt mid-stream. The best evaluative research does this as a
>matter of course, unless the primary purpose of it is to sit on a shelf.
>But for short-term programs, or when other stakeholder groups define the
>end goals (e.g. increased test scores), this is much harder to do.
>
>Bill
>----------------------------------------------------------------------
>Bill Penuel, PhD
>Research Social Scientist
>Center for Technology in Learning
>SRI International
>333 Ravenswood Avenue, Mailstop BS116
>Menlo Park, CA 94025
>tel: 650-859-5001
>fax: 650-859-4605
>
>Check out our websites at:
>
>
>
>http://www.sri.com/policy/ctl/
>
>http://www.cilt.org
>----------------------------------------------------------------------