Mike et al,
What is normal evaluation anyways? I don't have a favorite piece but tend
toward the utilization focused approach.
i feel for you cuz, it's such a controverted problem with after school
programs. There is the assumption that such would tap indirectly on the big
three determinants of school ach.
Mentors/tutors are supplanting personnel and activities that for the
middle class reflect the parent involvement effect. One would think too that
the teacher student ratio is also being tapped at least for a few hours a
month and that time on tasks connect with some of the formal
content/instruction in school.
But how does one really know ASProgs. are useful/effective?
Would they have to be organized around "homework" and projects where the
class-based parent involvement differential grows.? Yes if its improvement
in grades we seek....or
should they be organized to assist in knowledge/skills tests like the CTBS
tap?? rarely
should the evaluation be "goal free" (Scriven)
and inclusive of others forms of positive development that are not
necessarily test driven?
Should social development be counted? or character education? (with all its
nuances?)
In planning to do an eval. for a couple of after school programs here in
Jefferson Co. that are being sustained by the school district, I found
little to go on because of the service oriented mentaility that prevails.
Given scarce resources, knowing what works and why seems a curiously low
priority. The "important" thing seems to be to launch and "have" such
programs EXIST and to show that something is being done for them, ie those
placed at risk.
After years of implementation, no one knows if such programs (or which ones)
work or why. The scientific approach now seems passe' unless some
accountability oriented source is willing to fund evaluation, independent or
not.
In proposing a new approach here recently, it was clear that what is already
going on needs to be assessed, at least for a baseline in judging a new
prog. a la college student pipeline notion.
The fact that the district already had ASPs was a plus yet the proposal was
turned down...
Comparing something to naught was the 1st step before looking more
carefully into what the somethings were...
That would have been the rationale for the eval yet could it be that we are
living in a late modern climate where R & D activities related to reducing
the gap via such logical initiatives are no longer a priority. Development
yes but not evaluation research unless there is a political economy behind
it.
A longitudinal design is sorely needed for a meaningful evaluation but the $
seems to go elsewhere for service, mediators etc usually. But even if we
found some, who then evaluates, what drives the evaluation, who defined the
goals, the criteria for success? who consumes the eval. results and what
comes of it?
Just look at the class reduction literature. Even if there is evidence that
it tends to help hose on free lunch the most...policy wise, don't see much
coming from those evaluations net of the 12.4 billion to produce more
teachers (100k) and the wish for class sizes of 18 in elementary grades in
some districts..
(when everyone almost knows that above 15-16 it does'nt make that much of a
difference..), Yep, that's how we choose to organize it..
In any case, to stop the ranting... it seems to me that normal evaluation,
by which i understand ;
that evaluation is to be connected with the service delivery activities
before in the planning, during the initial trials and implementation, and
thereafter at regular periods in a feedback loop for utilization
is not so bad if it were actually (well) carried out...
What has caused this impasse or demonization of all things scientific? The
demise of log. positivism and its errors? the idea that evaluations are
themselves constructed and can be at any time with any method? for certain
communities by certain communities for particular purposes etc..
What should the eval. questions center on?
How much AfterSchoolProgramming is needed to make a difference? For How
long will a positive effect last if assisted development is not maintained?
If reducing the literacy gap is the goal, then what should a normal or not
so normal evaluation measure? ach.. beliefs, motivation related changes?
Have not seen much either from the National Center for Family Literacy that
is solid, lots of projects and testimomials, cases and tons of un-evaluated
data..
So no good news here, only empathy...pedro
----- Original Message -----
From: "Mike Cole" <mcole@weber.ucsd.edu>
To: <xmca@weber.ucsd.edu>
Sent: Sunday, August 26, 2001 6:09 PM
Subject: Community Literacy Programs
> Thanks for the Grabill tip, Peter. I have ordered the book from Amazon
> for 15 bucks.
>
> The more I looking into evaluation issues for outside of
> school programs, where there is voluntary participation, the
> more convinced I become that the problem cannot be solved by
> normal evaluation means, and the curiousor and curiousor the
> work arounds I am discovering become.
> If anyone else is working with such situations and has
> some favorite piece on evaluation to suggest, it would be most
> appreciated.
> mike
>
This archive was generated by hypermail 2b29 : Mon Oct 01 2001 - 01:02:20 PDT