International expert provides evaluation advice

An internationally recognised expert in evaluation, Aucklander Jane Davidson is helping shape an evaluative approach to quality assurance which is to be introduced across the tertiary sector as part of the government's new investment system for tertiary education.

Dr Davidson is a member of an Expert Advisory Group established earlier this year to provide advice on current good practice in quality assurance.

She is well qualified for her role, having launched and directed the world's first fully interdisciplinary PhD in evaluation at Western Michigan University in the United States. She is also the author of a textbook used internationally in post-graduate evaluation courses.

E-valu-ation involves collecting descriptive information about an educational course or programme, using "values" to determine what information should be collected, and drawing explicitly evaluative inferences from the data. The inferences say something about the quality, value or importance of the course or programme, Dr Davidson says.

"Put it another way, research can tell us 'what's so' but only evaluation can tell us 'so what?'," she says.

And it's that "so what?" that is really important, says NZQA Deputy Chief Executive, Quality Assurance, Mike Willing.

"So what are the programmes being delivered by Tertiary Education Organisations (TEOs) achieving in terms of outcomes for learners and the broader economy? What is New Zealand getting for its investment in tertiary education?"

This outcome focus marks a shift away from an emphasis on increasing participation in tertiary education, although ensuring access remains important.

NZQA has been charged with developing the quality assurance part of the wider reforms. The judgements about quality made by NZQA, and other quality assurance bodies, will feed into investment decisions made by TEC.

What does an evaluative approach look like?

While all organisations undertake quality assurance, certain underlying attitudes, beliefs and values can be seen in those organisations with an evaluative approach to quality assurance according to Dr Davidson.

"An organisation that 'gets' evaluation believes it is valuable to distinguish good from poor performance or quality. It believes that you can't attain excellence without serious feedback and critique. It embraces an evaluative attitude and a real need to know the truth about quality and how good programmes are. It values diverse perspectives on quality including external views."

Dr Davidson says there is no precise prescription for getting people to share the attitudes, beliefs and values that create an evaluative organisational culture.

"It's like a flock of birds in flight. There's no manual but rather a shared consciousness of what they're trying to do to get to their desired destination," she says.

There may not be a manual or a tick-box list of instructions but evaluation is a distinct approach and discipline, she says. The move to an evaluative approach in New Zealand reflects international trends where organisations manage their own quality, and use the process and results to improve educational outcomes for learners and the broader community, says Dr Davidson.

The way ahead

Several pieces of work are underway at NZQA to develop the new evaluative approach to quality assurance and a small trial of evaluative self-assessment tools. The periodic external evaluation and review of key system components will be carried out in 2008.

The tertiary reforms will roll out over several years. It’s expected TEOs will start evaluative self-assessment to some degree in 2009, and by the end of 2013, all will be engaging in robust self-assessment and will have completed a full cycle of external evaluation and review.