National External Moderation Summary Report for 2008, Semester 2

Prescription: 644 Buyer Behaviour and Communication Strategies

Introduction

This report provides a national perspective on the moderation of 644 Buyer Behaviour and Communication Strategies

Assessment materials from seven Tertiary Education Organisations (TEOs) were moderated for this prescription. A new moderation report format was piloted, which required moderators to consider four key questions:

do assessment activities meet prescription requirements?
are assessments clear, appropriate and fair to learners?
is there sufficient guidance provided for assessors in assessment schedules?
are assessor decisions about learner work consistent?

Of the seven submissions for 644 Buyer Behaviour and Communication Strategies :

one met all key assessment requirements
another one met key assessment requirements but required some modification before next course delivery
the remaining five did not meet one or more key assessment requirements and modification/redevelopment before next course delivery, and post-assessment resubmission is required.

Key areas where modification or redevelopment was required were:

assessment activities not meeting prescription requirements (three submissions) generally owing to a high percentage of marks being allocated to non-prescription activities (presentation and referencing) and therefore weightings being outside of the aggregate variance allowed
insufficient guidance provided for assessors in assessment schedules for allocation of partial marks (five submissions).

Presentation of assessment materials

Only one provider did not submit all the required materials. However, in several cases, presentation of materials made it difficult to locate material in the submissions. It is recommended that material is placed in the same order as the submission checklist and that colour coded and labelled separating pages are used.

Assessment grids

All providers submitted assessment grids. However, in some cases, it appeared that the grid was completed after assessments had been designed, as weightings in the assessments did not reflect the weightings in the grid.

Assessment activities

The key considerations for moderators in deciding whether prescription requirements were met were whether tasks:

assessed all prescription learning outcomes, with appropriate weightings
were at the appropriate level.

Learning outcomes

Learning outcomes indicate assessment outcomes and specify what learners need to know and be able to do. Key elements indicate assessment coverage and specify how the related learning outcome should be evidenced. Key element assessment evidence should be provided in the context of the learning outcome.

A 10% aggregate variance is allowed in assessment weightings. That is, the percentage variation in total across all learning outcomes should not be more than 10%.

One submission showed no evidence of learning outcome three being assessed.

Four submissions exceeded the 10% aggregate variance, in one case only marginally, but in three cases by 5-10%. The allocation of marks to presentation and referencing was a contributing factor. An NZQA guideline regarding this is available in tertiary circular T2009/001, 3.1.2 (see links opposite).

Two submissions did not cover both the benefits and limitations of segmentation (learning outcome two, key element a) and in two cases, organisational buyer behavior appeared not to have been assessed. This is because student topic choice would dictate whether organisational or consumer buyer behavior was assessed.

Level

In all cases, learners were given the chance to apply their learning to a real case that involved a level of complexity and problem solving characteristic of a level 6 paper. However, it is noted that, in several cases, the prescription requirement that learners justify their answer was not clearly specified in instructions and marking guides.

General guidance about prescription format is available in tertiary circular T2009/004, Appendix 1.

Assessment conditions and instructions

The key consideration for moderators was whether assessment conditions and instructions were clear, appropriate and fair to learners.

There were a variety of issues with respect to assessment conditions and instructions. Two of the main issues were:

two submissions included statements as to how certain marks would be awarded in the assessment instructions; however, this was not reflected in the marking guide
other submissions included typographical errors and spelling mistakes which could have a significant impact on comprehension by second language learners.

Assessment schedules

The key considerations for moderators in deciding whether sufficient guidance was provided for assessors were whether schedules:

gave examples of learner responses (e.g. model answers, and/or a range of appropriate answers, and/or quality criteria for answers) that met prescription requirements
provided a sufficiently detailed breakdown of marks to ensure consistent marking.

Although marking guides cannot be overly prescriptive, the majority of marking guides required considerable improvement. The main issue was a lack of quality criteria with which to differentiate between learner answers. Although most marking guides provided guidance as to content requirement, the guides provided in five submissions lacked sufficiently detailed criteria or breakdown of marks with which to differentiate between the quality of learner answers. In one case, the opposite was evident where there were very good quality criteria, but no guidelines as to the type of content required. Some submissions contained model/sample answers, which is acceptable provided sufficient scope for variations in answers is provided.

Assessor decisions

The key considerations for moderators in deciding whether assessor decisions were consistent were whether:

marking rewarded a similar quality of work with similar marks
marking rewarded learner work in a manner consistent with prescription requirements.

There were no major issues in this respect. However, the flawed marking guides, as noted above, made verification of consistency of marking between learners and with the prescription difficult.

Conclusions

Overall, the prescription was adequately assessed at the correct level. Learners were given the opportunity to apply their learning to a selection of varied and interesting cases. However, there were deficiencies in the marking guides provided in all but one submission, which affected the reliability of marking guides. Consistency of assessor decisions can therefore not be assured.

Page updated: 16 June 2009