METRIC: ASSESSMENT AND FEEDBACK IN IMPROVISATION TEACHING

In the development of the joint modules in the framework of the METRIC Project, an important point of discussion was a common approach to assessment. This discussion resulted in the development of two important outcomes: a set of common assessment criteria for improvisation and an approach towards the exchange of (international) external examiners.

Methodology

The project working group suggests that the METRIC assessment criteria will be used during assessments taking place in the METRIC joint modules, but they could also be used by the partner institutions during assessments of their own improvisation courses. Furthermore, in order to maintain a common approach to the assessment criteria, it is also suggested that the METRIC partners make use of (international) external examiners from the consortium partners, who will be members of the assessment panels for the joint modules and the institutional improvisation courses.

Evaluation areas

The criteria below represent a framework for the assessment of improvisation in higher education and, more specifically, for use in the international joint modules being developed in the METRIC project. Although hotly debated in academia, assessing creativity has largely been acknowledged as a valid, helpful and deeply artistic form of feedback for students engaged in the study of improvisation.

The criteria developed and finally arrived at here are the product of several years of research carried out by members of the METRIC consortium. The members of METRIC represent decades of experience in the teaching, performance and assessment of improvisation. While a wide range of stylistic approaches to improvisation are represented within the METRIC group, common musical values and approaches eventually emerged, making it possible to propose this as a specific framework for the assessment of improvisation.

Those completing modules in improvisation will be expected to demonstrate:

Musicality

Richness and creativity of ideas
Tonal control and variety
Awareness and control of form
Development of material
Dynamic sensitivity
Sense of flow and shape in phrasing
Rhythmic control and character
Contrast
Originality
Fantasy

Performance

Expressive intent
Instrumental control, technical confidence
Concentration
Control and variation of energy
Clarity of ideas
Dynamic control and variety
Variety of touch and articulation
Sense of character and commitment
Freedom
Virtuosity

Collaborative and Solo skills

Control of texture
Speed of thinking
Responsive sense of ensemble, listening and interplay
Projection
Awareness of function – solo, leading, background, provocation, silence
Presence and purpose in ensemble

Stylistic awareness

Stylistic authority and command
Personal language, confident identity
Taste
Command of tonal or atonal genres
Adhere to, cross bridges or push stylistic boundaries

Reflection

Perceptive and accurate awareness of performance
Aesthetic and philosophical awareness
Critical openness and flexibility
Insightful peer feedback

[> Click here to download the assessment table]

METRIC and (International) External Examiners

In this paragraph it is described what exactly is meant with the term International External Examiner, what the relevance is for institutions in the use of such examiners, how the use of International External Examiners relates to the developed assessment criteria and which practical details should be considered when operating a system that uses International External Examiners.

What do we exactly mean with an International External Examiner? In the document ‘International External Examiners in Higher Music Education: Role, Purpose and Case Studies’ published by the ERASMUS Network for Music ‘Polifonia’, information is provided on the various types of external examiners. As has been identified through research in the ‘Polifonia’ project, there are many different practices in the use of International External Examiners and their role(s) in relation to the assessment of student performance.

In order to avoid confusion with the terminology used in some countries where a tradition of external examiners/evaluators/assessors exists (e.g. in the UK), it is important to be clear about the different roles and types of external examiners:

In the context of the METRIC joint modules or the improvisation courses of METRIC partners, an external examiner is normally a specialist in improvisation with the task to serve on assessment panels in formative and/or summative performance assessments.
In some countries, another practice exists in which an external expert (often called an ‘overall external examiner’ or a ‘programme external examiner’) is overseeing final examinations in different departments as a generalist to examine the overall comparability of standards and procedures between departments within one single degree.
The meaning of the word ‘external’ only refers to the fact that the external examiners comes from another institution.

Whereas the ‘overall external examiner’ can be seen more as a quality assurance tool to review the institution’s internal standards and procedures for assessment, the specialist external examiner will be more directly involved with the actual assessment of individual students. Therefore, the profile of such a specialist external examiner will be more connected to the actual content of the study programmes and its curriculum, and it is this type of external examiner that will be active in the framework of the METRIC joint modules or the improvisation courses in METRIC partner institutions.

Purpose: why International External Examiners?

The presence of International External Examiners will be important to ensure that the assessment criteria will be applied in the same way across METRIC joint modules. As to the existing improvisation courses in METRIC partner institutions, the presence of the International External Examiners will also facilitate the exchange of expertise between teachers and can play a role in the continuing professional development of teachers with regards to assessment and feedback. This way, international collaboration on assessment is an effective way of enhancing teachers’ expertise through engaging with colleagues at international level and gaining understanding of other assessment systems. Teachers gain experience of different pedagogic approaches but above all learn to develop their own ideas of what standards are applied internationally.

There are also other more general benefits in the use of the International External Examiners:

Music is a very specialist discipline and there may be only a small number of instrumental practitioners in a particular country. It is therefore possible that cross-institutional assessments with institutions within one country will have only limited benefit in terms of objectivity. The engagement of International External Examiners enlarges the number of practitioners and thus enhances objectivity.
Higher music education is very much an international discipline. To ensure that the programmes are continually updated in line with the requirements of international professional practice it is not only important that there are foreign students and teachers present in the institution, but also that the study programmes are continuously benchmarked at international level. International cross-institutional assessment can provide valuable information as to a programme’s outcomes in relation to international standards.
Lastly, this international approach can make a positive contribution to the accreditation and programme or institutional review procedures institutions will have to undergo as part of their national quality assurance systems. Not only can this approach show stronger links between the assessment, quality assurance and internationalisation policies of the institutions involved, but also provide an useful tool for comparing standards of student achievement at the international level.

International external examiners and the METRIC Assessment Criteria

A crucial requirement is a careful preparation of a visit by an international external examiner. It is important that the international external examiner is informed fully about the assessment procedure and the grading system in advance of the examinations. Even in the case when the agreed METRIC Assessment Criteria are being used, additional marking/grading criteria provided by the host institution may be essential tools for the external examiner. It is expected that over time teachers will learn about the assessment practices in other METRIC institutions, but even so there may be changes or developments that he/she should be informed about.

When using the METRIC Assessment Criteria, it may be suggested to use a common methodology with regards to the actual assessment process. There are two ways in which the criteria can be used:

As a tool for feedback. In this case, the Assessment Criteria will be used as a basis for an assessment and a subsequent written or aural feedback to the students. It may also be considered to give copies of the criteria to the students beforehand, which would be the most transparent way of informing students about the assessment process.
As a tool for feedback and grading. Apart from using the criteria to do an assessment and give feedback, there may be a situation in which the performance of the student needs to be graded. This will then need to be done with the grading scale as used in that particular institution and the table as presented underneath the assessment criteria above can be used as an example. It is recommended to relate the institutional grading scale to the ECTS Grading Scale when the conversion of grades is required. With regards to grading, it is suggested that the members of the panel first starts with blind marking (i.e. putting a grade on a piece of paper confidentially and handing this paper to the panel chair), which is then followed by the announcement of the chair of the grades given and by a discussion of the panel of the performance and the subsequent grading. This discussion should be based on the common assessment criteria. If the grades of the panel members differ substantially from each other, the panel chair may ask (following the discussion) for a second round of blind marking. Usually this will resolve any differences, but if not, the panel chair should be in the position to propose a final grade. In any case, the final assessment and grading should always be related to the assessment criteria, so that, in case a student asks for a further clarification, a reference to the assessment criteria can always be made.

Practical considerations

There may be concerns of institutions regarding the high costs when using International External Examiners. Here are some further thoughts on this issue:

The use of the regular teaching staff exchanges in ERASMUS+ can be considered for this purpose on the basis that the visiting teacher is carrying out a role in the evaluation of teaching. At the moment, there are no regulations that would contradict this assumption, as the regulations usually only speak of limits in terms of time to be spent at the host institution. It is prudent to check this with the ERASMUS+ National Agency beforehand. A solution could be to combine the role of the teacher as an international external examiner with a few hours of teaching in the host institution. If ERASMUS+ can cover the costs of the international external examiner, than the international external examiner could even be more financially attractive than employing one from within the same country.
Another financial arrangement could be to use the principle of ‘closed purses’. This means that the sending institutions pays the travel costs, while the hosting institution covers local costs (hotel and meals).
In some countries it is standard practice to pay external examiners a fee. If this is the case, the international external examiner should be treated in the same way. It is important to agree very precisely on such details beforehand.

The criteria for the assessment of improvisation cover 5 areas for evaluation.

They are:

Musicality
Performance
Collaborative and solo skills
Stylistic awareness
Reflection