Development of the IAG reference material programme
In the early years of the IAG, the relatively poor practices then current in the development of geological reference materials were discussed. Up to that time, very few organisations characterised certified reference materials, NIST being an important exception, but a large number of other reference materials were available that made an essential contribution to the characterisation of geoanalytical measurement results. In this respect, the USGS and CRPG (as well as other organisations) were very active. At the same time, REMCO, the ISO committee of reference materials, was developing guides that described good practice in the characterisation of reference materials. These guides were designed to be applicable to all branches of science, not just to the field of geoanalysis.
It occurred to a number of us that the geological community was no longer following best practice in programmes for the development of geological reference materials. Issues of concern were:
- The frequent contribution of results to characterisation programmes by laboratories that had not been evaluated for competence or asked to provide validation data.
- The frequent practice of periodically revising reference values based on compilations of data abstracted from the research literature, including results which had themselves often been published to demonstrate compliance with the results of previous compilations! These revisions were often undertaken without the involvement or approval of the original issuing organisation.
- The problem that often there was no proper assessment of uncertainties in reference values, with estimates often based on a simple statistical evaluation of results with outlier rejection.
In an attempt to resolve these issues, we organised a workshop at GFZ Potsdam in November 2002 that was attended by Phil Potts, Jean Kane, Jean Carignan and Michael Wiedenbeck. The result of these discussions was the development and subsequent publication of the IAG Certification Protocol (Kane et al., 2003), with Jean Kane as the lead author based on her extensive experience in the development of reference materials at both NIST and the USGS. The main features of this protocol were: (i) the qualification of laboratories invited to participate on the basis of their recent performance in the GeoPT proficiency testing programme; (ii) the requirement to co-analyse an established reference material to link results with the existing geological metrology system; (iii) the requirement of laboratories to undertake replicate measurements on multiple test portions; leading to (iv) a full and comprehensive assessment of all sources of uncertainty.
Having established this protocol, it was important to assess it in practice. The first opportunity arose in the certification of OU-6 (Penrhyn Slate), a material that had been collected from North Wales and prepared as a powder at The Open University. This material had previously been circulated in 2001 as the GeoPT 9 test material, which simplified the selection of laboratories invited to contribute further measurement results to its certification. The resultant certificate of analysis was published in 2004.
The next stage in the development of IAG reference materials grew out of the increased popularity of the GeoPT proficiency testing programme, which benefitted from regular participation by over 100 laboratories worldwide. In addition, meticulous procedures had been developed for the assessment of data distributions in GeoPT, to identify the consensus value that represented the best estimate of the true value, largely based on the statistical expertise of Mike Thompson. At the time, other organisations were distributing similar materials as ‘quality assurance’ materials. However, we realised that consensus ‘assigned’ values derived from GeoPT were fully compliant with the definition of reference values associated with reference materials. Where surplus material was available from the GeoPT programme, reference material data sheets were developed and these materials made available to the wider geoanalytical community through IAGeo Limited. A relatively conservative approach was taken in deciding which values should be listed on these reference material data sheets. GeoPT ‘assigned’ values were allocated as reference values, ‘provisional’ values as information/indicative values and GeoPT ‘information’ values might not be listed at all. Although the reference material data sheet summarised key information, it benefitted from the corresponding GeoPT report in giving the users of these reference materials full access to all the source data and assessment procedures.
The next development was stimulated by the publication by ISO-REMCO of a revised version of Guide 35:2017 (Reference Materials: Guidance for characterisation and assessment of homogeneity and stability). Traditionally, REMCO had recommended the certification of reference material using an inter-laboratory study with significant emphasis on the use of ‘primary’ analytical procedures. These are procedures that provide a route for tracing measurement results directly to the relevant SI base unit. However, this 2017 guide accepted for the first time that certification was possible through proficiency testing programmes. This proposal was subject to various criteria, which were not particularly well described in the Guide. The issue for the IAG was – could this certification approach be adopted using the GeoPT programme?
Two IAG workshops were organised by Phil Potts at Chicheley Hall in 2014 and Horwood House in 2016 to explore the possibilities. There were a number of issues that needed to be resolved if GeoPT assigned values could be ascribed as certified values. The most important were: (i) is it possible to demonstrate the traceability of GeoPT assigned values? (ii) in circumstances where, for some elements, most GeoPT data is contributed by a single technique, how can confidence be gained that assigned values are not influenced by technique bias? and (iii) do GeoPT uncertainties account for all uncertainty contributions?
The culmination of these workshop discussions, with contributions from both members of the IAG Certification Committee and guest speakers, led to the publication of a protocol for the certification of geological materials using the GeoPT programme (Potts et al., 2019). In this protocol, it was argued that:
- Although it is not possible to demonstrate the traceability of an entire proficiency testing procedure (traceability is the property of a measurement result), the properties of the GeoPT scheme had been demonstrated on a significant number of occasions when established certified geological reference materials (with traceable certified values) had been used anonymously as GeoPT test materials. In addition, traceability of individual results could be demonstrated by participating laboratories that were accredited or had appropriate validation/calibration procedures, although this information was not traditionally recorded in the GeoPT scheme.
- Procedural bias in the GeoPT scheme had been observed on a number of occasions, most notably associated with either incomplete digestion of refractory mineral grains, such as zircon, leading to variably low measurement results or positive bias in the reporting of XRF data as concentrations approached the detection limit. In these cases, the degree of bias reported by different laboratories was variable, leading to a clearly distinguishable tailing of the data distribution and allowing appropriate judgement to be made in identifying the true consensus value.
- In terms of uncertainties, one of the characteristics of the GeoPT programme with its large number of participating laboratories is that all forms of uncertainty, including inhomogeneity effects, will affect the data distribution, so that the derived standard error of data selected to define the consensus value represents a realistic assessment of the uncertainty in the certified value.
The second of these workshops also discussed ways in which this certification protocol could be applied to the certification of microbeam materials, characterised using the G-Probe programme. In this application there are issues still to be resolved.
With all these developments, the IAG has been able to demonstrate leadership in the development and characterisation of reference materials, to the benefit of the geoanalytical community.
Full references to the papers cited above and others resulting from IAG activities in the field of reference materials and their certification can be found here.
Phil Potts (February 2021)