The bigger picture
Is conformity assessment becoming more difficult – even for the best auditors? Increasing questions from auditees about governance, corporate responsibility and reputation risk management issues are putting new strains on assessors at the same time as the boundaries between quality, safety and environmental issues are blurring. And is the assessment community losing sight of the underlying principles of the ISO management system standards? Fraser Paterson asks: how do we rebuild credibility and start adding real value to the certification process?
Since the start of this decade, there have been several key developments among the main conformity assessment standards. Firstly, ISO 9001 emerged with its brand new process-based approach. Then others began to follow suit with, for example, QS 9000 morphing into ISO/TS 16949. The various quality and environmental management system auditing specifications were rolled up into a rather uninspiring ISO 19011 and, most recently, ISO 14001:2004 appeared with what on the surface were only fairly minor changes and a clearer definition of some of its requirements.
At the same time, certification bodies and registrars have been adapting to an evolving market-place and trying to add real value to client offerings. In addition to developing capabilities in integrated management systems assessment and risk-based auditing, they have been branching out into areas like personnel accreditation, verification of emissions trading schemes, and providing assurance for corporate non-financial and sustainability reporting. Further lucrative opportunities will undoubtedly present themselves in emerging disciplines like anti-corruption, where there is growing recognition that independent assessors will be essential for the success of integrity management systems and transparent pacts between all parties involved in development projects.
What’s the progress?
But how much real progress is being made? The debate about the credibility and value of certification continues with little sign of resolution and there is a certain amount of frustration on both sides of the auditor/auditee relationship. In some respects, it has been a difficult time, with increasing expectations of auditors from both their clients and their employers. Aligned, integrated and risk-based auditing sounds fairly straightforward in theory, but are we extending the individual auditor’s skills or diluting them? It is obvious that a fair proportion of quality management system (QMS) assessors have found the transition to the process approach difficult. For example, in 2002, persistent rumours circulated in the UK of extended stress leave among ISO 9001 auditors who found it hard to get to grip with issues like applicable statutory and regulatory requirements. But it is the conformity assessment industry’s lack of real buy in to the process approach which is the most significant failing.
But it is not just auditors that are at fault. Clients, certification bodies and accreditation agencies have unreasonable expectations of what can be achieved using the current process model. Is the assessment process itself fundamentally flawed and ripe for an overhaul?
Calibre of assessors
For whatever reason, conformity assessment to ISO’s management system standards is now a commodity service. A simple equation – complexity of business, number of sites and number of employees equals a certain number of assessor man-days at whatever rate per day – is often used to calculate what to charge the customer. Certification bodies compete as much on price as they do on the quality of their product and rigid control over costs is a vital element in maintaining competitive edge. Accordingly, one of the key performance indicators needs to be high auditor utilization rates. In the UK, at least, generally substantial overheads and an overly expensive accreditation process compel them to cap manpower costs. This is manifested in the levels of remuneration they offer assessors. While wages and benefits rates may be described as fair and comparable, is auditor pay set too low to attract or retain personnel of sufficient calibre to meet the evolving expectations of the clientele – let alone allow them to repeat the mantra of adding value? Admittedly, it can takes a peculiar type of individual to put up with the varied demands of constant travel, applying international standards and sector knowledge, having to produce error–free reports that will pass muster in internal and external scrutiny before leaving the premises, not to mention working alone and always in the face of a motley assortment of exacting customers.
If auditing were a truly valued profession, certification bodies would pay salaries closer to the levels that corporates and consultants offer. Remuneration packages do attract technically skilled auditors, but such recruits are not always especially business-aware or results-oriented. Once appointed, the best calibre candidates often quit the profession for better paid jobs on the other side of the divide, leaving the steady, reliable types who appear contented in their role, enjoying the sometimes fawning attention of the auditee and opportunities to learn from all types of business, and make their comments before disappearing at the end of the day with no further responsibility to that client until the next visit six months later.
Too many assessors lack the real interpersonal skills or experience to engage effectively with both main board directors and the shop floor workers. Too often, there is insufficient interaction with senior managers and main board directors. Are auditors reluctant to engage top management, or do they lack the preparation, training or gravitas to interface at this level? If assessors do not have sufficient intellectual power or political or cultural savvy, how can we expect them to bring about real change in client organizations? Part of this problem must be linked to the raw material quality, but it is likely that the certification bodies’ training and development schemes are another factor.
Auditor training
Technically – in terms of knowledge of standards and technical/compliance factors – training is generally of an acceptable if not good standard. But there are limitations. One example of this can be found where a company is registered to both ISO 9001 and ISO 14001. Despite claims of an aligned or integrated service, in practice there is often a distinct lack of teamwork and even a ‘disconnect’ between quality and environmental assessors. At best, it is accidental and can result in mixed messages to the auditee; at worst, the different disciplines push the auditee in conflicting directions and confusion reigns.
How can the same company pass 12 man-days of quality assessment, yet be found to be seriously adrift of compliance in respect of its identification of supplier or contractor competences, communicating its requirements and monitoring their performance by an environmental auditor whose assignment was only three days? What impression is an intelligent auditee left with? This type of situation is partly down to the imposed system - two certifications equals two separate reports, or one report with two components – and an absence of really effective teamwork.
Assessor training seems to cover the technical aspects of integrated audits, but does not appear to address fully the practicalities of different auditors working together in an effective team. Many in the assessment community now believe that there will be a trend towards fewer audits, but that they will be longer and of a multi-disciplinary character. A second area where the assessment process can fall down is where an auditor has to think outside of his comfort zone. When a client is paying up to US$1,000 per day for an assessor, he naturally expects value for money – an expert at least, if not a ‘super-auditor’.
It is therefore reasonable to believe that a quality assessor should be able to answer questions on management techniques like balanced scorecards and six sigma. Some QMS auditors – but not all – can. However, the stock response of ‘we have someone in our office who does’ fails to impress. The process for compiling certification bodies’ training needs and competency matrices seems to be somewhat flawed.
Subcontract assessors
Outsourcing is a recognized and well-established management strategy. Many certification bodies regularly subcontract work to associates and one major organization has even been rumoured to have tried to convert all its assessor employees into contractors. As associate rates can be as little as a third or less of the daily charge out fee to the client, their use can also be good business. But is their use another symptom of management system certification as a commodity product, or can it add value to the process?
There are pros and cons: On the plus side, associates can help a certification body meet peak demand, or provide specialist expertise that it would not be cost-effective to maintain in-house. As they do not assess on a day-in, day-out basis, their approach may be fresher and they are able to bring their wider business experience to the party. On the flip side, subcontracted auditors are cheap and plentiful, grateful to pick up work that only requires them to turn up, blast out a quick report and then disappear without any real long term responsibility to either the certification body or its client.
But how do clients perceive them? It can be quite amusing when you ask several certification bodies to tender for a contract and find the same associate included in two or more of their bids. Mind you, the real fun starts when you invite them for interview and try to guess which certification body has secured his or her services for the day. There is obviously a fine balance between exploitation and service delivery. Certainly, there are a lot of truly outstanding associates out there and the pity is that, as with assessor recruitment, remuneration levels do not always ensure that the best and most qualified people are secured for assessment work.
Auditing techniques
The days when clients accept detailed but fairly low level assessment reports are drawing to a close. Most auditees already know that their management system is imperfect and that anyone can find the errors, omissions and inaccuracies that get reported as non-conformance. Highlighting comparatively trivial matters neither impresses senior management nor adds value to the business of improving performance.
The apparent need to have auditors leave a report with the client before leaving site is another contributory factor to auditee concerns with value for money. Instant cut-and-paste audit reporting produced on laptops does not allow sufficient time for reflection and finely judged commentary on business performance. There is little allowance for strategic thinking or other value-adding components within the time allotted for the assignment. The assessor is placed under pressure to deliver the report and so will naturally concentrate on the immediate details, rather than the bigger picture. As a client, I would prefer to pay a little more and have the auditor leave, prepare the report off-site and formally present the findings to my top management.
A management system assessment should challenge management on its strategy, tactics and performance. In any case, should we not really leave the necessary concentration on basic compliance details to the client’s internal audit process? The whole assessment and certification process needs to be re-invigorated. The whole point about ISO 9001 is the application of the process approach to systems, breaking down organizational silos and barriers to effective communications in order to drive up performance. Yet when we apply this to the certification process, we are immediately confronted with limitations and barriers.
Looking at the audit standard ISO 19011, all the rules and conventions governing certification bodies and the latters’ own protocols, it quickly becomes obvious that auditors are not exactly encouraged to think imaginatively. Assessments are generally restricted to the scope of the management system and the auditor only interfaces with a company’s managers and accessible personnel who are covered by the system. Although there are occasional opportunities for assessors to interact with suppliers and contractors when they are present on site, we have forgotten the basic principles that underscore ISO 9001 and we have created new boundaries and silos around the audit process. Assessors do not really communicate with any of the organization’s customers, its shareholders, funders or other providers of finance, its partners, its trade associations, regulatory bodies and host governments, the local community or other affected by the business. So how can they really understand the auditee’s performance if they are denied direct access to the organization’s stakeholders?
Should we not practice what we preach? Those familiar with SA 8000 or other second party social compliance audits of company’s supply chains will know that different audit techniques have to be deployed. Some of these can be adapted and used in quality and environmental assessments. For example, when auditing supply chains, you need to establish to the client’s satisfaction that management has a fair and humane approach to employment conditions and workplace health, safety and welfare. As contract awards may be tied to a satisfactory outcome, there may be a certain amount of pressure and facility managers may try to dupe the auditor.
Tread carefully
The standard process is to start by checking management’s procedures and records against local legal requirements or the client’s code of conduct, whichever are the more rigorous. You then need to verify what management says but you cannot endanger workers or coerce them in any way. Consequently, social systems auditors employ techniques like the use of anonymous focus groups that get information in a way which will not expose the participants to retribution. Remember that in some countries, disciplinary measures may result in poorly paid workers losing their livelihood or being exposed to threats or even violence.
Try using focus groups in quality assessments. Bring together people from different parts of a process and challenge their performance and perceptions – the results can be amazing. Better still, go outside of the usual scope of assessment and obtain qualitative feedback from the organization’s key external stakeholders. Do not rely upon the stereotypical mundane customer satisfaction questionnaires for information. Go and speak to some real customers. It is not hard, once you establish suitable protocols, and, if you need a model, look at the GoodCorporation approach to verifying an organization’s corporate social responsibility performance. The GoodCorporation Standard is based on a core set of principles that defines a framework for a responsible approach to managing the needs and expectations of any organization’s key stakeholders. Since its launch, the GoodCorporation model has been used by a reasonably wide range of organizations including trades unions, charities and not-for-profit organizations, as well as businesses of every shape and size up to a multinational oil company.
Under each principle, the standard sets out assessable requirements in the guise of good management practices and/or minimum acceptable performance levels. The verification methodology consists of four stages:
- is there a policy in place?
- is there a system in place to implement the policy?
- do records show that the system works in practice?
- when asked, do the stakeholders agree that the system works and is fair?
Traditional management system auditing usually only goes as far as the third point – but it is the last step that can really add value. Naturally there are practical difficulties: for example, in identifying stakeholders you can contact during a limited assignment that are willing to help by providing feedback – but there is nothing that cannot be overcome.
Another element in the GoodCorporation scheme that I particularly like is the commendation, which can be awarded by the auditor in recognition of best practice and stimulates top management interest and buy in. Who says auditing must always be negative? Let’s embrace the new and exciting!
About the author
Fraser Paterson is principal corporate responsibility consultant and group environmental advisor with the international consultancy Scott Wilson, which has its head office in the UK. He has extensive worldwide consultancy and training experience in quality, environmental and integrated management systems and business improvement. For more information contact e: fraser.paterson@scottwilson.com or visit www.scottwilson.com