This website uses cookies primarily for visitor analytics. Certain pages will ask you to fill in contact details to receive additional information. On these pages you have the option of having the site log your details for future visits. Indicating you want the site to remember your details will place a cookie on your device. To view our full cookie policy, please click here. You can also view it at any time by going to our Contact Us page.

Baseefa Ltd

Better Investigations – getting value from investigation methods and process

31 May 2018

In this article, two very experienced ex-regulatory investigators working for the BP group give a practical view of investigation as a process and an activity, rooted in a largely non-methodological approach. BP International Human Factors Advisor (Downstream) Simon Monnington and Independent Human Factors Consultant John Wilkinson here build on an earlier joint paper (Ref. 1) and update the ongoing case study, presented at the Hazardex 2017 Conference.

Investigation can be a very challenging activity for many people, whether they are leading or just occasional team members.  Even after a good investigation, the actual impact of recommendations and their implementation can be disappointing.

Opportunity is anyway limited either by availability of incidents or by the time and resource made available to investigate.  This makes gaining experience and expertise a challenge. 

Investigation methods vary widely too and are often adopted for perceived (but then not fully realised) benefits.  For example, a method may offer better and more detailed root cause identification.  But in practice this is made difficult for users by having multiple drop-down menus and poorly-explained choices.  Methods may offer alleged improved validity and reliability, but again coupled with poor usability. 

Training is often more method- and software-focused and not well consolidated by structured on-the-job training or other practical support.  Human and organisational factors (HOF) are often not included effectively and are also very variably addressed in the methods themselves.  Even the best methods are in the end very reliant on the investigators’ own HOF understanding and experience – so claims of HOF integration (and few methods offer this) are often unrealised in practice.

Organisations may try to investigate lower level incidents and near-misses but there may be barriers, for example their internal incident classification system may only allow a site to deploy limited resource and capability to investigate.

The resulting outcomes can then lack depth and rigour.  Outcomes generally can just produce the usual suspects such as more and longer procedures, and narrow re-training i.e. ‘fixing the last accident, not the next’.  There is often a corresponding reluctance to improve design or take other measures higher up the hierarchy of control, or to tackle organisational factors such as resourcing, work planning and change issues.  Investigators can only find what they look for, and what they have the time, training, support and ‘permission’ to look for. In this context the behaviours of real concern are those of the organisation, not of those closer to the sharp end.

Investigation depth can also be affected by contributions from others in the system.  Potential recommendations/actions may be challenged or played down during review and discussions.  Lead investigators may feel under pressure not to push findings especially if a senior manager does not really understand the issues concerned.  This may be disguised by a “these recommendations are not SMART enough, too fuzzy” etc. response.  It is also difficult if investigation findings reflect poorly on leadership by having a proper focus on the organisational contribution.

So what is needed? 

In a nutshell: an adequate narrative (the story), a reasonable timeline and effective analysis.  Also proper consideration of what influenced performance on the day, the performance influencing factors (PIFs)?  These make human failures more or less likely on the day – if you have ever asked ‘Why this person, and why today?’ then the reasons lie in the PIFs.  Usually people get things right and often they work around sub-optimal conditions.  PIFs can be divided between the job, the person and the organisation, and often in practice will overlap across more than one, for example competence resides in the individual, the job competencies, and the competency management system.  So the organisation may often fail, not just the individual. 

Where methods are concerned, it’s vital to consider HOF.  In a perfect world all methods would have these integrated appropriately but in practice, few – if any – actually do.  There are a few standalone methods to consider HOF and human behaviour aspects, in addition to the normal investigation method.  Be careful with methods that just offer lists or dropdown menus - does anyone really understand?

Consider also the outputs – are they realistic (not just SMART) and are they actually likely to be implemented?  How clear are the recommendations and what happened to them?

What is the process?

Investigation is the mirror image of hazard and risk analysis but we are looking backwards instead of forwards, i.e. in the rear-view mirror, not through the windscreen.  The human contribution must be properly considered and this includes recognising that most of the time people get things right, or at least make them work right.  Often people (including you and me) are ‘working around’ difficult or sub-optimal conditions to get things done.  In practice, we are often compensating for poor design or arrangements.  When things go right all is well, when things go wrong then suddenly we are in the spotlight.  And investigators have 20:20 hindsight (one of our well-known cognitive biases). 

The Safety-2 approach (Ref. 2) captures this kind of thinking, taking a systems’ stance to reflect the inherent complexity, messiness and non-linear state of the real world.  Traditional investigation methods are said to be linear.  A Safety-2 stance is more forward looking, characterised by ‘chronic unease’ and a corresponding vigilance for system disturbances.  So for investigation what does this mean?  In essence it means making sure that as investigators we come to understand what made sense to those involved in the incident before things went wrong. In most cases those involved did not see the incident coming so it follows that their view was different to what has subsequently been established by investigation. 

Whether error or violation was involved, they were not trying to cause an incident and did not – with the information and experience they had - think it likely that one would occur. 

However, the few new Safety-2 investigation methods are as yet not tried and tested.  While there have been some case study reports it is to date hard to see what the new methods are adding if judged by their outcomes.

It will also help to consider using more neutral terms such ‘ as ‘debriefing’ witnesses and others post-incident, rather than the more formal word ‘interviewing’ with all its implications.  What you want to do is learn first, then any proper accountability can follow if justified.  But don’t start with this.

So what method to use? 

Keep it simple: does the method help you reliably establish a sufficient narrative, timeline and analysis. And does it encourage you to seek out corroboration and other support for key evidence to make sure hindsight bias is not distorting the investigation.  Are other stories (explanations) considered?  More than one story can explain a set of facts - so which is the right story and how does the method help you check this out?

It’s vital that the method has good usability.  Is it in fact used, and used consistently, and is it really understood by the users? Are the method’s limitations and advantages known?  And is there evidence that the method tail is wagging the dog? Check the outcomes to find out.  Is there clear organisational support and understanding and is this expressed through effective training, resourcing and management support?  Finally, is there sufficient HOF focus in the method – or allied approaches – and do investigators have sufficient (awareness of human and performance influencing factors?  If they don’t, then the method can’t achieve this on its own.

In practice, a single method may not be enough for all situations and so a toolbox approach is better.  As a minimum, this means having a simple method for everyday and something more sophisticated for larger or more complex incidents. Starting with the simple method will soon tell you if you need something else.

Cognitive biases

These biases are part of what it is to be human.  For investigation the most important ones are:

Hindsight bias: the fact that we know what went wrong can let us view the pre-incident picture very narrowly and without trying to understand why, for those involved, what was happening up until things went wrong made sense to them.

Confirmation bias: our tendency to be selective in the evidence we look for, or that we value on finding.  So if we think a violation occurred, we won’t look hard for evidence of contributing errors at the individual or organisational level.

Fundamental attribution error: our strong tendency to over-estimate an individual’s contribution to an incident, and underestimate external factors, and to do the reverse for ourselves. For example, you were speeding – I was in a legitimate hurry to get to X on time, I’d done my risk assessment, the traffic was light, I had a lot on my mind, the signing was poor, the light was poor etc. etc. 

There are others.  Investigators need to be aware of these for themselves and for witnesses and other parties.  They also need to look for corroboration and different sources of evidence, and look for other stories that could explain the facts.

Training and competency

Not everyone is a good fit for investigation – there should be some structured selection process (not just: ‘All supervisors will…’).  Investigators need to be able to gain experience and consolidate their training in a structured way.  Where actual incidents are not available then substitutes such as cold case reviews and, even better, applying investigation skills prospectively through, for example,  monitoring, audit and review, find the factors that matter before something goes wrong.

They need training that goes beyond just the method(s) being used, for example, how is this applied to our workplace, our processes, my role, my investigation team and so on.  They also need informed awareness of biases, of human and performance influencing factors including human behaviour, of method strengths and weaknesses.  If the difference between errors and violations is not well understood then there is a big problem because the solutions to these two fundamental types of human behaviour are different.

Investigation is usually a team process, so consider teams as well as individuals.  Soft (non-technical) skills are important such as communicating effectively with others, and good interviewing skills are essential and require training.

The BP approach

BP utilises a ‘logic tree’ method for analysis of higher severity incidents and HFAT (Human Factor Analysis Tools) for human factors analysis. The logic tree - think of it as a type of why / because analysis, similar to fault tree without the logic gates - is developed by the team using their choice of off-the-shelf software. It is a free-flowing method that involves early development prior to significant evidence collection allowing hypothetical causes or contributory factors to be identified.

The advantage of doing this is that the analysis tool can be used to frame lines of enquiry for investigation. The logic tree is then revisited and hypotheticals ruled in or ruled out. There follows a fresh round of hypothesis development and this cycle continues until sufficient depth is realised. BP uses the logic tree to drive identification of lines to follow, and to capture and map causes and contributions.  The completed tree then reflects all the causes identified and ruled in, ruled out or unverified. It therefore helps the investigator maintain a record of the breadth and depth covered.

The logic tree approach explicitly aims to address physical, human, management system and organisation / leadership factors. In addition, the method requires the investigator to look across the whole logic tree beyond the ‘linear’ cause and effect relationships. HOF are captured and analysed using LT linked to the BP human factors tool kit. For complex incidents, investigators are provided with and trained in HFAT, developed by The Keil Centre. The completed LT then drives the development of findings and recommendations.

Experience so far

The use of Logic Tree with HFAT, matched with a capable lead investigator, is driving breadth and depth in investigation.

Findings tend to fall out of the approach more readily and this makes it easier to record recommendations to help prevent recurrence.

So there is an ongoing program of HOF coaching during investigations, provision of training, discussions in investigator community forums and the deployment of HOF expertise as part of investigations which have a critical behavioural theme.

An example: in February 2015 at a petrochemicals site, there was a release of 25m³ of an acetic acid solution during the process of filling the process plant after routine maintenance works. The temperature of the solution released was 67°C, marginally above flashpoint. The solution was released into the unit sewer system and captured at the waste water plant.

Nobody was injured in the incident and the operation remained well within the boundaries of the site’s environmental permit.  Given the quantity of the release of the acetic acid solution, the incident was classified as High Potential. At BP, this level of investigation involved the deployment of a Master Level investigation leader and the utilisation of the logic tree incident investigation methodology.

The investigation identified that the outgoing operator opened three drains to drain the system - the operator’s normal method for draining the system. However, the incoming operator closed two drains and this was that operator’s normal method for draining the system. The solution was released through the open third drain. It was found that the SOP in use was imprecise, for example ‘close all drains’, updated frequently and not written with sufficient operator involvement. In one of the recent SOP updates, the unit leadership introduced the option to use the third drain in earlier steps of the SOP, allowing judgement of the operators. Unit leadership did not specify in writing in the SOP how to drain for the last steps of the SOP, nor did it specify expectations in another way, nor did it verify the task execution. 

The site took action in response to the incident, including:

• Organisational structure – new unit operations leader roles created, allowing verification of task execution

• SOP development and review tools and training rolled out

• Updated policy on SOP development to reflect new tools and training

• Enhanced self-verification processes implemented to check the area of operational rigour and line-ups before commissioning

• Learnings from this incident applied into wider organisation

The way forward

The quality of HOF analysis and recommendations in an investigation report are primarily dependent upon the capability and performance of the investigation team. Remember, investigation teams are human too and as such subject to HOFs. Investigation methods, training and support from the organisation are all important factors in enabling an investigation team to do a good job.

There are no perfect methods out there, and none that fully integrate human factors.  So take a toolkit approach and have more than one method available, for example one simple, one for more complex cases, and possibly one for helping understand behaviours of concern.  Make sure investigators have sufficient human factors’ awareness and understanding.  Choice of methods should be led by usability and simplicity – can those who are going to use them understand them, and will they be able to use them with understanding.  Method(s) should support and guide the team; the tail shouldn’t wag the dog.  Check the (real, implemented) outcomes to see if the method(s) are doing what you want or not. 

Effective training requires more than just training on the investigation method. Don’t neglect the soft skills - such as team working, interviewing, human factors / behaviour understanding (including key cognitive biases), and communication skills – needed to investigate well, as well as the need for structured practice and consolidation.  Use cold cases or the same principles - for example the performance influencing factors – to apply prospectively as part of ongoing audit, monitoring and review before something goes wrong.  Keep things neutral, for example use ‘debriefing’ rather than ‘interviewing’ and focus on the learning you want.  If that includes proper accountability issues then you can move to more formal approach.  Either way, debriefing or interviews should be well structured. 

Look for corroboration and make good use of walking and talking through the incident and understanding the baseline against which to assess errors and non-compliance.  Finally, even if you have a coherent story that explains the facts you have established, make sure that there isn’t another (perhaps better) story that can do so.

Finally, make sure that relevant human behaviour in an incident is as rigorously approached as the technical and engineering aspects.  This will produce better investigations, recommendations and learning. 

References

1. Investigation – What Does Good Look Like and Does It Really Have to Be Complicated? Simon Monnington and John Wilkinson.  IChemE Hazards 26 Conference, Edinburgh, 2016. http://www.icheme.org/communities/special-interest-groups/safety%20and%20loss%20prevention/resources/hazards%20archive/hazards%2026.aspx?sc_trk=follow%20hit,{0D792DA4-E5D3-4ADF-854C-9478C33ADA27},wilkinson%20monnington

2. List of Performance Influencing Factors via HSE website: http://www.hse.gov.uk/humanfactors/topics/pifs.pdf

About the authors

John Wilkinson is an Independent Human Factors Consultant and an Associate at The Keil Centre, Edinburgh. He was Principal Human Factors Specialist Inspector and Human Factors Team Leader at the HSE from 1999-2011 and was lead HSE human factors investigator at Buncefield.

Simon Monnington is Human Factors Advisor, Downstream Safety and Operational Risk, at BP. He is a Chartered Ergonomics and Human Factors Specialist with a track record in incident investigation, safety optimisation and human factors in hazardous industries.


Contact Details and Archive...

Print this page | E-mail this page