Can We Understand Complex Systems Without Mixed Methods?

You are here: Home » Can We Understand Complex Systems Without Mixed Methods?

Ari Antonovsky, Clare Pollock and Leon Straker, Curtin University, Perth Western Australia

Abstract

There is a tendency in both engineering and psychology to disdain information tainted with ‘subjectivity’. This attitude may be acceptable in regards to linear, loosely-coupled systems. However, the difficulty with analysing effectiveness in complex social-technical systems, relates to the need for contextual analysis of these systems, which Reiman and Oedewald (2007) describe as “socially-constructed and dynamic cultures” (p.747). Understanding performance in socio-technical systems may therefore require an interpretive analysis, rather than generalised models of organisational effectiveness.
The people who work intimately with and within a complex system are often in the best position to judge the ‘emergent’ properties in terms of safety, reliability, and other indices of organisational performance. In order to determine the current state of a workplace and how it may operate in future, Dekker, Cilliers and Hofmeyr (2011) considered it necessary to “gather multiple narratives from different perspectives inside of the complex system” (p.944).
In the current research, the aim has been to investigate the relationship between perceptions of the workplace, determined quantitatively and qualitatively, and the effectiveness of the work in the petroleum production facilities under study. Quantitative measures, as well as narratives from personnel that are responsible for operational performance, were used to relate specific human factors to objective measures of performance.

The study was conducted as part of a research project analysing the impact of human factors on the reliability of maintenance in petroleum industry workplaces (Antonovsky, Pollock & Straker, 2014). Participants surveyed included maintenance technicians, supervisors and coordinators in a large petroleum company. Survey questions were used to determine the perceptions of respondents concerning the operation of their workplace. Thematic analysis was used to extract themes from responses to an open-ended question within the same survey. Workplace effectiveness of nine workplaces within the target organisation was determined using standard measures of plant reliability.

Results

The quantitative survey data indicated that problem-solving behaviours and workplace design related significantly to operational reliability at the group level. The relationship between communication and organisation was significant. Decision-making was not significantly related to performance. The relationship between responses and reliability level in the thematic analysis assisted in the interpretation of data obtained from the survey question.

Quantitative survey data demonstrated a statistically significant and useful relationship between specific human factors and performance in a complex production system. However, questions remained as to why some factors related to performance while others did not. Thematic analysis, although more ‘subjective’, provided contextual data, and offered greater explanatory power in understanding how socially constructed workplaces might influence outcomes from complex technical systems. Furthermore, although past performance, a lag indicator, related to the quantitative data, predicting future performance could benefit from an understanding of the emergent properties of the workplace. The analysis of multiple narratives demonstrated their value in providing insight into the extant human factors that are likely to influence future performance.

Reiman, T. and Oedewald, P. (2007). Assessment of complex socio-technical systems – Theoretical issues concerning the use of organizational culture and organizational core task concepts. Safety Science, 45(7), 745-768.

Date & time

7-8 June 2016

NCTL Learning and Conference Centre, Nottingham

What is a Complex System?

Organised by

Follow me on Twitter