Cookies on this website

We use cookies to ensure that we give you the best experience on our website. If you click 'Accept all cookies' we'll assume that you are happy to receive all cookies and you won't see this message again. If you click 'Reject all non-essential cookies' only necessary cookies providing core functionality such as security, network management, and accessibility will be enabled. Click 'Find out more' for information on how to change your cookie settings.

BACKGROUND: Oasys-2 is a validated diagnostic aid for occupational asthma that interprets peak expiratory flow (PEF) records as well as generating summary plots. The system removes inconsistency in interpretation, which is important if there is limited agreement between experts. A study was undertaken to assess the level of agreement between expert clinicians interpreting serial PEF measurements in relation to work exposure and to compare the responses given by Oasys-2. METHOD: 35 PEF records from workers under investigation for suspected occupational asthma were available for review. Records included details of nature of work, intercurrent illness, drug therapy, predicted PEF, rest periods, and holidays. Simple plots of PEF and the Oasys-2 generated plots were available. Experts were advised that approximately 1 hour was available to review the records. They were asked to score each work-rest-work (WRW) period and each rest-work-rest (RWR) period for evidence of occupational effect. At the end of each record scores of 0-100% were given for evidence of "asthma" and "occupational effect" for the whole record. Kappa values were calculated for each scored period and for the opinions on the whole record. The scores were converted into four groups (0-25%, 26-50%, 51-75%, 76-100%) and two groups (0-50% and 51-100%) for analysis. This is relevant to scores produced by Oasys-2. Agreement between Oasys-2 scores and each expert was calculated. RESULTS: 24 of 35 records were analysed by seven experts in the allotted time. For whole record occupational effect, median kappa values were 0.83 (range 0.56-0.94) for two groups and 0.62 (0.11-0.83) for four groups. For asthma, median kappa values were 0.58 (0-0.67) and 0.42 (0.15-0.70) for two and four groups respectively. For all WRW and RWR periods kappa values were 0.84 (0.42-0.94) and 0.70 (0.46-0.87) respectively. Agreement between Oasys-2 and individual experts showed a median kappa value of 0.75 (0.50-0.92) for two groups and 0.50 (0.39-0.70) for four groups. Kappa values for the median expert score v Oasys-2 were 0.75 for two groups and 0.67 for four groups. Agreement was poor for records with intermediate probability, as defined by Oasys-2. CONCLUSION: Considerable variation in agreement was seen in expert interpretation of occupational PEF records which may lead to inconsistencies in diagnosis of occupational asthma. There is a need for an objective scoring system which removes human variability, such as that provided by Oasys-2.

Type

Journal article

Journal

Thorax

Publication Date

10/2002

Volume

57

Pages

860 - 864

Keywords

Asthma, Expert Systems, Humans, Observer Variation, Occupational Diseases, Peak Expiratory Flow Rate, Predictive Value of Tests, Professional Practice, Sensitivity and Specificity