Both in inter and intra-examiner reliability là gì

The main objective of this study was to evaluate intra- and inter-rater reliability in the analysis of digital images of donor areas for skin in burn patients using the CaPAS plugin in the ImageJ®.

METHODS:

Donor sites were reviewed by two independent reviewers in duplicate. The capture of images was standardized on the same device and distance (with a millimeter ruler), without a flash. The evaluators were trained to capture the images and use the plugin.

RESULTS:

We selected 70 images from donor areas, from men and women between 18 and 60 years old. In the analysis of intra-examiner reliability, eight of the nine variables exhibited excellent reliability (0.985–0.998) and one (entropy) exhibited good reliability (0.525). The same was true for the inter-examiner analysis: excellent reliability for eight variables (0.824–0.993) and good reliability for entropy (0.501).

CONCLUSIONS:

The CaPAS plugin has proven to be a reliable tool for use in research in skin donor areas in burns, as demonstrated by its excellent intra- and inter-examiner reliability values. This is a pioneering study in the quantitative assessment of skin donor areas in burn patients using the CaPAS plugin.

KEYWORDS: Reproducibility of results; Data accuracy; Burns

INTRODUCTION

Digital photographs have been used for skin analysis in research with an interest in more reliable investigations of tissue characteristics11 Choi JY, Kim EJ, Jang SI, Kim AR, Lee TJ, Lee HK. A new technique for evaluating heel xerosis grade and the effects of moisturizer on heel skin dryness. Skin Res Technol. 2018;24(4):557-61. https://doi.org/10.1111/srt.12466 https://doi.org/10.1111/srt.12466.... Creating new tools to quantify and validate these images is a challenge22 Li D, Mathews C, Zhang F. The characteristics of pressure injury photographs from the electronic health record in clinical settings. J Clin Nurs. 2018;27(3-4):819-28. https://doi.org/10.1111/jocn.14124 https://doi.org/10.1111/jocn.14124..., since a subjective analysis only provides qualitative data that are difficult to reproduce33 Sabetai MM, Tegos TJ, Nicolaides AN, Dhanjil S, Pare GJ, Stevens JM. Reproducibility of computer-quantified carotid plaque echogenicity: can we overcome the subjectivity? Stroke. 2000;31(9):2189-96. https://doi.org/10.1161/01.str.31.9.2189 https://doi.org/10.1161/01.str.31.9.2189.... The clinical evaluation of the burned patient (both donor and recipient areas) is routinely performed qualitatively through direct inspection44 Nascimento C, Hansen LD, Sandoval ML, Santos VN, Vieira ALN, Ramos FS. Treatment of sequelae of burn: a case study. Rev Bras Queimaduras. 2014;13(4):267-70..

The easy-to-use ImageJ® public domain software developed in Java programming at the U.S. National Institutes of Health is a well-known health tool for image processing that has also been used for the assessment of skin lesions and can, therefore, be very useful in the evaluation of burn patients55 Rodrigues DF, Mendes FF, Menezes LB, Carvalho WL, Sá S, Silva JA, et al. Tratamento de feridas excisionais de coelhos com extrato de barbatimão associado a células mononucleares autólogas da medula óssea. Arq. Bras. Med. Vet. Zootec. 2017;69(5):1243-50. https://doi.org/10.1590/1678-4162-9301 https://doi.org/10.1590/1678-4162-9301....

The CaPAS plugin was designed for carotid plaque analysis and was developed for the quantitative evaluation of videodensiometric images of atherosclerosis through computational measurements. This tool is sensitive to grayscale changes in cholesterol, collagen, and calcium, enabling the differentiation of groups66 Baroncini LA, Pazin Filho A, Murta Junior LO, Martins AR, Ramos SG, Cherri J, et al. Ultrasonic tissue characterization of vulnerable carotid plaque: correlation between video densitometric method and histological examination. Cardiovasc Ultrasound. 2006;4:32. https://doi.org/10.1186/1476-7120-4-32 https://doi.org/10.1186/1476-7120-4-32..., patients with and without symptoms have a different plaque composition77 Nandalur KR, Baskurt E, Hagspiel KD, Phillips CD, Kramer CM. Calcified carotid atherosclerotic plaque is associated less with ischemic symptoms than is noncalcified plaque on MDCT. AJR Am J Roentgenol. 2005;184(1):295-8. https://doi.org/10.2214/ajr.184.1.01840295 https://doi.org/10.2214/ajr.184.1.018402....

The use of the CaPAS plugin in burn patients was reported in a study that analyzed images of skin donor areas (e.g., thigh and scalp) in two groups (i.e., sham and intervention) to investigate re-epithelialization88 Gomes RC, Guirro ECO, Gonçalves AC, Farina Junior JA, Murta Junior LO, Guirro RRJ. High-voltage electric stimulation of the donor site of skin grafts accelerates the healing process. A randomized blinded clinical trial. Burns. 2018;44(3):636-45. https://doi.org/10.1016/j.burns.2017.10.006 https://doi.org/10.1016/j.burns.2017.10..... The plugin made it possible to quantify the images and differentiate the texture of the re-epithelialized tissue. However, no plugin properties or variables were evaluated.

The aim of this study was to evaluate intra- and inter-rater reliability in the analysis of images of skin donor areas in burn patients using the CaPAS plugin implemented in ImageJ®.

METHODS

A total of 70 images (32 scalp and 38 thigh) of 70 patients were taken from skin donor areas of burn patients admitted to the emergency unit of the hospital of the Ribeirão Preto School of Medicine. One was randomly selected using a simple lottery system for analysis by two evaluators, according to previous health reliability studies99 Bassi D, Santos-de-Araújo AD, Camargo PF, Dibai-Filho AV, Fonseca MA, Mendes RG, et al. Inter and intra-rater reliability of short-term measurement of heart rate variability on rest in diabetic type 2 patients. J Med Syst. 2018;42(12):236. https://doi.org/10.1007/s10916-018-1101-8 https://doi.org/10.1007/s10916-018-1101-....

Images were taken using a 12-megapixel digital camera (DSC-W510, Sony, Manaus, Brazil) with 72 dpi. The images were taken without flash with the camera perpendicular to the area to be photographed at a standardized distance of 30 cm apart (determined using a millimeter ruler) on the same plane as the sample. This was performed by two independent evaluators who had undergone training for the use of the software. All images were analyzed twice, with a 1-week interval between evaluations. The evaluation and digitization of the analyses followed the same order (1–70) for both evaluators. The image evaluation procedure is described in Figure 1.

Figure 1 Importing the image in the File icon. After inserting the image, click twice on the CaPAS icon. At this moment, the image will appear in grayscale.

Outline the area of interest with the computer mouse. Select the ANALYSIS icon. Then, the graph with the nine measurements will be generated: Count: number of pixels within the area; Average: average pixel value calculated on the pixels in the selected region; Standard Deviation: of the gray values of the pixel in the selected region; Skewness: asymmetry (when positive, asymmetry is toward the dark side; when negative, asymmetry is toward the light side); Kurtosis: graph symmetry, Energy: angular value, when the elements of the co-occurrence matrix are very unequal; Inertia: intensity contrast between a pixel and its neighbor; Entropy: degree of uncertainty in the uniformity of the selected region (its value increases when homogeneity reduces); and Homogeneity: dissimilarity and contrast.

These measurements are represented by texture parameters involving mean and standard deviation, and grayscale parameters (i.e., entropy, energy, and homogeneity). The mean gray level (MGL) represents the average grayscale value of the pixels in the region of interest on a scale of 256 shades of gray, with zero being the darkest possible tone (hypoechoic) and 255 the lightest possible tone (hyperechoic)66 Baroncini LA, Pazin Filho A, Murta Junior LO, Martins AR, Ramos SG, Cherri J, et al. Ultrasonic tissue characterization of vulnerable carotid plaque: correlation between video densitometric method and histological examination. Cardiovasc Ultrasound. 2006;4:32. https://doi.org/10.1186/1476-7120-4-32 https://doi.org/10.1186/1476-7120-4-32....

Intra- and inter-rater reliability were determined using the intraclass correlation coefficient (ICC2,1), with the calculation of the 95% confidence interval, standard error of measurement (SEM), and minimal detectable change (MDC). ICC values were interpreted based on Weir1010 Weir JP. Quantifying test-retest reliability using the intraclass correlation coefficient and the SEM. J Strength Cond Res. 2005;19(1):231-40. https://doi.org/10.1519/15184.1 https://doi.org/10.1519/15184.1...: 1.00–0.81 indicates excellent reliability; 0.80–0.61 indicates very good reliability; 0.60–0.41 indicates good reliability; 0.40–0.21 indicates reasonable reliability; and 0.20–0.00 indicates poor reliability. The statistical analyses were performed using the Statistical Package for the Social Sciences (SPSS), version 17.0 (Chicago, IL, USA).

RESULTS

A total of 70 digital photographic images of 70 patients of both sexes between the ages of 18 and 60 years were evaluated. The mean values are described in Table 1.

Table 1 Description of the mean of the variables by both evaluators.

Table 2 shows intra- and inter-rater ICC values. Intra-rater ICC values ranged from 0.985 to 0.998 (excellent reliability) for eight of the nine variables. The intra-rater ICC for “entropy” was 0.525 and inter-rater ICC for “entropy” was 0.501, thus good reliability was found for entropy, while excellent reliability was found for other variables (ICC=0.824–0.993).

Table 2 Summary of intra- and inter-examiner reliability obtained.

DISCUSSION

The evaluation of burns is commonly performed by local visual inspection, being a quick and easy method, but subjective, not being the most reliable or accurate1111 Pape SA, Skouras CA, Byrne PO. An audit of the use of laser Doppler imaging (LDI) in the assessment of burns of intermediate depth. Burns. 2001;27:233-9. https://doi.org/10.1016/s0305-4179(00)00118-2 https://doi.org/10.1016/s0305-4179(00)00...,1212 Droog EJ, Steenbergen W, Sjöberg F. Measurement of depth of burns by laser Doppler perfusion imaging. Burns. 2001;27(6):561-8. https://doi.org/10.1016/s0305-4179(01)00021-3 https://doi.org/10.1016/s0305-4179(01)00..., as it strongly depends on the experience of the evaluator.

The use of digital photographs is a viable alternative to overcome this limitation. It has been used in research on skin erythema1313 Cho M, Lee DH, Doh EJ, Kim Y, Chung JH, Kim HC, et al. Development and clinical validation of a novel photography-based skin erythema evaluation system: a comparison with the calculated consensus of dermatologists. Int J Cosmet Sci. 2017;39(4):426-34. https://doi.org/10.1111/ics.12393 https://doi.org/10.1111/ics.12393..., skin cancer1414 Rosenberg A, Meyerle JH. Total-body photography in skin cancer screening: the clinical utility of standardized imaging. Cutis. 2017;99(5):312-6. PMID: 28632800