Analysis of short-term variation and long-term drift during reagent kit lot change in an NABL accredited clinical biochemistry laboratory

Background Kit lot change in clinical biochemistry labs leads to variations in patient results. This study planned to identify variations during 60 reagent lot changes in our laboratory during the period from June 2018 to May 2019. Methods A statistical analysis was performed to identify the difference between patient samples results variations and QC results. The long term drift was analyzed using a regression test. Results There was a significant difference between the patient and QC results in 16.7% of reagent lot changes. Moreover, the extent of variation in QC results was 3.3%. No long-term drift was seen in three analytes which were studied using regression analysis. Conclusions Our results showed that, during reagent kit lot change, along with QC material, the patient samples should also be run in order to identify the variation. However, this practice is presently ignored by most of the laboratories. There was no accumulated effect in our laboratory due to reagent kit lot change.


Introduction
Delivering consistent and reliable test results is the primary objective of a clinical chemistry laboratory, for which various steps of quality control (QC) are to be followed properly. In an automated clinical biochemistry lab, the autoanalyser uses various consumables like reagents, calibrators and QC materials for the routine testing of patient samples. The reagents used in the tests are the most consumed product in the laboratory. The amount of reagents consumed is determined by the total number of tests a laboratory receives, and the frequency of reagent kit change further depends on this. Among various analytes, plasma glucose is one of the analytes which is more frequently asked for and because of this, there will be a frequent reagent kit lot change of this analyte. A specific lot number is given to reagents of a particular batch which is manufactured using similar conditions (1). The manufacturing conditions in which each of these lots is produced will be slightly different and it can lead to a slight change in the analytical process which is generally acceptable (2). There is a chance that these changes can gradually build up over a period of time and affect the patient results without our knowledge (2). In order to avoid these variations, proper measures have to be taken routinely, each time when a new reagent lot is introduced into a machine (3,4). As per the guidelines of CLSI (Clinical and Laboratory Standards Institute), laboratories should analyse and correct the changes in the performance of an analyte after the reagent lot change (5). In Indian laboratories, National accreditation board for testing and calibration laboratories (NABL) is the agency which gives the certification and accreditation for the quality of the work they perform. According to NABL, the kit lot change should be accompanied by QC run, and the values of QC should not surpass the range set for the old lot (6).
During a reagent lot change, QC values with a new lot should fall within the same range (±2SD) set for the old lot (7). This can be done with various numbers of levels of QC materials as per the lab's QC protocol. However, the QC material does not represent the patient sample, and in order to rule out the variation in this context, a number of patient samples should also be performed with the old and new reagent lots (3). There is no definite international criterion which states the minimum sample size to be used for this purpose. What is more, this procedure varies from laboratory to laboratory in terms of the number of samples and the level of QC used. The reagents with a new lot number will be used when there is a consistency in these results based on the acceptance criteria put up by each lab (7). Generally, reagents with a new lot will be accepted for use in a lab if the results obtained from different levels of QC material run using a new lot come within the ±2SD range of QC material set for the old reagent lot and if the variation between the results of the patient sam-ple with the new lot and the old lot is less than 10 per cent (8). If any of these criteria are not met, then the reagent lot change becomes invalid.
In this study, we have examined the variation in QC materials' results when compared to patient samples while a lot change of a reagent occurs. We have also analysed chances of long term drift in three analytes during five consecutive reagent lot changes in our laboratory.

Study design
The current study was performed in the clinical biochemistry laboratory of the Department of Biochemistry, Armed Forces Medical College, Pune, for a period of one year, from the 1 st of June 2018 to the 31 st of May 2019. All procedures performed in the studies involving human participants were in accordance with the ethical standards of the institutional research committee (Institutional ethics committee, AFMC, Pune, reference number -IEC/2019/292) and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards. Informed consent was obtained from all individual participants included in the study. Our laboratory has been accredited by NABL for medical testing since 2010. For 9 years, we have been receiving re-accreditation based on our performance and quality of work in the laboratory. The last accreditation was given in February 2018 and it is valid until February 2020. The test results of patients and QC samples after 60 reagent lot change events were included in the analysis of this study. These reagent lot changes were performed in two Siemens Dimension EXL 200 auto analyzers and there were routine biochemistry analytes for which kit lot change was performed. The reagents used were purchased from the Siemens Company and the laboratory procedure for each analyte was followed according to the manufacturing company's instructions. The QC materials for Level 1 and 2 were purchased from Biorad (USA) and the mean and standard deviation was set according to the manufacturing company's instructions. Our laboratory validates the effect of change in the lot number of a reagent using the results of 10 patients' samples and QC materials with 2 levels (one with a normal and another with an abnormal range). The new reagent lot will be in use only when it meets the variation as per the cut-off values provided by Henry's textbook of clinical chemistry (9,10). For the patients' samples, the percentage of variation for each sample is obtained from the following equation: (Patient result with the new lot -Patient result with the old lot)/Patient result with old lot] X 100. The results of QC are considered only when it falls within the range set for the old reagent lot. Our lab used to set the range for each analyte as mean ±2 SD from the data provided by the company. All the data were collected and double-checked for any variation. There was no QC material kit lot change during this period in our laboratory.

Statistics
The results obtained from the patient samples and QC with the new and old lots during 60 reagent lot changes were first log-transformed. Both P (difference of patient sample) and ΔQC were calculated from the above data. In order to see the difference between the variation in QC and patient sample results, Wilcoxon sum rank test was performed in R 3.5.2 statistical software. Statistical significance was set at a P value less than 0.05. The analysis of DQC was performed to see the size and variation. The results of QC material with an old lot of a reagent were subtracted from a new lot of a reagent in order to get the DQC values. Then these DQC results were divided by their own SD values of each analyte which is used in monitoring the daily QC. This was expressed in terms of SD multiples. There were five classes of these values ranging from <0.5 to >2.0.
Next, the long-term effect of kit lot change for 3 analytes was analysed in which consecutive five reagent lot change data were available. Out of the 19 analytes which underwent reagent kit lot change, only three analytes had reagent lot change for 5 consecutive times during the study period of one year in our lab. This was due to the high demand for tests and a smaller number of tests per reagent lot. So only those were included in the long term drift analysis. The data of serum calcium, ALT and AST during their five consecutive reagent lot changes were collected. Using Weighted Deming Regression (WDR), regression coefficients and Jack-knife standard error were derived. WDR was performed using the mcr package of R 3.5.2 software. The intercept and slope of regression with a standard error during the change of reagent lots were analysed and t statistics was performed (7) in order to find out the cumulative shift in lot change.

Results
In our study, practically none of the analytes which underwent kit lot change showed variation beyond acceptable limits as shown in Table I. The P and QC results of 60 kit lot change results with patient samples and QC were analysed and it was found that 10 out of the 60 reagent lot changes (16.7%) differed significantly (P<0.05). These include albumin, phosphorus, calcium, total cholesterol, ALP, BUN, glucose, amylase and total protein. The complete data is given in Table I. The kit lot change result for phosphorous was significantly differ-ent in two consecutive times. The results are grouped according to the nature of the method into pure chemical method results and enzymatic method results and presented in Table I. A total of 120 D QC data were available with two levels of QC during 60 reagent lot change procedures. The sizes of these were determined by dividing them with SD set for each analyte. The D QC size was represented as multiples of SD values. There were 4 D QC results above 2SD range and they were for Albumin and BUN. The data are shown in Table II.
In order to evaluate the degree of bias between the results of two different lots of a reagent, regression analysis was performed. The regression analysis provides the values of a slope and an intercept which represent the degree of proportional and constant bias respectively. The data of the regression analysis of three analytes for 5 consecutive times are given in Table III In the regression analysis of AST, there was a positive regression intercept and a positive regression slope in the 2 reagent lot change events during June 2018 and May 2019. There were three negative regression intercepts with a positive regression slope. In regression analysis, the values of slopes were close to 1 in all three analytes which shows that there was no proportional bias in these analytes. But the values of intercepts in these cases were not close to 0 which shows the occurrence of a constant error in all the cases. This indicates the chances of a cumulative drift in the data of 3 analytes over a period of time.
The t-statistics was performed for all 3 analytes, with the regression data and the results presented in Table IV. In the case of calcium, the t-statistic was less than critical t-value of proportional bias (1.66; 2.10) and constant bias (-1.45; 2.10) at 95% CI. So, there was no cumulative drift observed in serum calcium during this period. In the case of ALT, t-statistics was less than critical t-value of proportional bias (-0.37; 2.04) and constant bias (0.06; 2.05) at 95% CI. So, there was no cumulative drift observed in serum ALT during this period. In the case of AST, t-statistics was less than critical t-value of proportional bias (-0.60; 2.05) and constant bias (0.45; 2.05) at 95% CI. So, there was no cumulative drift observed in serum AST during this period.

Discussion
Good laboratory practice consists of validating a new reagent lot before using it in the analyzer to ensure that patient sample results and QC results are consistent during the change in lots of reagents in a measurement procedure (9). During kit lot change, a number of patient samples used to be run before and after the change of lot, as a part of routine quality assurance. These test results which were obtained were examined to verify the consistency between the results after the usage of each lot of reagents. Ideally, the set of patient samples used in this procedure should verify the consistency over the entire measuring interval of the analyte. Laboratory faces difficulty in getting the test samples which encompass the entire measuring interval; thus, patient samples in smaller intervals are taken to symbolize the consistency of consequences over the complete interval (9). Preferably, when QC and the patients' test samples are measured with an old and new lot of reagent, their values are expected to be the same and this never happens. Therefore, every laboratory will have its own guidelines to check the consistency of QC and patient results during reagent lot change. General criteria for accepting a new reagent lot is that the result with a new kit falls within the predefined target range of ±2SD of QC material set for the old lot and the variation in the results of patient samples with both   kits is less than 10 % (8). This criterion is not followed by most of the laboratories as these are not the mandatory guidelines given by the accreditation agencies.
The current study examined the variation of QC results when compared to the results of variation in patient sample results when there is a reagent lot change in the laboratory. Our data demonstrated that there was a 16.7% variation between the patient and QC results during the 60 reagent lot changes. It means that if we had done only the analysis with QC material, we would have missed the variation that had been identified with patient samples. In contrast, one previous research work by Cho et al. (9) showed that there was a significant difference in 7.8% of lot change in 360 reagent lot change events (11). In another research paper, Miller et al. (9), presented that there was a variation in the 40.9% of events of reagent lot change In both studies, they had used two assay methods, general chemistry and immuno-assay, but, in our study we have included only general chemistry assays. Our laboratory uses a strategy in which we run two levels of QC along with 10 patients' samples when there is a reagent lot change. Previous studies used only five or fewer patients' samples during reagent lot change along with multiple levels of QC. We have used the Wilcoxon sum rank test for our analysis. The disparity in the results might be due to the variation in the level of QC and the number of samples tested during each lot change rather than due to the difference of the statistical method being used.
There are non-commutability problems when we use QC material to assess the impact of reagent lot change. This is because during the manufacturing, the matrix which is used for QC material making does not represent the patient sample. Therefore, using QC material result to validate the impact of reagent lot change is debatable (12)(13)(14). For example, in one of the previous studies of alkaline phosphatase assay, the non-commutability problem was caused due to the usage of QC materials prepared from a non-human source (15). Subsequently, the results obtained from QC material run during lot change event of a reagent are useful in adjusting the predefined range. It will minimize false negative or positive errors rates, which may further accumulate over some time. The false-negative and negative rates for particular QC material can be estimated after dividing D QC results by SDs of corresponding analytes QC yielding multiples of SD (11).
As per our laboratory policy, the prede ned target range differs from the mean value by 2 SD, and there is a false detection rate of 3.3% during a reagent lot change which has a difference of SD more than 2 (Table III). There were 120 DQC results available with two levels of QC during 60 events of lot change. After evaluating our DQC size, it was found that 96.7% QC results were inside the DQC size of 2. Thus, during 3.3% of lot change procedures, the range of QC should be readjusted. The occurrence of these non-commutable results with QC products is sufficient enough to prove that this alone should not be used to check the reliability of results when a reagent lot change occurs.
The current approaches used in clinical chemistry laboratory failed in detecting lot-to-lot change because the variation pointed out in this lot-to-lot change is small and neglectful. Moreover, if the lot-tolot change remains undetected over time, it may accumulate to a significant level and become a longterm drift (2,5). A recent report has identified an undetected long-term drift during reagent lot change in Mayo Clinic (7). Therefore, the long-term drift in reagent lots could be an important contributor to result in inconsistencies in clinical laboratories. To find out the long-term drift in reagent lots after consecutive reagent lot changes, we evaluated the results of the patient result data of three analytes using regression analysis. In the regression analysis, we couldn't find any cumulative shifts in any of the reagent lot changes in the analytes serum calcium, AST and ALT between June 2018 and May 2019. This shows that all three analytes we analysed were in the acceptable limits.
In conclusion, there was variability in QC results and patient results during reagent lot change in the 16.67% of analytes, suggesting that the use of QC material to find the impact of a new reagent lot doesn't have much utility in a clinical biochemistry lab. The difference in the values of patient samples is significant to validate a new lot of reagent, as there is a difference in values of QC material performed during kit lot change. It was found that there was no cumulative shift in the results of consecutive reagent lot change in our laboratory, and this may not be the case in all other laboratories. Hence, individual laboratories should verify their QC and patient results during reagent kit lot change, the step which is presently totally ignored by most of the laboratories.