Research focus

Measures of self-reported illicit drug use in research

Studies of illicit drug use often rely on people's self-reported use. The collection of self-report measures provides a convenient, relatively inexpensive, and non-invasive method for obtaining information about illicit drug use and, in some cases, is the only feasible methodology that can address research questions of interest. However, concerns about the reliability of such reports are often raised^1-3. Reasons proposed for their potential inaccuracy relate to stigma, negative consequences of disclosing use, poor recall on the participant's part, and the variable content of illicit drugs^3,4.

How do we evaluate the accuracy of self-reported illicit drug use?

Biochemical markers of illicit drugs in biological samples are often used as an adjunct measure of illicit drug use. Blood, urine, and saliva are some of the more common samples used for this purpose. Most drugs are detectable for up to 48–72 hours after use in these samples, but the detection window may be potentially extended for some drugs (e.g., cannabis) and considerably longer for other biological matrices (e.g., hair)^5,6. The recommended method for detecting biochemical markers in biological samples typically includes confirmation using laboratory testing procedures, however, point-of-care screening tests alone are often used as a less costly and faster alternative.

Although biological measures of illicit drug use may be considered more ‘objective’ than self-report, they are not perfect. Practical limitations relating to when drug use last occurred, the metabolite(s) being tested, the detectability of the metabolite in the biological sample and the quality of the testing method might mean that discrepancies, when compared with self-report, cannot always be attributed to misreporting of use by a person⁷.

A common method for comparing self-reported drug use and biological tests results involve the construction of a 2x2 table (see four cells in the top right of Figure 1). For a given sample, the total number of cases in which the measures of drug use are both positive (a), both negative (d), or the two scenarios in which they differ (b and c) are evaluated. False positives (b) and false negatives (c) are determined according to the way in which the index test (self-report) misaligns with the clinical reference or ‘gold’ standard (biological sample). Values in the 2x2 table can then be used to calculate a range of diagnostic testing measures, including overall agreement, sensitivity, specificity, false omission rates and false discovery rates.

Figure 1. Visual description of measures evaluated from a mock 2x2 table  

Reproduced from Bharat et al. Addiction (2023)⁸.

How does self-reported illicit drug use and biologically tested measures of illicit drug use compare?

A study recently published in Addiction (titled ‘Agreement between self-reported illicit drug use and biological samples: A systematic review and meta-analysis’) comprehensively examines the evidence for agreement between self-reported and biologically measured illicit drug use across all major illicit drug classes and biological indicators and the extent to which this varies across the populations and settings in which these measures are collected.

From 7,924 studies, data from 207 eligible studies were extracted. Pooled estimates of diagnostic measures were evaluated by drug class using random-effect regression models. Heterogeneity was assessed by inspecting forest plots. Analyses were restricted to the most prevalent biological samples and drug classes available. For biological samples, this included urine, saliva, and hair. Drug classes included cannabis, cocaine, methamphetamine, heroin, benzodiazepines, methadone, MDMA, buprenorphine and fentanyl.

Overall agreement ranged from good to excellent i.e., kappa estimates were all >0.79. However, there was substantial between-study variability in other summary estimates of sensitivity, specificity, false omission rates, and false discovery rates. False omission rates were generally low, while false discovery rates varied by setting. Specificity was generally high, but sensitivity varied by drug, sample type and setting. Detailed results for all major drug classes, biological indicators, populations, and settings can be found in the published article.

What are some of the characteristics that might impact agreement of these measures?

For urine and saliva, the above review focused analyses on two time intervals of self-reported drug use: first, past 1-4-day self-reported drug use, as this aligns most closely with the detection window for most testing procedures; and secondly, past month (28-31 days) self-reported drug use, as this is a common time-frame used in epidemiological studies. Findings indicated that longer timeframes of self-reported drug use tended to provide better agreement. This suggests individuals may be more comfortable reporting drug use that could be perceived as being more temporally distant to the time of self-report.

Willingness to disclose illicit drug use was expected to vary based on a person’s perception that disclosing use would lead to adverse consequences. Included studies were assessed for whether the purpose of data collection, the person/s conducting the interview, or the way findings would be used could be perceived by participants as potentially resulting in adverse impacts. Agreement was found to be generally higher in randomised clinical trials and in situations in which there were no consequences. This reinforces the importance of conducting research that puts in place strategies to ensure individuals feel confident they will not experience stigma or negative consequences because of reporting use.

Agreement also appeared to be influenced by the level of drug use in the population, with sensitivity higher among cohorts with a high proportion of people who use drugs. This is consistent with what one might expect from studies recruiting people based on their having dependence or high drug use, where prior drug use is already acknowledged as a condition of study participation.

Although there was no evidence to suggest that agreement between self-report and biological measures was influenced by the quality of the clinical reference standard, half of all studies used methods for testing the biological samples that are considered at high risk of bias. Most studies used drug screening tests which may misclassify results in either direction; that is, an index test result that is classified as a false positive may in fact be a true positive, and a false negative index test result may be a true negative. The implications of this will vary depending on the application. Nevertheless, it highlights the importance of using recommended methods in cases where there are consequences for a given result (e.g., in cases where the outcome of a test determines whether to initiate an individual on a specific program).

Conclusion

While there are limitations associated with self-report and biological testing to measure illicit drug use, the overall agreement between the two methods is high. Given the relative benefits of measuring drug use via self-report in terms of reduced cost and increased convenience, self-report provides a useful measure of illicit drug use for research purposes.

More information     

The full open-access publication is available at Addiction. 

For more information, please contact Dr Chrianna Bharat.

References

1. Percy A, McAlister S, Higgins K, McCrystal P, Thornton M. Response consistency in young adolescents’ drug use self‐reports: a recanting rate analysis. Addiction 2005; 100(2): 189-96.
2. Johnson TP. Sources of error in substance use prevalence surveys. International Scholarly Research Notices 2014; 2014.
3. Althubaiti A. Information bias in health research: definition, pitfalls, and adjustment methods. Journal of Multidisciplinary Healthcare 2016; 9: 211.
4. Richter L, Johnson PB. Current methods of assessing substance use: A review of strengths, problems, and developments. Journal of Drug Issues 2001; 31(4): 809-32.
5. United Nations International Drug Control Programme. Recommended Methods for the Detection and Assay of Heroin, Cannabinoids, Cocaine, Amphetamine, Methamphetamine, and Ring-Substituted Amphetamine Derivatives in Biological Specimens. United Nations: New York, 1995.
6. United Nations Office on Drugs Crime. Guidelines for Testing Drugs under International Control in Hair, Sweat and Oral Fluid. United Nations New York; 2014.
7. Wolff K, Farrell M, Marsden J, et al. A review of biological indicators of illicit drug use, practical considerations and clinical usefulness. Addiction 1999; 94(9): 1279-98.
8. Bharat C, Webb P, Wilkinson Z, et al. Agreement between self-reported illicit drug use and biological samples: a systematic review and meta-analysis. Addiction 2023; n/a(n/a).