Reliability and Clinical Validity of a Technique to Assess Lifetime Illicit Injection Drug Use

  A lifetime illicit injection drug history covers time periods when chemical assays for illicit substances (eg, urine, blood, or sputum analysis) cannot provide meaningful data. Self-report of the drug history allows for a historical perspective of complex data not observable by means of other data collection methods.

A lifetime injection drug history is often necessary when examining the impact of injection drug use on a physical health problem. For example, a lifetime injection drug history is useful when examining the lower extremities for changes related to chronic venous disorders (CVD). These disorders represent a clinical spectrum of manifestations to the lower extremities due to venous damage and include edema, change in skin color and texture, varicose veins, and painful, difficult to heal ulcers. Previous research conducted by the authors showed a strong relationship between years of injecting in the groin, legs, and feet and the development of CVD (Spearman’s rank order correlation coefficient = .47, P < .001) but this work did not include a reliability phase or account for the years a person did not inject drugs.¹

 The purpose of this study was to examine a technique of questioning persons about lifetime illicit injection drug use history including its reliability and relation to CVD.

Literature Review

  Self-report of drug use has long been used by researchers and clinicians to study and diagnosis substance abuse, to develop the treatment plan, and to explore risk for disease.² Long-term self report of drug use can be affected by memory failure, concealment of parts of one’s life, and over-reporting.³ Hser et al³ examined the reliability of self report of 323 men (M age = 24.4 years) addicted to narcotics and admitted to the California Civil Addict Program. Participants completed two face-to-face interviews 10 years apart about an overlapping period from 1970 to 1974-1975. Before the interview, the interviewers prepared a schematic time chart of the person’s legal history; this was used to provide temporal reference points for structuring the reported data. Interviews were conducted in multiple locations including the university, respondents’ homes, and other institutions. Frequency of narcotic use (eg, abstinence, less than once per week) was examined but not specific drugs or route of use. For categories in narcotic use with sufficient sample sizes to evaluate correlations for reliability, the test-retest reliabilities were .71 for abstinence, .65 for number of fixes per month, and .63 for daily use. The age of participants did not have an effect on reliability. Larger amounts of alcohol and narcotic use and extensive criminal records were associated with less consistency in self-report of illicit drug use.

  Shillington et al⁴ examined the stability of self-reported drug use for 223 persons (M age = 38 years; 53%, women) over 10 years at the St. Louis site of the National Institute of Health Epidemiologic Catchment Area study. Interviews were performed during 1981-1982 and 1992-1993. Participants were asked if they had ever used a certain substance more than five times and their age when they began use. Rates of total agreement plus incident cases (ie, saying no at baseline and yes at the 10-year follow-up) were 73.6% for amphetamines; 85.2%, cannabis; 85.7%, cocaine; and 82.5%, opiates. Shillington et al⁴ concluded that even after 10 years, participants were consistent in reporting substance abuse and that providing a time line may help participants pinpoint periods based on events occurring within their lives.

  Sobell et al² noted that few instruments are available to assess lifetime drug use. At a residential treatment facility, they interviewed 20 persons (M age = 23.8 years) twice approximately 19.5 days apart; the sample size was achieved after 12 persons (37.5% of sample) left the treatment facility. For the nine drug categories, participants were asked if they ever used drugs in the category, age first used, total years used, most typical route of administration, year last used, frequency of use in a specified past number of days, and if ever prescribed. Because the most typical route of use was requested, determining years of use for specific route was not possible. Intraclass correlations (ICC) for self report at times 1 and 2 for different drug classes for number of years of use were significant and ranged from .66 (hallucinogens) to .93 (narcotics). Demographic variables and drug-related events also had significant ICC values. The authors concluded that drug abusers in residential treatment could reliably report information about demographic variables, drug-related events, and past drug use. Participants had better recall for more frequently used drugs or the primary drug of choice.

  Czermak et al⁵ examined test-retest reliability (30- day time period) for the Lifetime Drug Use Questionnaire. For seven drug categories, the instrument examined abstinence-corrected total duration of regular use, mean number of consumption days per month of regular use, variability in consumption days per month of regular use, mean daily amount, and month variability in daily amount. The definition of regular consumption was left to the respondent; it was not standardized. The route of drug use was not assessed. Participants included 47 residents (M age = 29 years) in a drug rehabilitation center. This young sample had a mean duration of drug use of 4 to 7 years. The test-retest ICC values for abstinence-corrected total duration of regular use were significant for cannabinoids (.95), heroin (.94), and cocaine (.87). Significant ICC values were present for all drug variables (ie, mean number of consumption days per month of regular use, variability in consumption days per month of regular use, mean daily amount, and month variability in daily amount). The authors concluded the high reliability coefficients were partly due to the administration of the instrument in a familiar atmosphere of a therapeutic community.⁵ The details on drug use of the past can be reliably obtained from abstinent drug users in treatment.

  In summary, the reliability of drug histories has been examined for long-term drug use with various instruments. Participants tended to be young and as a result had shorter drug use histories.^2-5 Sobell et al² did not specifically examine injection use for different drug categories but instead asked participants the most usual route of drug administration. Czermak et al⁵ examined regular use, which could be different from a total history of injection drug use. Generally, the authors concluded that drug users accurately reported their drug history. Studies were not found about instruments that assessed lifetime years of injection drug use. Examination of reliability and validity of a lifetime self-report instrument of drug type and route that includes questions about nonuse history and site of injecting use may advance the evidence for drug histories in clinical practice.

  In the study described herein, the following research questions were examined: 1) What are the lifetime drug use and route of use for various drug categories? 2) What are the test-retest reliabilities for injection drug use questions and scales? 3) What is the construct validity for scales assessing years of injecting drug use into various sites of the body and clinical evidence of CVD?

Materials and Methods

  Design. A test-retest design was used to estimate test reliability. To approximate the ideal situation for test-retest reliability, the location of the interview, research staff, questionnaire, and research protocol were the same at Time 1 and Time 2. An interval of at least 30 days was enforced to limit bias related to memory for specific responses given in the prior test administration.

  Procedure. Participants were individually interviewed by two registered nurses at a methadone maintenance treatment center located in a large industrial city in the Midwest. The first 113 persons were told they were participants in the two-part phase of the study and were given an appointment to return for Time 2 approximately 1 month after Time 1. The second interview occurred a mean of 45.9 days (SD =12.9) from the first interview. Of these 113 persons from Time 1, nine (7.9%) were no longer receiving care in the treatment center and could not be contacted; thus, the reliability/validity sample consisted of 104 persons. Participants were not told that Time 2 was to examine their consistency in answering the questions. The entire research protocol was completed at both Time 1 and Time 2.

  The study began with completion of a questionnaire. The questionnaire included demographic, health history, and drug use items. The drug use questions followed the demographic and health history information so questions would appear less threatening. The questionnaire packet was read to participants to avoid frustration if a participant had a low reading level. Completion of the entire questionnaire packet took 45 to 60 minutes; the drug history questions involved 10 to 15 minutes of the total time. Following the questionnaire, participants removed clothing from their knees to feet to facilitate clinical assessment of the lower extremities for manifestations of CVD. The legs were assessed using the clinical portion of the Clinical-Etiology-Anatomy-Pathophysiology (CEAP) classification for CVD.⁶

  The two nurses obtained all data related to the drug history and performed the clinical leg assessment for CVD. The participant’s Time 1 research papers were in a folder and present at Time 2. After data were collected for Time 2, the research nurse opened the person’s folder and compared responses. If discrepancies were found, the participant was told what was said at Time 1 and Time 2 and asked to explain the difference, if possible. These notations were recorded on a separate form; the original responses on the research documents were not changed.

  Each participant was given an explanation of the study and assured that all information provided would be kept confidential, not reported to the treatment center staff, not placed in his/her treatment center record, and not reported to law enforcement. The study received a Certificate of Confidentiality from the National Institute of Health and was approved by a university’s Institutional Review Board. Participants were compensated $40 for each phase.

  Instruments. The Demographic Questionnaire obtained information about the gender, race, and age of each participant. The Health History Questionnaire asked the participant about 23 medical diagnoses/health problems such as hypertension, heart disease, diabetes mellitus, and arthritis. The medical diagnoses/health problems were tabulated for a total comorbidity score. The demographic and health history instruments have been previously used and have face validity. In this study, the test-retest reliability values for the demographic and health history questionnaires were .99 and .86, respectively.

  The Lifetime Injection History (LIH) Questionnaire was developed for this study to focus on lifetime injection drug use. When attempting to develop questions that examine lifetime history of injection drug use, challenges exist. This type of questioning requires the individual to make a quantitative judgment while remembering drug use from years past. In addition, the person may become confused by the type of questions such as total years of using illicit drugs, years using heroin in any manner, and years injecting heroin. The potential for measurement error is ever-present. The quantity of interest (years injecting any type of illicit drug excluding extended periods of nonuse,) is unlikely to be a number that many injection drug users have taken time to determine on their own.

  The LIH was not designed to be inclusive of all illicit drugs; it included major drug/route categories: amphetamines/ oral, amphetamines/injected, cocaine/snorted, cocaine/smoked, cocaine/injected, heroin/injected, heroin/ smoked or snorted, heroin plus cocaine/injected, nonprescription methadone/oral, nonprescription methadone/injected, other opiates/oral, other opiates/injected, and marijuana/smoked. The drug categories were those used in the National AIDS Research Project Risk Behavior Assessment Questionnaire.⁷ For each, the participant responded if the drug was ever used, age at first use, age of last use, and if there were years from first use to last use that the specific drug was not used. Upon completion of the specific drug/route category portion, the research nurse perused the responses and asked the person to verify the youngest and oldest age of ever injecting a drug. The youngest age of first injecting any drug was subtracted from the oldest age of injecting any drug to obtain the initial/trial value for total years of injection drug use. Previous work by this author indicated that periods of time existed within this initial/trial value that the participant verbalized not injecting any drug(s) – for example, while in jail, pregnant, or in treatment. In addition, because of the overlap of drug use, the lifetime years of not injecting drugs could be different than the years not injecting the specific drug categories. In order to obtain a total lifetime years of injection drug use that took into account the periods of nonuse, it was necessary to obtain an estimate of the total period of not injecting drugs. The participant was asked to think across all the years of injected drugs and identify the number of years, if any, injected drugs were never used. The years of not injecting obtained in this way was subtracted from the initial/trial years of injection drug use to get the total years of injecting. This method of questioning provided: 1) age at first use, age at last use, and nonuse of all drugs queried; 2) a self-report estimate of the total years not injecting any drugs during the lifetime period of injection drug use; and 3) total lifetime years of injecting any drug excluding years of nonuse (see Table 1).

  After obtaining this final quantity – total lifetime years of injecting any drug excluding periods of nonuse – the participant was asked three additional questions: 1) Of those years you injected drugs, on average how many times per day did you do so? 2) For the years you injected, how many years did you inject into the veins in your hands, arms, and above the waist? 3) For the years you injected, how many years did you inject in the veins of the groin, legs, and feet? (see Table 1). These estimates take into account periods of nonuse because of how total years injecting were calculated.

  Because venous system damage manifests in dysfunction primarily below the knee, the clinical portion of the CEAP classification was used to evaluate the legs for CVD.⁶ This is a descriptive classification. The clinical assessment has the following components: Class 0 – no visible or palpable signs of venous disease; Class 1 – telangiectasies or reticular veins; Class 2 – varicose veins, distinguished from reticular veins by diameter of >3 mm; Class 3 – edema; Class 4a – pigmentation or eczema; Class 4b – lipodermatosclerosis or atrophie blanche; Class 5 – healed venous ulcers; Class 6 – active venous ulcer.⁶ Each leg was assessed and assigned a CEAP clinical score according to the most marked clinical evidence. Chronic venous disorder assessment was scored by the registered nurse who was trained by one of the investigators. In these authors’ previous work, the interrater reliability of the CEAP was kappa = 1.0.¹ In the current study, the interrater reliability was .97 for the right leg and .94 for the left leg.

  Sample. The sample consisted of 104 clients (60 men, 44 women; age range from 30 to 60 years, M = 49.3, SD = 6.5) from a methadone maintenance treatment center. Of these participants, 66% were African- American (the remainder were Caucasian), 44.2% were single, and 60.6% were not employed. They had a mean comorbid health conditions score of 3.6 (SD = 2.3); the most common were hypertension (n = 47), arthritis (n = 44), depression (n = 35), hepatitis C (n = 30), and deep vein thrombosis (n = 23).

  Drug users included 88 persons who injected drugs and 16 persons who never injected drugs but used illegal substances by other routes. Among the 88 persons who injected drugs, 18 persons injected only in the hands, arms, and above the waist. The other 70 persons injected one or more years in the arms/upper-body and in the legs/lower-body – ie, injected in arms and legs for one or more years, either serially or concurrently.

  Data analysis. Separate analyses were conducted for: 1) the questionnaire items pertaining to lifetime use of the different drug type/route categories, 2) items identified as variables used in computation (eg, age of first injection, age last injecting, and years not injecting), and 3) the primary scales assessing years injecting in the arms/upper-body and years injecting in the legs/lower body. Analysis of covariance was used to examine the relationship between years of injecting in the arms/upper- versus legs/lower-body and a clinical measure of venous disorders (CEAP Clinical).

  Reliability of the primary variables and scales was assessed using ICC. Although r, the Pearson product moment correlation, may be used for this purpose, r for test-retest reliability assessment is limited in that it is invariant up to a linear transformation of scale – ie, it is unaffected by a constant shift in scale.^8,9 As a result, any tendency to report higher or lower scores at the second test administration or to report proportionally higher/lower scores would result in the same reliability in either case. The ICC is sensitive to these kinds of shifts in scale and often is recommended when absolute agreement is of interest. While conceptually the same ratio of the true variance to total variance is involved, Shrout and Fleiss¹⁰ reported six different ways of computing the ICC that give very different estimates. In their example, ICC ranged from .17 to .91 for the same data set. Advances in computing software technology (eg, SPSS, Chicago, Ill) have made it easy to report any one of these six ICCs. The ICC of interest in the current study assumes 1) random participants and fixed occasions of measurement, 2) absolute rather than relative agreement, and 3) reliability for a single test administration rather than an average of multiple administrations.

  Reliability of the dichotomously scored questions involving lifetime use and route were assessed using percent agreement (concordance), phi, and kappa. The phi statistic is by definition equal to r; because only two categories are available, a certain amount of agreement is expected by chance alone. The kappa statistic is a measure of percent agreement adjusted for chance.¹¹ When used for reliability assessment, kappa values >.81 are regarded as almost perfect and values from .61 to .80 are regarded as substantial.¹²

Results

  Types of drugs/route used. Table 2 shows the relative frequency of the different drug/route categories reported. Because all subjects were recruited from one clinic, rates are only representative of the pattern of drug use behavior of clients in a particular urban setting; however, they are useful in interpreting the reliability statistics. The highest drug/route of choice was injection of heroin (84%). Snorting heroin was the second most prevalent choice (77%). Participants were consistent in reporting this kind of information with concordance ranging from 86.5% for marijuana use to 100% for injecting heroin. The phi and kappa statistics were comparably high and in virtually perfect agreement. The kappa for injecting heroin was 1.0. This was followed by ever used inject cocaine (.87), other oral opiates (.85), oral amphetamines (.84), and mixing heroin and cocaine (speed balling) (.83). The lowest kappa was ever injected nonprescription methadone (.50).

  Primary scales and variables: reliability. Table 3 presents the descriptive statistics and reliability results of the injection drug use variables and scales for respondents who used injected drugs. As shown in the Table 3, ICC ranged from .70 for years injecting in the veins or the groin, legs, or feet to .96 for age of first injecting. This indicates good absolute agreement. For example, the mean reported age at first injection was 21.9 years (SD = 6.67) at Time 1 and 22.14 years (SD = 6.76) at Time 2. The difference between the Time 1 and Time 2 report of this variable was only -0.24 which is 3.57% (= 0.24/[(6.67 + 6.76) (0.5)] (100)) of the scale deviation. Similarly, small differences in agreement were found for other variables. The mean difference as a proportion of the variable/scale standard deviation ranged from 3% to 12%. Reliability for years of injecting (total years injecting .90) was higher than the reliability for years injecting at specific sites (eg, upper [.79] versus lower [.70] body).

  The descriptive statistics on injection drug history, using Time 1 data for illustrative purposes, are revealing. The mean lifetime years of injecting any drug in any part of the body excluding years of not injecting drug use (Years Injecting-Total) was 15.81 years (SD = 9.93). The mean number of years of not injecting any drugs was 4.15 (SD = 5.40). This figure is 21% (= 4.15/[41.84 - 21.9] x 100) of the raw or unadjusted years of injection drug use; if not taken into account, it would result in a substantial overestimation in lifetime use.

  Validity. The CEAP clinical scores were used to examine validity of the LIH scales that distinguish arm/upper-body versus leg/lower-body years of injection. The CEAP clinical assessment was performed on both legs at both interviews (see Table 4). Across both time periods and legs, 5% to 7% of participants had no evidence of venous disease (Class 0); 45% to 48%, mild disease (Classes 1-3), and 47% to 49%, advanced disease (Classes 4 - 6). Apparent in the table is the high correlation between left and right leg classification at Time 1, r = .90, and Time 2, r = .90. The test-retest correlations were high for each leg; r = .96 and .95, left and right legs, respectively. Given the high correlations between legs and time of testing, the CEAP clinical classification of the worst leg from Time 1 was used to examine validity.

   The authors’ prior work¹ suggests that years of leg injection should be more strongly related to CVD risk than years of arm injections. If the LIH scales have validity for distinguishing years injecting in different parts of the body, a classification based on these scales should be predictive of participants at more or less risk depending on years of injection drug use (IDU) and site of injection. To test this, the participants were classified into one of six risk groups based on a cross classification of years of injecting in arms/upper-body and legs/lower-body. The scales were dichotomized at the median years of injected drug use. Each scale had three levels; 0 years IDU, below median years IDU, and above median years IDU. The median of years of injecting in the arms/upper-body was 10.5 and the median of years injecting in the legs/lower-body was 7. This cross-classification resulted in nine groups, but only six were needed to classify all participants – two of the cells were empty (no participants injected in only the legs) and the two subgroups of participants who injected only in arms were collapsed into a single group in order to simplify the comparisons. The decision of collapsing these groups was done before data were examined.

  A one-way analysis of covariance (ANCOVA) was performed with IDU groups as the six-level independent variable (IV) and the CEAP clinical scores of the worst leg from the Time 1 assessment as the dependent variable (DV). The ANCOVA included four covariates: age, gender, body mass index, and comorbidity score. The omnibus main effect of IDU group was highly significant, (F[5, 94] = 3.80, P = .004), with a partial regression coefficient = .32, indicating good validity for the LIH scales and classification. The classification accounted for 32% of the variance in CVD. Age was the only covariate that reached significance (F[1,94] = 4.64, P = .034). Post hoc analysis of differences between group means was performed using the Newman-Keuls (P <.05) procedure. The results, as can be seen in Figure 1, showed disease severity increased with increasing years of leg injection but not with increased years of arm injection. Each of the significant post hoc comparisons supports this interpretation. Briefly, Group 3 was significantly greater than Group 2 indicating that leg injection, even for 7 years or less, resulted in more severe disease than any number of years of arm injection. Group 6 was significantly greater than Group 4, indicating more severe disease with increasing years of leg injection controlling for years of arm injection. Lastly, regardless of years injecting in the arms, persons who injected in their legs for more than 7 years (Groups 5 and 6) were diagnosed with more severe disease than those who did not use injected drugs (Group 1) or those who injected only in the arms (Group 2).

Discussion

  The psychometric properties of the LIH were developed and examined to study CVD in persons with a history of injection drug use. Because many years may pass between the first use of injected drugs and the appearance of CVD symptoms, biochemical markers in urine, blood, or saliva of recent drug use would not provide the information needed about lifetime drug use. A lifetime injection drug use history was needed to assess CVD risk and potential venous damage in a cross-sectional design. Participants were able to consistently recall drug/route history information as evidenced by high kappa values (median = .75). This finding is similar to reports by others of the recall of drug use irrespective of route. Czermak et al⁵ reported high test-retest reliabilities for heroin (.82) and cocaine (.82) but they focused on the most recent 30-day period. Sobell et al² reported a high interclass correlation (.93) for the number of years of narcoticuse but the study was limited by a small sample size (20 persons). Hser et al³ reported test-retest reliabilities for frequency of narcotic use with interviews performed 10 years apart; heavier narcotic and alcohol use, not age, affected reporting. Needles et al¹³ noted high test-retest reliabilities for crack (.69), cocaine (.78), and heroin (.78). Examining the stability of substance use over a 10-year follow up period, Shillington et al⁴ reported high agreement for cocaine (85.7%) and opiates (82.5%). These data strengthen the use of self report of drug histories even for long periods of time.

  With regard to injection drug use per se, ICC reliability values ranged from .70 for years injecting in the veins of the groin, legs, or feet to .96 for age of first injecting. The mean lifetime years of injecting any drug in any part of the body excluding years of noninjection drug use also had a high test-retest reliability (.96). It is important to reiterate that the lifetime years of injecting would be 21% higher if years of nonuse were not taken into account.

  The LIH Questionnaire was examined for validity with the clinical classification of the CEAP, which noted that clinical scores increased with increasing years of leg injection but not with increased years of arm injection. In the authors’ previous work with a different sample of methadone treatment center clients, a high correlation (Spearman’s rank order correlation = .47) was found between the CEAP and years injecting in the lower extremities.¹ This further substantiates the link between CVD and injecting in the groin, legs, or feet.

  Clinicians using this or another similar instrument need to realize that many factors affect accuracy of responses (eg, confidentiality, rapport with client, physical and psychological condition of the client); thus, such information needs to be carefully elicited in a nonjudgmental manner. An injection drug use history can be important for medical conditions such as CVD that evolve over time. Clinical manifestations of damage to veins of the lower extremities may not occur for years after the initial insult and may continue to evolve with age. Clinicians need to be aware of drug use questions to ask and to consider the effect of drug use on the development of CVD. Patients can easily hide their legs and not discuss problems they are having with them. Although more research is needed, implementing leg protective strategies such as leg elevation, ankle exercises, and support stocking use may promote leg health and lessen the clinical manifestations of CVD.

Limitations

  This study had limitations. The participants were volunteers from an urban methadone treatment program. It is not known how representative they were of persons who have used injected drugs. Some responses may have been influenced by the research environment because it was one in which the participants felt comfortable. The LIH Questionnaire was read to participants; thus, it is not known whether these persons can complete it on their own or if it must remain a tool that the practitioner reads. Because the questionnaire takes about 10 minutes to administer, practitioners would need to decide if the instrument would fit into the clinic practice where time spent with patients is limited.

  Because the sample size was small (N = 104), researchers were not able to examine each lower extremity site (ie, groin, legs, and feet) of injecting separately for its relation to the clinical CEAP. Regardless, damage to the venous system of the lower extremities is related to CVD. In future research, asking about years of injecting in specific sites of the legs may be warranted.

Conclusion

  The results of this study add support to information that persons who used illicit drugs can report accurate lifetime drug use histories, including response to specific substance abuse questions regarding, drug, route, site of injection, years of use, and years of nonuse. Self-report of long-term use appears to be a reliable and valid measure. The LIH provided reliable and valid information on the specific lifetime use of injection drugs including drug, route, site of injection, years of use, and years of nonuse. In addition, LIH scales accounted for 32% of the variance in the clinical CEAP scores and showed that injecting in the legs but not the arms was related to disease severity. Interrater reliability for the clinical portion of the venous disease assessment tool (the CEAP) was high. Data analysis showed that manifestations of venous disease were highly correlated across legs. Hence, the LIH can be used in research and clinical studies seeking to understand complications evolving from injection drug use.

Acknowledgments

  This project was funded by the National Institute of Nursing Research/National Institute of Health (NINR/NIH), Effect of Drug Use on the Legs: Chronic Venous Insufficiency, Mobility and Pain, R01 NR009264-01.   The authors thank Joyce Peck, BSN, RN; Terri Gibbons; and Cynthia Birk, BA, CET, research assistants; and Lucilla Ryder, Executive Director, and her staff at Star Center Inc. for their support and encouragement of research in addiction healthcare.

1. Pieper B, Templin T. Chronic venous insufficiency in persons with a history of injection drug use. Res Nurs Health. 2001;24(5):423-432.

2. Sobell LC, Kwan E, Sobell MB. Reliability of a drug history questionnaire (DHQ). Addict Behav. 1995;20(2):233-241.

3. Hser Y, Anglin MD, Chou C. Reliability of retrospective self-report by narcotics addicts. Psychol Assess. 1992;4:207-213.

4. Shillington AM, Cottler LB,Mager DE, Compton III WM. Self-report stability for substance use over 10 years: data from the St. Louis Epidemiologic Catchment study. Drug Alcohol Depend. 1995;40(2):103-109.

5. Czermak C, Lehofer M, Gasser-Steiner P, et al. Test-retest reliability of a lifetime drug use questionnaire. Addict Behav. 2005;30(2):361-368.

6. Eklof B, Rutherford RB, Bergan JJ, et al. Revision of the CEAP Classification for chronic venous disorders: Consensus statement. J Vasc Surg. 2004;40(6):1248-1252.

7. National Institute on Drug Abuse. National AIDS Research Project. Risk Behavior Assessment Questionnaire. National Institute on Drug Abuse, Washington, DC, 1993.

8. Lord FM, Novick MR. Statistical Theories of Mental Test Scores, with Contributions by Alan Birnbaum. Reading,Mass: Addison-Wesley;1968;314.

9. McDonald RP. Test Theory: A Unified Treatment. Mahwah, NJ: Lawrence Erlbaum Association;1999:66-71.

10. Shrout PE, Fleiss JL. Intraclass correlations: uses in assessing reliability. Psycho Bull. 1979;86:420-428.

11. Fleiss JL. Statistical Methods for Rates and Proportions. Somerset, NJ: John Wiley and Sons;1981.

12. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33(1):159-174.

13. Needle R, Weatherby N, Chitwood D, et al. Reliability of self-reported HIV risk behaviors of drug users. Psychol Addict Behav. 1995;9(4):242-250.