Urology Nurses Online: ARTICLES

Have you thought about using or developing continence assessment tools to track treatment outcomes for your patients? Are measurement tools in the literature inadequate for your purposes? Would you be interested in easy to use assessment/documentation tools you could start using now? If you answered "yes" to any of the above questions, the information provided in this article about new continence tools and tool development, could be very useful in your clinical practice and in designing your own tools. Particularly, this article discusses the process of tool development, the importance of tool validity and reliability, when and how to get help with tool design, and the impact proper documentation tools have on continence therapies.

CONTINENCE PROGRAM DEVELOPMENT

A new Spina Bifida Continence Program was developed in 1994 at the Spina Bifida Center, St. Michael Hospital, Milwaukee, Wisconsin. The program objective was improvement of urinary and fecal continence in people with myelomeningocele. Incontinence is a lifelong challenge for many in this population. The three primary components of the Continence Program were: Continence Coaching; Therapeutic Electrical Stimulation (TES) Therapy; and Neuromuscular Re-education (biofeedback).

"Continence Coaching" is the active process of: (1) determining a patient's elimination habits and patterns, (2) identifying the changes needed to improve function, and (3) after exploring options to achieve the changes, choosing an appropriate treatment plan. Following implementation of the treatment plan and a reasonable amount of time for improvement to occur, outcomes should be assessed and documented. The treatment plan is then continued or revised, based on the outcomes.

TES is a home treatment administered to a child at night while the child sleeps. Two sets of electrodes are placed on the skin in specific areas at bedtime. A pocket-sized stimulator is set to deliver a low level of electrical stimulation intermittently throughout the night. Left and right side electrode placement is alternated six out of seven consecutive nights. The intent is to provide a low level of stimulation to muscles and nerves involved with continence, and to reverse any disuse muscle atrophy. Follow-up occurs at six-month intervals to evaluate progress towards specific continence goals.

Neuromuscular re-education is instituted to teach the child how to control the muscles involved with continence, when those muscles have been strengthened. It is the final component of the Spina Bifida Continence Program. Again, at regular intervals, progress towards specific continence goals is evaluated.

Depending on diagnosis and circumstances, these primary components may be used independently or collectively. Evaluation tools to assess and document clinical outcomes and progress towards goals are essential to each component.

CONTINENCE ISSUES ARE COMPLEX

Continence issues are complex, highly individualized, and influenced by many variables. Assessment and documentation of a patient's continence status must address the variables relevant to that patient. For example, dietary habits, exercise, medications, time of meals, type of clothing, cognitive abilities, sensation level, and degree of independent functioning may all affect continence. Use of protective pads is another example. The number of pads used a day, type of absorbency, and degree of saturation all need to be considered. Another variable is the need to change clothing due to incontinence. Frequency of change, time of change, and whether the change is due to urine, feces, or both, need to be determined.

IN SEARCH OF TOOLS

Quality measurement tools - those that assess and document functional characteristics related to continence - need to be applied before interventions are started, as well as after, to determine effectiveness of the interventions. Useful measurement tools have several key characteristics. The tools need to be: 1) clear in content so that questions and answer options are understood by both the interviewer and the respondent, 2) comprehensive as to the variables being measured, 3) easy to use, 4) time efficient in terms of administering the tool, scoring patient responses, and applying outcome data to the patient's treatment plan, and 5) valid and reliable (See Table 1). Useful tools are constructed in an organized way that will allow data to be analyzed and conclusions drawn regarding treatment effectiveness.

Finding standardized measurement tools that have the desired characteristics can be very challenging. Review of the relevant literature revealed a variety of continence assessment tools as well as recommended content for such tools (Sidani & Woodtli, 1994; Sackett, 1993; USDHHS, AHCPR, 1992; McDowell, 1994; Joseph, 1992; ICSCS, 1990).^1,2,3,4,5,6 Unfortunately, none of the tools was sufficiently comprehensive or adaptable to address the authors' needs, such as serial evaluations of treatment outcomes, so the authors decided to develop treatment specific measurement tools.

Using information in the literature, ten years of experience with neurourologically impaired patients and their families, as well as observed and reported signs and symptoms related to incontinence, a preliminary continence assessment tool was drafted in 1994. The draft tool was a questionnaire, designed for nurses to use as they interviewed and examined patients. It also served as a documentation tool. The intent was twofold: 1) to achieve consistency in asking the same questions each time a patient was evaluated, and 2) to compare current function with past function. The patient's or parent's answers and comments were written directly on the tool, and the tool was attached to the patient's medical record.

The draft tool contained the following five categories related to continence: voiding characteristics; functional independence; sensation level; motivational level; and bowel characteristics. Each category had 1 to 6 items, for a total of 13 items. Using a Likert scaling procedure, each item was a specific question related to continence and was scored as 1 to 6. For example, the question, "What is the degree of urinary incontinence?" offered the following scoring options: 1) always wet, 2) moderately wet, 3) damp, 4) wet with activity, 5) usually dry, 6) always dry. The lowest score described the "worst" or "least favorable" function, and the highest score described the "best" or "most favorable" function.

Over the next year, the draft tool was used approximately fifty times by three different Spina Bifida Clinic nurses who were trained to use the tool. The questionnaire was used to evaluate and document patient progress related to TES therapy. Unfortunately, more questions than answers were generated when the nurses met as a group to discuss patient and family responses. Variability in the way questions were asked, for example, presented inconsistencies in scoring. A lack of clarity within the tool similarly prevented uniform application and interpretation. The tool was also limited in scope regarding number of variables addressed. Recognizing that the draft tool's limitations could potentially compromise measurement outcomes related to the new Continence Program, the authors decided to seek help from an off-site research agency.

FORGING FORWARD IN TOOL DESIGN

Following the advice of a colleague, the authors contacted Non-Profit Research Consulting Services (NRCS) in January 1996, and a meeting was arranged. The authors' primary needs were identified as: 1) statistical analysis of TES data, and 2) improvement of the draft tool. NRCS was asked to draft an action plan to accomplish what was needed, with specific activities, time lines, and fees. Based on NRCS's plan, in February 1996, the authors submitted a proposal to the St. Michael Hospital Foundation requesting a financial grant to cover NRCS expenses. The request was approved, and NRCS was contracted to analyze clinical outcome data related to TES, and to facilitate refinement of the continence assessment tool. Two NRCS consultants worked on the project. One was an expert in test construction, and the other was a statistician. The authors worked very closely with the NRCS consultants to facilitate statistical analysis of the TES data, and to refine the tool. This was accomplished through frequent mail and facsimile correspondence, extended telephone conversations, and periodic meetings. (Preliminary outcomes of TES for improved continence have been published by Balcom, et al.)⁷

During the ensuing months, the authors concentrated on improving the quality of the assessment tool by identifying unclear, non-discriminating areas; discussing and changing wording; expanding categories and items; and by incorporating feedback from the NRCS consultants. Comments from patients and parents were also helpful. When the nurses asked the questions from the draft tool, they made notations about which items were confusing to patients or parents and what their respective ideas were about removing the confusion. For example, the items addressing stool consistency and amount needed graphic description before patients and parents could differentiate answers. Revisiting the literature was also helpful, particularly in deciding how to further describe and differentiate items to be scored. (O'Brien, Bradford & Gibb, 1995; Leibold, Braun, Peterson & Col, 1995; Beckman, 1995; Dmochowski, 1996; Ewalt & Bauer, 1996). ^8,9.10.11.12

Incorporating the foregoing, the authors continued to revise the tool. The goal was to have the revised tool ready for the June 1996 Spina Bifida Association of America (SBAA) Annual Conference. The tool was ready, and had expanded from 5 categories to 12, and from 12 items to 76. It was comprehensive and tested what the authors agreed needed to be tested, that is, whether TES was effective to improve continence outcomes in the myelomeningocele population.

At the 1996 SBAA Conference, members of the SBAA Nursing Council were invited to critique the Continence Assessment - Nursing Guide for clarity, usefulness, and completeness. Those who were interested in offering feedback signed a participant list and received a copy of the assessment tool. The participant list documented the number of copies that were in circulation and who had them, so that a return reminder could be sent to a participant if necessary. The following instructions were printed at the top and far right of each page, "Note: Please write comments next to the area of concern regarding clarity, usefulness, and completeness." One-third of the page from top to bottom was left blank for comments. Participants were provided an addressed, postage prepaid envelope in which to return their responses to the authors.

To gain broader representation and more feedback, Pediatric Nurse Urology Specialists who attended the American Academy of Pediatrics' Nurse Affiliate Program in October 1996 were also invited to critique the tool. The authors varied their approach for this group. Each person who attended the Nurse Affiliate Program received a copy of the continence assessment tool, accompanied by a cover letter that asked them to critique the tool. Again, the potential respondents were provided an addressed, postage prepaid envelope to return their responses.

A total of 35 advance practice nurses (15 from the Spina Bifida Group and 20 from the Pediatric Urology Group) responded. Many helpful recommendations were offered. All responses were considered important. The authors met several times to review, analyze, and synthesize responses for each item. Item content changes were based on consensus of the authors, guided by the NRCS tool construction consultant. Although the categories remained the same, most item changes occurred in the way items were stated.

ESTABLISHING USEFUL TOOLS

Validity and reliability are considered vital elements in establishing the accuracy of a measure (Lo-Biondo-Wood & Haber, 1994).¹³ A valid instrument measures what it purports to measure. Reliability is the consistency with which the instrument measures the constructs for which it has been created. Inter-rater reliability is the extent to which two or more independent raters agree in assigning a particular rating to a particular measure. Internal reliability is also an important dimension of reliability. It is the extent to which measures are consistent with one another within the instrument.

Based on the processes used to develop and refine the Continence Assessment - Nurses Guide (e.g., the literature, extensive experience, expert/advanced practitioners, instrument design consultants), the authors were confident that the finalized tool had content validity, a non-statistical type of validity. To verify that the tool had content validity and to obtain help establishing the tool's reliability, the authors contacted Research Associates (RA), an independent research and consulting firm (the NRCS contract had expired). RA was known for its test construction and research methodology in the areas of health and psychology. After reviewing the instrument, RA concurred that, because of the methodology used to construct the tool, the finalized continence assessment tool had adequate content validity. Reliability, however, was yet unknown.

Between June and September 1997, the principal author met with six nurse raters and instructed them on how to use the tool. The instruction included: 1) reading through the tool with the nurses, 2) discussing patient selection, 3) explaining that with the patient's permission, two nurses would be present during the assessment, but only one nurse would ask the questions, 4) explaining that the nurses should avoid discussing which number was circled on the continence assessment tool after the patient responded to the question, 5) directing the nurses to ask only questions 1.1 - 4.6 and 10.1 - 10.4, for a total of 41 items (the remaining items were either TES specific, urodynamic/radiographic interpretations, or physical exam findings), and (6) addressing any remaining concerns the nurses had. Day shift nurses from three separate areas at St. Michael Hospital - a Spina Bifida Clinic, Sub-Acute Care Unit, and a Rehabilitation Unit - participated in testing the tool. These areas were selected because the nurses from those units shared a common interest in outcome measurements related to continence, patients on their units had incontinence problems, the nurses wanted to participate in research activities, and because the units ranged from out-patient, to in-patient, to "in-between." This variety in the experiences of the raters provided the potential for demonstrating the versatility of the tool. Each test pair consisted of a Nurse Manager and a Staff Nurse, who were trained to use the tool at the same time. Each nurse pair was instructed to interview ten patients, using alphabetically coded photocopies of the tool. Coding the tool was necessary to determine inter-rater reliability. Although 30 patients yielded a small sample size, it provided an adequate base upon which to calculate appropriate reliability statistics. Data was forwarded to RA for statistical analysis.

Data was analyzed using SPSS, a commonly used statistical analysis package (SPSS, Inc., 1996 ).¹⁴ Split half reliability was computed for the entire sample, using Cronbach Alpha. This was a gross measure for the entire tool. A correlation coefficient of .93 was obtained. Inter-rater reliability was computed by correlating the scores of the raters in each nurse pair. Correlation coefficients ranged from .73 to 1.0, and were considered to reflect a moderately high to high level of reliability (Satterlee, William, 1997).¹⁵

To increase the tool's reliability, clarification between a rating of "5" (best or most favorable function) and "NA" (not applicable) was advised, as raters were not consistent in their use of those ratings. The authors appreciated the advice, and clarification was achieved with a minor revision of the written instructions for use of the tool. The outcome of this change in improving reliability remains untested.

CONTINENCE ASSESSMENT NURSES' GUIDE (CAN-GUIDE) EMERGES

The final version of the tool is "Continence Assessment Nurses' Guide" (CAN-Guide) (Figure 1). The acronym not only applies to the assessment tool, but has significant implications for the importance nursing practice has in improving the continence of the patients nurses serve. CAN-Guide is composed of 76 items in 12 categories related to continence. The 12 categories are: Usual Bladder Patterns, including degrees of dryness, pads used, and clothing change; Usual Bowel Patterns, including toileting, evacuation method, and use of laxatives or stool softeners; Habits Contributing to Independent Self-Care, such as changing pads and clothing; Dietary Contributions to Bowel Continence, including fiber foods, fluids, and fiber supplements; Physical Exam Related to TES, such as skin integrity, bulbocavernous reflex, and rectal tone; Sensory Exam, Somatosensory Evoked Potentials (SSEPs) to determine baseline and subsequent sensory level, expected to improve with TES ; Kidney and Bladder Testing, such as renal ultrasound and voiding cystourethrogram; Urodynamic Studies, including urine testing, cystometrogram, and urethral pressure profile studies; Motivation Level, including the patient, parent, and interviewer perspectives; Medications; Summary of Over-All Progress, again, including the patient, parent, and interviewer perspectives; and Continence Treatments, which lists the total hours on TES and the number of Intravesical Bladder Stimulation treatments since the last evaluation (see Table 2).

At first glance, the tool may appear to be too TES specific to have any general value. A closer look, however, reveals that most of the items can provide useful information to help any person improve his or her continence. In addition to the 41 broadly applicable items used to determine inter-rater reliability, motivational levels, radiographic outcomes, urodynamic findings, compliance with the treatment plan, perceived progress toward bladder goals, and perceived progress toward bowel goals, together raise the count to about 50 items relevant to continence assessment. Using the same format, patient specific item modification could be easily accomplished.

CAN-Guide items are scored using a 1 to 5 scale. One is the least desirable characteristic (worst function) and 5 is the most desirable characteristic (best function). Each of the 12 categories is scored independently, listing the highest total score possible and the patient's score. If an item or items listed in a particular category are not appropriate for a specific patient, those items can be omitted. For example, if a patient does not self-catheterize, items 1.4 and 1.5 are omitted, reducing the composite score from 15 to 13 items in that category. Based on the five point scale, the patient's total possible points in that category is 65. The patient is queried about only those items that are pertinent to him or her, and the total possible score is adjusted accordingly.

The patient's composite score compared to the highest possible score gives the patient and practitioner data regarding which continence variables to focus on, what goals to establish, and what interventions need to occur. A score on a single item is not very useful. Collectively, however, the items in a category do portray patterns of function. Data from the continence tool may expedite the determination of the most appropriate treatment plan, which is especially important to the patient. Since the treatment plan should be evaluated over time, the development of a companion tool seemed advantageous.

CONTINENCE ASSESSMENT NURSES' OUTCOME RECORD (CANOR)

CAN-Guide has a companion scoring tool called the Continence Assessment Nurses' Outcome Record (CANOR) (Figure 2). CANOR is used to numerically document outcomes (responses to treatment) over time. The tool has six columns where data can be entered on six separate dates. Initial (baseline) data are entered on the tool prior to treatment. Using this method, the patient serves as his or her own control. As data are entered at subsequent follow-up evaluations, longitudinal effects of the interventions can be meaningfully monitored. This can help prevent arbitrary and unnecessary changes in the treatment plan, as well as form a basis for changes. A systemic record, such as CANOR, is also an easy way to show progress to the patient - if not in all categories, at least in some. Some progress is likely to encourage and motivate patients to keep trying.

As previously stated, CAN-Guide and CANOR were both developed to help better evaluate continence outcomes related to TES. Patients and their families were initially told that they would be on TES for two to four years for improved continence. This conveyed the fact that TES was not a fast process. Patients and families were also told that because of the newness of the treatment and the lack of published evidence of outcome effectiveness, they were "pioneers," and that it was possible that no change would occur. Considering mutual patient, family, and professional goals, if reasonable progress is not made at the one year point, based on the emerging functional pattern on CANOR, continuing TES for continence is discouraged.

TES patients are re-evaluated every six months, which includes repeating CAN-Guide and documenting outcomes on CANOR. CAN-Guide is placed in the patient's chart upon initial assessment. Thereafter, it is used only to ask the questions and gather data. The answers are documented in the patient's chart on CANOR. Nurses (the same ones who were involved with improving the instrument) are now consistent in what they ask and how they score answers. As new nurses participate, their reliability in this area will have to be tested.

Three years of data can be tracked on CANOR. This helps practitioners identify functional trends long before the conclusion of four years of therapy. Regarding TES, these tools will continue to help better predict: 1) how long it takes for TES to be helpful, 2) how many people TES will actually help, and 3) who the best candidates are for functional continence improvement, using TES.

TOOL LIMITATIONS

CAN-Guide was carefully developed over a three year time span. While CAN-Guide is the best tool available to meet the authors' needs in assessing the variables related to continence, it has limitations. These limitations include the following: 1) small sample size, 2) many items based on subjective reporting, 3) cost constraints and sample size prevented factor analysis, 4) re-testing the impact of changing the tool's directions regarding a score of "5" or "NA" remains to be done, and 5) the tool does not address some adult issues, such as financial impact, sexuality, and transportation related to incontinence. These limitations can be reduced as more practitioners use and test CAN-Guide and CANOR.

QUALITY OUTCOMES AND BEYOND

Factors which influence and affect continence are complex. In an effort to add continuity and strength to our evaluation methods related to TES, a new therapy to improve continence, the authors sought comprehensive continence assessment tools. Since those tools found in the literature were inadequate for their needs the authors decided to develop an original tool. After much time and effort and many rewrites, as well as help from research and statistical consultants, the Continence Assessment Nurses' Guide (CAN-Guide) and Continence Assessment Nurses' Outcome Record (CANOR) were developed. Although many assessment items are subjective in nature because they are reported by patients and families, the way in which these tools were designed and the way in which they are used adds objectivity. The authors' outcome measurement methodology is now strong, valid, and reliable. CAN-Guide and CANOR produce a "quick look" at the many variables which affect continence, including which areas are improving and which areas may need treatment modifications.

The authors are excited about the CAN-Guide and CANOR. Beyond meeting their particular therapy outcome needs, the authors believe these companion tools can contribute to the assessments and follow-up evaluations performed by many practitioners working in the continence field. Both patients and practitioners have expressed satisfaction with the tool when used as a basis for patient feedback and mutual goal planning related to treatment modalities.

The usefulness of the CAN-Guide and CANOR extend beyond the limits of the neurogenic bladder and bowel. Because the items on CAN-Guide may be used selectively, the tool can be applied to most situations involving incontinence. Interested readers are invited to use and critique the new tool and offer feedback.*

Note: The authors wish to acknowledge the following nurses who participated in testing CAN-Guide for inter-rater reliability: Connie Bradley, Sheri Katzer, Julie Shields, Nancy Sonney, Nancy Zoltowski; also, Kimberly Rauen, J.D., for editorial advice.

*To obtain more information about CAN-Guide and CANOR, please call (414) 527-8556 and ask for Karen Rauen.

REFERENCES

1. Sidani, S., Woodtli, M.S. Testing and Validation of an Instrument to Assess the Defining Characteristics of Stress and Urge Incontinence. Classification of Nursing Diagnosis: Proceedings of the Tenth Conference. J.B. Lippincott Company, 179, 1994.
   
   
2. Sackett, C.K. Spina Bifida Part 3. Implications for Bladder and Bowel Management. Urol Nurs 13: 104, 1993.
   
   
3. Urinary Incontinence In Adults - Clinical Practice Guideline. Rockville, MD : AHCPR Pub. No. 92-0038, 27, 1992.
   
   
4. McDowell, B.J. Assessment of Urinary Incontinence - Basic Elements of Continence Assessment. Urol Nurs 14: 120, 1994.
   
   
5. Joseph, A.C. Clinical Insights, Joseph Continence Assessment Tool. Urol Nurs 12: 144, 1992.
   
   
6. International Continence Society Committee for the Standardization of Terminology of the Lower Urinary Tract Function. Br J Obstet Gynaecol, Suppl 6: 1, 1990.
   
   
7. Balcom, A., Wiatrak, M., Biefeld, T., Rauen, K., Langenstroer, P. Initial Experience with Home Therapeutic Electrical Stimulation for Continence in the Myelomeningocele Population. J. Urol 158: 1272, 1997.
   
   
8. O'Brien, B., Bradford, J., Gibb, J. Nine Steps to Better Nursing Management of Incontinence. Cont Nurs 4: 131, 1995.
   
   
9. Leibold, S., Braun, P., Peterson, P., Cole, J. Bowel Continence and Spina Bifida. Washington, DC: Spina Bifida Association of America: 19, 1995.
   
   
10. Beckman, N.J. An Overview of Urinary Incontinence in Adults: Assessments and Behavioral Interventions. CNS 9: 241, 1995.
    
    
11. Dmochowski, R. Cystometry. Urol CNA 23: 243, 1996.
    
    
    
12. Ewalt, D.E., Bauer, S.B. Pediatric Neurourology. Urol CNA 23: 501, 1996.
    
    
13. Lo-Biondo-Wood, G., Haber, J. Nursing Research Methods, Critical Appraisal and Utilization, 3rd ed. St. Louis: C.V.Mosby, 1994.
    
    
14. Statistical Package for the Social Sciences, version 7.5 Chicago: SPSS, Inc., 1996.
    
    
15. Satterlee, William. Inter-rater Reliability of the Continence Assessment Nurse's Guide. Glendale, WI: Research Associates, 1997.

TABLE 1

TABLE 2