You are here

Nurse in patients’ health status assessment: Data from a pilot study assessing agreement among nurse and gastroenterologist in computing IBD-clinical scores

Digestive and Liver Disease, October 2017, Volume 49, Issue 10, Pages 1110-1114

1 Introduction

Crohn’s Disease (CD) and Ulcerative Colitis (UC) are chronic, systemic Inflammatory Bowel Diseases (IBDs). IBDs affect the gastrointestinal tract and may involve different organs such as skin, eye or bones and joints [1] . Both diseases may compromise the quality of life, as in operated UC patients with stoma or in those who have fistulas and perianal CD [1 2] . Psychological assistance or therapy may also be necessary during the course of disease [3] . Patients have to manage IBD for life, particularly if diagnosed in pediatric age. Thus, there is a need for a multidisciplinary approach to lifelong IBDs, involving not only different medical specialists but also particularly qualified nurses [4 5] . In Europe, a role for an “advanced” nurse is increasingly accepted for management of IBD patients so that ECCO experts elaborated some guidelines on the nurse’s role [6] . Nurses strictly deal with inpatients and outpatients too, i.e. when the biologic therapy is scheduled and administered in a Day Hospital room [7] . In this context, the nurse is typically the first health care provider to evaluate the IBD patient before starting treatment. Nurses must be able to answer patient’ questions through an adequate knowledge of IBD and its impact on the quality of life, and have to face with clinical problems such as complications with sufficient autonomy. Within certain limits, they may substitute the doctor in the clinical encounter. Clinical scores as the Harvey–Bradshaw-Index (HBI) for CD patients [8] and the partial Mayo score for UC patients [9] have proven effective in evaluating the disease clinical activity and up to now they are useful in everyday clinical practice as well as for long term follow-up of IBD patients. During very busy days of outpatient clinics, a registered nurse may routinely apply these clinical scores in order to assess the disease activity instead of physicians’ evaluation [7] , helping them with the assessment and management of patients in remission or with active disease. However, the agreement between the nurse and physician has not been investigated. In this study, we explored the concordance between IBD-nurses and physicians in computing HBI and partial Mayo score in patients treated with biologic drugs.

2 Material and methods

We enrolled all consecutive IBD-patients treated with biologics at two IBD referral centers from July to September 2016. For each patient a gastroenterologist and a nurse blindly filled-out a form to assess the HBI in CD patients or the partial Mayo scores in UC patients. All nurses came from the team routinely involved in IBD patients’ treatment and they had received an adequate training in IBD clinical scores before starting the study. All the involved physicians were expert in the management of IBD and in HBI or partial Mayo score assessment. Table 1 shows the component items of the HBI and the partial Mayo score. Each clinical score allows to categorize each patient into one of four classes of activity: remission, mild activity, moderate activity, and severe activity.

Harvey–Bradshaw Index General well-being 0 = very well Sum of variable scores:
<5 = clinical remission
5–7 = mild disease
8–16 = moderate disease
>16 = severe disease
  1 = slightly below par
  2 = poor
  3 = very poor
  4 = terrible
Abdominal pain 0 = none
  1 = mild
  2 = moderate
  3 = severe
Number of liquid stools per day 1 per occurrence
Abdominal mass 0 = none
  1 = dubious
  2 = definite
  3 = definite and tender
Complications 1 per item:
  –Erythema nodosum
  –Aphthous ulcer
  –Pyoderma gangrenosum
  –Anal fissure
  –New fistula
  –Abscess1 Abscess
Partial Mayo score (non-invasive components) Stool frequency 0 = normal Sum of variable scores:
<2 = clinical remission
2–4 = mild disease
5–7 = moderate disease
>7 = severe disease
  1 = 1–2 stools/day more than normal
  2 = 3–4 stools/day more than normal
  3 = >4 stools/day more than normal
Rectal bleeding 0 = none
  1 = visible blood with stool less than half the time
  2 = visible blood with stool half of the time or more
  3 = passing blood alone
Physician rating of disease activity 0 = normal
  1 = mild
  2 = moderate
  3 = severe

Table 1


2.1 Data collection and statistical analysis

All data were recorded in an electronic database and used to measure the beyond chance agreement (concordance) between two observers using the kappa statistic [10] . A k value of 1 indicates perfect agreement, whereas a k of 0 indicates agreement equivalent to chance ( Table 2 ). In order to consider the test reliable, we hypothesized a substantial level of agreement between assessors. We set up a sample size to detect a kappa of 0.75 against a null (not acceptable) hypothesis that kappa is 0.5 (better than ‘moderate’), thus requiring at least 78 ratings made by the two raters to achieve 90% power [11] .

Kappa agreement
<0 Less than chance agreement
0.01–0.20 Slight agreement
0.21– 0.40 Fair agreement
0.41–0.60 Moderate agreement
0.61–0.80 Substantial agreement
0.81–0.99 Almost perfect agreement

Table 2


3 Results

We enrolled 87 patients at two centres. The mean age of patients was 46 (SD 34). Thirty-nine (45%) were male, 46 (55%) were female. Fifty-three (61%) were treated for CD, 34 (39%) for UC. The biologic drugs used were infliximab in 70 patients (80%) and adalimumab in 17 (20%) patients. Before the study, the disease activity was classified as mild in 55%, moderate in 40% and severe in 5%. Table 3 summarizes the results of the concordance analysis. The overall agreement for disease activity between nurse and physician in all 87 IBD patients was compatible with moderate to substantial agreement (k value 67%, 95% CI from 51% to 80%). Both in CD and UC patients there was an area of disagreement about the scoring of remission versus mild activity, and about mild activity versus moderate activity. Severe disease activity was too low in the sample to lead to reliable analysis.

  Class Sample k Statistics a
Activity scores calculation Remission, mild disease, moderate disease, severe disease 87 67% (51–80%)
HBI calculation   53 67% (49–84%)
General wellbeing (yesterday) Very well, slightly below par, poor, very poor, terrible 53 64% (48–80%)
Abdominal pain (yesterday) None, mild, moderate, severe 53 61% (39–83%)
Number of liquid or soft stools per day (yesterday)   53 80% (68–92%)
Abdominal mass None, dubious, definite, definite and tender 53 34% (<0–68%)
Complications None, one among arthralgia, uveitis, erythema nodosum, aphtous ulcers, pyoderma gangrenosum, anal fissure, new fistula, abscess 53 74% (53–95%)
Partial MAYO calculation   34 63% (40–87%)
Stool frequency (based on the past three days) Normal, 1–2 stools more than normal, 3–4 stools more than normal, 5 or more stools more than normal 34 58% (37–79%)
Rectal bleeding (based on the past three days) No blood seen, streaks of blood with stool less than half the time, obvious blood with stool most of the time, blood alone passed 34 47% (20–73%)
Global assessment Normal, mild disease, moderate disease, severe disease 34 61% (38–84%)

a <0: less than chance agreement; 1–20%: slight; 21–40%: fair; 41–60%: moderate; 61–80%: substantial; > 80%: almost perfect.

Table 3


With regard to the single items of HBI, the best agreement was for the number of liquid or soft stools per day (k: 80%, 95% CI 68%–92%) and for the general well-being on the previous day (k:62%, 95% CI 39%–84%), while the least agreement was for the presence of abdominal mass (k 35%, 95% CI 0%–71%). Finally, the overall agreement on calculating partial Mayo score in the subsample of 34 patients with UC, based on stool frequency and rectal bleeding in the previous three days, was substantial (k 63%). However, due to the small sample size, the resulting k was compatible with fair to almost perfect agreement.

4 Discussion

Studies in gastroenterology nursing evaluating the effectiveness of advanced practice nurses have been limited to the performance of procedures, such as screening flexible sigmoidoscopies, and assisting with percutaneous endoscopic gastrostomy tube placement or videocapsule 12 13 14 15 16 17 . Other kind of studies assessing inter-observer reliability between bedside nurses and attending physicians in evaluating clinical practices is lacking. The advent of nurse specialists devoted to IBD has been a very recent development [18 19] . Several aspects of IBD care such as the disease prevalence, the long course with the risk of cancer requiring surveillance, the availability of drug therapy and the encouragement of drug compliance are all factors perceived to lend themselves to nurse management. In other areas, nurse specialists have long fulfilled an important function in chronic disease and complication management [20] . The advent of nurse specialists devoted to IBD has been, a recent development also in Italy [21] . Some evidence suggests their importance in IBD management. When disease control and quality of life were compared for the year before and the year after the employment of an IBD specialist nurse in one centre, it was reported that hospital visits and the length of inpatient stays were reduced in the second year and the number of patients in remission increased. Although the quality of life did not change significantly, there was a modest increase in patients’ satisfaction with prescribed drug information and other aspects related to emotional support [22] .

Patients with IBD can also be taught to self-medicate when they have a flare-up and to telephone for an urgent appointment if symptoms are not controlled within 5 days [23] . In a controlled study, this reduced clinic visits by 30%, decreased the delay between symptom onset and treatment from 4 days to less than 24 h, increased quality of life scores, and decreased costs. Virtually all patients preferred the new system [24] . Some physicians would argue that IBD patients could have, if needed, a sufficient number of telephone consultations with the IBD-nurse, and should still have at least an annual formal outpatient visit, for review of drug treatment, cancer surveillance, and other issues of concern. For unselected IBD patients the provision of educational reading material alone does not alter the quality of life, but a nurse-supported educational package can increase the adherence to the prescribed treatment [20] . So nurses, that are perceived to have more time to answer patients’ queries, may play a pivotal role in the clinical management of these patients.

Collaborative practice models using advanced practice nurses have been developed to provide high-quality healthcare in a cost-effective manner. In an audit study on the management of a cohort of patients in the year prior to the employment of the specialist nurse and the year immediately after, there was a 38% reduction in hospital visits and a 19% reduction in patients’ length of stay measured in bed-days. The number of IBD patients in remission also increased from 63% to 69%. Patient satisfaction improved in key areas, in particular, access to information on IBD and advice on illness avoidance and health maintenance. Out of a total of 251 calls to the telephone helpline, only 19 patients were referred for a medical opinion and five patients required hospital admission [22] .

Thus, better disease management is conceivable by advanced nurse involvement in IBD care. In another audit study on the management of a cohort of IBD patients in the year prior to the employment of the specialist nurse and the year immediately after, there was a 38% reduction in hospital visits and a 19% reduction in patients’ length of stay measured in bed-days. The number of patients in remission also increased from 63% to 69%. Patient satisfaction improved in key areas, in particular, access to information on IBD and advice on illness avoidance and health maintenance. When calls to the telephone helpline were reviewed, only 8 percent of patients were referred for a medical opinion and 2 percent required hospital admission [25] . However, more trials of IBD specialist nursing interventions are needed to assess their real impact on the care and management of patients with IBD.

Here we focused on a particular aspect of the N-ECCO statement E1 regarding the nursing assessment process. There is a paucity of quantitative studies of the validity and clinical implications of clinical assessments conducted by nurses [26] . We explored their reliability when substituting the physician in the evaluation of disease activity through clinical scores in a set of patients undergoing biologic treatment at two centres.

The HBI score [8] includes bowel-related symptoms, complications and general well-being, and permits simple assessment during the long-term clinical follow-up of CD patients facilitating standardized measurement in outpatient clinical practice as well as cross-centre comparisons 27 28 29 . The Mayo score, commonly used in placebo-controlled clinical trials, is composed of clinical and endoscopic categories to give a total score that ranges from 0 to 12 in UC patients. In recent years, several clinical trials have utilized the non-endoscopic components of this index as a partial index at interim visits to assess the time to onset of response [30] .

Data on the accuracy of IBD clinical scores in predicting the disease activity are well-established, but less is known on the reproducibility of their assessment among physicians or among physicians and nurses. Some published studies report data on agreement between physicians on histological, endoscopic or imaging scoring of activity either in CD or UC patients 31 32 33 34 , but we have found no data on agreement between physicians and nurses. Our study shows a moderate to substantial strength of agreement between gastroenterologist and nurse in evaluating the disease activity in clinical practice. Due to the known simplicity of score administration, there is a need to improve concordance in order to optimize patient care. We hypothesized that nurses could routinely assess the disease activity in IBD patients, but the study suggests that better performance is likely needed. It should be noted that the least agreement was for the diagnosis of abdominal mass in Crohn’s disease (k 35%, 95% CI <0%–71%). It is likely that such a fair agreement reflects the inability of nurses to perform a correct physical examination, so that their performance should be improved.

There are other limitations of this study. First, one general question arises about the level of agreement between nurse and physician to be required as a threshold of acceptability for the assessment of IBD disease activity in clinical practice. Our study had a power of 90% to detect substantial agreement, that was observed overall but not according to disease subgroups. Second, the agreement on single items needs to improve, both qualitatively by developing common criteria of assessment, and quantitatively by increasing the power of future studies. Finally, these studies need to include more patients with severe disease activity, eventually not undergoing biologic therapy at the time of assessment. We currently don’t know if concordance may increase in presence of severe disease.

In conclusion, although the relatively small number of patients studied may affect the results of the analysis of single items of UC and CD scores, our data are encouraging and support a possible use of the nurse-performed HBI and partial Mayo score to assess the disease activity of IBD patients. There is a need to increase the reproducibility of the assessment both testing inpatients in busy referral IBD clinics and outpatients on treatment with expensive therapies.


  • [1] F. Gomollón ,A. Dignass ,V. Annese ,et al. 3rd European evidence-based consensus on the diagnosis and management of Crohn's disease 2016: part 1: diagnosis and medical management. J Crohns Colitis. 2017;11 :3-25
  • [2] A. Dignass ,R. Eliakim ,F. Magro ,et al. Second European evidence-based consensus on the diagnosis and management of ulcerative colitis part 1: definitions and diagnosis. J Crohns Colitis. 2012;6 :965-990 Crossref
  • [3] A.M. McCombie ,R.T. Mulder ,R.B. Gearry. Psychotherapy for inflammatory bowel disease: a review and update. J Crohns Colitis. 2013;7 :935-949 Crossref
  • [4] A. Hillier. The advanced practice nurse in gastroenterology. Identifying and comparing care interactions of nurse practitioners and clinical nurse specialists. Gastroenterol Nurs. 2001;24 :239-245 Crossref
  • [5] T.I. Waters. The role of the nurse practitioner in the gastroenterology setting. Gastroenterol Nurs. 1998;21 :198-206 Crossref
  • [6] M. O'Connor ,J. Gaarenstroom ,K. Kemp ,et al. N-ECCO survey results of nursing practice in caring for patients with Crohn's disease or ulcerative colitis in Europe. J Crohns Colitis. 2014;8 :1300-1307 Crossref
  • [7] W.R. Connell ,T. Samyue ,P.R. Gibson ,et al. Changing face of care for patients with moderate to severe inflammatory bowel disease: the role of specialist nurses in the governance of anti-TNF prescribing. Int Med J. 2015;45 :1161-1166 Crossref
  • [8] R.F. Harvey ,J.M. Bradshaw. A simple index of Crohn's-disease activity. Lancet. 1980;1 :514 Crossref
  • [9] K.W. Schroeder ,W.J. Tremaine ,D.M. Ilstrup. Coated oral 5-aminosalicylic acid therapy for mildly to moderately active ulcerative colitis. A randomized study. N Engl J Med. 1987;317 :1625-1629 Crossref
  • [10] J.R. Landis ,G.G. Koch. The measurement of observer agreement for categorical data. Biometrics. 1977;33 :159-174 Crossref
  • [11] A. Donner ,M.A. Rotondi. Sample size requirements for intervals estimation of the kappa statistic for interobserver agreement studies with a binary outcome and multiple raters. Int J Biostat. 2010;6 :1-11
  • [12] A. Lahad ,E. Levy-Lahad. Nurse practitioners as endoscopists. N Engl J Med. 1994;330 :1534 Crossref
  • [13] P. Schoenfold ,S. Lipscomb ,J. Crook ,et al. Accuracy of polyp detection by gastroenterologists and nurse endoscopists during flexible sigmoidoscopy: a randomised trial. Gastroenterology. 1999;117 :312-318 Crossref
  • [14] V. Moshakis ,R. Ruban ,G. Wood. Role of the nurse endoscopist in colorectal practice. Br J Surg. 1996;83 1399–1309
  • [15] J.A. DiSario ,R.A. Sanowski. Sigmoidoscopy training for nurses and resident physicians. Gastroint Endosc. 1993;39 :29-32 Crossref
  • [16] W.F. Maule. Screening for colorectal cancer by nurse endoscopists. N Engl J Med. 1994;330 :183-187 Crossref
  • [17] British Society of Gastroenterology. The nurse endoscopist. Report of a BSG Working Party. (BSG, London, 1994)
  • [18] A. Guarinia ,F. De Marinis ,A. Kohn ,et al. Inflammatory bowel disease nurse specialists for patients on biological therapies: a nationwide Italian survey. Ann Gastroenterol. 2016;29 :492-496
  • [19] M. O'Connor ,P. Bager ,J. Duncan ,et al. N-ECCO consensus statements on the European nursing roles in caring for patients with Crohn's disease or ulcerative colitis. J Crohns Colitis. 2013;7 :744-764 Crossref
  • [20] C. Norton ,M.A. Kamm. Specialist nurses in gastroenterology. J R Soc Med. 2002;95 :331-335 Crossref
  • [21] A. Guarinia ,F. De Marinis ,A. Kohn ,et al. Inflammatory bowel disease nurse specialists for patients on biological therapies: a nationwide Italian survey. Ann Gastroenterol. 2016;29 :492-496
  • [22] A.J. Nightingale ,W. Middleton ,S.J. Middleton ,et al. Evaluation of the effectiveness of a specialist nurse in the management of inflammatory bowel disease (IBD). Eur J Gastroenterol Hepatol. 2000;12 :967-973 Crossref
  • [23] A. Nightingale. Audit of guided patient-managed care. D. Rampton. IBD: Clinical Diagnosis and Management. (Martin Dunitz, London, 2000)
  • [24] A. Robinson ,D.G. Thompson ,D. Wilkin. Guided self-management in ulcerative colitis: quicker treatment and more efficient use of health services. Lancet. 2001 Sep 22;358 :976-981 Crossref
  • [25] M.J. Rothman ,A.B. Solinger ,S.I. Rothman ,et al. Clinical implications and validity of nursing assessments: a longitudinal measure of patient condition from analysis of the Electronic Medical Record. BMJ Open. 2012;2
  • [26] R. Belling ,S. McLaren ,L. Woods. Specialist nursing interventions for inflammatory bowel disease. Cochrane Database Syst Rev. 2009;7 :CD006597
  • [27] G. D'Haens ,W.J. Sandborn ,B.G. Feagan ,et al. A review of activity indices and efficacy end points for clinical trials of medical therapy in adults with ulcerative colitis. Gastroenterology. 2007;132 :763-786 Crossref
  • [28] W.R. Best ,J.M. Becktel ,J.W. Singleton ,et al. Development of a Crohn's disease activity index: National Cooperative Crohn's Disease Study. Gastroenterology. 1976;70 :439-444
  • [29] S. Vermeire ,S. Schreiber ,W.J. Sandborn ,et al. Correlation between the Crohn's disease activity and Harvey–Bradshaw indices in assessing Crohn's disease severity. Clin Gastroenterol Hepatol. 2010;8 :357-363 Crossref
  • [30] J.D. Lewis ,S. Chuai ,L. Nessel ,et al. Use of the noninvasive components of the Mayo score to assess clinical response in ulcerative colitis. Inflamm Bowel Dis. 2008;14 :1660-1666 Crossref
  • [31] D. Allende ,M. Elmessiry ,W. Hao ,et al. Inter-observer and intra-observer variability in the diagnosis of dysplasia in patients with inflammatory bowel disease: correlation of pathological and endoscopic findings. Colorectal Dis. 2014;16 :710-718 Crossref
  • [32] L.K. Wanders ,E. Mooiweer ,J. Wang ,et al. Low interobserver agreement among endoscopists in differentiating dysplastic from non-dysplastic lesions during inflammatory bowel disease colitis surveillance. Scand J Gastroenterol. 2015;50 :1011-1017 Crossref
  • [33] M. Daperno ,M. Comberlato ,F. Bossa ,et al. Inter-observer agreement in endoscopic scoring systems: preliminary report of an ongoing study from the Italian Group for Inflammatory Bowel Disease (IG-IBD). Dig Liver Dis. 2014;46 :969-973 Crossref
  • [34] N. Horvat ,C.C. Tavares ,A.R. Andrade ,et al. Inter- and intraobserver agreement in computed tomography enterography in inflammatory bowel disease. World J Gastroenterol. 2016;22 :10002-10008 Crossref