Clinical Trials in Surgery

1
Clinical Trials in Surgery

• Mazen S. Zenati, MD, MPH, Ph.D.
• University of Pittsburgh,
• Department of Surgery and Epidemiology

2
Modern Clinical Trials in History

• Clinical trials began in 1800 onward to
  proliferate and more attention was paid to study
  design.
• Placebos were first used in 1863
• The idea of randomization was introduced in 1923.
  
• The first trial using properly randomized
  treatment and control groups also featured double
  blind assessment was carried out in 1948 by the
  Medical Research Council, and involved the use of
  streptomycin to treat pulmonary tuberculosis..

3
The Need for Clinical Trials in Surgery

• In surgery many plausible clinical theory are
  institutionalized into practice before as
  systematic risk-benefit, cost-effectiveness and
  long term quality of life have been measured and
  evaluated
• Many surgical procedures implemented for years
  before we discovered the ineffectiveness to
  eventually discredited
• Halsted radical mastectomy
• Kidney decapsulation (for hypertension)
• Uterine suspension
• Legation of internal mammary arteries (for
  coronary insufficiency)

4
The Need for Clinical Trials in Surgery.. cont.

• New surgical procedures are not regulated in the
  same way new drugs or devices are
• New procedures can be approved in the hospital
  based on the surgeon ability to convince the
  patients and some time upper management of the
  potential benefit of the new idea or the
  technique.
• No enough cases or controls,
• Protocols and IRB in many cases are often not
  required
• No proper documentation of the outcome in
  systematic manner
• No real plan for study or follow up

5
The Unique Nature of Surgical Trials

• Blinding of patients or Surgeon is almost
  infeasible
• Placebo-controlled procedures are ethically
  controversial specially if invasive
• The placebo effect is powerful in surgical
  procedures and should be identify
• The bias of the investigator due to the need for
  that procedure or program to be successful
• Patient selection, extra skills in performance,
  and extra care and attention (efficacious vs.
  effective)

6
The Unique Nature of Surgical Trials.. cont.

• Difficulty in selecting centers to represent the
  practicing community (VA sites vs. academic
  centers)
• Standardization across the sites is difficult
• When we compare a well-established technique with
  an alternative, the skills and experience in the
  first might bias the comparison (the need for
  video tapes and operative reports for QC)

7
Clinical trial in Surgery Necessary for

• To test the safety and efficacy of a new
  technology in the short and long term outcome
• To test if the new technology is Cost effective
• To test if the new technology is at least as
  effective and/or safe as existing proven
  techniques
• To test for the feasibility of implementing the
  new technology in relation to the needed
  skills/training, equipment, and other factors.

8
Barriers to Clinical Trials in Surgery

• Underutilized due to difficulties in design and
  recruitments
• The high cost hospitalization, operation,
  personnel, and related special method of data
  collection and analysis
• Funding is difficult to obtain from public and
  agencies
• Peer-reviewers, IRB, and other over criticisms
  (protecting patients right and safety) delay the
  process and takes years that the timeliness of
  introduction may be lost

9
Barriers to Clinical Trials in Surgery.. cont.

• Multicenter trials might starts in different
  point with unbalance recruitments)
• Funding fro the industry carries its own risk of
  design and ownership
• Mastering complex procedures related to learning
  curve
• Consent documents very sensitive (the risk is
  more of an open ended format)

10
Forces Opposing Clinical Trials in Surgery

• The magnitude of potential benefits might subside
  the necessity for wait and for the total prove
  through a clinical trial
• The high risk might hold recruitment for a
  control clinical trial
• The wide range and high number of potential
  recipients also a factor in determine the need
  compared to a rare disease that may require
  special surgical procedure.

11
Exception

• When we deal with disease that carry very bad
  prognosis as in oncology, cardiology, and
  transplant where the intervention is highly
  morbid and invasive
• The evidence-based medicine appraised by patients
  themselves and third-party payers might modify a
  total embracement traditionally done by the
  surgical community

12
Randomized Control Trial

• Can we really have a controls in surgical
  clinical trials?
• Controls is it
• Not to operate on half of the patients?
• Mock operation not related to the indication?
• Can we really deny half of the patients in need
  for surgical interventions just for the sake of
  research?
• Are we then doom half of them to their death?
• But which half is that? (will be really exposed
  to a greater eventful hazard?)
• The interventions one or the controls one do we
  really know?

13
The Basic design of randomized Controlled
Clinical Trial
Population
Eligibility Criteria
Ineligible
Eligible
Decline to Participate
Recruitment
Agree to Participate, Informed Consent
Randomization
Controls Current standard No therapy/placebo
Intervention New surgical technique
Good Outcome
Bad Outcome
Good Outcome
Bad Outcome
14
Barriers to Performing Randomized Trials

• Surgeon bias for or against specific procedures
• Morbidity associated with surgery
• The for granted acceptance of lesser forms of
  clinical evidence by the surgical community
• Difficulties in recruitments
• Ethics and legality consents

15
Threats of the Lack of Randomized Control Group

• Internal validity related to the temporal trends
• Surgical learning curve effects
• Regression to the mean
• The frequent lack of equipoise among surgeons
• Amplifying the fact that the clinical practice
  outpaces research evaluation

16
RCT vs. Quasi-experimental Study Design

• RCT minimize the measurable confounding that
  potentially biasing the study's conclusions.
• RCT reduces selection bias
• RCT however, restricts the generalizablity of the
  results because of the strict inclusion and
  exclusion criteria
• Some worthy to consider designs in case of
  unlikely to achieve RCT
• The use of properly selected nonrandomized
  control groups
• Using pre and post comparisons

17
Bias and Validity

• Bias Systematic error within the study that
  results in mistaken estimate of the effect of
  therapy on disease
• Internal validity The ability of a trial to
  come to the correct conclusion regarding the
  question being studied
• External validity The ability of trial to
  produce results that are generalizable to a
  larger population of patients with disease

18
Formulating Research Question

• RCT to answer as narrow as a possible clinical
  research question
• To answer an important question justifying
• Recruitment of large number of patients,
• The high expenses of the trial
• The need to compare the exist of legitimate
  uncertainty between the effective of at least two
  therapies of the same disease (clinical
  equipoise)
• In surgery there are many patients avoid surgery
• Many surgeons are and perhaps patients reluctant
  to relinquish the control of choice of treatment
  of deeply held beliefs of most appropriate
  treatment
• Ideal problems to study by RCT
• Similar morbidity of the procedure using for
  example similar incisions
• The disease has high morbidity, new procedure
  promise to improve outcome
• Disease with current treatment result in
  potential long term morbidities that will may
  justify the risk for exploring a new choice

19
Issues in Formulating Research Question

• What are the end points
• Can these end points be accurately and reliably
  measured
• What potential surrogate end points are available
  and at what expenses to the internal validity
• All the above in consideration to practicality,
  expenses, and feasibility of RCT
• Pragmatic trials vs. explanatory trials

20
Pragmatic vs. Explanatory

• Pragmatic attempt to stimulate clinical
  realities more accurately. The outcomes can be
  easily generalized and accepted into clinical
  practice
• Explanatory attempts to answer more specific and
  narrow question (strict inclusion and exclusion
  criteria, homogeneous population)

21
Eligibility Criteria

• Inclusions and exclusions
• Important to determine the external validity
• The study population should not be too narrow or
  to broad
• It is a balance between statistical advantage and
  the generalizability of the results
• Pragmatic trial tend to be broader in inclusion
• Explanatory trials are narrower in focus

22
Outcome Measurement in Surgery

• Traditional outcome measures
• Mortality
• Measures
• Mortality rate, crude mortality rates, specific
  mortality rates, adjusted mortality rates,
  standardized
• Morbidity
• Measures
• Prevalence, point prevalence, period prevalence,
  incidence, cumulative incidence, attack rate,
  incidence rate
• Speed of recovery ICU LOS, Hospital LOS
• Health related quality of Life
• Measuring patients satisfactory

23
Example on Eligibility

• A study in vascular surgery to compare between a
  newly MIT and the conventional approach in
  treating occlusive peripheral vascular disease of
  lower extremities
• Inclusion criteria patientslt 65 yr with clinical
  and radiographic evidence
• Randomly allocated for the above interventions
• End points angiographic resolution of the
  occlusion

• Typical patients may be older than 65 yr, lack of
  generalizability lower the external validity
• Eligibility criteria should be more objectively
  defined
• Severity of lesion, symptoms
• Endpoint might not be the best to determinant of
  therapeutic success
• Patient symptoms, activity levels, and wound
  healing (more appropriate and more relevant
  clinically)

24
Example on selecting Endpoint

• A RCT to comparing between prothrombin complex
  concentrate and FFP in emergency need (surgery/
  trauma) for reversing the effect of warfarin. The
  INR has been suggested as an endpoint but the FDA
  did not accept the proposal and asked for more
  clinically related outcome. Any suggestion?

• A RCT to test the effect of preoperative oral
  Citrulline on surgical wound healing . The speed
  of wound healing, strength of the healing or LOS
  was though of as endpoint outcomes. Anything
  wrong with these?

25
Sample Size

• We might need in surgery to conduct first a pilot
  nonrandomized observational study on homogenous
  population to determine the differences
  anticipated in outcome between the randomized
  groups in case we do not have enough data
• The above will be also a good source to estimate
  the variability in the outcome data i.e.
  standard deviation, or standard error
• What is clinically relevant is more important of
  just statistically significant in power analysis
• Statistical consultation in this regard is very
  essential and should be obtained early in the
  course of study preparation

26
Controls and Level of Evidence

• Controls to increase the level of evidence
  without controls is just as case series study
  which is important in surgery to obtain all the
  descriptive analysis of outcome but will not
  provide evidence of superiority (biased)
• Without control we can not account for the
  placebo effect
• Without control we can not account for
  confounding variables
• Historical controls might be used frequently in
  surgery when a prospective controls is beyond the
  reach, specially when performed by the same
  surgeon and in the same institution
• Population and the time change in technology
  still differ

27
Controls and Level of Evidence

• Variables that we usually match with in selecting
  controls in surgical trials beside all inclusion
  and exclusion criteria
• Age, sex, BMI,
• Comorbidities, preexisting illness
• Disease stage, duration, severity, ISS, mechanism
  of injury, site
• Surgeon, procedure
• Others smoking, alcohol use, depression,
  insurance coverage, etc
• Still the superior level of evidence attributed
  to RCT is contingent on a successful
  randomization

28
Randomized Trial in Surgery

• Random allocation of the subjects to the trial
  groups
• Breaks the link between any unmeasured
  confounding variables and treatment status
• Assume all differences in effect between
  treatment groups to be a result of the
  differences in treatment (absent confounding)
• Randomization is the major challenge clinical
  trials in surgery and causes considerable
  weakness the design when not satisfactory
  attained
• The Challenge is convincing patients to
  relinquish control of their care to random
  process by whether or not they receive surgery or
  even which kind of surgery they actually received
• Randomized trials in surgery can be either
  Pragmatic or Explanatory

29
Pragmatic vs. Explanatory

• It is similar to effectiveness vs. efficacy in
  drug trials where
• Effectiveness is the ability of an intervention
  to accomplish its intended outcome in population
  under real-life circumstances
• Efficacy Is the ability of an intervention to
  obtain its intended outcome under ideal situation
• Most randomized trials in surgery tend to be
  categorized as pragmatic effectiveness,
  comparing surgical technique and outcome under
  usual clinical conditions

30
Randomization

• Stabilizes the internal validity of the study
• Randomization process can be predicted by
  specific logistics and practical constraints of
  the study. The strong desire to operate or not
  by a resident my tempt for the selection bias.
• The process must be truly unpredictable and
  concealed from the investigator. Allocate the
  patient to randomization after obtaining the
  consent. Using random number table/a computer
  generated and then enclosed in opaque, serially
  numbered enveloped.
• Investigator selection bias in selecting the
  intervention to subjects whenever more than one
  patient is presented at the same time as in a
  busy trauma center for instance
• Use distance randomization by phone or internet

Randomized Trials appear to annoy human
nature-if properly conducted, indeed they should
31
Randomization, cont..

• Block Randomization to result equal distributed
  patients between the arms of the study
• Stratified Randomization to assure certain
  traits are equal among treatment groups

32
Example on Randomization

• In a busy trauma center, RCT took place to assess
  whether abdominal ultrasound or diagnostic
  peritoneal lavage was more accurate in diagnosing
  intra-abdominal injury after blunt trauma. In the
  ER a clinical indicator was used to determine the
  need for rapid abdominal assessment. If met, a
  resident would open the next opaque, serially
  numbered envelope stored in a designated folder
  in the ED to indicate which assessment would be
  used. In such busy trauma centers, two or more
  patients may presented simultaneously. At the end
  of trial, it was noted that, more often than not,
  diagnostic peritoneal lavage was performed on
  thinner patients!

33
Example on Randomization

• Two cases or more at the same time might offer
  the busy resident an opportunity to open more
  than one envelope at once and subsequently decide
  which patients receive which assessment.
• The process was corrupted by a third part
  motivated by self interest. Likely/hopefully not
  understanding the potential effects on the
  outcome of the trial.

34
Masking. Blinding

• To prevent or reduce observation bias and placebo
  effect.
• Observation bias is systematic variation between
  the true outcome and the outcome observed
  (consciously or unconsciously)
• Placebo effect exaggerate the effects of
  treatment based on their enthusiasm and
  expectations
• Single blinded treatment assignment is concealed
  from subject only
• Double-blinded treatment assignment is concealed
  from both subject and the investigator who
  ascertains the study end point
• Contrary to studies of pharmacologic therapies
  for a disease, blinding in surgery is not that
  easy
• Logistic Challenges we do not have placebo
  surgery similar to placebo tablets. We have to
  use sham surgery
• Ethical Challenges many of the sham-surgery
  interventions are considered unethical by todays
  standards

35
Masking. Blinding.. Cont.

• Using blinded third party (not the operating
  surgeon) might be necessary
• We could verify the effectiveness of blinding by
  asking all study participants to guess which
  treatment they had received.
• If the patients guesses were no better than
  random chance.. indicated that the blinding
  remained successful
• Oftentimes, it is not possible to perform a
  randomized trial in blinded fashion different
  surgical techniques, therapies, approaches, or
  incisions

36
Example on Blinding

• In 1959, Cobb et al. reported a RCT designed to
  determine if the internal mammary artery ligation
  was effective in the treatment of angina. The
  researchers enrolled 17 patients with classic
  symptoms and recorded preoperative severity of
  symptoms as well as stress test electrocardiogram
  data. The patients then underwent surgery. After
  the internal mammary arteries were isolated, an
  envelope was opened and directed to either ligate
  or simply end the operation. The patients were
  informed preoperatively that the ligation
  procedure was experimental, but they unaware of
  the randomized nature of the study. Post
  operatively data were collected and found that
  the patients undergoing the sham procedure had
  greater symptomatic relief than those undergoing
  ligation, and one patient in the sham group had
  reversal of stress test electrocardiogram
  abnormalities. Overall, patients in both groups
  showed only modest improvement.

37
Example on Blinding.. cont.

• The importance of sham surgery, without it (as
  case series design) the author could incorrectly
  concluded that intervention had a merit
• The differences might be related to
• Placebo effect caused the improvement in both
  groups
• Something beside the trial intervention caused
  the improvement
• The observed benefit was part of the natural
  history of the disease

38
Patient Crossover and Intention-To-Treat Analysis

• Intention-To-Treat Analysis Analyze the outcome
  based on the original randomization assignments,
  regardless of treatment actually received
• The alternative Analyze the data based on the
  treatment actually received not by random
  allocation (as if here we are breaking the
  randomization introducing selection bias since
  crossover does not occur by chance)
• A high crossover rates questioned the validity of
  the study
• High crossover rates in surgical clinical trials
  is common
• Especially in trials comparing surgical with
  medical interventions

39
Patient Crossover and Intention-To-Treat
Analysis.. cont.

• There is a specific need to multiple statistical
  methods and mixed modeling techniques to deal
  with crossover
• We stick with Intention-To-Treat Analysis
  approach whenever randomization is preserved and
  in order to keep the statistical assumptions
  valid for the analysis approaches
• What is the nonrandom factor that caused patients
  to crossover
• These nonrandom factors will biases the study
  results

40
Example on Crossover

• From 1972 through 1974, the Veterans
  Administration Coronary Artery Bypass Surgery
  Cooperative Study Group enrolled patients into a
  randomized study comparing coronary artery bypass
  surgery with medical therapy in patients with
  angina and radiographic evidence of CAD. Patients
  were randomized to receive either surgical or
  medical therapy. After 14 yr of follow up, 55 of
  patients assigned to receive medical therapy had
  crossed over to receive surgical treatment,
  whereas 6 of the patients assigned to receive
  surgical treatment decided to not undergo the
  procedure and were thus managed clinically.
  Should we analyze the data based on the
  intention-to-treat (original randomization) or
  analyze the data based on the treatment actually
  received (not by random allocation)?

41
Example on Crossover.. Cont.

• Should we analyze based on the intention-to-treat
  with such large cross over. Is not that seem
  counterintuitive of the randomization process?
• Should we accept to break the randomization and
  allowing the introduction of selection bias by
  analyzing according to the actual treatment
  received?

42
Conclusions

• Surgeons have fallen behind the rest of medical
  community partly because of the challenges
  required to complete a valid randomized trial of
  surgical therapy
• Clinical Trials in surgery is underutilized
• Recent era of increase in medical evidence demand
  from policy maker and caregivers and even
  patients resulted in an increase in the demand of
  randomized clinical trials for publications and
  quality assurance. And partly due to our
  acceptance and reliance on lesser forms of
  evidence
• Clinical Trials in surgery is urgently needed
• Randomized clinical trials in surgery are not
  easy but still can be accomplished with the
  appropriate planning and selecting suitable
  design
• Good controls, selection of end point,
  randomization, blinding are the keys for
  successful CT
• We need to standardized the ascertainment of
  outcome, attempting to quantify outcome as
  objectively as possible
• We also should minimize the potential observation
  bias and placebo effect by selecting the most
  appropriate design

43
Final Thoughts

• Only until clinical trails in surgery became the
  gold slandered to prove safety and effectiveness
  of the treatment, the current practice will
  continue to be dominant by opinions or the
  enthusiasm for a procedure.
• Still however the surgeon's Judgment will never
  be a dispensable
• The results still need to be interpreted in light
  of the study design and clinical reasonableness
• The better commitment to evidence-based medical
  practice and the persistence and the application
  of novel approaches to overcoming difficult
  methodological hurdles will be continuously
  necessary to advance medicine

44
References and recommended Reading

• John Gallin, Principles and Practice of Clinical
  Research, Academic Press, 2002
• H. Troidl, Surgical Research, Springer, 1987
• David Machin, Text book of Clinical Trials,
  Wiley, 2006
• John Wei, Clinical Research Methods for Surgeons,
  Humana Press, 2006
• Friedman LM, Fundamentals of clinical
  Trials-Third Edition, PSH Inc, 1998
• Ovid MEDLINE
• http//www.hsls.pitt.edu/resources/ovid/
• PubMed http//www.ncbi.nlm.nih.gov/sites/entrez?h
  oldingupittlib
• FDA http//www.fda.gov/cder/guidance/iche6.htm
• Clinical Research Office for Surgery and Trauma
  (CROST), University of Pittsburgh, Department of
  Surgery http//www.surgery.upmc.edu/General/resea
  rch.htm

45
Clinical Trials in Surgery