EBM Application

AEIOU & Y
EBM Application

• AEIOUY Background

  During medical training, Evidence Based Medicine is often discussed. Most of the focus is on study design and quality. In general, very little time is given to applicability of the study. In other words, most people do not learn ways to apply research studies to their patients.  This page addresses this problem. Below outlines Dr. Stanley’s AEIOU&Y tool for EMB Applicability. 

  This tool uses the mnemonic AEIOU & Y. What each letter stands for is shown in the image above. You will find information about each element of the AEIOU&Y mnemonic below. 

   

   

• A = Acceptable

   

  

  Acceptability focuses on the internal aspects of a study that determine it’s quality. These qualities are traditionally discussed during EBM education. Acceptability is the first step to determining applicability, you must accept the study. Acceptability is essentially internal validity.  If the study is poorly designed or full of significant bias, it is unwise to accept it and even more unwise to apply it to your patients. 

  Dr. Stanley breaks “Acceptability”  down into several themes. 

  The tabs below discusses each theme in more detail. 

• Randomization

  Randomization give clinicians the best chances to match all variables (except for those being studied) between the test group and control group.

  

  When a patient is placed in a group deliberately is called selection bias (reasons can include physician choice, patient request, and disease severity). 

  

   

  This creates bias because this type of selection can create groups that are different from each other with regards to baseline prognostic factors NOTE: This is how observational studies work.

  

   

  Two questions to ask about randomization

  

  Question 1: Is the randomization method acceptable? The process of randomization must be truly random.

  

  Acceptable methods include a computer random number generator, dice, or a coin toss.

  Any method that does not allow true random group assignment is known as pseudo-randomization. Examples of pseudo-randomization include group assignments by days of week, hours of day or medical record number.

  

  Pseudo-randomization can lead to subtle, unforeseeable reasons that cause the groups to have significant differences in baseline prognostic factors.

  Question 2: Was the randomization concealed from clinicians? If clinicians can defeat the randomization process, they will try to do so. We should never underestimate the physicians drive to regain control over their patient’s care. 

  With the randomization not concealed, patients with different disease severities can be selectively allocated to groups which creates a systematic error. The best way to ensure randomization is to keep it concealed. This is best done by having the patients assigned off site (i.e. the treating clinician calls number and is given the proper treatment group)

   

• Baseline Group Similarity

  Comparing the variables of the groups being studied is important for determining the  Acceptability of a study. 

  Even with well performed randomization, chance can still cause differences in baseline prognostic factors between the treatment and control groups (this is even worse in non-randomized studies).

  Tip: Look for a Table. In well done studies, the researchers will provide a table of the most important hypothesized variables for each group. This allows for comparison of frequencies of these variables between the 2 groups. 

  

  Baseline mismatching: This is when study group and control group are imbalanced with respect to prognostically important characteristics. If this happens, there are some corrective measures that can be taken including: direct adjustment of continuous variables, multivariate analysis, and interim modification of group assignments.

  • NOTE: These corrective measures to correct baseline mismatching are only stopgap measures and employing them can disrupt randomization process. When these are used, the authors should show out-comes with and without the use of these corrective measures. 

  There is never a complete absence of confounders. 

  In addition, studies should have populations that span the entire disease severity spectrum.

  

   

• Criterion Standard

  Comparisons should always be made against the Criterion Standard.

  This is especially important in diagnostic studies.

  Is the criterion standard validated?

• Blinding

  In the study, who is blinded? 

  • Patient’s?
  • Clinicians?
  • Assessors of outcomes?
  • Statisticians?
  • Authors?

  In some studies an interim analysis is done to ensure patient safety. This should be done by an independent analyzer. 

• Appropriation

  Check to see if the control group received any treatment or testing. If they did, was it appropriate?

  

  If the control group is exposed to detrimental or ineffective treatment, then it can artificially elevate the effect of the treatment on the treatment group. 

  In placebo –controlled trials, the placebo treatment should have no effect on the group.

  

  In comparison controlled trials (i.e comparing two treatments), the non-experimental treatment may be ineffective, which could artificially elevate the efficacy of the experimental treatment

  

• Legit CI

  Legit CI: Confidence intervals are the best measure of precision (better than p values). 95% CI is standard (It is the rage at which the true value will be found 95% of the time). CI is also reported with a point value estimate of the true value. CI is reported like this: Positive effect was 17% (95% CI 5-23)

  

   

  You are looking for a narrow CI. A narrow CI means more trustable results. A narrow CI is proportional to the sample size.

  

  A narrow CI tells us that the sample size is large enough to assure us that the point estimate is probably close to the true value. In the image above, you can see that the top CI is wider as compared to the bottom CI. The bottom CI (14 to 23) is better. 

  You don’t want the low end of the CI to cross the line of no difference

  

  In the above example, the lower end of the CI range does not cross the line of no difference. This is what you want when you are looking at the results of a study to determine significance. Below shows an example of a CI that indicates the results show no significant difference. 

  

  You also want the low end of the CI to stay above the line of patient importance.

  

  This is because the line of no-difference is usually not enough to make a patient care decision. The line of patient importance is determined by you. 

   

• Intention to Treat

  Intention to treat: It is generally accepted that intention to treat it the best way to ensure a valid study. Were all subjects analyzed in the groups they were assigned to at the start of the study? The answer should be YES.

• Outcome Assessment

  Outcome assessment: Was outcome assessed in all patients? A key to this is look at follow up.

  

  If high # of patients are lost to follow up, be wary a chance for bias is high.

• Primary Outcome Only

  Primary Outcome Only: Determine if only the primary outcome was the focus of the assessment and discussion. 

  

  Studies are designed (powered) only to study the primary outcome. Any conclusions about the secondary outcomes can only be thought of as points of interest. Focus on secondary outcomes is a red flag for data dredging. 

  • Data dredging is when the study finds no positive effect with the primary , so the author plays with data to find some positive outcome.

  For the secondary outcomes to be valid, they need to be studied as primary outcomes.

• Sub-Group Analysis

  Sub-Group Analysis: Sub group analysis especially after study was completed (post-hoc) is also a red flag for data dredging. 

  

• Patient Oriented

  Patient Oriented: Patient oriented studies are much more applicable than disease oriented. 

  • Patient Oriented: Morbidity, Mortality, Hospitalizations
  • Disease Oriented: Blood pressure, Cholesterol level, Cortisol level

• E & I = Exclusion and Inclusion

   

  

  Looking at Exclusion & Inclusion criteria is the first step to determine applicability.

  Would have your patient been included in the study?

  Would have your patient been excluded from the study?

• O = Oldness

  

  Did the study include people the same age as your patient?

  If your patient is at an age extreme (really old or really young), you need to carefully consider applicability.

  Did the study include people enough people of the same gender as your patient?

• U = Unhealthness

  

  How does your patient level of unhealthiness compare to the study patients?

   

   

• Stage/Severity of disease

  • When looking at treatments, patients who are sicker than those in the treatment study will tend to have an increased positive effect from treatment
  • When looking at treatments, patients who are sicker than those in the treatment study may also have an increased risk of harm from treatment

• Comorbidities

  • If your patient has a comorbidity that was not in the treatment study, your patient will likely have an increased risk as compared to the study population. Be carful with applicability.
  • Warning: Apply with caution If your patient has renal, cardiac, or hepatic disease and these comorbidities were not in study population

  NOTE: You can use your patients baseline level of sickness and the findings from the study (RRI or RBI/RRR) to determine the impact of the treatment (if you feel the study is applicable) (See MDM section in patient care bucket)

• Y = Yes Can

  

  Yes I/patient/institution can do it. 

  Watch out for factors that would prevent reproducing study conditions. These can be physician factors, patient factors and/or institution factors. Some examples include:

  • Study requires a procedure that clinician is not comfortable doing
  • Patient can’t follow up (like those in study)
  • Patient can’t be monitored like those in study
  • Patient can’t comply to what was done in study