BME PhD Dissertation Defense - Clark Freifeld

2:00 pm on Friday, April 11, 2014
LSEB 804, 24 Cummington
John Brownstein (Harvard Medical School, Advisor)
Simon Kasif (BME Co-advisor)
Charles DeLisi (Committee Chair)
Douglas Densmore
Isaac Kohane (Harvard Medical School)

“MedWatcher: novel informatics tools for detecting adverse events of medical products in post-marketing”

Half of Americans take a prescription drug, regulated medical devices serve both at home and in the clinical setting, and vaccination coverage for many vaccines is over 90%. Nearly all medical products carry risk of adverse events (AEs), sometimes severe. However, pre-approval trials use small populations and often exclude participants who don’t fit specific criteria. The result is that trials are often insufficient to determine the true risks of a product as it is used in the population. In fact, over half of approved drugs have serious side effects not detected before approval. Existing post-marketing reporting systems are critical but suffer from significant underreporting by industry, clinicians, and patients. Meanwhile, recent years have seen an explosion in adoption of Internet tools and smartphones across the US population. We present MedWatcher, a new system that harnesses emerging information technologies for early detection and tracking of adverse events in the general population. MedWatcher consists of two components, a data mining module, MedWatcher Social, and a crowdsourcing module, MedWatcher Personal. With the data mining component, we acquire public data from the Internet and apply algorithms to filter and classify the information, and extract AE signals. With the crowdsourcing application, we provide a mobile and Web application that allows users to track safety information on chosen products, and submit AE reports directly.

For MedWatcher Social, our algorithm for identifying symptoms in posts performs with 77% precision and 88% recall on a sample of Twitter posts. Our document classification algorithm, for identifying posts containing AE information, performs with 68% precision and 89% recall on a labeled Twitter corpus. For three drugs, zolpidem tartrate, certolizumab pegol, and dimethyl fumarate, we compared AE profiles from Twitter with reports from the FDA spontaneous reporting system. We find some degree of concordance (Spearman’s rho = 0.85, 0.77, 0.82, respectively, for symptoms at the MedDRA System Organ Class level). Where the sources differ we find that less severe side effects are overrepresented in Twitter. For the three products we also performed qualitative comparison of post-marketing sources with clinical trial results and found very little concordance.

With MedWatcher Personal, we saw substantial user adoption, and received 550 direct AE reports in a one year period, including over 400 for a single medical device, Essure. We categorized 400 Essure reports by symptom and compared them to a similarly categorized set of 129 reports from the FDA spontaneous reporting system, and found high concordance (Spearman’s rho = 0.65) using MedDRA Preferred Term symptom granularity. We also compared Essure Twitter posts with both MedWatcher direct reports and FDA reports, and found rho = 0.25 and 0.31 respectively.

MedWatcher represents a novel system for post-marketing AE surveillance; our analysis is the first to compare AE profiles across social media, direct reporting, FDA spontaneous reports, and pre-approval trials.