{"id":9657,"date":"2023-12-14T10:50:37","date_gmt":"2023-12-14T14:50:37","guid":{"rendered":"https:\/\/www.bu.edu\/photonics\/?p=9657"},"modified":"2024-10-07T15:19:59","modified_gmt":"2024-10-07T19:19:59","slug":"when-cutting-edge-microscopes-meet-deep-learning-algorithms","status":"publish","type":"post","link":"https:\/\/www.bu.edu\/photonics\/2023\/12\/14\/when-cutting-edge-microscopes-meet-deep-learning-algorithms\/","title":{"rendered":"When Cutting-Edge Microscopes Meet Deep Learning Algorithms"},"content":{"rendered":"
Lei Tian\u2019s Computational Imaging Systems group is calculating new ways to see molecules, cells, and life in action\u00a0<\/strong><\/h5>\n

By Kat J<\/span><\/i>. McAlpine<\/em><\/p>\n

Research images provided by Lei Tian | Photos by Kelly Pe\u00f1a<\/em><\/p>\n

Lei Tian<\/span><\/a> \u2013 a faculty member at BU\u2019s <\/span>Photonics Center<\/span><\/a> and a BU <\/span>College of Engineering (ENG)<\/span><\/a> assistant professor of electrical and computer engineering \u2013 has spent his career harnessing physics, electronics, and optics to create new imaging systems capable of applications ranging from the enormous (illuminating the flow of massive oil spills within oceanwater) to the minute (detecting precise cellular activity within biological tissues).\u00a0<\/span>\u00a0<\/span>\u00a0<\/span><\/p>\n

\"\"He credits holography \u2013 which manipulates laser light to display 3D images \u2013 as his entry point into the field of computational imaging at a time when it was still a nascent idea, while he was earning his PhD at Massachusetts Institute of Technology (MIT). There, he was focused on digital holography, employing sensors to capture real 3D information and then reconstructing objects computationally by modeling the way light diffracts through space.\u00a0<\/span>\u00a0<\/span><\/p>\n

Using that approach to develop holography systems for submersible vehicles, Tian recalls that the 2010 Deepwater Horizon oil spill added a new layer of interest and urgency to his work. \u201cWe wanted to design better 3D cameras for large-scale applications\u201d like detecting where the oil had spread throughout ocean columns and currents, he says.<\/span>\u00a0<\/span><\/p>\n

Working on the macroscale, however, led to a new inspiration. \u201cWe also realized these techniques would be good for looking at very, very small things on the microscopic scale.\u201d\u00a0<\/span>\u00a0<\/span><\/p>\n

After completing his PhD, he pursued postdoctoral work at University of California, Berkeley, where he developed computational microscopy techniques for imaging cells, leveraging tiny distortions of light that happen as it travels through cells.<\/span>\u00a0<\/span><\/p>\n

Since then, Tian\u2019s passion for inventing novel computational microscopy tools has been ablaze. Upon arriving at BU in 2016 to start his own lab, \u201call my projects, both large and small, focused on microscopy.\u201d Now, leading BU\u2019s <\/span>Computational Imaging Systems Lab (CISL)<\/span><\/a>, Tian is making incredibly powerful and extremely small new types of microscopes possible \u2013 with the goal of visualizing how cells and the brain\u2019s neurons communicate and behave with higher resolution and clarity.<\/span>\u00a0<\/span><\/p>\n

\u201cThrough computational imaging, we are augmenting imaging hardware with advanced algorithms to increase what microscopes are capable of capturing,\u201d Tian says. \u201cHarnessing deep learning to understand the way light travels through space, the way light travels through tissue\u2026 that\u2019s the special sauce of what we\u2019re doing.\u201d<\/span>\u00a0<\/span><\/p>\n

And it\u2019s taking a village of research partners. \u201cIt\u2019s a nexus of research that\u2019s bringing together many different people \u2013 to solve big problems like this, understand and construct the hardware, and computationally extract and analyze gigantic amounts of information, we need people from all these different backgrounds.\u201d\u00a0<\/span>\u00a0<\/span><\/p>\n

Tian received a Dean\u2019s Catalyst Award from ENG, which promotes such cross-disciplinary research, that helped him launch a hub of collaborations. CISL has stretched its tendrils into a network of collaborations across BU\u2019s Charles River and Medical campuses. Tian is affiliated with BU\u2019s <\/span>Department of Biomedical Engineering<\/span><\/a>, the <\/span>Neurophotonics Center<\/span><\/a>, the <\/span>Center for Information and Systems Engineering (CISE)<\/span><\/a>, the <\/span>Hariri Institute for Computing and Computational Science and Engineering<\/span><\/a>, and the <\/span>Nanotechnology Innovation Center<\/span><\/a>.\u00a0<\/span>\u00a0<\/span><\/p>\n

The lab\u2019s goals have evolved to be \u201cabout 80 percent focused on neuroscience applications,\u201d Tian says, crediting the support of <\/span>David A. Boas<\/span><\/a>, director of BU\u2019s Neurophotonics Center, and research partnerships with BU Neurophotonics faculty members <\/span>Jerry L. Chen<\/span><\/a> and <\/span>Ian G. Davison<\/span><\/a> as essential to his team gaining momentum in brain science applications, which Tian had no prior experience in.\u00a0<\/span>\u00a0<\/span><\/p>\n

These days, CISL\u2019s work is getting noticed near and far.<\/span>\u00a0<\/span><\/p>\n

In January 2023, Tian\u2019s Computational Imaging Systems Lab was awarded more than $1.3 million over two and a half years in funding from the <\/span>Chan Zuckerberg Initiative (CZI)<\/span><\/a>, part of a grant program that backs advancements in monitoring biological processes in motion and across time and space. With the CZI support, the team is designing miniaturized microscopes that can image whole mouse brains at single-cell resolution, seeking to visualize molecular and cellular activity like never before.<\/span>\u00a0<\/span><\/p>\n

Their petite, lightweight microscope designs are empowered by special miniaturized lenses to enable powerful imaging despite their compact size. Aboard each \u2018scope, the team typically employs an array of several microlenses to peer into samples from multiple angles, combining those readouts computationally to build 3D images of the brain.<\/span>\u00a0<\/span><\/p>\n

\"\"They call their devices mesoscopes because they can achieve both single-cell imaging of neurons and a large field of view across brain tissue; the prefix \u201cmeso-\u201d originates from the Greek word for middle, and used here it describes a mesoscope\u2019s ability to image both individual cells and whole tissues.\u00a0<\/span>\u00a0<\/span><\/p>\n

\u201cWe\u2019re building these as wearable devices for fundamental research in mice, with the goal that we can mount one of our mesoscopes on a mouse\u2019s head, allowing us to image neural activity across its entire brain as the mouse moves around and performs various behaviors,\u201d Tian says.\u00a0<\/span>\u00a0<\/span><\/p>\n

Within his lab, teammates are working on several approaches to creating their miniaturized wearable microscopes. PhD student Joseph Greene is experimenting with different architectural designs of non-conventional optics and also investigating which algorithms best enhance their imaging capabilities. In May 2023, Tian and Greene were authors of a paper published in <\/span>Neurophotonics<\/span><\/i> describing a new low-cost, head-mounted miniscope<\/span><\/a> that can capture images from deeper within mouse brain tissue using miniature diffractive optics and an algorithm that compensates for the effect of light scattering through tissue.<\/span>\u00a0<\/span><\/p>\n

Meanwhile, Yujia Xue, a former Tian lab member (who earned his PhD and is now developing new cameras at Apple, Inc.), and Qianwan Yang, a current PhD student, have been dedicated to the algorithm and deep-learning aspect of microlens arrays, seeking to computationally extract the most 3D information from each sample as accurately as possible. In August 2023, Xue, Yang and Tian published a study in <\/span>Optica<\/span><\/i> describing a <\/span>computational miniature mesoscope design<\/span><\/a> that utilizes deep learning image reconstruction to achieve high resolution, wide-field microscopy.<\/span>\u00a0<\/span><\/p>\n

\"\"In October 2023, Tian\u2019s team <\/span>reported in <\/span>arXiv<\/span><\/i><\/a> an ultrafast imaging technique using single-shot, wide-field 3D imaging augmented with a sensor that detects changes in neural activity, or \u201cevents\u201d. It gives its readout in response to these events, rather than frame by frame like more conventional \u2018scopes. They further enhanced the information of the event-based readout by designing an algorithm to accurately interpret imaging data both spatially and in the context of time. In the lab, they demonstrated how this technique can capture clear and contextually rich images of freely moving <\/span>C. elegans <\/span><\/i>worms. Tian and his co-authors say the method could have wide-ranging applications across biological and biomedical research.<\/span>\u00a0<\/span><\/p>\n

Beyond fundamental research, Tian\u2019s lab is also closely collaborating with researchers at BU\u2019s Chobanian & Avedisian School of Medicine (MED), including faculty members Ann McKee, B. Russell Huber, and Jonathan Cherry, leaders at BU\u2019s dedicated research centers focused on Chronic traumatic encephalopathy (CTE), Alzheimer\u2019s, and other neurodegenerative diseases. Together, they\u2019re working to advance computational imaging systems and deep learning algorithms to image and interpret human brain data.<\/span>\u00a0<\/span><\/p>\n

\u201cWe\u2019re building multi-scale datasets and looking to cover the entire human brain so that we can gain more insights into disease development and progression in people,\u201d Tian says. The collaborators envision that computational imaging could help pinpoint the exact location of molecules like tau proteins, the spread of which is a key catalyst in both Alzheimer\u2019s and CTE.\u00a0<\/span>\u00a0<\/span><\/p>\n

Undergraduate researcher Sunni Lin, who was mentored by former Tian lab member Shiyi Cheng, is using deep learning to extract new information from human brain images, essentially looking to map out structural data using digital \u2013 rather than traditional chemical \u2013 staining. (Cheng defended his thesis this year and is now at Apple, prototyping camera features and video algorithms.) Digital staining could simplify sample preparation, making neural imaging more broadly accessible to researchers with diverse training. It could also create more bandwidth within tissue samples for prioritizing other types of labels and markers attached to molecules of interest.<\/span>\u00a0<\/span><\/p>\n

\u201cThe focus on deep learning algorithms continues to grow within my group,\u201d Tian says. \u201cWe\u2019re not just using off-the-shelf datasets or general-purpose neural networks for training our [microscopy] algorithms. We\u2019re really trying to integrate as much physical knowledge as we can through experimental data \u2013 and that\u2019s the core philosophy of computational imaging, to synergistically combine algorithms with imaging methods.\u201d\u00a0<\/span>\u00a0<\/span><\/p>\n

Within the last year, Tian\u2019s team has also received funding support from the National Institute of Neurological Disease and Stroke at the National Institutes of Health, National Institute of Biomedical Imaging and Bioengineering, and Samsung.<\/span><\/i>\u00a0<\/span><\/p>\n","protected":false},"excerpt":{"rendered":"

Lei Tian\u2019s Computational Imaging Systems group is calculating new ways to see molecules, cells, and life in action\u00a0 By Kat J. McAlpine Research images provided by Lei Tian | Photos by Kelly Pe\u00f1a Lei Tian \u2013 a faculty member at BU\u2019s Photonics Center and a BU College of Engineering (ENG) assistant professor of electrical and […]<\/p>\n","protected":false},"author":22337,"featured_media":0,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[7001,6990],"tags":[],"_links":{"self":[{"href":"https:\/\/www.bu.edu\/photonics\/wp-json\/wp\/v2\/posts\/9657"}],"collection":[{"href":"https:\/\/www.bu.edu\/photonics\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.bu.edu\/photonics\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.bu.edu\/photonics\/wp-json\/wp\/v2\/users\/22337"}],"replies":[{"embeddable":true,"href":"https:\/\/www.bu.edu\/photonics\/wp-json\/wp\/v2\/comments?post=9657"}],"version-history":[{"count":7,"href":"https:\/\/www.bu.edu\/photonics\/wp-json\/wp\/v2\/posts\/9657\/revisions"}],"predecessor-version":[{"id":9688,"href":"https:\/\/www.bu.edu\/photonics\/wp-json\/wp\/v2\/posts\/9657\/revisions\/9688"}],"wp:attachment":[{"href":"https:\/\/www.bu.edu\/photonics\/wp-json\/wp\/v2\/media?parent=9657"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.bu.edu\/photonics\/wp-json\/wp\/v2\/categories?post=9657"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.bu.edu\/photonics\/wp-json\/wp\/v2\/tags?post=9657"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}