“Healthcare is moving very fast. And what often happens in adults, also happens in children. Unfortunately, most of the research is directed initially at adults, and then whittles down to children. At Children’s National, we’re trying to turn that around. We’re trying to do research for children that will expand its way up to adults, turning it on its head.”
Anthony Sandler, MD, senior vice president and surgeon-in-chief, Joseph E. Robert Jr. Center for Surgical Care, and director of the Sheikh Zayed Institute for Pediatric Surgical Innovation highlighted the exciting research and innovation happening at Children’s National – including demonstrating a technology, led by Raj Shekhar, PhD, that uses real-time imaging with augmented reality to project live ultrasound visualization of a patient within the surgeon’s field of view. This enhances surgical precision and ultimately supports positive patient outcomes.
This conversation was a part of Axios’ inaugural Future of Health Summit – an event bringing together the top voices in healthcare, policy and technology to explore the biggest challenges and innovations shaping the future of medicine.
https://innovationdistrict.childrensnational.org/wp-content/uploads/2025/05/Sandler-Axios-CNRI.jpg385685Innovation Districthttps://innovationdistrict.childrensnational.org/wp-content/uploads/2023/12/innovationdistrict_logo-1-1030x165.pngInnovation District2025-05-15 15:55:332025-05-15 16:13:36In the news: Axios’ Future of Health Summit
Children’s National experts developed a novel visualization platform, called “HoloUS,” for ultrasound-guided procedures using commercially available see-through augmented reality (AR) glasses, according to a pre-clinical study published in the International Journal of Computer Assisted Radiology and Surgery. The finding suggests that the HoloUS application enhanced the user experience and simplified hand-eye coordination.
“By eliminating head-turning and back-and-forth viewing of the patient and the ultrasound screen, our solution could improve the safety, accuracy, intuitiveness, and length of ultrasound-guided procedures, while making them easier to learn and perform,” said Raj Shekhar, M.S., Ph.D., principal investigator for Sheikh Zayed Institute for Pediatric Surgical Innovation at Children’s National and senior author in the study.
Ultrasound-guided procedures demand a high degree of hand-eye coordination to advance a needle with one hand while holding the ultrasound probe with another as they visualize the live ultrasound image. With this application, the practitioner will wear the AR glasses that will allow them to visualize the ultrasound image rendered live in their field of view.
“We are excited about the possibility of improving the accuracy and efficiency from augmented reality visualization because these benefits may translate clinically in the upcoming years after conducting carefully designed clinical studies,” said Trong Nguyen, staff scientist for Sheikh Zayed Institute for Pediatric Surgical Innovation at Children’s National and lead author.
Microsoft’s HoloLense headset, often used for hands-free work and video gaming, set the stage to develop the custom application that would display information from a portable ultrasound machine connected through a wireless network in the form of a hologram. The voice controls on the HoloLense allowed scaling and movement of the ultrasound imaging while displaying in real-time.
Largely driven by video gaming and consumer applications, AR and virtual reality technologies have been growing and are expected to grow at an even faster clip in the coming decade.
“We are a handful of researchers exploring the innovative application of the technology for ultrasound visualization,” said Shekhar. “We are trying to establish ourselves as the leader in the use of AR for pediatric ultrasound procedures.”
To further advance this technology, Shekhar’s team has an IRB protocol pending that will continue to shed light on the benefits through more bench testing and an NIH grant proposal that is also pending. In the meantime, they will adopt second-generation glasses to continue to improve the technology.
Children’s National Hospital researchers combined augmented reality (AR) and ultrasound imaging technologies to improve visualization during laparoscopic procedures. The patent-pending hybrid tracking method improved accuracy compared with hardware-based and computer vision-based approaches, according to the pre-clinical study published in the Journal of Medical Imaging.
“The system that we developed can work reliably under challenging intraoperative conditions, which is exciting for us,” said Raj Shekhar, M.S., Ph.D., principal investigator at Children’s National and senior author in the study. “The surgical view is usually very complex with various tissues, tools, blood and smoke in the view. Our system has been evaluated on pre-clinical models and has shown to provide acceptable overlay accuracy even during extreme conditions.”
Augmented reality for surgery is an emerging trend, consisting of a computer-generated image for the operating surgeon to see. While using ultrasound during surgery allows the doctor to see the organ’s internal structures, when combined, the AR system significantly improves the surgeon’s comprehension of the ultrasound image in the context of the surgical view.
“The AR system that we developed will benefit patients because AR-guided surgeries can be more precise, safer and faster,” said Shekhar. “Through sustained research and development effort and robust clinical-technical partnership, Children’s National is leading in clinical translation and technology transfer of the laparoscopic AR visualization technology. Other solutions are more laboratory-based while our focus has been on building a clinically practical and viable solution from day one.”
Laparoscopic surgery is known to improve outcomes, cause less scarring and speed the recovery process since the surgeons create small incisions with the aid of a camera. Still, doctors have a limited view of anatomic structures and surgical targets below the exposed surfaces.
Given that AR is an emerging technology, this work would not have been possible until recently. According to the researchers, no other scientists in the field take advantage of both tracking hardware and computer vision-based approaches.
“This work is important because it features best of both worlds: relying on tracking hardware to ensure robustness while using computer vision to enhance accuracy. This will generate a more accurate and reliable AR system which is more feasible for clinical use,” said Shekhar.
The hybrid tracking mount for the LUS transducer. The mount contains a six-DOF EM sensor and an AB with 21 markers fixed on three flat surfaces.
https://innovationdistrict.childrensnational.org/wp-content/uploads/2021/07/The-hybrid-tracking-mount-for-the-LUS-transducer.png300400Innovation Districthttps://innovationdistrict.childrensnational.org/wp-content/uploads/2023/12/innovationdistrict_logo-1-1030x165.pngInnovation District2021-07-02 10:31:582021-07-02 10:31:58New hybrid approach for AR visualization of ultrasound images has surgical use
