Google cloud expands generative AI capabilities for health care

Google Cloud expands generative AI capabilities: ©Sundryphotography - stock.adobe.com

Google Cloud announced new generative AI features for Vertex AI Search for health care, including Visual Q&A, a tool that can analyze tables, charts, and diagrams, as well as the integration of Gemini 2.0, the latest iteration of Google’s AI model. The updates are designed to help health care organizations provide clinicians with a more comprehensive view of patient health through multimodal AI technology, according to the company.

“We’re helping clinicians to work more efficiently with new multimodal capabilities embedded in Vertex AI Search for health care,” said Aashima Gupta, global director of Healthcare Strategy & Solutions at Google Cloud, in a statement. “Multimodal analysis processes diverse sources of patient data, like medical images and genetic information, for a more comprehensive understanding and improved decision-making.”

Enhancing multimodal AI for health care

Multimodal AI integrates information from various sources, including text, images, and videos. This approach is particularly relevant in health care, where nearly 90% of data is in image form, such as x-rays and scans. By incorporating medical images, patient history, and genetic data, AI-powered tools can enhance diagnostic accuracy, personalize treatment plans, and improve patient outcomes.

Vertex AI Search for health care, a specialized component of Google Cloud’s Vertex AI platform, aids developers in building assistive technologies that retrieve information from complex medical records and documents. The introduction of Visual Q&A and Gemini 2.0 further strengthens these capabilities, according to the company.

Visual Q&A allows Vertex AI Search for health care to process images like medical diagrams and charts as direct inputs, eliminating the need to convert them into text. This functionality enables the system to analyze patient records more effectively. Meanwhile, Gemini 2.0 enhances the speed and accuracy of AI-driven searches, improving the retrieval and processing of multimodal data, including visual medical content.

Industry adoption and partnerships

Several health care technology companies have already begun leveraging Google Cloud’s latest AI advancements.

Counterpart Health, an AI-powered physician enablement company, aims to enhance value-based care through proactive management of chronic diseases. The company’s Counterpart Assistant integrates real-time patient insights into physicians’ workflows.

“Counterpart Health is dedicated to helping clinicians deliver the highest quality of care to their patients,” said David Tsay, MD, chief medical officer at Counterpart Health, in a statement. “The advancements in Vertex AI Search for health care provide us with powerful new tools to achieve these goals. Multimodal AI capabilities will enable clinicians to integrate diverse types of clinical data sources, providing a more comprehensive understanding of their patients’ health and facilitating personalized care management.”

Meditech, a global EHR company, has incorporated AI-powered search and summarization tools into its Expanse EHR system. The new Google Cloud features further enhance these capabilities, providing care teams with advanced search functionalities.

“Meditech’s collaboration with Google Cloud will bring the latest advancements in multimodal AI to our Expanse EHR,” said Cathy Turner, chief marketing and nursing executive at Meditech, in a statement. “Building on our successful work with Google Cloud in developing AI-powered search and summarization, these advancements will further enhance our ability to deliver cutting-edge tools that empower clinicians and improve workflows.”

Suki, a provider of AI-driven clinical assistants, recently integrated patient summarization and clinical Q&A into its Suki Assistant, powered by Google Cloud technology. These features streamline access to essential patient data, reducing administrative burdens for clinicians, according to the company.

“Google Cloud is continuing to push the boundaries of what’s possible with multimodal AI in health care,” said Punit Soni, founder and CEO of Suki. “The advancements in Vertex AI Search for health care complement Suki’s mission to make health care technology invisible and assistive. By combining Suki’s expertise in ambient AI with Google Cloud’s powerful multimodal capabilities, we are equipping clinicians with instant access to the critical information they need, enabling them to deliver the best possible patient care.”