Google announced upgrades to its multimodal Gemini 1.5 Pro model, which, according to the company, will make it better at reasoning, translating, and helping with code generation. The company also introduced Gemini 1.5 Flash, a model as powerful as the main Gemini but more efficient for "narrow, low-latency, high-frequency tasks" like local on-device search.

This summer, Google will introduce a new feature to Photos that enables Gemini to sift through users' photos and reply to queries based on analyzing the pictures' visual content. During a demo, Google CEO Sundar Pichai asked Gemini what his license plate was, and the model replied by pulling a picture of the plate from his Photo Library. We doubt that was Pichai's actual plate, though.

Courtesy Google

Google has updated Google Lens to enable video search in addition to the usual image-based search capabilities. Users can now record a video of the subject they want to know more about and ask a question with their voice during the video. Google's models will interpret the video and audio content to provide a contextual and relevant answer based on Google Search results.

Project Astra is a new advanced AI assistant designed to function as an all-encompassing virtual helper. It sports multimodal capabilities to observe and interpret visuals through your device's camera, keep track of belongings, and perform tasks for the user. Google's vision for Project Astra is to evolve it into a fully functional AI agent that can communicate with users and carry out actions on their behalf.

Veo is a new video generation model that competes directly with OpenAI's Sora. The model can output 1080p footage based on video, photo, or text inputs and in many different styles, from aerial shots to timelapses. The company has already started offering Veo to some selected creators on YouTube.