What you need to know
- Google is introducing software that enables users to detect unknown phishing/spam calls in real-time, alerting users to hang up the call before they give out personal details.
- Circle to search will now be able to solve complex math and physics problems, by providing a step by step process and examples that help the user understand concepts better.
- The already existing Talk Back feature will now be able to provide accurate descriptions of images shared by friends or family.
- Gemini is said to pop up in other apps as and when needed and help users generate images that can be dropped onto Gmail, Google messages, or other apps.
- The new Gemini 1.5 can also provide more information on YouTube videos when paused through "Ask this video"
AI took center stage at this year's Google I/O, so much so that it was used 121 times throughout the presentation. Since AI is now at the core of most Android devices and operating systems, here are some of the new updates that were discussed at this year's event—specifically catering to Android users.
One thing that caught our eye at this year's I/O, which is the most useful add-on to Android phones, is phishing scam detection.
Google is introducing software that enables users to detect unknown phishing/spam calls in real time. They're currently testing a feature that uses Gemini Nano to provide alerts during calls if it detects that the unidentified caller uses speech patterns associated with scams.
Imagine a user getting a call from an unknown/private number asking them to transfer funds as their current account could be compromised, requesting personal information like card numbers, pins, or passwords. Gemini will immediately show users a red warning sign on the phone screen that reads "likely spam," asking the user to end the call, along with a short description of why Gemini thinks it could be a threat.
Google says the conversation is encrypted and private to the user. They will share more information for users to opt-in "later this year." This feature could come in handy specifically for older users or even children, who might be unaware and get looped into multiple scams that have been doing the rounds.
We've all heard about the magic of AI with the Circle to search ever since Samsung's Unpacked event. Google says it's now taking this further by getting its Language Learning Model to solve math or physics problems when prompted. This turns the feature into a personalized tutor that shows you how to solve a math problem and gives you examples from your daily life to help you understand tough physics concepts.
The tech giant says this will give users the answer to those word problems and a step-wise explanation of how to solve them. The company also teases more upgrades later this year that will help students analyze and solve problems involving symbolic formulas, diagrams, and graphs.
Additionally, Google announced a faster Gemini 1.5 model, and with it, Android users will soon be able to bring up Gemini on top of any app. With AI integrated into Android phones, users can get creative by generating fun AI images and dropping them into various apps like Gmail and Google messages or even creating memes to add to their chats.
With "Ask this video" and "Ask this PDF," users will be able to get more information about a YouTube video they're watching or get answers about a PDF without the need to scroll through pages of information.
While Google didn't give us details about when we can see these tricks on our phones, we can expect them to roll out in the "next few months."
Finally, Gemini Nano and Multimodality will give the already existing TalkBack option more "powers." This means that Android phones can process text with a better understanding of sounds, sights, and even spoken language.
TalkBack is a tool that helps people who experience blindness or low vision by reading out loud accurate descriptions of what’s happening in an image, so an upgrade like this could be very beneficial. For instance, when an image of a dress is shared, the TalkBack option will describe the dress's exact color, the length of the sleeves, and other accessories that go with it, helping the user visualize the dress effectively without looking at it.
Google says these descriptions are generated in real-time and don't need a network connection, as Gemini Nano will work directly on the user's device.
Gemini Nano with Multimodality will be coming to Pixel phones later this year. However, it remains unclear which models will receive this. We'll have to watch for those monthly updates to see when new features start rolling out.
