Google Breaks AI Out of the Chat Window | PYMNTS.com

For most of computing history, the pointer has tracked where users look. Google now wants it to understand what they see. The company calls the concept AI Pointer, a cursor that reads on-screen content and understands why it matters to the user, without any manual prompting. Google has already built experimental demos into Google AI Studio.

The Problem With the Prompt Box

Current artificial intelligence tools require users to do the translation work themselves: open the chatbot, describe the document, paste the relevant section, ask the question. Each step is a break in workflow. DeepMind’s framing is blunt: Today’s AI tools make users drag their world into them. AI Pointer flips the model by capturing visual and semantic context directly from wherever the pointer sits, across documents, websites and applications, so users can point at a table of figures and say “make this a…