How Anthropic’s new ‘computer use’ ability could further AI automation

“Up until now, LLM developers have made tools fit the model, producing custom environments where AIs use specially-designed tools to complete various tasks. Now, we can make the model fit the tools,” Anthropic company wrote in a blog post, adding that the idea is to fit Claude into computer environments that people use daily and allow it to use software on that environment as a human user would.  

The message has clearly been heard by robotic process automation (RPA) vendor UIpath, which said Tuesday that it has integrated Claude 3.5 Sonnet in three of its products: UiPath Autopilot for everyone, Clipboard AI, and a new medical record summarization tool.

Claude’s computer use capability could shake up the RPA market, wrote Paul Chada, co-founder of AI startup Doozer AI, because it is not held up by constraints such as requiring constant maintenance or breaking when user interfaces change. “Anthropic’s new approach addresses these core challenges: Adaptive Interaction: Instead of hard-coded scripts, it actually understands what it’s looking at,” Chada wrote in a LinkedIn post, adding that other advantages of the system include it working across any interface and its potential to improve through usage and feedback.