If you didn’t get a chance to listen to our most recent episode of the Collective Intelligence Community Podcast, go listen to our very own Thomas Joshi’s interview with Quinn Slack, CEO of the AI code intelligence platform Sourcegraph. They dive into his learnings from co-founding Blend Labs and discuss how code search is shaping the future of developer productivity!

While theorists and much of industry discourse has been framing the appropriate language, lens, and talking points to discuss the “AGI,” something HUGE just dropped in the AI world that feels like an effective means towards it. 

Anthropic quietly released what might be the biggest game-changer in AI capabilities since GPT-3: Computer Control for Claude via API, enabling AI to perform real computer actions—clicking, typing, scrolling, and browsing—beyond simple prompting. This tool opens the door for developers to build on this technology, making possibilities as vast as our budgets and imaginations.

On OSWorld, which evaluates AI models' ability to use computers like people do, Anthropic reports tClaude 3.5 Sonnet scored 14.9% in the screenshot-only category, notably outperforming the next-best AI system's score of 7.8%. When given more steps to complete the task, Claude’s score increased to 22.0%. These metrics highlight Claude's budding capability to interact with computer environments in a more human-like manner.

Let's keep it real – this isn't perfect yet. Some things that humans find super easy (like scrolling or dragging) are still tricky for Claude. As Anthropic puts it, they're encouraging developers to start with "low-risk tasks." Translation: Maybe don't have it control your nuclear power plant just yet.

While this capability is still in its infancy and Claude's current ability to use computers is imperfect, especially with actions like scrolling, dragging, and zooming, Anthropic encourages developers to start experimenting with low-risk tasks. Recognizing the potential risks—such as spam, misinformation, or fraud—Anthropic has proactively developed new classifiers to detect when computer use is being exploited and to identify any harmful activities. For an in-depth look into the research and safety measures, check out their post on developing computer use.

Unlike traditional RPA which follows rigid, pre-programmed workflows, generative AI automation represents a fundamental shift toward true cognitive automation. Where RPA excels at repetitive, rule-based tasks, AI agents can retrieve and synthesize information from multiple sources, engage in complex reasoning, and dynamically adjust their workflows based on context and intermediate findings. This enables them to handle intellectual tasks like research analysis, content creation, and strategic decision-making that were previously beyond the scope of automation.

Bloomberg Intelligence projects the generative AI market will skyrocket from $40 billion in 2022 to $1.3 trillion by 2032, a 42% CAGR. Adding computer control amplifies automation’s potential in industries like research, software development, and more.

The RPA market alone is projected to grow from $7.94 billion in 2024 to $19.21 billion by 2028. With AI like Claude integrating more versatile functionality, the combined market opportunity across AI and RPA is estimated at $1.8 trillion by 2030. Yes– with a “T!”

Legacy software compatibility and error-handling in automation will be pivotal for widespread adoption. Claude’s adaptability in non-standard scenarios could give it a competitive edge over traditional RPA.

So, if you’re a developer or entrepreneur, let’s just say your next VC pitch deck’s “market size” slide is going to have quite the appealing pie to picture.

Google is set to unveil its AI-powered "computer-using agent," codenamed “Project Jarvis,” as early as December 2024. This groundbreaking technology aims to revolutionize how we interact with our devices by automating everyday web-based tasks within Chrome, such as conducting research, making purchases, and booking flights. Jarvis operates by capturing frequent screenshots of the user's screen, interpreting the visual content, and executing appropriate actions based on user commands. Powered by the next-generation Gemini 2.0 model, Jarvis represents a significant leap in in the universal daily applications that will be widespread in the imminent future. 

While the technology promises increased productivity and efficiency, it also raises important questions about data privacy and security. The success of Jarvis will largely depend on its accuracy, speed, and ability to gain users' trust. 

The introduction of computer control capabilities in AI like Claude represents a monumental shift in how we interact with technology. Unlike traditional automation tools that require predefined scripts and are limited to specific tasks, Claude’s ability to navigate dynamic interfaces, understand context, and adapt to changes opens up endless possibilities for automation and efficiency. This advancement not only enhances productivity but also paves the way for more intuitive and seamless human-AI collaboration.

What do you think about this development? Drop your thoughts in the comments! How do you envision agentic frameworks and agent-based apps shaping the future of work? Reach out to share your thoughts on our community Slack or email [email protected] 

We’re thrilled to announce the launch of two new GenAI Collective chapters in Washington, DC and Los Angeles!

These cities represent incredible hubs of AI innovation and talent, and we’re excited to bring our mission to foster real, impactful connections to each. In DC, Catherine McMillan will be leading the charge, building a community that bridges government, academia, and private sector AI initiatives. On the West Coast, Brandon Hoffman will head up our LA chapter, connecting top AI talent with startups and entertainment tech innovators!

Make sure to follow our DC & LA Luma Calendars so you don’t miss out on what’s next!

This past week, we had the privilege of being part of the TEDAI San Francisco Hackathon for Good, hosted at Microsoft Reactor. The event brought together innovators and AI enthusiasts to develop real-world solutions, with projects ranging from AI-driven content for individuals with hearing challenges to generative AI for children's stories.

We were honored by the presence of Mayor London Breed, who stopped by to show her strong support for AI innovation and its growing role in shaping San Francisco's future!

Be sure to stay connected with future Bay Area events by following our Luma Calendar.

On October 16th, the GenAI Collective and AWS hosted the NYC Bedrock Top Startup Roadshow at Neuehouse Madison Square, bringing together some of New York’s brightest AI minds, founders, and investors.

Led by our NYC chapter heads, Sriram Ramakrishnan and Prithvi Raj, the event was a deep dive into AWS Bedrock’s capabilities, with sessions on data generation, deploying AI agents, and integrating top models like Anthropic and Stability AI.

After a full day of hands-on workshops and real talk about the future of AI, we capped things off with a happy hour—perfect for building new connections in NYC’s AI scene.

Huge thanks to AWS and everyone who joined us! Follow our NYC Luma Calendar so you don’t miss out on future events.

The GenAI Collective is growing and we need passionate, visionary community builders to join our team! If you want to join a team of 30+ organizers helping to shape the future of AI, we have tons of exciting ways to get involved. Read more about each opportunity below and learn how you can contribute to this vibrant community! 

About Aqeel Ali

Aqeel is currently COO at DeepAI. At the GenAI Collective, he focuses on co-leading the newsletter and systems building for the collective. When not immersed in AI, startup operations, or crafting satirical jokes, Aqeel delves into psychology and human creativity! 🎨

About Amal Irgashev

Once a professional music producer, Amal now develops custom AI applications for fast-growing tech and media startups at Triple G Ventures, focusing on optimizing operational efficiency, automating customer acquisition, and consulting on digital marketing.

As the Boston Chapter Co-Lead, he loves connecting students and aspiring founders to valuable resources, helping them navigate the AI landscape and launch innovative projects.