Search venue sites for concerts to attend
January 29, 2025. Task: January 29, 2025 Operator: ChatGPT in a VM Reviewing Operator as it is Today Looking to the future Browser navigation: It's obvious to us, but the web is actually quite messy. Interfaces aren't always clear, where we can or can't click depends on each website, etc. Operators struggle a lot with this to the point of failing tasks simply because they don't understand how to deal with some forms or navigation. The VM isn't your computer: I was hoping to have Operator connect to my Google Workspace Gmail account to automatically fetch receipts. I couldn't manage to sign in because of how difficult it is to interact with the VM. Reasoning: Supposedly, Operator uses GPT-4o under the hood. I struggle to believe it, as it feels like a huge step back. This is important because tasks on the web are more than a simple Google (or in this case Bing) search; if you need to look for alternatives to a certain product, you understand that some websites are going to offer better insights than others. You also know the importance of checking multiple results. Operator doesn't know any of these, making its output very low quality. #ai #productivity #review; tools: Operator. Human role: The original post does not say how a person stayed involved.; limitations: Operator is a research preview with significant limitations in browser navigation and reasoning. It cannot easily handle complex forms or date pickers. The VM environment prevents easy login and password pasting. The model often relies on low-quality search results rather than visiting specific product or event pages, leading to inaccurate outputs..
Tools
Published on Agent Work Atlas:
Overview
January 29, 2025
Workflow
January 29, 2025 Operator: ChatGPT in a VM Reviewing Operator as it is Today Looking to the future Browser navigation: It's obvious to us, but the web is actually quite messy. Interfaces aren't always clear, where we can or can't click depends on each website, etc. Operators struggle a lot with this to the point of failing tasks simply because they don't understand how to deal with some forms or navigation. The VM isn't your computer: I was hoping to have Operator connect to my Google Workspace Gmail account to automatically fetch receipts. I couldn't manage to sign in because of how difficult it is to interact with the VM. Reasoning: Supposedly, Operator uses GPT-4o under the hood. I struggle to believe it, as it feels like a huge step back. This is important because tasks on the web are more than a simple Google (or in this case Bing) search; if you need to look for alternatives to a certain product, you understand that some websites are going to offer better insights than others. You also know the importance of checking multiple results. Operator doesn't know any of these, making its output very low quality. #ai #productivity #review
Reported result
The task failed to produce usable results. The first attempt returned names and dates without URLs, making verification difficult. The second attempt, which requested URLs, took 33 minutes and returned three concerts that were completely irrelevant to the user's criteria. The author attributes this to poor reasoning and navigation skills.
Human role
The original post does not say how a person stayed involved.
Limitations
Operator is a research preview with significant limitations in browser navigation and reasoning. It cannot easily handle complex forms or date pickers. The VM environment prevents easy login and password pasting. The model often relies on low-quality search results rather than visiting specific product or event pages, leading to inaccurate outputs.
Try it
Act as a web research agent using OpenAI Operator. I want to find classical concerts in [City] for [Date Range]. Please visit the websites of major venues: [Venue 1 URL], [Venue 2 URL]. Navigate to their event calendars, set the date filters, and extract the concert title, date, time, and direct URL for each event. Do not rely solely on search engine snippets; you must visit the venue pages. If a date picker is difficult to use, try alternative navigation methods or report the failure. Output the results in a table format. Before finalizing, verify that the URLs are valid and the events match the criteria. If the agent struggles with navigation, stop and report the specific UI element causing the issue.
Source reports are not independently verified. Publication date is the catalog publication date, not the original article date.