Test OpenAI's Agent on online shopping
Spent my afternoon stress-testing the new “Agent” feature that’s supposed to handle shopping, travel, and reservations for you. Task: Spent my afternoon stress-testing the new “Agent” feature that’s supposed to handle shopping, travel, and reservations for you. Here’s the real-world outcome: What the Marketing Promised: AI agent that browses the web and completes tasks! What Actually Happened: A token-devouring Wikipedia wrapper that can’t access any major commercial site. My Test Results What Failed: Amazon: “Sorry, something went wrong” (classic Amazon error dog screen) Best Buy, Walmart, Target: All blocked Travel/Booking Sites: No bookings, no reservations Any JavaScript-heavy site: Non-functional What Worked: Wikipedia Some government sites Generating PowerPoints explaining its own failures Technical Architecture Exposed Agent uses two browsers (text and GUI). Both get shut down by anti-bot systems everywhere that matters. The “API Tool” (which should connect to partners) is disabled, with zero transparency on when or why. Token usage is wild: my first big task looped for 18 minutes, retrying the same failures until I killed it. No visibility on token consumption: Agent admits it cannot show you how many tokens it’s burning. Notable Moments Asked if Agent was worth $20/month. No answer—just endless “thinking” until my quota ran out. When confronted (“You can’t complete tasks, you’re not worth $20/month”), it only replied: Understood. Thank you for sharing your perspective. TL;DR You’re paying $20/month to beta test a product that: Can’t shop (blocked everywhere) Can’t book travel Can’t make reservations Burns tokens at a crazy rate (no tracking) Fails silently unless you force it to admit it https://preview.redd.it/g2ldfl47g9ff1.png?width=1626&format=png&auto=webp&s=c27ecd4de935876b4eea5e2d39429184a5f95ba9 https://preview.redd.it/7ej7fej9g9ff1.png?width=1536&format=png&auto=webp&s=44e9b4c05cf1a88133e69d0e2b87908727e91206 https://preview.redd.it/0jnxxdqfg9ff1.png?width=1189&format=png&auto=webp&s=436670df1e0f693c3cee7e2870b74a9526fb4680 https://preview.redd.it/37z2ctuhg9ff1.png?width=1680&format=png&auto=webp&s=f02ed5d641fbb2fa0a282738526b6fb07181b6aa https://preview.redd.it/tqm7d3xjg9ff1.png?width=1272&format=png&auto=webp&s=c241f99bfc420f6fc9100f672805bc7cdadc8671 I will continue to share on a new post in this thread I am limited by Reddit; tools: ChatGPT Agent. Human role: The original post does not say how a person stayed involved.; limitations: Prerequisites include a $20/month subscription. Constraints are severe: anti-bot systems block major retailers, CAPTCHAs, and login walls. The 'API Tool' is disabled without transparency. Token usage is untracked and high. Legal and policy barriers prevent scaling to real-world commercial tasks. Human approval is required for any potential transaction, though none succeeded..
Tools
Published on Agent Work Atlas:
Overview
Spent my afternoon stress-testing the new “Agent” feature that’s supposed to handle shopping, travel, and reservations for you.
Workflow
Spent my afternoon stress-testing the new “Agent” feature that’s supposed to handle shopping, travel, and reservations for you. Here’s the real-world outcome: What the Marketing Promised: AI agent that browses the web and completes tasks! What Actually Happened: A token-devouring Wikipedia wrapper that can’t access any major commercial site. My Test Results What Failed: Amazon: “Sorry, something went wrong” (classic Amazon error dog screen) Best Buy, Walmart, Target: All blocked Travel/Booking Sites: No bookings, no reservations Any JavaScript-heavy site: Non-functional What Worked: Wikipedia Some government sites Generating PowerPoints explaining its own failures Technical Architecture Exposed Agent uses two browsers (text and GUI). Both get shut down by anti-bot systems everywhere that matters. The “API Tool” (which should connect to partners) is disabled, with zero transparency on when or why. Token usage is wild: my first big task looped for 18 minutes, retrying the same failures until I killed it. No visibility on token consumption: Agent admits it cannot show you how many tokens it’s burning. Notable Moments Asked if Agent was worth $20/month. No answer—just endless “thinking” until my quota ran out. When confronted (“You can’t complete tasks, you’re not worth $20/month”), it only replied: Understood. Thank you for sharing your perspective. TL;DR You’re paying $20/month to beta test a product that: Can’t shop (blocked everywhere) Can’t book travel Can’t make reservations Burns tokens at a crazy rate (no tracking) Fails silently unless you force it to admit it https://preview.redd.it/g2ldfl47g9ff1.png?width=1626&format=png&auto=webp&s=c27ecd4de935876b4eea5e2d39429184a5f95ba9 https://preview.redd.it/7ej7fej9g9ff1.png?width=1536&format=png&auto=webp&s=44e9b4c05cf1a88133e69d0e2b87908727e91206 https://preview.redd.it/0jnxxdqfg9ff1.png?width=1189&format=png&auto=webp&s=436670df1e0f693c3cee7e2870b74a9526fb4680 https://preview.redd.it/37z2ctuhg9ff1.png?width=1680&format=png&auto=webp&s=f02ed5d641fbb2fa0a282738526b6fb07181b6aa https://preview.redd.it/tqm7d3xjg9ff1.png?width=1272&format=png&auto=webp&s=c241f99bfc420f6fc9100f672805bc7cdadc8671 I will continue to share on a new post in this thread I am limited by Reddit
Reported result
Actual outcomes show failure on major commercial sites due to anti-bot blocks and disabled API tools. Success is limited to Wikipedia and some government sites. The agent exhibits high token consumption with no visibility, often looping for 18+ minutes on failed tasks. It cannot complete bookings or purchases, failing silently or returning generic errors.
Human role
The original post does not say how a person stayed involved.
Limitations
Prerequisites include a $20/month subscription. Constraints are severe: anti-bot systems block major retailers, CAPTCHAs, and login walls. The 'API Tool' is disabled without transparency. Token usage is untracked and high. Legal and policy barriers prevent scaling to real-world commercial tasks. Human approval is required for any potential transaction, though none succeeded.
Try it
I want to evaluate ChatGPT Agent's capability for online shopping and travel booking. Please perform a stress test on [replaceable inputs: e.g., Amazon, Expedia, Wikipedia]. Steps: 1. Attempt to access [target site] and search for [specific product/service]. 2. Monitor for anti-bot blocks, CAPTCHAs, or login walls. 3. If the agent loops for more than 5 minutes without progress, terminate the task. 4. Record token usage if visible; otherwise, note the lack of visibility. 5. Generate a summary report of failures and successes. Output Format: A table listing Site, Task, Outcome (Success/Fail/Blocked), and Notes. Quality Check: Verify if any actual transaction or reservation was completed. If not, mark as 'Failed'. Safeguards: Do not enter any payment information or personal credentials. Stop immediately if the agent requests sensitive data. Confirm with me before attempting any site that requires login.
Source reports are not independently verified. Publication date is the catalog publication date, not the original article date.