VisualWebArena: Visual Web Agent Evaluation

[
  {
    "id": "vwa_001",
    "text": "Find the red sneakers shown in the homepage banner image and add them to the shopping cart in size 10.",
    "website": "E-commerce (Shopping)",
    "screenshots": "Screenshot 1: Homepage with large hero banner showing a lifestyle photo of a runner wearing red Nike Air Max sneakers on a trail. Navigation bar at top with categories: Men, Women, Kids, Sale. Search bar in upper right. Below the banner, a grid of 12 featured products with thumbnail images and prices.\n\nScreenshot 2: Search results page showing 8 sneaker products. The agent has typed 'red sneakers' in the search bar. Results include red Nike Air Max ($129.99), red Adidas Ultraboost ($149.99), red Puma RS-X ($89.99), and others. Each result has a product image, name, price, and star rating.\n\nScreenshot 3: Product detail page for Nike Air Max in red colorway. Large product image on left showing the shoe from side angle. Right side has title, price ($129.99), color selector with red highlighted, size dropdown menu expanded showing sizes 8-13, and 'Add to Cart' button.\n\nScreenshot 4: Same product page with size 10 selected in the dropdown. The 'Add to Cart' button is highlighted. A mini-cart popup in the upper right shows '1 item' with the red Nike Air Max thumbnail.",
    "trajectory": "Step 1: Observe homepage banner — identify red sneakers in hero image\nStep 2: Click search bar in upper right corner\nStep 3: Type 'red sneakers' and press Enter\nStep 4: Scan search results — identify Nike Air Max matching banner image\nStep 5: Click on Nike Air Max product listing\nStep 6: Click size dropdown menu\nStep 7: Select size '10' from dropdown options\nStep 8: Click 'Add to Cart' button\nStep 9: STOP — Task complete",
    "final_state": "Product page for Nike Air Max (Red) displayed. Size 10 selected. Cart indicator shows 1 item. Mini-cart popup confirms: Nike Air Max - Red - Size 10 - $129.99. Cart total: $129.99."
  },
  {
    "id": "vwa_002",
    "text": "On the travel website, find a hotel that looks similar to the one shown in the reference image (a beachfront resort with a pool) and book it for 2 nights.",
    "website": "Travel Booking",
    "screenshots": "Screenshot 1: Travel homepage with search form. A reference image is displayed at the top showing a tropical beachfront resort — white multi-story building, infinity pool facing the ocean, palm trees lining a white sand beach, thatched-roof cabanas by the pool.\n\nScreenshot 2: Hotel search results for 'Miami Beach' showing 6 hotels in a grid layout. Each has a large photo, name, star rating, price per night. Hotels include: The Palms Resort (beachfront with pool, $289/night), Ocean View Inn (city view, $159/night), Seaside Boutique (small beach, no pool, $199/night), Grand Beach Resort (large beachfront with infinity pool, $349/night).\n\nScreenshot 3: The agent has clicked on Grand Beach Resort. Detail page shows a photo carousel with 8 images — the main photo shows a white building with an infinity pool overlooking the ocean, closely matching the reference image. Room selection form visible below with date pickers and room type options.\n\nScreenshot 4: Booking confirmation page showing Grand Beach Resort, Deluxe Ocean View room, 2 nights, total $698. Check-in and check-out dates filled. Guest name form partially visible.",
    "trajectory": "Step 1: View reference image of beachfront resort with pool\nStep 2: Enter 'Miami Beach' in destination search field\nStep 3: Set check-in date to next available date\nStep 4: Set check-out date to 2 days later\nStep 5: Click 'Search' button\nStep 6: Scan hotel results — compare photos to reference image\nStep 7: Click on Grand Beach Resort (best visual match — beachfront, white building, infinity pool)\nStep 8: Review photo carousel to confirm visual similarity\nStep 9: Select 'Deluxe Ocean View' room type\nStep 10: Click 'Book Now' button\nStep 11: STOP — Task complete",
    "final_state": "Booking confirmation for Grand Beach Resort displayed. Room: Deluxe Ocean View. Duration: 2 nights. Total: $698. The selected hotel visually matches the reference image — beachfront location, white building, infinity pool facing the ocean."
  }
]

// ... and 6 more items