Swordfish

Recording Browser Interactions

Recording allows you to capture your manual interactions with a web page (clicks, typing, etc.) and include them in the conversation. This helps the agent understand what you're trying to accomplish.

How to Record

Try it: Record Browser Interactions Button
The button changes from gray to red with a pulsing animation when recording is active. Click to toggle between states.
  1. Enable Recording

    Click the "Record Browser Interactions" button in the Events panel. The button will turn red with a pulsing animation to indicate recording is active.

  2. Interact with the Page

    Click buttons, type in fields, navigate - do whatever you want the agent to understand or replicate.

  3. Stop Recording

    Click the button again to stop recording. Your interactions will appear in the Events panel.

⚠️ Important

Recording automatically cancels any ongoing agent execution. Make sure the agent has finished its current task before starting to record.

What Gets Recorded

When recording is enabled, the following interactions are captured:

  • Clicks - Button clicks, link clicks, element clicks
  • Typing - Text input in form fields
  • Navigation - Page navigations and URL changes
  • Other interactions - Various browser events

Including Interactions in Chat

Recorded interactions appear in the Events panel just like console logs and network requests. You can:

  • Click "Include in conversation" on specific interactions
  • Let the agent observe the interactions automatically
  • Use interactions to show the agent what you want it to do

Use Cases

  • Demonstrating a workflow - Record yourself completing a task, then ask the agent to automate it
  • Showing what to click - Record clicking a button, then ask the agent to do the same
  • Providing context - Record interactions to show the agent the current state of the page
💡 Tip

Recording is especially useful when you want the agent to replicate a specific user flow or understand a complex interaction pattern.

Recording vs. Direct Instructions

Sometimes it's easier to just tell the agent what to do:

  • "Click the submit button"
  • "Type 'test@example.com' in the email field"

But recording is useful when:

  • The interaction is complex or hard to describe
  • You want to show the exact sequence of actions
  • The page structure is dynamic or hard to target with selectors