# Changelog Source: https://docs.scoutqa.ai/changelog Latest updates and improvements to Scout ## Test Cases, Test Suites & the Test Runner This release turns Scout into a full test-management workspace. You can now save reusable **test cases**, group them into **test suites**, and run them—individually or as a batch—from a new **Test Runner** that executes cases in parallel and replays passing runs deterministically. ### What's New **Test Cases — reusable, two ways to write them** Save a scenario once and run it whenever you need to. Write a test case as explicit **step-by-step** instructions (with optional preconditions, expected results, and test data) or as a **high-level intent** that Scout interprets and adapts to your UI. Find them under **Library → Test Cases**. **Test Suites — group and run as a batch** Bundle related cases into named, ordered suites—*Smoke regression*, *Checkout flows*, *Auth*—under **Library → Test Suites**. Reorder cases, reuse a case across suites, and load a whole suite into the Test Runner in one click. **Test Runner — run everything in parallel** The new **Test Runner** runs your cases against any **Target URL**. Load cases from your library, from a suite, or by pasting JSON, then hit **Run**. Each case executes independently in its own browser session, so a batch runs **in parallel** with live status streaming into an execution grid—passed, failed, blocked, running, and queued—plus per-case step progress and links to the full execution and any bug reports. **Deterministic replay with Playwright** The first time a step-by-step case passes, Scout records a deterministic **Playwright script** of exactly what it did. After that, **Re-run with Playwright** replays the case verbatim—no AI, fully repeatable—and you can **Download script** to run it in your own pipeline. Parallelism respects your plan's limits: **Vibe** runs up to 5 concurrent cases and **Ship** up to 10. Extra cases wait in the queue and start as slots free up. ### Improvements * New **Test Runner**, **Test Cases**, and **Test Suites** entries in the project sidebar * Run history shows each case's **Last run** time at a glance * Paste-as-JSON support in the Test Runner for ad-hoc and CI-driven runs Learn how to write test cases, build suites, and run them in parallel. ## Smart Mode, Fix with Coding Agents & API Key Management This release introduces **Smart Mode** — a new way to test using natural language intent — along with one-click bug fixing via coding agents, CLI issue verification, and self-service API key management. ### What's New **Smart Mode — Intent-Based Testing (New Default)** Scout now understands *what you want to test*, not just *where*. Smart Mode is a new default testing mode that interprets your natural language input and automatically selects the right workflow, depth, and tools for the job. Just describe your intent and Scout figures out the rest: * **Test as a user persona** — "Test the checkout flow as a first-time mobile user" * **Execute test cases** — "Smoke test login, signup, and password reset" * **Verify bug fixes** — "Verify the cart total bug is fixed after my latest deploy" * **Validate a PRD or user story** — "Test the user story: users can reset their password via email link" * **Run an accessibility audit** — "Check WCAG compliance on the dashboard page" * **Exploratory testing** — "Explore the settings page and look for UX issues" Behind the scenes, Smart Mode classifies your intent into one of six workflow templates — quick-scan, deep-test, smoke-test, exploratory, a11y-check, or verification — and dynamically selects the right tools and depth level. You can still override by explicitly choosing Quick or Deep mode. Smart Mode is now the default for all new executions. Your existing Quick and Deep modes still work exactly as before. **Fix with Coding Agents** Found a bug? Fix it in one click. Scout now integrates with **Cursor**, **Codex**, and **Windsurf** — click "Fix with Agent" on any issue to launch your preferred coding agent with the bug context pre-loaded. Go from bug discovery to bug fix without leaving your flow. **CLI: Issue Verification** Two new CLI commands let you verify bugs directly from your terminal: ```bash theme={null} # List issues from a previous execution scoutqa list-issues --execution-id # Verify whether a specific issue is still reproducible scoutqa issue-verify --issue-id ``` Perfect for CI/CD pipelines where you want to confirm fixes before merging. **API Key Management UI** Create, view, and revoke API keys directly from **Account Settings** — no CLI required. Use API keys to authenticate the Scout CLI or integrate Scout into your automation workflows. **Mode Persistence** When you select a testing mode (Smart, Quick, or Deep) from the project page, Scout now remembers your selection and carries it through to execution. ### Improvements * Increased concurrency limits for Vibe and Ship plans * Custom instruction limit raised from 2,000 to 5,000 characters * Compact mode selector with hover attribute bars for a cleaner UI * Recurring issues now appear in execution summary emails * Docs and changelog links added to the header dropdown ### Bug Fixes * Fixed intent-based workflow crashing on certain error conditions * Fixed Playwright MCP middleware not being passed to instruction-based agent * Fixed plan resolution using the highest plan from user balances * Fixed stale plan display after subscription upgrade ## Roast Enhancements, Unattended Executions & Test Result Reports This release polishes the Website Roast experience, adds smarter background execution handling, and improves how you view and share test results. ### What's New **Roast UI Overhaul** The Website Roast experience is now more fun and engaging with memey language, playful animations, and improved sharing options. The roast landing page and result cards have been refreshed for a more entertaining experience. **Unattended Auto-Continue for Background Executions** Scheduled runs, evaluations, verifications, and roasts now automatically continue past step limits when time remains, instead of pausing and waiting for input. Background executions run more smoothly without manual intervention. **Report Summary Card** Test results now include a visual summary card showing your site's validation status, quality rating badge, and severity breakdown — both on the share page and in execution details. **Share Dialog for Reports** A new share dialog on report pages makes it easier to share your test results with teammates or stakeholders. **Early Finish Hint** A placeholder in the chat input now hints that you can finish a test session early, making the workflow more discoverable. **Stuck vs. Blocked Test Steps** The todo node UI now clearly distinguishes between stuck and blocked states, so you can quickly understand why a test step hasn't progressed. ### Improvements * Renamed "Execution report" to "Test result" across the platform for clearer terminology * Simplified execution header with a help popover and sign-in call-to-action * Replaced the login teaser modal with a less intrusive inline banner * Updated report header button styling for better visual consistency ### Bug Fixes * Fixed S+/S- grade handling and validation status logic * Fixed missing OG image metadata on report share pages ## Website Roast, Mode Selector & Observation Feedback This release introduces the Website Roast workflow, a new testing mode selector, smarter observation feedback, and many quality-of-life improvements across the platform. ### What's New **Website Roast** Get a fun, shareable quality assessment of any website. Scout's new Roast workflow analyzes your site and delivers a brutally honest (but helpful) quality report — complete with a shareable gallery, badges, and social-ready OG images. **Quick/Deep Mode Selector** You can now choose between Quick and Deep testing modes directly from the chat input. Quick mode gives you a fast overview, while Deep mode runs a thorough analysis. Anonymous users are prompted to sign up for Deep mode access. **Observation Feedback in Chat** When Scout identifies duplicate observations or dismisses non-issues, you now see exactly why — with clickable outcomes in the chat panel. Duplicates link to the original issue, making it easy to track recurring patterns. **Check Emails Tool** Scout can now check emails during testing sessions, with a dedicated mail icon and UI rendering support for email verification workflows. **Social Sharing for Reports** Share your test results on social media directly from the report share dialog with new sharing buttons. **Facehash Default Avatars** Users without profile images now get unique, auto-generated avatars based on their identity — no more generic placeholders. **Inline URL Input** The execution dialog has been replaced with a streamlined inline URL popover, making it faster to start new test sessions. **Bug Verification from Issue Detail** You can now trigger bug verification directly from the issue detail view, making it quicker to confirm whether an issue is still reproducible. **Critical Graph Issue Preview** Click on nodes in the critical issue path graph to preview issue details and open the full issue drawer without leaving the graph view. ### Improvements * Magic link expiry extended to 6 hours for a more flexible login experience * Added spam folder reminder to magic link success message * Improved login teaser dialog responsive layout * Enhanced execution report accessibility * Redesigned verification section with updated done step UI ### Bug Fixes * Fixed magic link invalidation by email scanners — links now support multiple uses * Fixed dynamic URL and description layout for OpenGraph images * Fixed og:title, og:url, and Twitter metadata for report shares * Fixed report completion card appearing when user continues chat * Added 404 page for inaccessible executions * Fixed agent prompt/response status leak ## Knowledge Base, Bug Verification & Enhanced Reporting This release introduces powerful knowledge management capabilities, smarter bug verification workflows, and significant improvements to reporting and user experience. ### What's New **Knowledge Base Management** Scout now features a dedicated Knowledge Base where insights from your testing sessions are organized and accessible. Combined with a new Project Summary view and Application Graph (moved to Knowledge tabs), you get a comprehensive understanding of your application's structure and testing history. **Bug Verification by Re-running Related Flows** When Scout finds a bug, you can now verify it by re-running only the specific flow that triggered the issue. This targeted approach saves time and provides faster confirmation of whether bugs are reproducible or have been fixed. **Smarter Issue Detection** Scout now automatically converts observations into trackable issues and links related executions, helping you understand patterns across multiple test runs. **Critical Issue Path Graph** Visualize the paths that lead to critical issues in your application. This new graph helps you understand how users might encounter bugs and prioritize fixes based on impact. **Quality Validated Badge** Share your testing results with a beautiful quality badge. Each execution now generates a shareable badge showing your site's quality rating, issue breakdown, and validation status. Quality Validated Badge **Report Enhancements** * Markdown download support for report share links * Automatic 30-word summary generation for execution reports **Account & Profile Settings** Users can now edit their full name directly in account settings for better personalization. **Agent Interaction Improvements** * Suggestion buttons for agent questions make it easier to respond during testing sessions * Loading states on chat submit/stop buttons provide clearer feedback **CLI Updates** The Scout CLI now supports unauthenticated access for `create-execution` and `send-message` commands, making it easier to integrate into CI/CD pipelines. **Tet Holiday Theme** Celebrate Lunar New Year with a special Tet holiday theme for the bunny hop game. ### Improvements * Improved Vibe CTA scrolling and URL input focus * Better table styling in chat markdown * Updated color scheme for better visibility * Streamlined welcoming email * Relaxed report schema requirements for more flexible report generation ### Bug Fixes * Fixed broken image display in lightbox when load fails * Fixed report display when classification is mixed * Improved project access validation * Fixed duplicate tool call handling in agent responses ## Scout CLI & Vibe Testing Tools This release introduces the Scout CLI for terminal-based testing, along with integrations for Claude Code and Kiro IDE. Plus, improvements to schedule management, issue tracking, and overall platform reliability. ### What's New **Scout CLI** Run AI-powered exploratory tests directly from your terminal. The new Scout CLI brings vibe testing to your command line—test web applications, run accessibility audits, and validate user flows using natural language commands. Perfect for CI/CD integration and developers who prefer the terminal. ```bash theme={null} npm install -g @scoutqa/cli scoutqa auth login scoutqa --url https://your-app.com --prompt "Test the login flow" ``` **Claude Code Skill** Integrate Scout directly into Claude Code with the new vibe testing skill. Ask Claude to test your web applications using natural language—no test scripts required. [View on GitHub](https://github.com/scoutqa-dot-ai/claude-code) **Kiro Power** Bring vibe testing to Kiro IDE with the Scout power. Test features as you build them without leaving your editor. [View on GitHub](https://github.com/scoutqa-dot-ai/kiro) **Improved Schedule Management** Enhanced UX for managing scheduled test runs with better controls and clearer status indicators. **Issues Page State in URL** Issue filters and pagination state now persist in the URL, making it easy to share specific views with teammates or bookmark filtered lists. **Open Issues Count** The executions table now displays open issue counts per execution, giving you quick visibility into which runs need attention. ### Improvements * Weekly summary email reminders to keep you informed of testing activity * Welcoming email for new users with helpful onboarding information * PostHog integration for A/B testing and user behavior insights * Agent connection status events with idle-aware keepalive * Performance improvements to agent tool result handling * BRIN index optimization for faster stream cleanup * Improved error classification and benchmark metrics ### Bug Fixes * Fixed preview image display in execution details * Resolved Playwright MCP download issues * Fixed chat sync context handling * Improved handling of server URLs with trailing slashes ## Automated Testing & Smarter Planning This release introduces powerful automation capabilities that let Scout work for you around the clock, plus significant improvements to test planning and overall reliability. ### What's New **Project Scheduling** Set up automated test runs that execute on your schedule. Configure recurring tests for your projects—daily smoke tests, weekly regression checks, or any custom schedule that fits your workflow. Scout will automatically run your configured tests and track results over time, helping you catch regressions before they reach production. Manage all your schedules from the new dedicated Schedules page in your project dashboard, where you can create, edit, pause, or delete scheduled runs with ease. ## Reliability, Polish & Performance This release focuses on making Scout more reliable on complex websites, streamlining issue management, and improving the overall experience with thoughtful UX enhancements. ### What's New **Improved Agent Reliability on Complex Websites** Scout now works reliably on complex e-commerce and content-heavy sites. We've optimized how the agent captures page snapshots—blocking 200+ ad/tracker domains and intelligently reducing page data without losing important context. Sites like Amazon that previously caused issues now work smoothly. **Streamlined Issue Management** Managing issues just got a whole lot easier: * **Dismiss with a reason**: When dismissing an issue, you can now explain why (up to 500 characters). Great for team communication and future reference. * **Unified status dropdown**: Manage issue status from a single control—Open, Dismiss, or Resolve—with undo support for safety. * **Quick issue review**: Click any issue row to open a detail panel without leaving the list. Review multiple issues faster. * **Cleaner issues table**: Reduced visual clutter with a more focused layout. **Scout Stuck? Now You'll Know** When Scout needs your input, it will now show a helpful dialog explaining what's happening and guiding you to the Chat tab. No more wondering why things paused. **Screenshot Lightbox** Click any screenshot to open it in a full-featured viewer with zoom (0.5x–3x), pan, and download capabilities. Perfect for inspecting details or sharing findings. ### Improvements * **Faster Return to Work**: Scout remembers the last project you were working on. When you log back in, you're taken straight there instead of starting from the project list. ### Security Updates * Next.js upgraded to 15.5.9 (security patches) ## A Fresh New Way to Watch Scout Work We've been listening to your feedback, and this release delivers a major upgrade to how you experience Scout. Say hello to a cleaner, more intuitive execution interface—plus some powerful new ways to manage your projects. ### What's New **Completely Redesigned Execution UI** This is the big one! We've split the execution view into two dedicated panels: one for tracking progress, one for the chat. No more juggling between what Scout is doing and what it's saying. Everything is exactly where you'd expect it to be, and monitoring long test runs has never felt smoother. New execution interface **Jump to Live** Scrolled back to review something? Hit the "Jump to Live" button to snap right back to the action. It's the little things that make a difference. **Your Issues, All in One Place** Remember those bugs Scout found last week? Now you can view and manage all discovered issues right from your project dashboard. Update severity, track status, and keep your whole team on the same page without leaving Scout. **Test File Uploads Like a Pro** Got a feature that handles image uploads? Profile pictures? Document attachments? Now you can upload files directly to your project, and Scout will actually use them during testing. Finally, a way to test those tricky file input flows end-to-end. Upload files to your project Manage your project files **Scout Gets Smarter Over Time** Here's something cool: Scout now builds a knowledge base from your testing sessions. The more you use it, the more it learns about your specific app. Think of it as Scout developing muscle memory for your product. ### Improvements * **Cleaner Screenshots**: Debug overlays now hide automatically during capture—no more visual clutter * **Better Cursor Tracking**: Improved positioning and labels make it easier to follow Scout's actions * **Faster Quick Bug Finder**: We've optimized page analysis for noticeably speedier results * **Rock-Solid Sessions**: Browser timeouts now work exactly as expected * **Graceful Recovery**: Tests keep running smoothly even when network hiccups happen * **Always Fresh Data**: Pages auto-refresh when you come back to them * **Security Updates**: Dependencies upgraded to keep everything locked down ### Bug Fixes * Screenshots now display correctly in historical chat views * Chat input auto-resize works properly (no more hydration weirdness) * Issue updates sync instantly across the UI * Project credentials inject reliably every time * Caching issues resolved—you'll always see the latest run info ## Session Replay & Enhanced History This release introduces powerful historical replay capabilities, improved chat history, and significant visual refinements. ### What's New **Historical Replay** We've added screenshot persistence and historical replay, allowing you to review past execution states and visually retrace Scout's steps. **Enhanced Chat History** Chat history now preserves tool calls and context, making it easier to understand previous interactions and pick up where you left off. **Scout Overlays** New visual overlays provide better visibility into the agent's actions and focus areas during execution. ## Enhancements & Bug Fixes This release focuses on enhancements to the chat experience, security improvements, and several key bug fixes. ### What's New **Resume Chat Stream** Users can now resume their previous chat sessions, allowing for a seamless continuation of testing workflows. **Security Enhancements** We've implemented additional security measures to ensure your data and testing environments remain secure. **Assistant Notifications** Scout now informs you when the maximum step limit is reached and requests your input on how to proceed, giving you more control over long-running executions. **New Todo/Issue Features** We've added new variants for the todo dot and improved how you track and manage findings for better issue visibility. ## Enhanced Reporting & Issue Management An important interim release preparing the foundation for next week's major UI overhaul. This update brings a completely reworked reporting system and improved issue management to help you track findings more effectively. ### What's New **Redesigned Report System** We've rebuilt our reporting infrastructure from the ground up in preparation for the new UI launching next week. The re-worked report system provides a cleaner structure and better organization, making it easier to understand Scout's findings at a glance. **Improved Issue List Management** Issues now append to the bottom of the issues panel, creating a more intuitive chronological flow. This improvement ensures you can track new findings as they appear without losing context from earlier discoveries. **Execution Timeline Component** Visualize your test execution progress with the new execution timeline component, providing better insights into how Scout navigates and explores your application over time. ### Improvements * Enhanced report structure for better readability and organization * More consistent status display between execution tables * Improved issue panel ordering for better tracking * Timeline visualization for execution progress * Foundation work for next week's major UI update ### Coming Next Week Stay tuned for a major UI refresh that will transform how you interact with Scout. This release sets the stage for those exciting changes! ## Quick Bug Fighter & Enhanced Reliability A focused release bringing a powerful new workflow for rapid bug discovery, enhanced authentication reliability, and numerous quality-of-life improvements across the interface. ### What's New **Quick Bug Fighter Workflow** Discover bugs faster with our new Quick Bug Fighter workflow. This streamlined testing mode is designed for rapid issue detection, complete with review and rerun capabilities to iterate quickly on your findings. **Credentials Management** Securely store and manage authentication credentials for your projects, making it easier to test authenticated areas of your applications without manual login steps. **Custom 404 Page** No more generic error pages—Scout now displays a helpful custom 404 page when you wander off the beaten path, making navigation errors less jarring. Custom 404 page ### Improvements * Google login now shows a proper loading state during redirects, eliminating confusion * Fixed redirect behavior after Google login and logout for a seamless experience * Resolved confusing redirects from the homepage for logged-in users * Extended magic link validity to 30 minutes, giving you more time to complete sign-in * Fixed rerun functionality for private executions—now you can reliably rerun any test * Improved Quick Bug Fighter report completion to properly enable review and rerun actions * Long URLs no longer break the layout—text wraps gracefully * Avatar images now load smoothly with skeleton placeholders for a more polished feel * Enhanced security with the latest authentication and dependency updates to keep your data safe ## Multi-Project Workspace & Agent Improvements A major release focused on organization and flexibility. Scout now supports multiple project workspaces, enhanced onboarding, and more powerful agent configuration options. ### What's New **Multi-Project Workspace** Organize your testing with dedicated project workspaces that keep everything connected. Create projects, switch between them seamlessly, and let Scout automatically populate project details from your URLs. Each project gets its own dashboard with test insights and quick access to create new executions. **Enhanced Test Execution Management** Search executions by URL, filter by status (completed, running, queued, failed, canceled), and rerun any execution with a single click. Share public execution results via `/t/{executionId}` links and enjoy a revamped execution details page with improved layout and information display. **Guided Onboarding Experience** New users now get a smooth two-step onboarding flow that walks them through creating their first project. Simply enter a URL, and Scout fetches metadata to auto-populate project names and logos. Enhanced sign-in with magic link email authentication and Google OAuth support. ### Improvements **Project Management** * Project switcher in navigation for quick context switching * Project dashboard with test insights and execution history * Project settings page for editing and deletion * Automatic website metadata fetching for project setup * Website metadata scraper endpoint (`/api/v1/meta`) for fetching titles, descriptions, and favicons **Authentication & Access** * Magic link email authentication * Google OAuth integration * API key authentication support for programmatic access **Agent & Automation** * Real-time streaming API for agent responses * Chrome DevTools MCP integration updated to v0.9.0 * Refactored agent streaming architecture with `startAgentStream` **UI Polish** * Improved color contrast and theming throughout the interface * Fixed button alignments and layout issues * Better URL normalization and validation * Optimized frontend build process ### Under the Hood * Introduced `@katalon-studio/scout-common` shared package for type definitions and schemas * Benchmark result caching during synchronization * Enhanced agent streaming architecture for better performance ## Smarter Testing & Smoother Experience We've been busy making Scout more intelligent and delightful to use. This release brings enhanced AI capabilities, mobile support, and a bunch of quality-of-life improvements. ### What's New **Mobile-Ready Execution Page** Monitor your testing sessions from anywhere! The execution page now adapts beautifully to mobile devices, so you can keep tabs on Scout's progress whether you're at your desk or on the go. **Shared Context Intelligence** Scout just got smarter. We've evolved our memory system to use shared context bundles, enabling Scout to generate more coherent and contextual test reports by understanding relationships across your entire testing session. **Dynamic Browser Management** Scout now intelligently manages browser resources, launching and cleaning up browser sessions on-demand for better performance and lower resource usage. ### Improvements **Cleaner, More Readable Reports** We've refined the structured report design with improved font contrast and removed redundant execution metadata, making it easier to scan and understand your test results at a glance. **Enhanced Email Experience** Updated our magic link email template with engaging "vibe testing" branding and clearer security messaging—because even authentication emails should spark joy. **Full AI Transparency** Ever wondered what parameters Scout is using? The webapp now displays tool call parameters, giving you complete visibility into how our AI agent makes decisions during exploration. **Polished Theming** Refreshed our UI theming standards for a more consistent and cohesive visual experience throughout the application. ### Under the Hood * Optimized report generation with a dedicated AI model for higher quality structured reports * Fixed structured report streaming for smoother real-time updates * Resolved flashing LiveView issues during sub-agent executions * Added TTL support for CDP connections and live view URLs to automatically clean up stale sessions * Updated to latest better-auth package with security improvements * Enhanced AI agent prompts for improved testing accuracy * Improved JSON Schema compatibility across the system ## Interactive Live View & Enhanced Reporting A major update focused on improving user interaction and transparency. ### Homepage Updates * **Scout by the Numbers**: Live statistics dashboard showing URLs tested, total executions, and bugs found across all Scout users * **Discord Channel**: Join our community to share feedback, ask questions, and connect with other Scout users * **Cal.com Integration**: Easy scheduling to book time with our team for support or demos ### Product Improvements * **Interactive Live View**: Users can now pause the chatbot and interact directly with the live view to help Scout bypass login screens, solve captchas, and navigate complex authentication flows * **Better Structured Reports**: Enhanced report format with clearer organization, categorized findings, and improved readability This release makes Scout more collaborative and transparent, giving you greater control over the exploration process. ## Initial Release We're excited to launch the first version of Scout, your AI quality companion for high-velocity development. ### Core Features * **Instant Run with URL Input**: Simply paste your application URL and Scout starts exploring immediately * **Simple Chat Window**: Interact with Scout through an intuitive chat interface to guide explorations * **Live View**: Watch Scout navigate your application in real-time with a view-only live feed * **Text-Based Reports**: Receive detailed, text-based exploration reports with findings and recommendations ### Supported Platforms * Quick Run support for Lovable and Replit applications * Simple URL transformation pattern for instant exploration This initial release focuses on making quality testing accessible and effortless for developers shipping at high velocity. *** New to Scout? Check out our [Quickstart guide](/quickstart) to launch your first exploration in under ten minutes. Have feedback or suggestions? [Contact our team](mailto:huy.tieu@scoutqa.ai) - we'd love to hear from you! # Credentials Source: https://docs.scoutqa.ai/essentials/credentials Pre-configure test credentials so Scout can authenticate during explorations. Credentials let you store reusable authentication details that Scout uses when it encounters login forms or protected areas during testing. Instead of getting blocked at sign-in screens, Scout can automatically apply the right credentials and continue exploring your application. **Security Notice**: Credentials are stored and transmitted as plain text so that Scout can use them during test execution. Only use test credentials created specifically for QA purposes. Never add production passwords, personal accounts, or sensitive API keys. ## Why use credentials When Scout explores your application, it may encounter: * Login forms that block access to authenticated pages * API endpoints requiring authentication headers * Protected admin areas or user dashboards By pre-configuring credentials, Scout can seamlessly authenticate and provide comprehensive coverage of your entire application—not just the public-facing pages. ## Supported credential types ### Basic authentication Use basic auth credentials for standard username/password login forms. Scout will automatically detect login pages and fill in the credentials you provide. A friendly name to identify this credential set (e.g., "Test Admin Account", "QA User"). The username or email for the test account. The password for the test account. Optional context about when to use this credential or any relevant details. ## Creating credentials Add new credential modal 1. Navigate to your project settings 2. Select the **Credentials** tab 3. Click **New Credential** 4. Choose the credential type (Basic Auth) 5. Fill in the required fields 6. Click **Create** ## Best practices Set up accounts specifically for Scout testing rather than using personal or shared credentials. This keeps your test data isolated and makes it easy to reset if needed. Name credentials clearly so you know which account does what. For example: "Admin - Full Access", "Standard User - Limited", "Guest - Read Only". Use the notes field to document which areas of the app require this credential, any special permissions the account has, or rotation schedules. Even for test accounts, update passwords periodically. Delete credentials you no longer need to keep your project tidy. ## How Scout uses credentials During an exploration, Scout: 1. Detects authentication requirements (login forms, 401 responses, etc.) 2. Matches the context to available credentials in your project 3. Automatically fills and submits login forms 4. Continues exploring authenticated areas of your application 5. Reports any authentication failures as issues Scout will attempt to use credentials intelligently based on the context. If you have multiple credentials, consider using clear labels and notes to help Scout select the right one. ## Managing credentials Credentials list view From the credentials list, you can: * **Edit** existing credentials to update usernames, passwords, or notes * **Delete** credentials you no longer need * View when each credential was created and last updated Changes take effect immediately for any new explorations. Running explorations will continue using the credentials they started with. # Files Source: https://docs.scoutqa.ai/essentials/files Upload test images and assets for Scout to use during explorations. Files let you pre-upload images that Scout can use when testing file upload features in your application. Instead of skipping upload forms or using placeholder data, Scout can test your file handling with real images you provide. ## Why use files Many applications include file upload functionality: * Profile picture uploads * Document attachment forms * Image galleries and media libraries * Receipt or invoice scanning features By uploading test files to your project, Scout can exercise these upload flows and verify they work correctly—catching issues like format validation errors, size limit problems, or broken upload handlers. ## Supported file types Scout currently supports image uploads: | Format | MIME Type | Use Case | | ------ | ------------ | --------------------------------------- | | PNG | `image/png` | Screenshots, graphics with transparency | | JPEG | `image/jpeg` | Photos, general images | | WebP | `image/webp` | Modern compressed images | | GIF | `image/gif` | Animated images, simple graphics | Maximum file size is **5 MB** per image. Files larger than this will be rejected during upload. ## Uploading files Add new file modal 1. Navigate to your project settings 2. Select the **Files** tab 3. Click **Upload File** 4. Choose an image from your computer 5. Add a descriptive label (e.g., "Sample Receipt", "Test Profile Photo") 6. Optionally add notes about the file 7. Click **Upload** ## Best practices Upload images that resemble what real users would submit. A sample receipt image tests your receipt scanner better than a random photo. Include files that test boundaries: images near the size limit, different aspect ratios, various formats. This helps Scout find validation issues. Use descriptive labels like "Large JPEG - 4.5MB" or "Portrait Photo - PNG" so you can easily identify files later. Document what each file is meant to test. For example: "Use for profile photo uploads" or "Tests receipt OCR feature". ## How Scout uses files During an exploration, when Scout encounters a file upload form: 1. Identifies the upload input and accepted file types 2. Selects an appropriate file from your project's uploaded files 3. Submits the upload and verifies the response 4. Reports any errors or unexpected behavior as issues Scout intelligently matches files to upload contexts based on accepted MIME types and file requirements. Having a variety of file types and sizes gives Scout more options to test thoroughly. ## Managing files Files list view From the files list, you can: * View all uploaded files with their labels, types, and sizes * See when each file was uploaded * **Delete** files you no longer need Deleted files are removed from future explorations but won't affect historical execution records. ## File storage Uploaded files are stored securely and associated with your project. They persist across explorations and can be reused as many times as needed. Files are project-specific. Each project maintains its own file library, so you can tailor test assets to each application you're testing. # Issues Source: https://docs.scoutqa.ai/essentials/issues Track and triage quality issues discovered during Scout explorations. Issues are quality problems that Scout discovers while exploring your application. From accessibility gaps to security concerns, Scout automatically detects, categorizes, and reports issues so your team can prioritize fixes. ## Issue categories Scout classifies issues into five categories: | Category | Description | Example | | ----------------- | ----------------------------------------------------------------- | -------------------------------------------------------------------- | | **Accessibility** | Barriers that prevent users with disabilities from using your app | Missing form labels, low color contrast, keyboard navigation issues | | **Security** | Potential vulnerabilities or unsafe practices | Exposed credentials, insecure form handling, authentication bypasses | | **Performance** | Slow or resource-intensive behaviors | Long page load times, unoptimized images, memory leaks | | **Usability** | Confusing or broken user experiences | Dead links, unclear error messages, inconsistent form validation | | **Other** | Issues that don't fit other categories | Unexpected behavior, edge case failures | ## Severity levels Each issue is assigned a severity to help you prioritize: Severe problems that block core functionality or pose significant security risks. Address immediately. Important issues affecting user experience or application stability. Plan to fix soon. Notable problems that should be addressed but aren't urgent. Add to your backlog. Minor issues or suggestions for improvement. Fix when convenient. ## Issue statuses Track resolution progress with three statuses: * **Open** — Issue needs attention. This is the default status when Scout discovers a problem. * **Resolved** — Issue has been fixed. Mark issues resolved after deploying a fix. * **Dismissed** — Issue is intentionally ignored. Use for false positives or accepted risks. ## Viewing issues ### Project issues list Issues list view Access all issues across executions from the **Issues** page in your project. From here you can: * **Search** by title to find specific issues * **Filter** by status (Open, Resolved, Dismissed) * **Filter** by severity (Critical, High, Medium, Low) * **Filter** by category (Accessibility, Security, etc.) * **Sort** by when issues were discovered ### Issue details Click any issue to see the full details: A brief description of what Scout found and why it's a problem. Bullet points explaining how this issue affects users or your application. The sequence of actions Scout took to discover the issue. This helps you reproduce and verify the problem. Guidance on resolving the issue, tailored to your specific context. ## Managing issues ### Update severity If Scout's initial severity assessment doesn't match your priorities, you can adjust it. Click the severity dropdown on any issue to change it. ### Resolve issues After fixing an issue: 1. Open the issue details 2. Click **Resolve** 3. The issue moves to Resolved status Run a new exploration after deploying fixes to verify issues are truly resolved. Scout will create new issues if problems persist. ### Dismiss issues For false positives or accepted risks: 1. Open the issue details 2. Click **Dismiss** 3. Optionally add a reason for dismissing Dismissed issues won't clutter your open issues list but remain in your history for reference. ### Fix with coding agents **New in v0.11.0:** Go from bug discovery to bug fix in one click. When Scout finds a bug, you can send it directly to your preferred coding agent for fixing: 1. Open the issue details 2. Click **Fix with Agent** 3. Choose your agent — **Cursor**, **Codex**, or **Windsurf** 4. The agent launches with the bug context (summary, reproduction steps, suggested fix) pre-loaded This creates a seamless loop: Scout finds bugs → coding agent fixes them → Scout verifies the fix. ### Verify bugs by re-running related flows You can verify whether a bug is reproducible by re-running only the specific flow that triggered it. **From the web UI:** 1. Open the issue details 2. Click **Verify Bug** 3. Scout re-runs only the specific flow that triggered the issue **From the CLI:** ```bash theme={null} # List issues from a previous execution scoutqa list-issues --execution-id # Verify a specific issue scoutqa issue-verify --issue-id ``` Use CLI verification in your CI/CD pipeline to automatically confirm bug fixes before merging. This targeted approach is faster than running a full exploration and helps you quickly confirm: * Whether the bug is consistently reproducible * Whether your fix resolved the issue * Whether the bug was a one-time occurrence ## Best practices Review new issues after each exploration. Quick triage keeps your issue list actionable and prevents important problems from getting lost. Focus on the most severe issues first. These typically have the biggest impact on users and may indicate broader problems. Filter by category when working on specific areas. For example, filter to Security issues before a release or Accessibility issues during an audit. When dismissing issues, add a reason. This helps teammates understand why an issue was ignored and prevents re-discovery confusion. Notice patterns in your issues. Repeated accessibility problems might indicate a need for developer training. Recurring security issues might suggest architecture improvements. ## Issue lifecycle ```mermaid theme={null} graph LR A[Scout discovers issue] --> B[Open] B --> C{Action} C -->|Fix deployed| D[Resolved] C -->|False positive| E[Dismissed] C -->|Needs work| B ``` Issues start as **Open** when discovered. Your team reviews and either **Resolves** them after fixing or **Dismisses** them if they're not actionable. Resolved and dismissed issues stay in your history for tracking and auditing. # Knowledge Base Source: https://docs.scoutqa.ai/essentials/knowledge-base Centralized insights from your testing sessions, organized and accessible. The Knowledge Base is where Scout stores and organizes insights gathered from your testing sessions. As you run explorations, Scout builds a comprehensive understanding of your application that helps improve future test runs. **New in v0.10.1:** The Knowledge Base now includes Application Graph, Project Summary, and automatic issue pattern detection. ## Knowledge tabs Access the Knowledge Base from the **Knowledge** tab in your project. It contains three main sections: ### Application Graph The Application Graph visualizes your application's structure based on Scout's explorations. It shows: * Pages and routes discovered during testing * Navigation paths between screens * User flows and common journeys This helps you understand how Scout perceives your application and identify areas that may need more exploration. ### Project Summary Get a high-level overview of your project's testing history: * Total executions and issues found * Issue trends over time * Most frequently discovered issue types * Coverage of your application ### Critical Issue Path Graph Visualize the paths that lead to critical issues in your application. The Critical Issue Path Graph shows you exactly how users might encounter bugs: * Traces the steps leading to critical and high-severity issues * Helps prioritize fixes based on how easily users can trigger them * Identifies bottlenecks where multiple issues converge ## How the Knowledge Base helps ### Smarter future explorations Scout uses Knowledge Base data to: * Avoid re-testing stable areas unnecessarily * Focus on sections that historically have more issues * Recognize when new issues are related to previously found problems ### Automatic issue linking When Scout finds a bug that's similar to a previous issue, it automatically links them together. This helps you: * Understand patterns across multiple test runs * Identify regressions when fixed bugs reappear * Track issue frequency over time ### Observation-to-issue pipeline Scout automatically converts observations into trackable issues. During exploration, Scout notes interesting behaviors. After analysis, significant observations become formal issues with: * Severity assessment * Reproduction steps * Suggested fixes ## Best practices The more you test, the smarter Scout becomes. Regular explorations build a richer Knowledge Base that improves issue detection and reduces false positives. Check the Application Graph periodically to ensure Scout has discovered all important areas of your application. If something is missing, run a targeted exploration. When planning fixes, check the Critical Issue Path Graph. Issues on frequently-traveled paths should be prioritized over those in rarely-used features. Pay attention to issues that Scout links together. Multiple linked issues often indicate a systemic problem that needs architectural attention rather than individual fixes. # Projects Source: https://docs.scoutqa.ai/essentials/projects Keep every product, execution, and report organized with multi-project workspaces. Projects group explorations, personas, and reports by product so fast-moving teams can keep quality work focused. Each project lives inside a workspace and comes with its own dashboard, execution history, and settings. ## Navigate with the project switcher * Open the project switcher from the global navigation to jump between products without leaving the page you are on. * Create a new project whenever you need coverage for another environment, brand, or major feature area. * Scout remembers your last selection so relaunching an exploration always starts in the right context. Project list and recent executions ## Project dashboards Project dashboards surface the current health of a product and provide one-click access to new runs. Project dashboard overview ### Key capabilities * Launch fresh explorations directly from the dashboard. * Review a summarized activity feed before diving into a specific execution. * Track completion status, duration, and personas for recent runs. ### Execution controls * Search executions by URL or run title to find prior coverage quickly. * Filter by status—completed, running, queued, failed, or canceled—to focus on the work that matters now. * Rerun prior executions with a single click or share read-only results via `/t/{executionId}` links. ## Project settings Use the settings tab inside a project to keep metadata accurate and clean up stale work. Update the name shown in navigation and reports. Great for differentiating staging versus production projects. Set or adjust the primary domain Scout should explore. Changing the URL triggers automatic metadata refresh. Upload a logo if you want to override the favicon and title that Scout fetched automatically. Removing a project deletes its executions and reports. Export anything you need before proceeding. ## Automated metadata Scout fetches titles, descriptions, and favicons for you during onboarding and whenever you update a project's base URL. If you need the same enrichment in other systems, call the `/api/v1/meta` endpoint to retrieve the metadata programmatically. New project modal capturing URL input ## Best practices * Create separate projects for each major surface area (marketing site, app, admin) so dashboards stay focused. * Use the status filters to keep an eye on runs that are still in progress before big launches. * Duplicate high-signal executions into new missions when you discover a regression worth tracking. * Schedule regular reviews of project settings to retire old environments and keep navigation tidy. # Executions Source: https://docs.scoutqa.ai/essentials/projects-executions Review every project run, pick up unfinished work, and launch new explorations without leaving the dashboard. Executions keep the pulse of a project. Each run—whether it completed, stalled, or was canceled—lands here with the context you need to react quickly. ## Execution list Execution list showing recent runs * Sort through the latest activity across the project, with status chips that highlight completed, running, queued, failed, or canceled runs. * Search by URL or execution title to surface prior coverage when you are validating a regression fix. * Open any execution to inspect the live transcript, download the structured report, or copy the public `/t/{executionId}` link. Use filters before big launches to make sure there are no runs stuck in the queue or waiting for credentials. ## Start a new execution Modal for launching a new execution * Click **New execution** to launch a guided mission from anywhere in the project. * Drop in the URL you want Scout to explore, tweak the objective, and confirm the persona or environment. * Kick off the run immediately, or duplicate a previous execution as a starting point when you need consistent coverage. ### Testing modes Scout offers three testing modes. Choose the one that matches your intent: Describe what you want to test in natural language. Scout interprets your intent and picks the right workflow, depth, and tools automatically. Fast scan with minimal depth. Best for quick sanity checks and smoke tests when you need results in minutes. Thorough, comprehensive testing. Explores edge cases, tests multiple paths, and produces detailed findings. #### Smart Mode — intent-based testing Smart Mode is the default and recommended way to run tests. Instead of choosing a depth level, just describe what you need: * **"Test the checkout flow as a first-time mobile user"** → Scout runs a focused deep test with a persona * **"Smoke test login, signup, and password reset"** → Scout runs a quick smoke test across three flows * **"Verify the cart total bug is fixed"** → Scout runs a targeted verification workflow * **"Check WCAG compliance on the dashboard"** → Scout runs an accessibility audit * **"Explore the settings page for UX issues"** → Scout runs exploratory testing Behind the scenes, Smart Mode classifies your input into one of six workflow templates — quick-scan, deep-test, smoke-test, exploratory, a11y-check, or verification — and selects the appropriate tools and depth level. Your selected mode persists across the project. Switch once and all new executions use that mode until you change it. ## Execution in progress Running execution with live transcript Once launched, Scout explores your application and reports findings in real-time. The execution view shows: * A visual progress indicator tracking Scout's journey through your app * Live transcript of actions, observations, and discovered issues * Issue details panel with summary, impact, and fix suggestions New executions inherit project defaults—environment, guardrails, integrations—so teams stay aligned even when multiple people launch runs. # Test Cases Source: https://docs.scoutqa.ai/essentials/test-cases Write reusable manual test cases—step-by-step or high-level intent—that Scout can run on demand and replay deterministically. A **test case** is a reusable unit of testing work. Instead of describing what you want every time you launch a run, you save the scenario once and run it whenever you need it—after a deploy, before a release, or as part of a [test suite](/essentials/test-suites). Find your test cases under **Library → Test Cases** in the project sidebar. ## Two ways to write a test case When you click **New test case**, choose the style that fits the scenario: Spell out each action as an explicit step. Best when you know exactly what should happen and want repeatable, predictable coverage. Describe the goal in plain language and let Scout figure out the steps. Best for flows that change often or when you care about the outcome, not the clicks. ## Writing the instruction Every test case starts with a **Title** (e.g. *Login flow*). From there the fields depend on the mode you picked. ### Step-by-step cases | Field | Required | What it's for | | -------------------- | -------- | ------------------------------------------------------------------------------------------------------------------------------- | | **Steps** | Yes | The ordered actions Scout performs, one per line — e.g. *"Open the login page"*, *"Enter valid credentials"*, *"Click Submit"*. | | **Preconditions** | Optional | State that must be true before the steps run — e.g. *"User must be logged in"*. | | **Expected results** | Optional | What you expect to be true afterward — e.g. *"Dashboard is visible"*. | | **Test data** | Optional | A JSON object Scout can reference during the run — e.g. `{"email": "demo@example.com"}`. | A step-by-step case needs **at least one step** to be valid. ### Intent cases For an intent case, you write a single **Intent** describing the goal: > *"Verify a user can reset their password from the login page."* Scout interprets the intent, plans the steps, and adapts as it explores—so the case keeps working even when your UI shifts. Preconditions and test data are still available if you want to pin down context. Reach for **intent** when you're testing an outcome (*"a user can check out with a saved card"*) and **step-by-step** when the exact sequence matters (*"the wizard advances only after every required field is filled"*). ## Running a test case Test cases run in the [**Test Runner**](/essentials/test-runner). Open it, set your **Target URL**, load the case from your library, and click **Run**. Scout spins up a browser, executes the case, and reports pass or fail with the supporting evidence. You can also re-run a case directly from the Test Cases list once it has a recorded script (see below). ## Deterministic replay with Playwright The first time a step-by-step case passes in the Test Runner, Scout automatically records a **deterministic Playwright script** of exactly what it did. After that, the case shows a **Re-run with Playwright** action in the list. * **Re-run with Playwright** replays the saved script verbatim—no AI, fully repeatable, fast. Ideal for regression checks. * **View script** opens the recorded script, and **Download** saves it so you can run it in your own pipeline. A script is only recorded on a **passing** run. If a case has never passed, the replay and view-script actions stay disabled until it does. ## Managing test cases From the Test Cases list you can **Edit**, **Delete**, and see each case's **Last run** time at a glance. Editing a case applies to all future runs; deleting one removes it permanently but leaves any [suites](/essentials/test-suites) it belonged to intact. # Test Runner Source: https://docs.scoutqa.ai/essentials/test-runner Run one test case or a whole suite against your app—cases execute in parallel and results stream into a live grid. The **Test Runner** is where your [test cases](/essentials/test-cases) and [test suites](/essentials/test-suites) actually run. Each case executes independently in its own browser session, so a batch runs **in parallel** and results stream back into a single grid as they finish. Open it from **Test Runner** in the project sidebar. ## Run your test cases The runner walks you through three phases: pick the cases, watch them run, review the results. ### 1. Select what to run First, enter the **Target URL** you want to test (e.g. `https://app.example.com`). Then load the cases to run from one of three sources: Browse your saved test cases and tick the ones to run. Use **Select all** / **Deselect all** to move fast. Pick a suite and click **Load suite into selection** to load and pre-select every case in it. Paste an array of test cases as JSON—handy for ad-hoc runs or CI. Sample buttons (*1 passing*, *5 cases*, …) give you a starting shape. Selected cases appear in the **Staged cases** list. When you're happy with the selection, click **Run case(s)**. ### 2. Watch the run Each case becomes a card in the execution grid with a live status badge: | Status | Meaning | | --------------------------------- | -------------------------------------------------------------------------------------------------------------------------------- | | **Queued** | Waiting for an execution slot. | | **Starting…** | The browser session is being created. | | **Running…** | Scout is executing the case—you'll see step-by-step progress (✓ done, ● current, ○ pending, ✗ failed) and a live activity label. | | **Passed** | Every step and assertion succeeded. | | **Failed** | At least one step or expectation failed. | | **Blocked** / **Could not start** | The case couldn't run—often a bad URL or missing precondition. | A summary bar at the top tallies the run: how many **passed**, **failed**, **blocked**, **running**, and **queued**, plus a **Done** badge when everything finishes. ### 3. Review results From each card you can: * **View execution** — open the full transcript, screenshots, and any bug reports Scout filed for a failure. * **Download script** — grab the deterministic Playwright script recorded for a passing case. When the run is complete, click **Start new run** to go again. ## Parallel runs Scout runs your selected cases **concurrently**—each in its own isolated browser session—rather than one after another, so a suite finishes much faster than running cases by hand. You don't configure a worker count; Scout throttles the run automatically and respects your plan's concurrency limits. Concurrency is bounded by your plan. **Vibe** allows up to 5 concurrent executions and **Ship** up to 10. If you queue more cases than your limit, the extras wait in **Queued** and start as slots free up. Hitting the ceiling shows a *"too many active executions"* message—let the in-flight cases finish, then continue. Because each case runs in a fresh session, cases don't share state. Put anything a case depends on (login, seed data) in its **preconditions** or **test data** rather than assuming an earlier case set it up. ## Run state is per-session A run's progress is stored in your browser session, so a refresh won't lose it. Opening a new tab or clearing browser storage starts fresh. # Test Suites Source: https://docs.scoutqa.ai/essentials/test-suites Group related test cases into reusable suites you can run as a batch from the Test Runner. A **test suite** is a named, ordered collection of [test cases](/essentials/test-cases). Suites let you organize coverage—*Smoke regression*, *Checkout flows*, *Auth*—and run the whole set in one go from the [Test Runner](/essentials/test-runner). Find your suites under **Library → Test Suites** in the project sidebar. ## Create a suite 1. Click **New suite**. 2. Give it a **Name** (e.g. *Smoke regression*) and an optional **Description** of what it covers. 3. Click **Create suite**. The suite starts empty—you add cases from its detail page. ## Add and order cases Open a suite to manage its cases. The detail page has two columns: * **All test cases** — every case in the project. Tick the ones you want to include. * **Selected cases (in order)** — the cases currently in the suite. Use the up/down arrows to set the run order, or remove a case with the trash icon. Click **Save changes** when you're done. Order matters: cases run in the sequence shown here. A case can belong to more than one suite. If a case is deleted from the project, the suite shows **"Missing test case"** in its place—just remove it and save. ## Run a suite In the [**Test Runner**](/essentials/test-runner), choose **From library → Test suites**, pick a suite, and click **Load suite into selection**. Every case in the suite is loaded and pre-selected. Set your **Target URL** and **Run**—Scout executes the cases independently and shows results in the execution grid. Build a small **smoke suite** of your most critical flows and run it against every deploy. Keep deeper, slower scenarios in a separate suite you run before releases. ## Managing suites From the suites list you can **Edit** a suite's name and description or **Delete** it. Deleting a suite removes only the grouping—the underlying [test cases](/essentials/test-cases) are preserved. # Scout - Find Bugs in Your Lovable, Replit, V0 & Base44 Apps Source: https://docs.scoutqa.ai/index Scout is the quality companion for vibe coders, solo founders, and side project builders. Find bugs and test your Lovable, Replit, V0, or Base44 app instantly - no QA team required. ## Quality for the age of vibe coding AI pair programmers make it effortless to ship features at warp speed—but quality often gets left behind. Scout explores your app like the people who will actually use it: curious newcomers, power users, and even mischievous bad actors. It flags usability friction, functional gaps, and risky flows before they reach production. > Give Scout a URL at [scoutqa.ai](https://scoutqa.ai) and it returns focused feedback instead of noise, so you can keep moving fast without gambling on product quality. ## Why teams send a Scout first Launch an autonomous exploration with almost no setup—perfect for vibe coding sessions and rapid demos. Receive structured insights with reproduction steps, screenshots, and suggested fixes. Turn exploratory learnings into repeatable health checks as your product matures. ## How Scout fits your workflow Point Scout at the environment you'd like explored—share staging credentials or pre-authenticated sessions when needed. Set intent, guardrails, and risk appetite. Scout adapts its journey to match hacking weekends, internal QA, or go-live smoke tests. Findings land in your workspace within minutes. Assign owners, capture context, and stack up follow-up explorations directly from each insight. Need to share Scout with someone new? Point them to the [Quickstart](/quickstart) so they can launch their first exploration in under ten minutes. # Quick Run Source: https://docs.scoutqa.ai/quick-run Get started with Scout in minutes using our simple URL pattern ## Overview Scout makes it incredibly easy to start exploring your web applications. No complex setup, no configuration files—just a simple URL pattern and you're ready to go. ## Scout URL Format Scout uses a simple URL pattern to explore your applications: **Original URL**: `yourapp.lovable.app` **Scout URL**: `yourapp.lovable.scoutqa.app` **Original URL**: `yourapp.replit.app` **Scout URL**: `yourapp.replit.scoutqa.app` Simply replace your app's domain with the Scout-enabled version to enable automated exploration. ## How It Works Find your application's current URL (e.g., `myproject.lovable.app` or `myrepl.replit.app`) Replace the domain with the Scout-enabled version: * Lovable: Add `.scoutqa` before `.app` * Replit: Add `.scoutqa` before `.app` Visit the Scout URL in your browser or use it in your CI/CD pipeline to start automated exploration Scout will explore your application and provide detailed reports with issues, screenshots, and reproduction steps ## Quick Examples ### Lovable App ```bash theme={null} # Original app URL https://todo-app.lovable.app # Scout-enabled URL https://todo-app.lovable.scoutqa.app ``` ### Replit App ```bash theme={null} # Original app URL https://portfolio-site.replit.app # Scout-enabled URL https://portfolio-site.replit.scoutqa.app ``` ## Use Cases Simply visit your Scout URL in a browser to trigger an immediate exploration. Perfect for quick checks during development. Share the Scout URL with your team members so they can trigger explorations without any setup. Bookmark your Scout URL for one-click access to quality checks whenever you need them. Use the Scout URL in your GitHub Actions, GitLab CI, or other automation tools for continuous quality monitoring. ## Supported Platforms Full support for all Lovable-hosted applications with automatic exploration capabilities. Complete integration with Replit-deployed applications for seamless quality checks. ## What Scout Explores When you access your Scout URL, Scout will: * **Navigate your application** like a real user would * **Test user flows** including authentication, forms, and navigation * **Identify usability issues** that affect user experience * **Find functional bugs** that could impact your app's reliability * **Check for security concerns** based on common vulnerabilities * **Generate detailed reports** with screenshots and reproduction steps ## Personas Scout can explore your app from different user perspectives: Explores as a typical user, focusing on common workflows and usability. Looks for security vulnerabilities, authentication issues, and data exposure risks. Checks for accessibility issues affecting users with disabilities. Tests advanced features, edge cases, and complex workflows. ## Next Steps Set up automated Scout runs in your CI/CD pipeline See what Scout findings look like Automated PR reviews with Scout (launching soon) ## FAQ No! The Quick Run URL pattern works immediately without any API keys or configuration. Just transform your URL and go. Most explorations complete within 5-10 minutes, depending on your application's complexity and the persona selected. For advanced customization (personas, timeouts, specific flows), use our [GitHub Actions integration](/quick-run/github-actions) or wait for the [Scout CLI](/tooling/cli). Scout only interacts with publicly accessible pages by default. We don't store sensitive data and all explorations are ephemeral. For authenticated apps, you'll need to use our [GitHub Actions integration](/quick-run/github-actions) with auth tokens, or wait for the [Scout CLI](/tooling/cli). ## Get Help Have questions or need assistance? * Check out our [Quickstart guide](/quickstart) * Review the [GitHub Actions setup](/quick-run/github-actions) * [Contact support](mailto:huy.tieu@scoutqa.ai) Ready to explore? Transform your URL and give Scout a try! # Quick Start - Test Your Lovable, Replit, V0, or Base44 App Source: https://docs.scoutqa.ai/quickstart Find bugs in your Lovable, Replit, V0, or Base44 app in under 5 minutes. No setup required. Perfect for solo founders, vibe coders, and side projects. ## Two ways to start Scout offers two paths to get quality feedback fast—pick the one that fits your workflow. Transform your app URL and get results immediately. Zero setup. Create a project for ongoing coverage, history, and team collaboration. *** ## Option 1: Instant URL (30 seconds) The fastest way to see Scout in action. Perfect for quick checks during development. Locate your deployed application URL (works with Lovable, Replit, and more). Insert `.scoutqa` before `.app` in your domain: | Platform | Original URL | Scout URL | | -------- | ------------------- | --------------------------- | | Lovable | `myapp.lovable.app` | `myapp.lovable.scoutqa.app` | | Replit | `myapp.replit.app` | `myapp.replit.scoutqa.app` | Open the Scout URL in your browser. Scout automatically begins exploring and reports findings within minutes. Bookmark your Scout URL for one-click quality checks anytime you push changes. *** ## Option 2: Full Dashboard (5 minutes) Create a project to track explorations over time, collaborate with your team, and configure custom personas. ### Step 1: Sign in Scout sign-in screen One-click authentication with your Google account. No passwords to manage. Enter your email to receive a magic link. Click it to sign in instantly—no password needed. ### Step 2: Create your first project Create a new project in Scout Enter the primary URL you want Scout to explore (e.g., `https://myapp.com` or `https://staging.myapp.com`). Scout automatically fetches your app's name, description, and favicon. Adjust if needed. Click **Create** and you'll land on your project dashboard, ready to launch. ### Step 3: Configure the mission Before launching, tell Scout how to explore your app. Select who Scout should emulate during exploration: | Persona | What Scout does | | ------------------------- | ------------------------------------------------- | | **General User** | Tests typical workflows like a first-time visitor | | **Power User** | Exercises advanced features and edge cases | | **Security Tester** | Hunts for vulnerabilities and auth issues | | **Accessibility Auditor** | Checks for a11y issues affecting all users | Define what success looks like: completing onboarding, finishing checkout, exploring settings, etc. Specify areas Scout should avoid: payment flows, admin actions, or any sensitive sections. ### Step 4: Launch the exploration Project dashboard with launch button Click **Launch Scout** from your project dashboard. The exploration typically completes in 5-10 minutes. Watch Scout work in real-time via the live transcript, or close the tab—results will be waiting when you return. ### Step 5: Review findings Scout findings report Scout delivers a structured report with: * **Issues organized by severity** — Critical, High, Medium, Low * **Screenshots** — Visual proof of what Scout encountered * **Reproduction steps** — Exact actions to recreate each issue * **Suggested fixes** — Recommendations to resolve problems Generate a public link (`/t/{executionId}`) to share findings with anyone—no login required. Tag team members on specific issues to track resolution. *** ## What Scout explores During each exploration, Scout:
**Functionality** * Navigates pages like a real user * Fills out and submits forms * Tests authentication flows * Clicks buttons and interactive elements
**Quality checks** * Usability friction and confusing UI * Broken links and missing pages * Console errors and API failures * Performance bottlenecks * Accessibility violations * Security red flags
*** ## Scheduling recurring explorations Set up automated runs to catch regressions before users do. Navigate to your project and click the **Schedules** tab. Choose frequency (daily, weekly, or custom), select a persona, and save. Enable email notifications to receive a summary after each scheduled run. Scheduling is available from your [project dashboard](/essentials/schedules). Perfect for staging environments before each deploy. *** ## Next steps Organize explorations by product or environment. View run history and relaunch past explorations. Track, triage, and resolve discovered issues. *** ## FAQ Most explorations complete in 5-10 minutes, depending on your app's size and complexity. No. The Instant URL approach works immediately with no setup. The dashboard requires only a free account. Yes. Use the [Credentials](/essentials/credentials) feature to provide login details, or share a pre-authenticated session. Scout doesn't store sensitive data. Explorations are ephemeral, and you control what pages Scout can access. Scout works with any publicly accessible web app. The Instant URL pattern currently supports Lovable and Replit, with more platforms coming soon. Yes. Use the Instant URL in GitHub Actions, GitLab CI, or any automation tool. API access is coming soon. Every exploration generates a shareable link (`/t/{executionId}`). Recipients can view findings without creating an account. *** ## Need help? Stuck or have questions? Reach out and we'll get you unblocked fast. # Claude Code Skill - Vibe Testing Integration Source: https://docs.scoutqa.ai/tooling/claude-code-skill Vibe testing integration for Claude Code - test web apps using natural language directly from your AI coding assistant. **GitHub Repository**: [scoutqa-dot-ai/claude-code](https://github.com/scoutqa-dot-ai/claude-code) The **Scout skill for Claude Code** brings vibe testing to your AI coding workflow. Ask Claude to test your web applications using natural language commands - no test scripts, no complex setup. ## What is Vibe Testing with Claude Code? **Vibe testing** is a new approach to software testing where you describe what to test in plain English, and AI handles the execution. With the Scout skill installed in Claude Code, you can: * Ask Claude to "test the login flow" and get real results * Run accessibility audits by saying "check WCAG compliance" * Validate user journeys without writing a single test script This is **testing for the AI-assisted development era** - as fast and fluid as vibe coding itself. ## Features Describe tests conversationally. "Test the checkout flow" or "Find accessibility issues on the homepage." Automatically identify WCAG compliance issues and accessibility barriers. Verify critical journeys like signup, login, checkout, and onboarding work correctly. Detect UX friction, confusing interfaces, and potential user frustrations. ## Installation ### Prerequisites First, install the Scout CLI: ```bash theme={null} npm install -g @scoutqa/cli@latest scoutqa auth login ``` ### Add the Skill to Claude Code ```bash theme={null} # Add the Scout marketplace claude plugin marketplace add https://github.com/scoutqa-dot-ai/claude-code # Install the Scout skill claude plugin i scoutqa-plugin@scoutqa-marketplace ``` ## Usage Once installed, simply ask Claude to run tests: ``` Test the login flow on https://staging.myapp.com - Try valid credentials - Try invalid credentials - Check error messages are helpful ``` ``` Run an accessibility audit on https://myapp.com/signup Check for WCAG 2.1 AA compliance ``` ``` Run a quick smoke test on https://staging.myapp.com Verify the homepage loads and main navigation works ``` ``` Test the checkout process on https://myapp.com Add items to cart, go through checkout, verify order confirmation ``` ## Example Workflow ``` You: Test the user registration on https://staging.myapp.com/signup Claude: I'll run exploratory tests on the signup flow. Starting Scout... [Scout executes tests] Claude: Here's what I found: Issues Detected: 1. Email validation accepts invalid formats (high severity) 2. Password requirements not clearly displayed (medium) 3. Missing aria-label on submit button (accessibility) Recommendations: - Add regex validation for email field - Display password requirements before user types - Add aria-label="Create account" to submit button ``` ## Troubleshooting | Issue | Solution | | ---------------------- | ----------------------------------------------- | | Skill not found | Run `claude plugin list` to verify installation | | Authentication errors | Run `scoutqa auth login` to re-authenticate | | Tests failing to start | Verify target URL is publicly accessible | For debugging, run: ```bash theme={null} claude --debug scoutqa --version ``` ## Why Use Vibe Testing with Claude Code? | Traditional Testing | Vibe Testing with Scout | | ------------------------------- | ------------------------------------- | | Write and maintain test scripts | Describe tests in natural language | | Brittle selectors break often | AI adapts to UI changes | | Limited to predefined scenarios | Exploratory - finds unexpected issues | | Requires testing expertise | Anyone can write tests | ## Frequently Asked Questions A Claude Code skill is a plugin that extends Claude's capabilities. The Scout skill adds vibe testing functionality, allowing Claude to run real tests on your web applications when you ask. When you ask Claude to test something, the Scout skill executes AI-powered exploratory tests on your target URL. Scout's agent navigates your app, performs actions, checks for issues, and reports findings back to Claude - who then summarizes results for you. Yes! Scout performs WCAG accessibility audits. Just ask Claude to "check accessibility" or "audit for WCAG compliance" on any URL. Scout requires publicly accessible URLs. For localhost testing, use a tunneling service like [ngrok](https://ngrok.com/our-product/secure-tunnels) to expose your local server, then test using the tunnel URL. The Scout CLI is a standalone terminal tool. The Claude Code skill integrates Scout directly into your Claude Code workflow, so you can test without leaving your conversation with Claude. ## Related * [Scout CLI](/tooling/cli) - Run Scout from any terminal * [Cursor Rules](/tooling/cursor-rules) - Use Scout in Cursor IDE * [Kiro Power](/tooling/kiro-power) - Use Scout in Kiro IDE *** Scout requires a **publicly accessible URL** to run tests. Deployed environments like staging and production work immediately. For localhost testing, use a tunneling service like [ngrok](https://ngrok.com/our-product/secure-tunnels): ```bash theme={null} # Start your app, then create a tunnel ngrok http 3000 ``` Then ask Claude to test using the ngrok URL: ``` Test the login flow on https://abc123.ngrok.io ``` \*Third-party tunneling services are not affiliated with Scout. Use at your own risk. # Scout CLI - Vibe Testing from Your Terminal Source: https://docs.scoutqa.ai/tooling/cli Find bugs in your Lovable, Replit, V0, or Base44 app from the command line. AI-powered vibe testing for solo founders, vibe coders, and side project builders. The Scout CLI brings **vibe testing** directly to your terminal. Vibe testing is a new approach to QA where you describe what to test in natural language, and AI handles the rest - no test scripts required. **What is Vibe Testing?** Vibe testing lets you test web applications by simply describing what you want tested. Instead of writing brittle test scripts, you tell Scout "test the login flow" or "check accessibility" and AI explores your app like a human tester would. ## Installation ```bash theme={null} npm install -g @scoutqa/cli ``` ## Authentication The CLI supports multiple authentication methods (in priority order): ### 1. Environment Variables (recommended for CI/CD) ```bash theme={null} export SCOUT_API_KEY=sk-scout-xxx export SCOUT_SERVER_URL=https://scoutqa.ai # optional, this is the default ``` ### 2. OAuth Login (for local development) ```bash theme={null} scoutqa auth login ``` Opens your browser for authentication. Stores access and refresh tokens locally. ### 3. API Key Set an API key manually: ```bash theme={null} scoutqa auth set-api-key ``` **New in v0.11.0:** You can now create and manage API keys from the web UI at **Account Settings → API Keys**. No CLI required. Check your authentication status: ```bash theme={null} scoutqa auth status ``` ### Unauthenticated Access **New in v0.10.1:** The `create-execution` and `send-message` commands now support unauthenticated access for quick testing. This is ideal for trying Scout without signing up or for simple CI/CD pipelines. ```bash theme={null} # Run a test without authentication scoutqa --url https://your-app.com --prompt "Test the homepage" ``` Unauthenticated executions have some limitations: * Results are publicly accessible * No project organization * Limited execution history For production use, we recommend authenticating to access full features. ## Commands ### Create a Test Execution ```bash theme={null} # Test a specific URL with a prompt scoutqa --url https://your-app.com --prompt "Test the login flow" # Run accessibility audit scoutqa --url https://your-app.com --prompt "Check accessibility compliance" # Exploratory testing scoutqa --url https://staging.your-app.com --prompt "Find usability issues in checkout" ``` `create-execution` is the default command and can be omitted. ### List Executions ```bash theme={null} scoutqa list-executions ``` ### Continue an Execution Send follow-up messages to an ongoing execution: ```bash theme={null} scoutqa send-message --execution-id exec_123 --prompt "Now test the logout button" ``` ### Issue Verification **New in v0.11.0:** Verify bugs directly from the CLI — perfect for CI/CD pipelines and post-fix validation. List issues from a previous execution: ```bash theme={null} scoutqa list-issues --execution-id ``` Verify whether a specific issue is still reproducible: ```bash theme={null} scoutqa issue-verify --issue-id ``` This is especially useful in a fix-then-verify workflow: 1. Scout finds a bug → developer fixes it → run `issue-verify` to confirm the fix 2. CI/CD pipeline runs `list-issues` after a test, then `issue-verify` on any critical issues before merging ### Authentication Commands | Command | Description | | ----------------------------------------- | ---------------------------------------- | | `scoutqa auth status` | Show authentication status | | `scoutqa auth login` | Login via browser (OAuth) | | `scoutqa auth set-api-key ` | Set API key manually | | `scoutqa list-issues --execution-id ` | List issues from an execution | | `scoutqa issue-verify --issue-id ` | Verify if an issue is still reproducible | | `scoutqa auth logout` | Clear stored credentials | ## CI/CD Integration Add Scout to your GitHub Actions workflow: ```yaml theme={null} name: Scout QA on: [push] jobs: test: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - uses: actions/setup-node@v4 with: node-version: '20' - run: npm install -g @scoutqa/cli - run: scoutqa --url ${{ env.STAGING_URL }} --prompt "Smoke test critical flows" env: SCOUT_API_KEY: ${{ secrets.SCOUT_API_KEY }} ``` ## Troubleshooting | Issue | Solution | | --------------------- | --------------------------------------------------- | | Authentication failed | Run `scoutqa auth logout` then `scoutqa auth login` | | Command not found | Ensure npm global bin is in your PATH | | Tests timing out | Check network connectivity to target URL | Check your auth status with `scoutqa auth status`. ## Use Cases for Vibe Testing * **Smoke Testing** - Quick validation before deployments * **Regression Testing** - Verify existing features still work after changes * **Accessibility Testing** - WCAG compliance audits * **Exploratory Testing** - Discover edge cases and unexpected behaviors * **User Flow Validation** - Ensure critical paths like checkout and signup work ## Frequently Asked Questions Vibe testing is AI-powered exploratory testing where you describe tests in natural language instead of writing code. Scout's AI agent navigates your application, identifies issues, and reports findings - similar to how a human QA tester would explore your app. Traditional automated testing requires writing and maintaining test scripts. Vibe testing uses AI to understand your intent from natural language and autonomously explores your application. This means faster test creation, less maintenance, and better coverage of edge cases. Yes! Scout CLI integrates with GitHub Actions, CircleCI, Jenkins, and any CI/CD system. Use it to gate deployments with quality checks. Scout CLI works with any web application accessible via a publicly accessible URL - SPAs, server-rendered apps, staging environments, and production sites. For localhost testing, you'll need to use a tunneling service like [ngrok](https://ngrok.com/our-product/secure-tunnels) to expose your local server. ## What's Next * See [Claude Code Skill](/tooling/claude-code-skill) to use Scout directly in Claude Code * See [Kiro Power](/tooling/kiro-power) to integrate Scout into Kiro IDE Questions? Reach out at [huy.tieu@scoutqa.ai](mailto:huy.tieu@scoutqa.ai). *** Scout requires a **publicly accessible URL** to run tests. Deployed environments like staging and production work immediately. For localhost testing, use a tunneling service like [ngrok](https://ngrok.com/our-product/secure-tunnels): ```bash theme={null} # Start your app npm run dev # In another terminal, create a tunnel ngrok http 3000 # Use the tunnel URL with Scout scoutqa --url https://abc123.ngrok.io --prompt "Test the login flow" ``` \*Third-party tunneling services are not affiliated with Scout. Use at your own risk. # Cursor Rules - Vibe Testing in Cursor IDE Source: https://docs.scoutqa.ai/tooling/cursor-rules Vibe testing integration for Cursor IDE - test your Lovable, Replit, V0, or Base44 apps without leaving your editor. **GitHub Repository**: [scoutqa-dot-ai/cursor-rules](https://github.com/scoutqa-dot-ai/cursor-rules) **Awesome Cursorrules**: [View on awesome-cursorrules](https://github.com/PatrickJS/awesome-cursorrules) The **Scout rules for Cursor** bring vibe testing directly into your favorite AI code editor. Ask Cursor to test your web applications using natural language - no test scripts, no context switching. ## What is Vibe Testing with Cursor? **Vibe testing** is AI-powered exploratory testing designed for solo founders, vibe coders, and side project builders. With Scout rules installed in Cursor, you can: * Say "test my Lovable app" and get real bug reports * Run accessibility audits without leaving your editor * Validate user flows before shipping This is **QA that matches the speed of vibe coding** - test as fast as you build. ## Features Describe tests conversationally. "Test the checkout flow" or "Find bugs on the homepage." Works with Lovable, Replit, V0, Base44, Bolt.new, Vercel, Netlify, and any web app. Automatically identify WCAG compliance issues and accessibility barriers. Drop the rules file in your project and start testing immediately. ## Installation ### Step 1: Install Scout CLI First, install the Scout CLI globally: ```bash theme={null} npm install -g @scoutqa/cli scoutqa auth login ``` ### Step 2: Add Scout Rules to Cursor Create the rules directory and download the Scout rules: ```bash theme={null} mkdir -p .cursor/rules curl -o .cursor/rules/scout.mdc https://raw.githubusercontent.com/scoutqa-dot-ai/cursor-rules/main/scout.mdc ``` 1. Create a `.cursor/rules` directory in your project root 2. Create a file called `scout.mdc` inside it 3. Copy the rules from [our GitHub repository](https://github.com/scoutqa-dot-ai/cursor-rules) To enable Scout across all your projects: 1. Open Cursor Settings (`Cmd+,` or `Ctrl+,`) 2. Search for "Rules" 3. Add the Scout rules to your User Rules ### Step 3: Restart Cursor After adding the rules, restart Cursor to load them. ## Usage Once installed, simply ask Cursor to test your application: ``` Test my app at https://myapp.lovable.app Find any bugs or issues ``` ``` Test the login flow on https://staging.myapp.com Try valid and invalid credentials ``` ``` Run an accessibility audit on my Replit app https://myapp.replit.app ``` ``` I'm about to deploy - can you run a quick smoke test? https://staging.myapp.com ``` ## Example Workflow Write your code as usual in Cursor with AI assistance Push to Lovable, Replit, Vercel, or your hosting platform "Test the new checkout flow on [https://myapp.lovable.app](https://myapp.lovable.app)" Cursor runs Scout and reports issues - fix them in the same session ``` You: I just deployed my Lovable app. Can you test the signup flow? https://myproject.lovable.app Cursor: I'll run Scout to test the signup flow on your Lovable app. [Running: scoutqa --url https://myproject.lovable.app --prompt "Test signup flow"] Here's what Scout found: Issues Detected: 1. Email validation accepts invalid formats (high severity) 2. Password field doesn't show requirements (medium) 3. Submit button missing aria-label (accessibility) Passed: ✓ Form submits successfully with valid data ✓ Error messages display correctly ✓ Redirect to dashboard works Recommendations: - Add regex validation for email: /^[^\s@]+@[^\s@]+\.[^\s@]+$/ - Display password requirements above the field - Add aria-label="Create account" to the submit button Want me to help fix these issues? ``` ## Platform-Specific Examples ### Testing Lovable Apps ``` Test my Lovable app for bugs https://myproject.lovable.app Focus on: - User authentication - Form submissions - Mobile responsiveness ``` ### Testing Replit Deployments ``` Run a smoke test on my Replit project https://myapp.replit.app Make sure the main features work before I share it ``` ### Testing V0 Components ``` I built this UI with V0, can you test it? https://myapp.vercel.app Check if all the interactive elements work correctly ``` ## Troubleshooting | Issue | Solution | | ---------------------- | --------------------------------------------------------- | | Rules not loading | Restart Cursor after adding the `.cursor/rules` directory | | Scout CLI not found | Run `npm install -g @scoutqa/cli` | | Authentication errors | Run `scoutqa auth login` in your terminal | | Tests failing to start | Ensure your URL is publicly accessible | Check Scout CLI status: ```bash theme={null} scoutqa auth status scoutqa --version ``` ## Why Use Scout with Cursor? | Without Scout | With Scout | | ------------------------------ | --------------------------- | | Deploy and hope for the best | Test before every deploy | | Find bugs from user complaints | Catch bugs before users do | | Manual accessibility testing | Automated WCAG audits | | Context switch to run tests | Test without leaving Cursor | ## Frequently Asked Questions Cursor rules are instructions that customize how Cursor's AI behaves in your project. The Scout rules teach Cursor how to run vibe tests on your web applications using the Scout CLI. Scout works with any publicly accessible URL. Deploy your Lovable or Replit app, then ask Cursor to test it. Scout will explore your app and report any issues it finds. Scout requires publicly accessible URLs. For localhost testing, use a tunneling service like [ngrok](https://ngrok.com/our-product/secure-tunnels): ```bash theme={null} ngrok http 3000 # Then test the ngrok URL ``` The Cursor rules integrate Scout into Cursor IDE, while the Claude Code skill integrates Scout into Claude Code CLI. Choose based on your preferred editor. Yes! Scout offers a free tier for solo founders and side projects. Install the CLI and start testing immediately. ## Submit Your Own Rules Using Scout with a specific framework or platform? Create custom rules and submit them to: * [awesome-cursorrules](https://github.com/PatrickJS/awesome-cursorrules) - Community collection of Cursor rules * [cursor.directory](https://cursor.directory) - Cursor rules directory ## Related * [Scout CLI](/tooling/cli) - Run Scout from any terminal * [Claude Code Skill](/tooling/claude-code-skill) - Use Scout in Claude Code * [Kiro Power](/tooling/kiro-power) - Use Scout in Kiro IDE *** Scout requires a **publicly accessible URL** to run tests. Deployed environments like staging and production work immediately. For localhost testing, use a tunneling service like [ngrok](https://ngrok.com/our-product/secure-tunnels): ```bash theme={null} # Start your app, then create a tunnel ngrok http 3000 ``` Then ask Cursor to test the ngrok URL: ``` Test my app on https://abc123.ngrok.io ``` \*Third-party tunneling services are not affiliated with Scout. Use at your own risk. # Kiro Power - Vibe Testing in Your IDE Source: https://docs.scoutqa.ai/tooling/kiro-power Vibe testing integration for Kiro IDE - test web apps without leaving your editor using natural language. **GitHub Repository**: [scoutqa-dot-ai/kiro](https://github.com/scoutqa-dot-ai/kiro) The **Scout power for Kiro** brings vibe testing directly into your IDE. Test your web applications with natural language while you code - no context switching, no test scripts. ## What is Vibe Testing with Kiro? **Vibe testing** is AI-powered exploratory testing designed for modern development workflows. With the Scout power installed in Kiro, you can: * Say "test the feature I just built" and get instant feedback * Run accessibility audits without leaving your editor * Catch bugs before they reach production This is **QA that matches the speed of AI-assisted development** - test as fast as you code. ## Features The AI agent executes tests based on natural language instructions without manual scripting. Automatically test features after implementation to catch issues before they ship. Functional testing, accessibility audits, and exploratory testing in one tool. See test progress and findings live as Scout explores your application. ## Installation ### Prerequisites First, install the Scout CLI: ```bash theme={null} npm install -g @scoutqa/cli@latest scoutqa auth login ``` ### Add the Power to Kiro 1. Open Kiro IDE 2. Click the Powers panel (ghost icon with lightning bolt) 3. Select **Add Custom Power** 4. Choose **Import power from GitHub** 5. Enter: `https://github.com/scoutqa-dot-ai/kiro` 6. Press Enter to install Installing Scout power in Kiro ## Usage Once installed, talk to your AI agent naturally: ``` Test the login flow on https://staging.myapp.com ``` ``` Check accessibility of the homepage ``` ``` Run exploratory tests on the checkout process ``` ``` Quick smoke test on staging before I deploy ``` ## Example Workflow Write your code as usual in Kiro "Test the new user profile page I just built" Scout runs tests and reports issues directly in your IDE Address issues and re-test until everything passes ``` You: I just finished the checkout flow. Can you test it? Kiro: Running Scout tests on the checkout flow... [Scout executes parallel test scenarios] Kiro: Testing complete. Here are the findings: Passed: - Add to cart works correctly - Cart updates quantities properly - Payment form validates inputs Issues Found: 1. Shipping address not saved between sessions (bug) 2. No loading indicator during payment processing (UX) 3. Order confirmation missing for screen readers (a11y) ``` ## Parallel Execution Scout runs multiple test scenarios simultaneously for faster feedback: ``` Testing checkout flow... ├─ Cart functionality ✓ Passed ├─ Address validation ✓ Passed ├─ Payment processing ⚠ 1 issue └─ Order confirmation ⚠ 2 issues Completed in 45 seconds ``` ## Troubleshooting | Issue | Solution | | --------------------- | ---------------------------------------- | | Power not appearing | Restart Kiro after installation | | Authentication errors | Run `scoutqa auth login` in terminal | | Tests not starting | Ensure target URL is publicly accessible | ## Why Use Vibe Testing with Kiro? | Traditional Testing | Vibe Testing with Scout | | --------------------------- | ------------------------- | | Context switch to run tests | Test without leaving Kiro | | Write and maintain scripts | Natural language commands | | Delayed feedback loop | Instant results in IDE | | Manual accessibility audits | Automated WCAG checks | ## Frequently Asked Questions A Kiro power is an extension that adds capabilities to Kiro IDE's AI agent. The Scout power adds vibe testing functionality, enabling Kiro to run real exploratory tests on your web applications. When you ask Kiro to test something, the Scout power executes AI-powered tests on your target URL. Scout's agent navigates your app, interacts with elements, checks for issues, and reports findings directly in your IDE. Scout requires publicly accessible URLs. For localhost testing, use a tunneling service like [ngrok](https://ngrok.com/our-product/secure-tunnels) to expose your local server, then test using the tunnel URL. Yes, Scout executes multiple test scenarios simultaneously for faster feedback. You'll see real-time progress as each scenario completes. The Kiro power integrates Scout into Kiro IDE, while the Claude Code skill integrates Scout into Claude Code CLI. Choose based on your preferred development environment. ## Related * [Scout CLI](/tooling/cli) - Run Scout from any terminal * [Cursor Rules](/tooling/cursor-rules) - Use Scout in Cursor IDE * [Claude Code Skill](/tooling/claude-code-skill) - Use Scout in Claude Code *** Scout requires a **publicly accessible URL** to run tests. Deployed environments like staging and production work immediately. For localhost testing, use a tunneling service like [ngrok](https://ngrok.com/our-product/secure-tunnels): ```bash theme={null} # Start your app, then create a tunnel ngrok http 3000 ``` Then ask Kiro to test the ngrok URL: ``` Test my checkout flow on https://abc123.ngrok.io ``` \*Third-party tunneling services are not affiliated with Scout. Use at your own risk. # PR Review with GitHub Actions Source: https://docs.scoutqa.ai/tooling/pr-review Automatically review pull requests with Scout quality checks **Coming Soon** - Automated PR review with GitHub Actions is under active development. This page outlines the planned experience so you can prepare your workflows. ## Overview Scout will be able to automatically explore your application when pull requests are created or updated, providing quality feedback before code is merged. This helps catch usability issues, functional bugs, and security concerns early in the development cycle. ## Why PR Review with Scout? Identify quality problems before they reach your main branch or production. Get immediate feedback during development, not days later from QA or users. Keep shipping fast without sacrificing quality or accumulating technical debt. Automated exploration frees up human reviewers to focus on code architecture and logic. ## Setup ### 1. Create GitHub Action Workflow Create `.github/workflows/scout-pr-review.yml`: ```yaml theme={null} name: Scout PR Review on: pull_request: types: [opened, synchronize, reopened] branches: - main - develop jobs: scout-review: runs-on: ubuntu-latest permissions: pull-requests: write contents: read steps: - name: Checkout code uses: actions/checkout@v4 - name: Wait for preview deployment id: preview run: | # Replace with your preview deployment logic echo "url=https://pr-${{ github.event.pull_request.number }}.preview.myapp.com" >> $GITHUB_OUTPUT - name: Run Scout Exploration id: scout uses: katalon-studio/scout-action@v1 with: api_key: ${{ secrets.SCOUT_API_KEY }} url: ${{ steps.preview.outputs.url }} persona: 'general-user' timeout: 10 - name: Post Scout Report to PR uses: actions/github-script@v7 if: always() with: script: | const fs = require('fs'); const report = JSON.parse(fs.readFileSync('scout-report.json', 'utf8')); const statusEmoji = report.critical_count > 0 ? '🚨' : report.high_count > 0 ? '⚠️' : '✅'; const body = `## ${statusEmoji} Scout Quality Report **Exploration Status:** ${report.status} **Persona:** ${report.persona} **Duration:** ${Math.round((new Date(report.completed_at) - new Date(report.started_at)) / 1000 / 60)} minutes ### Issues Found - 🚨 Critical: ${report.critical_count} - ⚠️ High: ${report.high_count} - 📝 Medium: ${report.medium_count} - 💡 Low: ${report.low_count} ${report.issues.slice(0, 3).map(issue => ` #### ${issue.severity === 'critical' ? '🚨' : '⚠️'} ${issue.title} ${issue.description}
Reproduction Steps ${issue.reproduction_steps.map((step, i) => `${i + 1}. ${step}`).join('\n')}
`).join('\n')} ${report.issues.length > 3 ? `\n_...and ${report.issues.length - 3} more issues_\n` : ''} [📊 View Full Report](${report.report_url}) `; // Find existing Scout comment const { data: comments } = await github.rest.issues.listComments({ owner: context.repo.owner, repo: context.repo.repo, issue_number: context.issue.number }); const scoutComment = comments.find(c => c.user.type === 'Bot' && c.body.includes('Scout Quality Report') ); if (scoutComment) { // Update existing comment await github.rest.issues.updateComment({ owner: context.repo.owner, repo: context.repo.repo, comment_id: scoutComment.id, body: body }); } else { // Create new comment await github.rest.issues.createComment({ owner: context.repo.owner, repo: context.repo.repo, issue_number: context.issue.number, body: body }); } - name: Fail on critical issues if: steps.scout.outputs.critical_count > 0 run: | echo "Scout found ${{ steps.scout.outputs.critical_count }} critical issues" exit 1 ``` ### 2. Configure Secrets Add the following secrets to your GitHub repository: Go to your repository → Settings → Secrets and variables → Actions Create a new secret named `SCOUT_API_KEY` with your Scout API key from [scoutqa.ai](https://scoutqa.ai) If using fixed staging URLs, add secrets like `STAGING_URL` or configure preview deployment URLs in your workflow ## Integration Patterns ### With Lovable Apps ```yaml theme={null} - name: Run Scout on Lovable App uses: katalon-studio/scout-action@v1 with: api_key: ${{ secrets.SCOUT_API_KEY }} url: 'yourapp.lovable.scoutqa.app' persona: 'general-user' timeout: 10 ``` ### With Replit Apps ```yaml theme={null} - name: Run Scout on Replit App uses: katalon-studio/scout-action@v1 with: api_key: ${{ secrets.SCOUT_API_KEY }} url: 'yourapp.replit.scoutqa.app' persona: 'security-tester' timeout: 10 ``` ### With Vercel Preview Deployments ```yaml theme={null} - name: Wait for Vercel Preview uses: patrickedqvist/wait-for-vercel-preview@v1.3.1 id: vercel with: token: ${{ secrets.GITHUB_TOKEN }} max_timeout: 300 - name: Run Scout on Vercel Preview uses: katalon-studio/scout-action@v1 with: api_key: ${{ secrets.SCOUT_API_KEY }} url: ${{ steps.vercel.outputs.url }} ``` ### With Netlify Deploy Previews ```yaml theme={null} - name: Wait for Netlify Deploy uses: jakepartusch/wait-for-netlify-action@v1 id: netlify with: site_name: 'your-site-name' max_timeout: 300 - name: Run Scout on Netlify Preview uses: katalon-studio/scout-action@v1 with: api_key: ${{ secrets.SCOUT_API_KEY }} url: ${{ steps.netlify.outputs.url }} ``` ### With Custom Preview Environments ```yaml theme={null} - name: Build and Deploy Preview id: deploy run: | ./scripts/deploy-preview.sh ${{ github.event.pull_request.number }} echo "url=$(cat preview-url.txt)" >> $GITHUB_OUTPUT - name: Run Scout uses: katalon-studio/scout-action@v1 with: api_key: ${{ secrets.SCOUT_API_KEY }} url: ${{ steps.deploy.outputs.url }} ``` ## Advanced Configuration ### Multi-Persona Review Run Scout with different personas to catch diverse issues: ```yaml theme={null} jobs: scout-review: runs-on: ubuntu-latest strategy: matrix: persona: [general-user, security-tester, accessibility-auditor] fail-fast: false steps: - name: Run Scout as ${{ matrix.persona }} uses: katalon-studio/scout-action@v1 with: api_key: ${{ secrets.SCOUT_API_KEY }} url: ${{ steps.preview.outputs.url }} persona: ${{ matrix.persona }} - name: Upload ${{ matrix.persona }} Report uses: actions/upload-artifact@v4 with: name: scout-report-${{ matrix.persona }} path: scout-report.json ``` ### Conditional Execution Only run Scout for certain file changes: ```yaml theme={null} on: pull_request: paths: - 'src/**' - 'pages/**' - 'components/**' paths-ignore: - 'docs/**' - '**.md' ``` ### Quality Gate Policies Define strict quality policies: ```yaml theme={null} - name: Evaluate Quality Gate run: | CRITICAL=$(jq '.critical_count' scout-report.json) HIGH=$(jq '.high_count' scout-report.json) if [ "$CRITICAL" -gt 0 ]; then echo "❌ Failed: $CRITICAL critical issues found" exit 1 fi if [ "$HIGH" -gt 5 ]; then echo "⚠️ Warning: $HIGH high-severity issues found" exit 1 fi echo "✅ Quality gate passed" ``` ## Comment Format Examples ### Clean PR (No Issues) ``` ✅ Scout Quality Report Exploration Status: completed Persona: general-user Duration: 8 minutes ### Issues Found - 🚨 Critical: 0 - ⚠️ High: 0 - 📝 Medium: 2 - 💡 Low: 5 Great work! No critical or high-severity issues detected. 📊 View Full Report ``` ### PR with Issues ``` 🚨 Scout Quality Report Exploration Status: completed Persona: security-tester Duration: 10 minutes ### Issues Found - 🚨 Critical: 2 - ⚠️ High: 3 - 📝 Medium: 8 - 💡 Low: 12 #### 🚨 SQL Injection vulnerability in search User input in the search parameter is not properly sanitized... [Reproduction Steps] 1. Navigate to /search 2. Enter payload: ' OR '1'='1 3. Submit form ...and 12 more issues 📊 View Full Report ``` ## Best Practices Begin with Scout running as informational only (`fail_on_critical: false`). Once your baseline is clean, enable strict gates. Match personas to your PR type: security-tester for auth changes, accessibility-auditor for UI work, etc. Set 5-10 minute timeouts for PR reviews to balance thoroughness with CI speed. Reuse the same comment for updates rather than creating a new comment on each push. Always save Scout reports as artifacts for later review and trend analysis. For authenticated apps, generate temporary tokens in CI and pass them to Scout via `auth_token`. ## Metrics and Reporting Track Scout's impact over time: ```yaml theme={null} - name: Track Scout Metrics run: | jq '{ pr_number: "${{ github.event.pull_request.number }}", commit_sha: "${{ github.sha }}", timestamp: now, critical: .critical_count, high: .high_count, medium: .medium_count, duration: .duration }' scout-report.json >> scout-metrics.jsonl - name: Upload Metrics uses: actions/upload-artifact@v4 with: name: scout-metrics path: scout-metrics.jsonl ``` ## Troubleshooting Ensure the workflow has `pull-requests: write` permission in the job configuration. Add appropriate wait/polling logic after deployment but before running Scout. Generate a session token in your workflow and pass it to Scout using the `auth_token` parameter. Adjust the persona or timeout, and consider implementing an issue allowlist for known non-issues. ## Next Steps * [Set up GitHub Actions integration](/quick-run/github-actions) * [View Scout reports](/essentials/markdown) Questions? [Contact support](mailto:huy.tieu@scoutqa.ai).