Course contentsShow
← Security
Lesson 1371 of 3,05228. Application Security - TestingFree lesson

Crawling and Application Discovery

How DAST tools discover endpoints, form inputs, and application attack surface through automated crawling.

Crawling and Application Discovery

What You'll Learn

You'll learn how DAST tools automatically map out a web application's structure to identify all the places attackers might strike.

The Exploration Challenge

Before a DAST scanner can test for vulnerabilities, it needs to know what exists. Imagine trying to secure a building without knowing how many doors, windows, and entry points it has. Application discovery through crawling solves this problem by systematically exploring your web application.

How Crawling Works

A DAST crawler acts like an automated user browsing your site:

  1. Starts at entry points – Usually your homepage or login page
  2. Follows links – Clicks every <a> tag, button, and navigation element it finds
  3. Submits forms – Identifies input fields (search boxes, login forms, comment fields)
  4. Maps API endpoints – Discovers AJAX calls and REST endpoints through JavaScript analysis
  5. Records the attack surface – Builds a complete map of URLs, parameters, and inputs

The crawler builds what's called the attack surface: every place where user input enters your application and every endpoint that responds to requests.

Discovery Techniques

Modern crawlers use multiple strategies:

  • Traditional spidering: Following HTML links recursively
  • JavaScript rendering: Executing JavaScript to find dynamic content (Single Page Applications)
  • Form field detection: Identifying every input, textarea, and file upload
  • Parameter discovery: Finding URL parameters and hidden form fields
  • API documentation parsing: Reading OpenAPI/Swagger specs if available

Why This Matters

Without thorough discovery, DAST tools miss vulnerabilities. A crawler that can't find your API endpoints won't test them. A crawler that ignores JavaScript-rendered content in your React app leaves blind spots an attacker could exploit.

Key Takeaway: DAST crawling automatically discovers and maps your application's attack surface—all endpoints, forms, and inputs—so security testing covers everywhere attackers might probe.