> ## Documentation Index
> Fetch the complete documentation index at: https://praison.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Firecrawl

> Web scraping, crawling, and data extraction for AI applications

```mermaid theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
graph LR
    U[Input] --> A[Agent]
    A --> T[Tool]
    T --> O[Output]

    classDef agent fill:#8B0000,color:#fff
    classDef tool fill:#189AB4,color:#fff

    class A agent
    class U,O tool
    class T tool
```

# Firecrawl Tools

Firecrawl provides powerful web scraping, search, crawling, and data extraction capabilities for AI applications.

## Installation

```bash theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
npm install firecrawl-aisdk
```

## Environment Variables

```bash theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
FIRECRAWL_API_KEY=fc-your-api-key
```

Get your API key from [Firecrawl](https://firecrawl.dev).

## Available Tools

| Tool | Description |
| - | - |
| `scrapeTool` | Scrape a single URL |
| `searchTool` | Search the web |
| `mapTool` | Discover URLs on a site |
| `crawlTool` | Crawl multiple pages |
| `batchScrapeTool` | Scrape multiple URLs |
| `extractTool` | Extract structured data |

## Quick Start

**1. Simple Usage**

```typescript theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
import { Agent } from 'praisonai';
import { firecrawlScrape, firecrawlCrawl } from 'praisonai/tools';

const agent = new Agent({
  name: 'WebScraper',
  instructions: 'You scrape and analyze web content.',
  tools: [firecrawlScrape(), firecrawlCrawl()],
});

const result = await agent.run('Scrape https://example.com and summarize the content');
console.log(result.text);
```

**2. With Configuration**

Adjust agent instructions, tool options, and provider settings for production — see the Configuration section below.

## Scrape Tool

Scrape a single URL and get clean markdown content.

```typescript theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
import { firecrawlScrape } from 'praisonai/tools';

const scrapeTool = firecrawlScrape({
  // Output format
  formats: ['markdown', 'html'],
  
  // Wait for page to load
  waitFor: 1000,
  
  // Include/exclude tags
  includeTags: ['article', 'main'],
  excludeTags: ['nav', 'footer'],
  
  // Screenshot options
  screenshot: true,
});

const agent = new Agent({
  name: 'Scraper',
  tools: [scrapeTool],
});
```

## Crawl Tool

Crawl multiple pages starting from a URL.

```typescript theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
import { firecrawlCrawl } from 'praisonai/tools';

const crawlTool = firecrawlCrawl({
  // Maximum pages to crawl
  limit: 10,
  
  // Crawl depth
  maxDepth: 2,
  
  // URL patterns to include/exclude
  includePaths: ['/docs/*', '/blog/*'],
  excludePaths: ['/admin/*'],
  
  // Allow external links
  allowExternalLinks: false,
});

const agent = new Agent({
  name: 'Crawler',
  tools: [crawlTool],
});
```

## Using with AI SDK Directly

```typescript theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
import { generateText } from 'ai';
import { openai } from '@ai-sdk/openai';
import { scrapeTool, searchTool, mapTool, crawlTool } from 'firecrawl-aisdk';

// Scrape a page
const { text } = await generateText({
  model: openai('gpt-4o'),
  prompt: 'Scrape https://firecrawl.dev and summarize what it does',
  tools: { scrape: scrapeTool },
});

// Search the web
const { text: searchResult } = await generateText({
  model: openai('gpt-4o'),
  prompt: 'Search for Firecrawl and summarize what you find',
  tools: { search: searchTool },
});

// Map a site
const { text: mapResult } = await generateText({
  model: openai('gpt-4o'),
  prompt: 'Map https://docs.firecrawl.dev and list the main sections',
  tools: { map: mapTool },
});
```

## Batch Scraping

Scrape multiple URLs efficiently.

```typescript theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
import { generateText } from 'ai';
import { openai } from '@ai-sdk/openai';
import { batchScrapeTool, pollTool } from 'firecrawl-aisdk';

const { text } = await generateText({
  model: openai('gpt-4o'),
  prompt: 'Scrape https://firecrawl.dev and https://docs.firecrawl.dev, then compare',
  tools: { 
    batchScrape: batchScrapeTool, 
    poll: pollTool 
  },
});
```

## Extract Structured Data

Extract specific data from web pages.

```typescript theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
import { generateText } from 'ai';
import { openai } from '@ai-sdk/openai';
import { extractTool, pollTool } from 'firecrawl-aisdk';

const { text } = await generateText({
  model: openai('gpt-4o'),
  prompt: 'Extract the main features from https://firecrawl.dev',
  tools: { 
    extract: extractTool, 
    poll: pollTool 
  },
});
```

## Response Format

### Scrape Result

```typescript theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
interface FirecrawlScrapeResult {
  url: string;
  markdown: string;
  html?: string;
  metadata?: {
    title?: string;
    description?: string;
    language?: string;
  };
  screenshot?: string;
}
```

### Crawl Result

```typescript theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
interface FirecrawlCrawlResult {
  status: string;
  total: number;
  completed: number;
  data: Array<{
    url: string;
    markdown: string;
    metadata?: object;
  }>;
}
```

## Advanced Example

```typescript theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
import { Agent } from 'praisonai';
import { firecrawlScrape, firecrawlCrawl } from 'praisonai/tools';

const agent = new Agent({
  name: 'ContentAnalyzer',
  instructions: `You are a content analyst. 
    1. Scrape the provided URL
    2. Extract key information
    3. Provide a structured summary`,
  tools: [
    firecrawlScrape({ formats: ['markdown'] }),
    firecrawlCrawl({ limit: 5, maxDepth: 1 }),
  ],
});

const result = await agent.run(
  'Analyze the documentation structure of https://docs.firecrawl.dev'
);
console.log(result.text);
```

## Error Handling

```typescript theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
import { firecrawlScrape } from 'praisonai/tools';

const tool = firecrawlScrape();

try {
  const result = await tool.execute({ url: 'https://example.com' });
  console.log(result);
} catch (error) {
  if (error.message.includes('FIRECRAWL_API_KEY')) {
    console.error('Missing API key');
  } else if (error.message.includes('rate limit')) {
    console.error('Rate limited - try again later');
  } else {
    console.error('Scrape failed:', error.message);
  }
}
```

## Best Practices

> [accordion] Guidelines
>
> 1. **Use appropriate tool** - Scrape for single pages, crawl for multiple
> 2. **Set limits** - Always set crawl limits to avoid excessive API usage
> 3. **Filter content** - Use includeTags/excludeTags to get relevant content
> 4. **Handle async jobs** - Use pollTool for crawl and batch operations
> 5. **Cache results** - Store scraped content to avoid repeated requests

## Related Tools

* [Tavily](https://praison.ai/docs/js/tools/tavily.md) - Web search and extraction
* [Exa](https://praison.ai/docs/js/tools/exa.md) - Semantic web search
* [Parallel](https://praison.ai/docs/js/tools/parallel.md) - Token-efficient web search


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.