OCR

Extract text from images

Agents can read text from images - receipts, documents, signs, and screenshots.

Quick Start

1. Simple Usage

   import { Agent } from 'praisonai';
   
   const agent = new Agent({
     instructions: 'Extract all text from images',
     llm: 'gpt-4o'  // Vision-capable model
   });
   
   await agent.chat([
     { role: 'user', content: [
       { type: 'text', text: 'What text is on this receipt?' },
       { type: 'image', url: 'https://example.com/receipt.jpg' }
     ]}
   ]);
   // "Store: Coffee Shop, Total: $4.50..."
   
2. With Configuration

   const agent = new Agent({
     instructions: 'Extract data as JSON',
     outputFormat: 'json'
   });
   
   await agent.chat([
     { role: 'user', content: [
       { type: 'text', text: 'Extract items and prices from this receipt' },
       { type: 'image', path: './receipt.jpg' }
     ]}
   ]);
   // { items: [...], total: 4.50, date: '...' }
   

---

User Interaction Flow

---

Configuration Levels

// Level 1: Bool - Enable with vision model
const agent = new Agent({
  llm: 'gpt-4o',
  vision: true
});

// Level 2: String - High detail for small text
const agent = new Agent({
  llm: 'gpt-4o',
  vision: 'high'
});

// Level 3: Dict - Full options
const agent = new Agent({
  vision: {
    detail: 'high',
    ocr: true,
    language: 'en'
  }
});

---

Common Uses

Use Case	Example
Receipts	Extract items, totals, dates
Business cards	Get contact information
Documents	Digitize scanned papers
Screenshots	Read displayed text

---

Dedicated OCRAgent class

OCRAgent calls a real OCR backend through a required extractor function.

import { OCRAgent, type OCRExtractor } from 'praisonai';

const mistralExtractor: OCRExtractor = async ({ source, document, model, options, apiKey, baseUrl }) => {
  // Call your OCR backend (Mistral OCR, Tesseract, Cloud Vision, ...)
  // Return { text, pages: [{ index, markdown }], images?, metadata? }
  return { text: '...', pages: [{ index: 0, markdown: '...' }] };
};

const agent = new OCRAgent({
  name: 'Reader',
  extractor: mistralExtractor,
  model: 'mistral/mistral-ocr-latest',
});

const result = await agent.extract('receipt.jpg');
const textOnly = await agent.read('receipt.jpg');

OCRAgentConfig

Field	Type	Default	Description
extractor	OCRExtractor	—	OCR backend. Required at call time — extract() throws without it.
name	string	'OCRAgent'	Agent name
instructions	string	—	Optional instructions
llm / model	string	'mistral/mistral-ocr-latest'	OCR model (model is an alias for llm)
baseUrl	string	—	Custom API endpoint URL
apiKey	string	—	API key for the provider
ocr	boolean | OCRConfig	—	OCR settings; true/false use defaults
verbose	boolean | number	true	Log progress messages

OCRConfig

Field	Type	Default	Description
pages	number[]	[]	Specific pages to extract (for PDFs)
imageLimit	number	0	Maximum images per page
timeout	number	600	Timeout in seconds
apiBase	string	''	Custom API endpoint URL
apiKey	string	''	API key for the provider

OCRExtractRequest

The extractor receives this request:

Field	Type	Description
source	string	URL or path the caller asked to read
document	{ type: string; ... }	Provider document reference derived from source
model	string	Model to use for this extraction
options	Required<OCRConfig>	Resolved OCR options
baseUrl	string	Custom API endpoint URL, if configured
apiKey	string	API key, if configured

OCRResult

The extractor returns this shape:

Field	Type	Description
text	string	Combined extracted text
pages	OCRPage[]	Per-page results (index, markdown, images?)
images	string[]	Optional base64 images
metadata	Record<string, any>	Optional additional metadata

OCRAgent.extract() (and .read()) now throw when no extractor is configured. Before, they returned placeholder text like "[OCR extraction from … - requires API integration]" while logging ✓ OCR complete. That fabricated path is gone.

---

API Reference

OCRConfig

Complete configuration options

OCRAgent

Full class documentation

---

Best Practices

Use high detail for small text

Set detail: 'high' when reading receipts or documents.

Be specific about what to extract

“Extract the total and date” works better than “read this”.

Use good quality images

Clear, well-lit images produce better results.

---

Related

Vision

Image analysis

Files

File operations