INTELISCAPE-X: AI-Powered Web Scraping in Excel

This xlwings Lite app turns Excel into a web scraping platform:

• Extract structured data from multiple websites
• Process batches of URLs with custom column specifications
• Use AI to identify and extract data elements from web pages
• Apply custom filtering instructions to refine results
• Track processing status and errors automatically
• Monitor performance metrics and token usage
• Format results directly into Excel tables

1. Install xlwings Lite from the Add-in button in Excel
2. Configure the MASTER sheet:
   • Add your Jina and Gemini API keys
   • Select your Gemini model (e.g. gemini-2.5-flash)
   • Set scraping parameters (timeout, retries, delays)
3. Define extraction columns in COLUMN_INPUTS:
   • Specify column names and descriptions for the data you want
   • Add custom filtering instructions if needed
4. Add target URLs in URL_LIST
5. Run the scraper: execute scrape_urls_from_list from the xlwings tab
6. Review results: DATA, DASHBOARD, ERROR_LOG, and URL_LIST sheets

The web scraper processes URLs through a three-stage pipeline:

Stage 1: Configuration

• xlwings Lite runs Python code directly in Excel via WebAssembly
• Reads configuration from MASTER sheet (API keys, model settings)
• Loads column definitions from COLUMN_INPUTS

Stage 2: Data Extraction

• Sends each URL to Jina API to fetch and render as markdown
• Passes rendered content to Gemini API with column specifications
• Gemini extracts structured data matching your definitions

Stage 3: Results

• Writes extracted data to DATA sheet as formatted table
• Logs errors to ERROR_LOG sheet with timestamps
• Generates DASHBOARD with performance metrics

Process Flow