AI-Investigator is a powerful Python framework designed for automated website content analysis. Leveraging the Claude 3.5 Sonnet API and Firecrawl, it efficiently maps website structures, extracts content from specific pages, and generates comprehensive, structured reports. While initially tailored for analyzing enterprise AI case studies, its adaptable architecture can be customized to meet a wide range of content analysis needs, making it a versatile tool for researchers and analysts.
The system operates in two primary modes, allowing users to either discover all website links or scrape specific page content. It processes website inputs using link analysis, content extraction and then utilizes Claude's AI for analysis before producing detailed reports. By streamlining the process of gathering and interpreting web-based information, AI-Investigator offers a reliable solution for anyone seeking to glean insights from online content quickly and accurately.