/ AI Website Assistant | 2026
A full-stack AI platform that turns website content into a searchable knowledge base and an embeddable customer-support assistant.
/ Project overview
Scrapcher lets a business submit its website, crawl and index its content, test an AI assistant against that knowledge, customize the assistant, and embed it back into the website. The product combines a Next.js dashboard with a multi-stage RAG pipeline and a separate Python crawling service.
/ Process
The main challenge was connecting several distinct systems into one reliable product flow: crawling external websites, processing and indexing content, retrieving the right context for each question, and delivering grounded answers through both the dashboard and an embedded widget.
The system uses Crawl4AI through a FastAPI service with a JavaScript fallback crawler, then chunks and embeds the extracted content before storing it in Supabase. Questions are matched through vector search, reranked with Cohere, and answered through Groq using the selected website context. Website-scoped identifiers keep retrieval isolated between indexed sites.
/ The result
The finished product demonstrates an end-to-end workflow from website ingestion to a configurable, embeddable AI assistant, combining product UI, backend orchestration, scraping, vector retrieval, reranking, and LLM integration.
/ Next project
Zentiva