The first version of Atlyz's setup flow was embarrassing. We made business owners manually copy-paste their service descriptions. Fill out a form. Upload a FAQ document. Review training data line by line. The whole process took about 40 minutes.
Nobody finished it.
We watched three test users during early beta. All three gave up in the first ten minutes. Not because they didn't want an AI chatbot — they were motivated enough to sign up for a beta. But because the setup felt like work, and small business owners already have more work than hours in the day.
That failure led to the insight that shaped everything about how Atlyz works today.
The Insight: The Website Is Already the Knowledge Base
Every business owner has spent hours getting their website right. Their prices are there. Their services are described. Their hours, location, policies, team bios — all of it is sitting in HTML files, already written, already accurate.
The problem isn't that the information doesn't exist. It's that it's trapped in static web pages instead of living inside your AI assistant.
So we asked a different question: instead of making business owners train an AI from scratch, what if we just taught the AI to read their website?
How It Works
When a business owner enters their URL, here's what happens under the hood:
We fetch the homepage and extract all internal links — service pages, about, FAQ, contact, product pages. We filter out external links, social profiles, and anything that looks like a category or pagination page.
We crawl up to 50 pages, stripping navigation, footers, cookie banners, and repeated boilerplate that appears on every page. What's left is the actual content — the stuff a customer would actually read.
We send the combined text to GPT with a prompt that asks it to extract and structure the key business information: what they do, what they charge, how to contact them, what their policies are, what questions customers commonly ask.
The output becomes the knowledge base. Every time a customer asks a question, the AI retrieves from this structured summary — not from raw HTML, which would be slow and unreliable.
Total time from URL to ready AI: about 60 seconds for a typical 10-page business website.
The Engineering Problems We Had to Solve
Reading websites sounds simple. In practice, the web is a mess.
JavaScript-rendered pages. Many modern sites built with Shopify themes, React storefronts, or Webflow deliver near-empty HTML to web scrapers. The actual content only appears after JavaScript runs. We handle this with a rendering layer that executes JavaScript before we read the page, though it adds a few seconds to the crawl.
Duplicate content everywhere. A navigation menu with twelve links appears on every single page. So does the footer. So does a cookie consent banner. If we naively concatenated all pages, about 40% of the text sent to GPT would be identical boilerplate. We deduplicate aggressively before extraction.
Image-only content. Some businesses — especially restaurants and salons — put their entire menu or service list in a PDF or image file. There's no text for us to read. When we detect this, we flag it and prompt the owner to add a simple text FAQ page. Businesses with a well-written FAQ page produce measurably better AI responses than businesses without one.
What we learned: Businesses with clear, text-heavy service pages and an FAQ section train AI that's about 40% more accurate than businesses with vague marketing copy. "We're passionate about delivering excellence" tells the AI nothing. "Haircuts from $35, balayage from $120, open Tuesday–Saturday 9am–7pm" tells it everything.
What Good Looks Like
After the scrape, you can send the AI a message that tests it against your own knowledge:
Atlyz: "Yes! We keep slots open each day for same-day bookings. You can call us at [number] or I can take your details and have someone reach out to confirm availability."
That answer came entirely from reading the website. No manual input. No training. The owner spent zero minutes on setup.
What's Next
The scraper is good, but we want it to be great. On our roadmap:
- Scheduled re-scraping — automatically detect when a page has changed and update the knowledge base without the owner doing anything
- PDF and menu support — ingest uploaded files for businesses whose key info isn't on the web
- Change detection — only re-ingest updated sections rather than re-reading the whole site each time
- Manual knowledge additions — let owners add custom Q&A pairs for questions the website doesn't cover
The goal is an AI that knows your business as well as you do — without you ever having to train it. Your website already contains that knowledge. We just need to keep reading it.
See it read your website
Enter your URL and watch Atlyz build a complete knowledge base in under 60 seconds. 7-day free trial.
Try it free →