Turn Any Website into a Smart Chatbot with AI-Powered Content Parsing!

Alright, class! Today, we are going to discuss something far more useful than just scrolling endlessly through the internet. It’s called Content Parsing and Q&A, and if you pay attention, you might just build something cooler than your last failed attempt at a to-do list app. So, sit straight, open your brains, and let’s get into it!

What Is This Magical Thing?

This project is a Flask application that allows users to scrape content from URLs, store it in a vector database (ChromaDB, if you must know), and interact with the content using a chatbox and also use Cross Encoder. Think of it as your personal AI librarian — except it doesn’t give you judgmental looks for asking weird questions.

It uses:

• ChromaDB for storing and retrieving the scraped content
• Bedrock API to generate answers based on the content
• Flask for serving this masterpiece to the world

Features That Make This Project Cool

Scrapes content from one or more URLs (Yes, even that shady-looking news site you love)

Stores scraped content in ChromaDB (because your brain can’t remember everything)

It lets you query the stored content using a chatbox (finally, a chatbot that makes sense!)

Uses the Bedrock API to generate smart answers (unlike your roommate)

Requirements (Because Nothing Works by Magic)

To run this beast, you need:

Python 3.7+ (Don't even try Python 2.7, I will find you.)
Flask
requests
beautifulsoup4
chromadb
sentence-transformers
boto3

If you don’t have these installed, don’t complain when things don’t work!

Installation (Follow This or Suffer!)

1. Clone the repository like a true hacker:

git clone…