update
All checks were successful
dongho-repo/Munich-news/pipeline/head This commit looks good

This commit is contained in:
2025-12-10 15:57:07 +00:00
parent 6e9fbe44c4
commit 7346ee9de2
2 changed files with 167 additions and 511 deletions

View File

@@ -1,56 +1,36 @@
# Quick Start Guide
# Quick Start Guide
Get Munich News Daily running in 5 minutes!
## Prerequisites
## 📋 Prerequisites
- **Docker** & **Docker Compose** installed
- **4GB+ RAM** (for AI models)
- *(Optional)* NVIDIA GPU for faster processing
- Docker & Docker Compose installed
- 4GB+ RAM (for Ollama AI models)
- (Optional) NVIDIA GPU for 5-10x faster AI processing
## Setup
## 🚀 Setup Steps
### 1. Configure Environment
```bash
# Copy example environment file
cp backend/.env.example backend/.env
# Edit with your settings (required: email configuration)
nano backend/.env
```
**Required:** Update `SMTP_SERVER`, `EMAIL_USER`, and `EMAIL_PASSWORD`.
**Minimum required settings:**
```env
SMTP_SERVER=smtp.gmail.com
SMTP_PORT=587
EMAIL_USER=your-email@gmail.com
EMAIL_PASSWORD=your-app-password
```
### 2. Start System
### 2. Start the System
```bash
# Option 1: Auto-detect GPU and start (recommended)
# Auto-detects GPU capabilities and starts services
./start-with-gpu.sh
# Option 2: Start without GPU
docker-compose up -d
# View logs
docker-compose logs -f
# Wait for Ollama model download (first time only, ~2-5 minutes)
# Watch installation progress (first time model download ~2GB)
docker-compose logs -f ollama-setup
```
**Note:** First startup downloads the phi3:latest AI model (2.2GB). This happens automatically.
### 3. Add RSS Feeds
### 3. Add News Sources
```bash
mongosh munich_news
# Connect to database
docker-compose exec mongodb mongosh munich_news
# Paste this into the mongo shell:
db.rss_feeds.insertMany([
{
name: "Süddeutsche Zeitung München",
@@ -65,11 +45,9 @@ db.rss_feeds.insertMany([
])
```
### 4. Add Subscribers
### 4. Add Yourself as Subscriber
```bash
mongosh munich_news
# Still in mongo shell:
db.subscribers.insertOne({
email: "your-email@example.com",
active: true,
@@ -78,90 +56,35 @@ db.subscribers.insertOne({
})
```
### 5. Test It
### 5. Verify Installation
```bash
# Test crawler
# 1. Run the crawler manually to fetch news
docker-compose exec crawler python crawler_service.py 5
# Test newsletter
# 2. Send a test email to yourself
docker-compose exec sender python sender_service.py test your-email@example.com
```
## What Happens Next?
## 🎮 Dashboard Access
The system will automatically:
- **Backend API**: Runs continuously at http://localhost:5001 for tracking and analytics
- **6:00 AM Berlin time**: Crawl news articles
- **7:00 AM Berlin time**: Send newsletter to subscribers
Once running, access the services:
- **Dashboard**: [http://localhost:3000](http://localhost:3000)
- **API**: [http://localhost:5001](http://localhost:5001)
## View Results
## ⏭️ What's Next?
The system is now fully automated:
1. **6:00 AM**: Crawls news and generates AI summaries.
2. **7:00 AM**: Sends the daily newsletter.
### Useful Commands
```bash
# Check articles
mongosh munich_news
db.articles.find().sort({ crawled_at: -1 }).limit(5)
# Check logs
docker-compose logs -f crawler
docker-compose logs -f sender
```
## Common Commands
```bash
# Stop system
# Stop everything
docker-compose down
# Restart system
docker-compose restart
# View logs for a service
docker-compose logs -f crawler
# View logs
docker-compose logs -f
# Rebuild after changes
# Update code & rebuild
docker-compose up -d --build
```
## New Features
### GPU Acceleration (5-10x Faster)
Enable GPU support for faster AI processing:
```bash
./check-gpu.sh # Check if GPU is available
./start-with-gpu.sh # Start with GPU support
```
See [docs/GPU_SETUP.md](docs/GPU_SETUP.md) for details.
### Send Newsletter to All Subscribers
```bash
# Send newsletter to all active subscribers
curl -X POST http://localhost:5001/api/admin/send-newsletter \
-H "Content-Type: application/json" \
-d '{"max_articles": 10}'
```
### Security Features
- ✅ Only Backend API exposed (port 5001)
- ✅ MongoDB internal-only (secure)
- ✅ Ollama internal-only (secure)
- ✅ All services communicate via internal Docker network
## Need Help?
- **Documentation Index**: [docs/INDEX.md](docs/INDEX.md)
- **GPU Setup**: [docs/GPU_SETUP.md](docs/GPU_SETUP.md)
- **API Reference**: [docs/ADMIN_API.md](docs/ADMIN_API.md)
- **Security Guide**: [docs/SECURITY_NOTES.md](docs/SECURITY_NOTES.md)
- **Full Documentation**: [README.md](README.md)
## Next Steps
1.**Enable GPU acceleration** - [docs/GPU_SETUP.md](docs/GPU_SETUP.md)
2. Set up tracking API (optional)
3. Customize newsletter template
4. Add more RSS feeds
5. Monitor engagement metrics
6. Review security settings - [docs/SECURITY_NOTES.md](docs/SECURITY_NOTES.md)
That's it! Your automated news system is running. 🎉