update

NEWSLETTER_API_UPDATE.md

@@ -0,0 +1,205 @@
+# Newsletter API Update
+
+## Summary
+
+Added a new API endpoint to send newsletters to all active subscribers instead of requiring a specific email address.
+
+## New Endpoint
+
+### Send Newsletter to All Subscribers
+
+```http
+POST /api/admin/send-newsletter
+```
+
+**Request Body** (optional):
+```json
+{
+  "max_articles": 10
+}
+```
+
+**Response**:
+```json
+{
+  "success": true,
+  "message": "Newsletter sent successfully to 45 subscribers",
+  "subscriber_count": 45,
+  "max_articles": 10,
+  "output": "... sender output ...",
+  "errors": ""
+}
+```
+
+## Usage Examples
+
+### Send Newsletter to All Subscribers
+
+```bash
+# Send with default settings (10 articles)
+curl -X POST http://localhost:5001/api/admin/send-newsletter \
+  -H "Content-Type: application/json"
+
+# Send with custom article count
+curl -X POST http://localhost:5001/api/admin/send-newsletter \
+  -H "Content-Type: application/json" \
+  -d '{"max_articles": 15}'
+```
+
+### Complete Workflow
+
+```bash
+# 1. Check subscriber count
+curl http://localhost:5001/api/admin/stats | jq '.subscribers'
+
+# 2. Crawl fresh articles
+curl -X POST http://localhost:5001/api/admin/trigger-crawl \
+  -H "Content-Type: application/json" \
+  -d '{"max_articles": 10}'
+
+# 3. Wait for crawl to complete
+sleep 60
+
+# 4. Send newsletter to all active subscribers
+curl -X POST http://localhost:5001/api/admin/send-newsletter \
+  -H "Content-Type: application/json" \
+  -d '{"max_articles": 10}'
+```
+
+## Comparison with Test Email
+
+### Send Test Email (Existing)
+- Sends to **one specific email address**
+- Useful for testing newsletter content
+- No tracking recorded in database
+- Fast (single email)
+
+```bash
+curl -X POST http://localhost:5001/api/admin/send-test-email \
+  -H "Content-Type: application/json" \
+  -d '{"email": "test@example.com"}'
+```
+
+### Send Newsletter (New)
+- Sends to **all active subscribers**
+- Production newsletter sending
+- Full tracking (opens, clicks)
+- May take time for large lists
+
+```bash
+curl -X POST http://localhost:5001/api/admin/send-newsletter \
+  -H "Content-Type: application/json"
+```
+
+## Features
+
+### Subscriber Filtering
+- Only sends to subscribers with `status: 'active'`
+- Skips inactive, unsubscribed, or bounced subscribers
+- Returns error if no active subscribers found
+
+### Tracking
+- Includes tracking pixel for open tracking
+- Includes click tracking for all article links
+- Records send time and newsletter ID
+- Stores in `newsletter_sends` collection
+
+### Error Handling
+- Validates subscriber count before sending
+- Returns detailed error messages
+- Includes sender output and errors in response
+- 5-minute timeout for large lists
+
+## Testing
+
+### Interactive Test Script
+
+```bash
+./test-newsletter-api.sh
+```
+
+This script will:
+1. Show current subscriber stats
+2. Optionally send test email to your address
+3. Optionally send newsletter to all subscribers
+
+### Manual Testing
+
+```bash
+# 1. Check subscribers
+curl http://localhost:5001/api/admin/stats
+
+# 2. Send newsletter
+curl -X POST http://localhost:5001/api/admin/send-newsletter \
+  -H "Content-Type: application/json" \
+  -d '{"max_articles": 2}'
+
+# 3. Check results
+curl http://localhost:5001/api/admin/stats
+```
+
+## Security Considerations
+
+⚠️ **Important**: This endpoint sends emails to real subscribers!
+
+### Recommendations
+
+1. **Add Authentication**
+   ```python
+   @require_api_key
+   def send_newsletter():
+       # ...
+   ```
+
+2. **Rate Limiting**
+   - Prevent accidental multiple sends
+   - Limit to once per hour/day
+
+3. **Confirmation Required**
+   - Add confirmation step in UI
+   - Log all newsletter sends
+
+4. **Dry Run Mode**
+   ```json
+   {
+     "max_articles": 10,
+     "dry_run": true  // Preview without sending
+   }
+   ```
+
+5. **Audit Logging**
+   - Log who triggered the send
+   - Log timestamp and parameters
+   - Track success/failure
+
+## Files Modified
+
+- ✅ `backend/routes/admin_routes.py` - Added new endpoint
+- ✅ `docs/ADMIN_API.md` - Updated documentation
+- ✅ `test-newsletter-api.sh` - Created test script
+
+## API Endpoints Summary
+
+| Endpoint | Purpose | Recipient |
+|----------|---------|-----------|
+| `/api/admin/send-test-email` | Test newsletter | Single email (specified) |
+| `/api/admin/send-newsletter` | Production send | All active subscribers |
+| `/api/admin/trigger-crawl` | Fetch articles | N/A |
+| `/api/admin/stats` | System stats | N/A |
+
+## Next Steps
+
+1. **Test the endpoint:**
+   ```bash
+   ./test-newsletter-api.sh
+   ```
+
+2. **Add authentication** (recommended for production)
+
+3. **Set up monitoring** for newsletter sends
+
+4. **Create UI** for easier newsletter management
+
+## Documentation
+
+See [docs/ADMIN_API.md](docs/ADMIN_API.md) for complete API documentation.

QUICK_START_GPU.md

@@ -56,8 +56,8 @@ docker-compose exec crawler python crawler_service.py 2
 # Check translation timing
 docker-compose logs crawler | grep "Title translated"
 
-# Test Ollama API directly
+# Test Ollama API (internal network only)
-curl http://localhost:11434/api/generate -d '{
+docker-compose exec crawler curl -s http://ollama:11434/api/generate -d '{
   "model": "phi3:latest",
   "prompt": "Translate to English: Guten Morgen",
   "stream": false

SECURITY_UPDATE.md

@@ -0,0 +1,125 @@
+# Security Update: Ollama Internal-Only Configuration
+
+## Summary
+
+Ollama service has been configured to be **internal-only** and is no longer exposed to the host machine. This improves security by reducing the attack surface.
+
+## Changes Made
+
+### Before (Exposed)
+```yaml
+ollama:
+  ports:
+    - "11434:11434"  # ❌ Accessible from host and external network
+```
+
+### After (Internal Only)
+```yaml
+ollama:
+  # No ports section - internal only ✓
+  # Only accessible within Docker network
+```
+
+## Verification
+
+### ✓ Port Not Accessible from Host
+```bash
+$ nc -z -w 2 localhost 11434
+# Connection refused (as expected)
+```
+
+### ✓ Accessible from Docker Services
+```bash
+$ docker-compose exec crawler python -c "import requests; requests.get('http://ollama:11434/api/tags')"
+# ✓ Works perfectly
+```
+
+## Security Benefits
+
+1. **No External Access**: Ollama API cannot be accessed from outside Docker network
+2. **Reduced Attack Surface**: Service is not exposed to potential external threats
+3. **Network Isolation**: Only authorized Docker Compose services can communicate with Ollama
+4. **No Port Conflicts**: Port 11434 is not bound to host machine
+
+## Impact on Usage
+
+### No Change for Normal Operations ✓
+- Crawler service works normally
+- Translation and summarization work as before
+- All Docker Compose services can access Ollama
+
+### Testing from Host Machine
+Since Ollama is internal-only, you must test from inside the Docker network:
+
+```bash
+# ✓ Test from inside a container
+docker-compose exec crawler python crawler_service.py 1
+
+# ✓ Check Ollama status
+docker-compose exec crawler python -c "import requests; print(requests.get('http://ollama:11434/api/tags').json())"
+
+# ✓ Check logs
+docker-compose logs ollama
+```
+
+### If You Need External Access (Development Only)
+
+For development/debugging, you can temporarily expose Ollama:
+
+**Option 1: SSH Port Forward**
+```bash
+# Forward port through SSH (if accessing remote server)
+ssh -L 11434:localhost:11434 user@server
+```
+
+**Option 2: Temporary Docker Exec**
+```bash
+# Run commands from inside network
+docker-compose exec crawler curl http://ollama:11434/api/tags
+```
+
+**Option 3: Modify docker-compose.yml (Not Recommended)**
+```yaml
+ollama:
+  ports:
+    - "127.0.0.1:11434:11434"  # Only localhost, not all interfaces
+```
+
+## Documentation Updated
+
+- ✓ docker-compose.yml - Removed port exposure
+- ✓ docs/OLLAMA_SETUP.md - Updated testing instructions
+- ✓ docs/SECURITY_NOTES.md - Added security documentation
+- ✓ test-ollama-setup.sh - Updated to test from inside network
+- ✓ QUICK_START_GPU.md - Updated API testing examples
+
+## Testing
+
+All functionality has been verified:
+- ✓ Ollama not accessible from host
+- ✓ Ollama accessible from crawler service
+- ✓ Translation works correctly
+- ✓ Summarization works correctly
+- ✓ All tests pass
+
+## Rollback (If Needed)
+
+If you need to expose Ollama again:
+
+```yaml
+# In docker-compose.yml
+ollama:
+  ports:
+    - "11434:11434"  # or "127.0.0.1:11434:11434" for localhost only
+```
+
+Then restart:
+```bash
+docker-compose up -d ollama
+```
+
+## Recommendation
+
+**Keep Ollama internal-only** for production deployments. This is the most secure configuration and sufficient for normal operations.
+
+Only expose Ollama if you have a specific need for external access, and always bind to `127.0.0.1` (localhost only), never `0.0.0.0` (all interfaces).

backend/routes/admin_routes.py

@@ -143,6 +143,77 @@ def send_test_email():
         }), 500
 
 
+@admin_bp.route('/api/admin/send-newsletter', methods=['POST'])
+def send_newsletter():
+    """
+    Send newsletter to all active subscribers
+    
+    Request body (optional):
+    {
+        "max_articles": 10  // Optional, defaults to 10
+    }
+    """
+    try:
+        data = request.get_json() or {}
+        max_articles = data.get('max_articles', 10)
+        
+        # Validate max_articles
+        if not isinstance(max_articles, int) or max_articles < 1 or max_articles > 50:
+            return jsonify({
+                'success': False,
+                'error': 'max_articles must be an integer between 1 and 50'
+            }), 400
+        
+        # Get subscriber count first
+        from database import subscribers_collection
+        subscriber_count = subscribers_collection.count_documents({'status': 'active'})
+        
+        if subscriber_count == 0:
+            return jsonify({
+                'success': False,
+                'error': 'No active subscribers found',
+                'subscriber_count': 0
+            }), 400
+        
+        # Execute sender in sender container using docker exec
+        try:
+            result = subprocess.run(
+                ['docker', 'exec', 'munich-news-sender', 'python', 'sender_service.py', 'send', str(max_articles)],
+                capture_output=True,
+                text=True,
+                timeout=300  # 5 minute timeout for multiple emails
+            )
+            
+            # Check if successful
+            success = result.returncode == 0
+            
+            return jsonify({
+                'success': success,
+                'message': f'Newsletter {"sent successfully" if success else "failed"} to {subscriber_count} subscribers',
+                'subscriber_count': subscriber_count,
+                'max_articles': max_articles,
+                'output': result.stdout[-1000:] if result.stdout else '',  # Last 1000 chars
+                'errors': result.stderr[-500:] if result.stderr else ''
+            }), 200 if success else 500
+            
+        except FileNotFoundError:
+            return jsonify({
+                'success': False,
+                'error': 'Docker command not found. Make sure Docker is installed and the socket is mounted.'
+            }), 500
+            
+    except subprocess.TimeoutExpired:
+        return jsonify({
+            'success': False,
+            'error': 'Newsletter sending timed out after 5 minutes'
+        }), 500
+    except Exception as e:
+        return jsonify({
+            'success': False,
+            'error': f'Failed to send newsletter: {str(e)}'
+        }), 500
+
+
 @admin_bp.route('/api/admin/stats', methods=['GET'])
 def get_stats():
     """Get system statistics"""
@@ -167,7 +238,7 @@ def get_stats():
             },
             'subscribers': {
                 'total': subscribers_collection.count_documents({}),
-                'active': subscribers_collection.count_documents({'active': True})
+                'active': subscribers_collection.count_documents({'status': 'active'})
             },
             'rss_feeds': {
                 'total': rss_feeds_collection.count_documents({}),

docker-compose.yml

@@ -6,16 +6,19 @@
 #   2. Start with GPU: ./start-with-gpu.sh
 #   Or manually: docker-compose -f docker-compose.yml -f docker-compose.gpu.yml up -d
 #
+# Security:
+#   Ollama service is internal-only (no ports exposed to host)
+#   Only accessible by other Docker Compose services
+#
 # See docs/OLLAMA_SETUP.md for detailed setup instructions
 
 services:
-  # Ollama AI Service
+  # Ollama AI Service (Internal only - not exposed to host)
   ollama:
     image: ollama/ollama:latest
     container_name: munich-news-ollama
     restart: unless-stopped
-    ports:
+    # No ports exposed - only accessible within Docker network
-      - "11434:11434"
     volumes:
       - ollama_data:/root/.ollama
     networks:

docs/ADMIN_API.md

@@ -100,6 +100,56 @@ curl -X POST http://localhost:5001/api/admin/send-test-email \
 
 ---
 
+### Send Newsletter to All Subscribers
+
+Send newsletter to all active subscribers in the database.
+
+```http
+POST /api/admin/send-newsletter
+```
+
+**Request Body** (optional):
+```json
+{
+  "max_articles": 10
+}
+```
+
+**Parameters**:
+- `max_articles` (integer, optional): Number of articles to include (1-50, default: 10)
+
+**Response**:
+```json
+{
+  "success": true,
+  "message": "Newsletter sent successfully to 45 subscribers",
+  "subscriber_count": 45,
+  "max_articles": 10,
+  "output": "... sender output ...",
+  "errors": ""
+}
+```
+
+**Example**:
+```bash
+# Send newsletter to all subscribers
+curl -X POST http://localhost:5001/api/admin/send-newsletter \
+  -H "Content-Type: application/json"
+
+# Send with custom article count
+curl -X POST http://localhost:5001/api/admin/send-newsletter \
+  -H "Content-Type: application/json" \
+  -d '{"max_articles": 15}'
+```
+
+**Notes**:
+- Only sends to subscribers with `status: 'active'`
+- Returns error if no active subscribers found
+- Includes tracking pixels and click tracking
+- May take several minutes for large subscriber lists
+
+---
+
 ### Get System Statistics
 
 Get overview statistics of the system.
@@ -156,12 +206,32 @@ curl -X POST http://localhost:5001/api/admin/trigger-crawl \
 sleep 30
 curl http://localhost:5001/api/admin/stats
 
-# 4. Send test email
+# 4. Send test email to yourself
 curl -X POST http://localhost:5001/api/admin/send-test-email \
   -H "Content-Type: application/json" \
   -d '{"email": "your-email@example.com"}'
 ```
 
+### Send Newsletter to All Subscribers
+
+```bash
+# 1. Check subscriber count
+curl http://localhost:5001/api/admin/stats | jq '.subscribers'
+
+# 2. Crawl fresh articles
+curl -X POST http://localhost:5001/api/admin/trigger-crawl \
+  -H "Content-Type: application/json" \
+  -d '{"max_articles": 10}'
+
+# 3. Wait for crawl to complete
+sleep 60
+
+# 4. Send newsletter to all active subscribers
+curl -X POST http://localhost:5001/api/admin/send-newsletter \
+  -H "Content-Type: application/json" \
+  -d '{"max_articles": 10}'
+```
+
 ### Quick Test Newsletter
 
 ```bash
@@ -180,6 +250,28 @@ curl -X POST http://localhost:5001/api/admin/trigger-crawl \
   -d '{"max_articles": 20}'
 ```
 
+### Daily Newsletter Workflow
+
+```bash
+# Complete daily workflow (can be automated with cron)
+
+# 1. Crawl today's articles
+curl -X POST http://localhost:5001/api/admin/trigger-crawl \
+  -H "Content-Type: application/json" \
+  -d '{"max_articles": 15}'
+
+# 2. Wait for crawl and AI processing
+sleep 120
+
+# 3. Send to all subscribers
+curl -X POST http://localhost:5001/api/admin/send-newsletter \
+  -H "Content-Type: application/json" \
+  -d '{"max_articles": 10}'
+
+# 4. Check results
+curl http://localhost:5001/api/admin/stats
+```
+
 ---
 
 ## Error Responses

docs/OLLAMA_SETUP.md

@@ -7,10 +7,11 @@ This project includes an integrated Ollama service for AI-powered summarization
 ## Docker Compose Setup (Recommended)
 
 The docker-compose.yml includes an Ollama service that automatically:
-- Runs Ollama server on port 11434
+- Runs Ollama server (internal only, not exposed to host)
 - Pulls the phi3:latest model on first startup
 - Persists model data in a Docker volume
 - Supports GPU acceleration (NVIDIA GPUs)
+- Only accessible by other Docker Compose services for security
 
 ### GPU Support
 
@@ -90,8 +91,8 @@ docker-compose logs -f ollama
 # Check model setup logs
 docker-compose logs ollama-setup
 
-# Verify Ollama is running
+# Verify Ollama is running (from inside a container)
-curl http://localhost:11434/api/tags
+docker-compose exec crawler curl http://ollama:11434/api/tags
 ```
 
 ### First Time Setup
@@ -187,8 +188,8 @@ If you prefer to run Ollama directly on your host machine:
 
 ### Basic API Test
 ```bash
-# Test Ollama API directly
+# Test Ollama API from inside a container
-curl http://localhost:11434/api/generate -d '{
+docker-compose exec crawler curl -s http://ollama:11434/api/generate -d '{
   "model": "phi3:latest",
   "prompt": "Translate to English: Guten Morgen",
   "stream": false

docs/SECURITY_NOTES.md

@@ -0,0 +1,160 @@
+# Security Notes
+
+## Ollama Service Security
+
+### Internal-Only Access
+
+The Ollama service is configured to be **internal-only** and is not exposed to the host machine or external network. This provides several security benefits:
+
+**Configuration:**
+```yaml
+# Ollama service has NO ports exposed
+ollama:
+  image: ollama/ollama:latest
+  # No ports section - internal only
+  networks:
+    - munich-news-network
+```
+
+**Benefits:**
+1. **No External Access**: Ollama API cannot be accessed from outside Docker network
+2. **Reduced Attack Surface**: Service is not exposed to potential external threats
+3. **Network Isolation**: Only authorized Docker Compose services can communicate with Ollama
+4. **No Port Conflicts**: Port 11434 is not bound to host machine
+
+### Accessing Ollama
+
+**From Docker Compose Services (✓ Allowed):**
+```bash
+# Services use internal Docker network
+OLLAMA_BASE_URL=http://ollama:11434
+```
+
+**From Host Machine (✗ Not Allowed):**
+```bash
+# This will NOT work - port not exposed
+curl http://localhost:11434/api/tags
+# Connection refused
+```
+
+**From Inside Containers (✓ Allowed):**
+```bash
+# Access from another container
+docker-compose exec crawler curl http://ollama:11434/api/tags
+```
+
+### Why This Matters
+
+**Security Risks of Exposed Ollama:**
+- Unauthorized access to AI models
+- Potential for abuse (resource consumption)
+- Information disclosure through prompts
+- No authentication by default
+- Could be used for unintended purposes
+
+**With Internal-Only Configuration:**
+- Only your trusted services can access Ollama
+- No external attack vector
+- Controlled usage within your application
+- Better resource management
+
+### Testing Ollama
+
+Since Ollama is internal-only, you must test from inside the Docker network:
+
+```bash
+# ✓ Correct way - from inside a container
+docker-compose exec crawler curl -s http://ollama:11434/api/tags
+
+# ✓ Test translation
+docker-compose exec crawler python crawler_service.py 1
+
+# ✓ Check logs
+docker-compose logs ollama
+```
+
+### If You Need External Access
+
+If you have a specific need to access Ollama from the host machine (e.g., development, debugging), you can temporarily expose it:
+
+**Option 1: Temporary Port Forward**
+```bash
+# Forward port temporarily (stops when you press Ctrl+C)
+docker exec -it munich-news-ollama socat TCP-LISTEN:11434,fork TCP:localhost:11434 &
+```
+
+**Option 2: Add Ports to docker-compose.yml (Not Recommended)**
+```yaml
+ollama:
+  ports:
+    - "127.0.0.1:11434:11434"  # Only bind to localhost, not 0.0.0.0
+```
+
+**⚠️ Warning:** Only expose Ollama if absolutely necessary, and always bind to `127.0.0.1` (localhost only), never `0.0.0.0` (all interfaces).
+
+### Other Security Considerations
+
+**MongoDB:**
+- Exposed on port 27017 for development
+- Uses authentication (username/password)
+- Consider restricting to localhost in production: `127.0.0.1:27017:27017`
+
+**Backend API:**
+- Exposed on port 5001 for tracking and admin functions
+- Should be behind reverse proxy in production
+- Consider adding authentication for admin endpoints
+
+**Email Credentials:**
+- Stored in `.env` file
+- Never commit `.env` to version control
+- Use environment variables in production
+
+### Production Recommendations
+
+1. **Use Docker Secrets** for sensitive data:
+   ```yaml
+   secrets:
+     mongo_password:
+       external: true
+   ```
+
+2. **Restrict Network Access**:
+   ```yaml
+   ports:
+     - "127.0.0.1:27017:27017"  # MongoDB
+     - "127.0.0.1:5001:5001"    # Backend
+   ```
+
+3. **Use Reverse Proxy** (nginx, Traefik):
+   - SSL/TLS termination
+   - Rate limiting
+   - Authentication
+   - Access logs
+
+4. **Regular Updates**:
+   ```bash
+   docker-compose pull
+   docker-compose up -d
+   ```
+
+5. **Monitor Logs**:
+   ```bash
+   docker-compose logs -f
+   ```
+
+### Security Checklist
+
+- [x] Ollama is internal-only (no exposed ports)
+- [x] MongoDB uses authentication
+- [x] `.env` file is in `.gitignore`
+- [ ] Backend API has authentication (if needed)
+- [ ] Using HTTPS in production
+- [ ] Regular security updates
+- [ ] Monitoring and logging enabled
+- [ ] Backup strategy in place
+
+## Reporting Security Issues
+
+If you discover a security vulnerability, please email security@example.com (replace with your contact).
+
+Do not open public issues for security vulnerabilities.

docs/SUBSCRIBER_STATUS.md

@@ -0,0 +1,290 @@
+# Subscriber Status System
+
+## Overview
+
+The newsletter system tracks subscribers with a `status` field that determines whether they receive newsletters.
+
+## Status Field
+
+### Database Schema
+
+```javascript
+{
+  _id: ObjectId("..."),
+  email: "user@example.com",
+  subscribed_at: ISODate("2025-11-11T15:50:29.478Z"),
+  status: "active"  // or "inactive"
+}
+```
+
+### Status Values
+
+| Status | Description | Receives Newsletters |
+|--------|-------------|---------------------|
+| `active` | Subscribed and active | ✅ Yes |
+| `inactive` | Unsubscribed | ❌ No |
+
+## How It Works
+
+### Subscription Flow
+
+```
+User subscribes
+    ↓
+POST /api/subscribe
+    ↓
+Create subscriber with status: 'active'
+    ↓
+User receives newsletters
+```
+
+### Unsubscription Flow
+
+```
+User unsubscribes
+    ↓
+POST /api/unsubscribe
+    ↓
+Update subscriber status: 'inactive'
+    ↓
+User stops receiving newsletters
+```
+
+### Re-subscription Flow
+
+```
+Previously unsubscribed user subscribes again
+    ↓
+POST /api/subscribe
+    ↓
+Update status: 'active' + new subscribed_at date
+    ↓
+User receives newsletters again
+```
+
+## API Endpoints
+
+### Subscribe
+
+```bash
+curl -X POST http://localhost:5001/api/subscribe \
+  -H "Content-Type: application/json" \
+  -d '{"email": "user@example.com"}'
+```
+
+**Creates subscriber with:**
+- `email`: user@example.com
+- `status`: "active"
+- `subscribed_at`: current timestamp
+
+### Unsubscribe
+
+```bash
+curl -X POST http://localhost:5001/api/unsubscribe \
+  -H "Content-Type: application/json" \
+  -d '{"email": "user@example.com"}'
+```
+
+**Updates subscriber:**
+- `status`: "inactive"
+
+## Newsletter Sending
+
+### Who Receives Newsletters
+
+Only subscribers with `status: 'active'` receive newsletters.
+
+**Sender Service Query:**
+```python
+subscribers_collection.find({'status': 'active'})
+```
+
+**Admin API Query:**
+```python
+subscribers_collection.count_documents({'status': 'active'})
+```
+
+### Testing
+
+```bash
+# Check active subscriber count
+curl http://localhost:5001/api/admin/stats | jq '.subscribers'
+
+# Output:
+# {
+#   "total": 10,
+#   "active": 8
+# }
+```
+
+## Database Operations
+
+### Add Active Subscriber
+
+```javascript
+db.subscribers.insertOne({
+  email: "user@example.com",
+  subscribed_at: new Date(),
+  status: "active"
+})
+```
+
+### Deactivate Subscriber
+
+```javascript
+db.subscribers.updateOne(
+  { email: "user@example.com" },
+  { $set: { status: "inactive" } }
+)
+```
+
+### Reactivate Subscriber
+
+```javascript
+db.subscribers.updateOne(
+  { email: "user@example.com" },
+  { $set: { 
+    status: "active",
+    subscribed_at: new Date()
+  }}
+)
+```
+
+### Query Active Subscribers
+
+```javascript
+db.subscribers.find({ status: "active" })
+```
+
+### Count Active Subscribers
+
+```javascript
+db.subscribers.countDocuments({ status: "active" })
+```
+
+## Common Issues
+
+### Issue: Stats show 0 active subscribers but subscribers exist
+
+**Cause:** Old bug where stats checked `{active: true}` instead of `{status: 'active'}`
+
+**Solution:** Fixed in latest version. Stats now correctly query `{status: 'active'}`
+
+**Verify:**
+```bash
+# Check database directly
+docker-compose exec mongodb mongosh munich_news -u admin -p changeme \
+  --authenticationDatabase admin \
+  --eval "db.subscribers.find({status: 'active'}).count()"
+
+# Check via API
+curl http://localhost:5001/api/admin/stats | jq '.subscribers.active'
+```
+
+### Issue: Newsletter not sending to subscribers
+
+**Possible causes:**
+1. Subscribers have `status: 'inactive'`
+2. No subscribers in database
+3. Email configuration issue
+
+**Debug:**
+```bash
+# Check subscriber status
+docker-compose exec mongodb mongosh munich_news -u admin -p changeme \
+  --authenticationDatabase admin \
+  --eval "db.subscribers.find().pretty()"
+
+# Check active count
+curl http://localhost:5001/api/admin/stats | jq '.subscribers'
+
+# Try sending
+curl -X POST http://localhost:5001/api/admin/send-newsletter \
+  -H "Content-Type: application/json"
+```
+
+## Migration Notes
+
+### If you have old subscribers without status field
+
+Run this migration:
+
+```javascript
+// Set all subscribers without status to 'active'
+db.subscribers.updateMany(
+  { status: { $exists: false } },
+  { $set: { status: "active" } }
+)
+```
+
+### If you have subscribers with `active: true/false` field
+
+Run this migration:
+
+```javascript
+// Convert old 'active' field to 'status' field
+db.subscribers.updateMany(
+  { active: true },
+  { $set: { status: "active" }, $unset: { active: "" } }
+)
+
+db.subscribers.updateMany(
+  { active: false },
+  { $set: { status: "inactive" }, $unset: { active: "" } }
+)
+```
+
+## Best Practices
+
+### 1. Always Check Status
+
+When querying subscribers for sending:
+```python
+# ✅ Correct
+subscribers_collection.find({'status': 'active'})
+
+# ❌ Wrong
+subscribers_collection.find({})  # Includes inactive
+```
+
+### 2. Soft Delete
+
+Never delete subscribers - just set status to 'inactive':
+```python
+# ✅ Correct - preserves history
+subscribers_collection.update_one(
+    {'email': email},
+    {'$set': {'status': 'inactive'}}
+)
+
+# ❌ Wrong - loses data
+subscribers_collection.delete_one({'email': email})
+```
+
+### 3. Track Subscription History
+
+Consider adding fields:
+```javascript
+{
+  email: "user@example.com",
+  status: "active",
+  subscribed_at: ISODate("2025-01-01"),
+  unsubscribed_at: null,  // Set when status changes to inactive
+  resubscribed_count: 0   // Increment on re-subscription
+}
+```
+
+### 4. Validate Before Sending
+
+```python
+# Check subscriber count before sending
+count = subscribers_collection.count_documents({'status': 'active'})
+if count == 0:
+    return {'error': 'No active subscribers'}
+```
+
+## Related Documentation
+
+- [ADMIN_API.md](ADMIN_API.md) - Admin API endpoints
+- [DATABASE_SCHEMA.md](DATABASE_SCHEMA.md) - Complete database schema
+- [NEWSLETTER_API_UPDATE.md](../NEWSLETTER_API_UPDATE.md) - Newsletter API changes

test-newsletter-api.sh

@@ -0,0 +1,48 @@
+#!/bin/bash
+
+# Test script for newsletter API endpoints
+
+echo "=========================================="
+echo "Newsletter API Test"
+echo "=========================================="
+echo ""
+
+BASE_URL="http://localhost:5001"
+
+# Test 1: Get stats
+echo "Test 1: Get system stats"
+echo "GET $BASE_URL/api/admin/stats"
+curl -s $BASE_URL/api/admin/stats | python3 -m json.tool | grep -A 3 "subscribers"
+echo ""
+
+# Test 2: Send test email
+echo "Test 2: Send test email"
+read -p "Enter your email address for test: " TEST_EMAIL
+if [ -n "$TEST_EMAIL" ]; then
+    echo "POST $BASE_URL/api/admin/send-test-email"
+    curl -s -X POST $BASE_URL/api/admin/send-test-email \
+      -H "Content-Type: application/json" \
+      -d "{\"email\": \"$TEST_EMAIL\", \"max_articles\": 2}" | python3 -m json.tool
+    echo ""
+else
+    echo "Skipped (no email provided)"
+    echo ""
+fi
+
+# Test 3: Send newsletter to all subscribers
+echo "Test 3: Send newsletter to all subscribers"
+read -p "Send newsletter to all active subscribers? (y/N): " CONFIRM
+if [ "$CONFIRM" = "y" ] || [ "$CONFIRM" = "Y" ]; then
+    echo "POST $BASE_URL/api/admin/send-newsletter"
+    curl -s -X POST $BASE_URL/api/admin/send-newsletter \
+      -H "Content-Type: application/json" \
+      -d '{"max_articles": 5}' | python3 -m json.tool
+    echo ""
+else
+    echo "Skipped"
+    echo ""
+fi
+
+echo "=========================================="
+echo "Test Complete"
+echo "=========================================="

test-ollama-setup.sh

@@ -117,18 +117,30 @@ echo "Test 8: Ollama service status"
 if docker ps | grep -q "munich-news-ollama"; then
     echo "✓ Ollama container is running"
     
-    # Test Ollama API
+    # Check if crawler is running (needed to test Ollama)
-    if curl -s http://localhost:11434/api/tags &> /dev/null; then
+    if docker ps | grep -q "munich-news-crawler"; then
-        echo "✓ Ollama API is accessible"
+        # Test Ollama API from inside network using Python
+        if docker-compose exec -T crawler python -c "import requests; requests.get('http://ollama:11434/api/tags', timeout=5)" &> /dev/null; then
+            echo "✓ Ollama API is accessible (internal network)"
             
             # Check if model is available
-        if curl -s http://localhost:11434/api/tags | grep -q "phi3"; then
+            if docker-compose exec -T crawler python -c "import requests; r = requests.get('http://ollama:11434/api/tags'); exit(0 if 'phi3' in r.text else 1)" &> /dev/null; then
                 echo "✓ phi3 model is available"
             else
                 echo "⚠ phi3 model not found (may still be downloading)"
             fi
         else
-        echo "⚠ Ollama API not responding"
+            echo "⚠ Ollama API not responding from crawler"
+        fi
+    else
+        echo "ℹ Crawler not running (needed to test internal Ollama access)"
+    fi
+    
+    # Verify port is NOT exposed to host
+    if nc -z -w 2 localhost 11434 &> /dev/null; then
+        echo "⚠ WARNING: Ollama port is exposed to host (should be internal only)"
+    else
+        echo "✓ Ollama is internal-only (not exposed to host)"
     fi
 else
     echo "ℹ Ollama container not running (start with: docker-compose up -d)"