dongho/Munich-news
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

cleanup and ui change

Browse Source
This commit is contained in:
Dongho Kim
2025-11-16 20:59:14 +01:00
parent f0e552b1b1
commit 9f167a6534
15 changed files with 165 additions and 155 deletions
Download Patch File Download Diff File Expand all files Collapse all files

34
cleanup.sh Executable file
View File

@@ -0,0 +1,34 @@
#!/bin/bash
# Cleanup unnecessary files from the project
echo "🧹 Cleaning up unnecessary files..."
# Remove Python cache files
echo "Removing Python cache files..."
find . -type d -name "__pycache__" -exec rm -rf {} + 2>/dev/null
find . -type f -name "*.pyc" -delete 2>/dev/null
find . -type f -name "*.pyo" -delete 2>/dev/null
# Remove local virtual environments (should use Docker)
echo "Removing local virtual environments..."
rm -rf backend/env 2>/dev/null
rm -rf news_crawler/env 2>/dev/null
rm -rf news_sender/env 2>/dev/null
rm -rf transport_crawler/env 2>/dev/null
# Remove local database files (should use MongoDB in Docker)
echo "Removing local database files..."
rm -f backend/munich_news.db 2>/dev/null
rm -f backend/*.db 2>/dev/null
# Remove macOS files
echo "Removing macOS files..."
find . -name ".DS_Store" -delete 2>/dev/null
# Remove editor files
echo "Removing editor backup files..."
find . -name "*~" -delete 2>/dev/null
find . -name "*.swp" -delete 2>/dev/null
find . -name "*.swo" -delete 2>/dev/null
echo "✅ Cleanup complete!"

4
docker-compose.yml
View File

@@ -206,6 +206,10 @@ services:
restart: unless-stopped
ports:
- "3000:3000"
# Traefik Configuration (commented out - uncomment when using Traefik)
# Remove port exposure when using Traefik
# ports:
# - "3000:3000"
environment:
- API_URL=http://backend:5001
- PORT=3000

56
fix-categories.js
View File

@@ -1,56 +0,0 @@
// MongoDB script to fix article categories
// Run with: docker exec -i munich-news-mongodb mongosh -u admin -p changeme --authenticationDatabase admin < fix-categories.js
use munich_news
print("=== RSS Feeds and their categories ===");
db.rss_feeds.find({}, {name: 1, category: 1, _id: 0}).forEach(feed => {
print(`${feed.name}: ${feed.category || 'NO CATEGORY'}`);
});
print("\n=== Current article category distribution ===");
db.articles.aggregate([
{$group: {_id: "$category", count: {$sum: 1}}},
{$sort: {count: -1}}
]).forEach(result => {
print(`${result._id || 'null'}: ${result.count} articles`);
});
print("\n=== Fixing null categories ===");
// Update articles based on their RSS feed source
var feedsUpdated = 0;
db.rss_feeds.find().forEach(function(feed) {
if (feed.category) {
var result = db.articles.updateMany(
{source: feed.name, category: null},
{$set: {category: feed.category}}
);
if (result.modifiedCount > 0) {
print(`Updated ${result.modifiedCount} articles from ${feed.name} to category: ${feed.category}`);
feedsUpdated += result.modifiedCount;
}
}
});
// Set remaining null categories to 'general'
var remainingNull = db.articles.updateMany(
{category: null},
{$set: {category: "general"}}
);
if (remainingNull.modifiedCount > 0) {
print(`Set ${remainingNull.modifiedCount} remaining null articles to 'general'`);
}
print(`\nTotal articles updated: ${feedsUpdated + remainingNull.modifiedCount}`);
print("\n=== Updated article category distribution ===");
db.articles.aggregate([
{$group: {_id: "$category", count: {$sum: 1}}},
{$sort: {count: -1}}
]).forEach(result => {
print(`${result._id || 'null'}: ${result.count} articles`);
});
print("\n✓ Done!");

194
news_sender/newsletter_template.html
View File

@@ -83,6 +83,101 @@
</td>
</tr>
<!-- TL;DR Section -->
<tr>
<td style="padding: 30px 40px;">
<h2 style="margin: 0 0 20px 0; font-size: 22px; font-weight: 700; color: #1a1a1a;">
📋 TL;DR - Quick Summary
</h2>
<p style="margin: 0 0 15px 0; font-size: 14px; color: #666666;">
Here's everything in one sentence each:
</p>
{% for section in category_sections %}
{% for article in section.articles %}
<div style="margin-bottom: 12px; padding-left: 20px; position: relative;">
<span style="position: absolute; left: 0; top: 0; color: #667eea; font-weight: 700;">{{ loop.index }}.</span>
<p style="margin: 0; font-size: 14px; line-height: 1.5; color: #333333;">
<strong>{{ article.title_en if article.title_en else article.title }}</strong> —
{{ article.summary.split('.')[0] }}.
</p>
</div>
{% endfor %}
{% endfor %}
</td>
</tr>
{% if transport_disruptions and transport_disruptions|length > 0 %}
<!-- Divider -->
<tr>
<td style="padding: 0 40px;">
<div style="height: 2px; background-color: #e0e0e0;"></div>
</td>
</tr>
<!-- Transport Disruptions Section -->
<tr>
<td style="padding: 30px 40px;">
<h2 style="margin: 0 0 20px 0; font-size: 22px; font-weight: 700; color: #1a1a1a;">
🚆 S-Bahn Disruptions Today
</h2>
<p style="margin: 0 0 20px 0; font-size: 14px; color: #666666;">
Current service disruptions affecting Munich S-Bahn:
</p>
{% for disruption in transport_disruptions %}
<!-- Disruption Card -->
<table role="presentation" width="100%" cellpadding="0" cellspacing="0" border="0" style="margin-bottom: 15px; background-color: #fff8f0; border-left: 4px solid #ff9800; border-radius: 4px;">
<tr>
<td style="padding: 15px 20px;">
<!-- Severity and Lines -->
<p style="margin: 0 0 8px 0; font-size: 13px; color: #666666;">
{{ disruption.severity_icon }} <strong style="color: #000000;">{{ disruption.lines_str }}</strong>
</p>
<!-- Title -->
<p style="margin: 0 0 8px 0; font-size: 15px; font-weight: 700; color: #1a1a1a; line-height: 1.4;">
{{ disruption.title }}
</p>
<!-- Description -->
{% if disruption.description %}
<p style="margin: 0 0 8px 0; font-size: 14px; color: #333333; line-height: 1.5;">
{{ disruption.description }}
</p>
{% endif %}
<!-- Time -->
{% if disruption.start_time_str or disruption.end_time_str %}
<p style="margin: 0; font-size: 13px; color: #666666;">
⏰
{% if disruption.start_time_str %}
From {{ disruption.start_time_str }}
{% endif %}
{% if disruption.end_time_str %}
until {{ disruption.end_time_str }}
{% endif %}
</p>
{% endif %}
</td>
</tr>
</table>
{% endfor %}
<p style="margin: 15px 0 0 0; font-size: 12px; color: #999999; font-style: italic;">
💡 Plan your commute accordingly. Check <a href="https://www.mvg.de" style="color: #667eea; text-decoration: none;">MVG.de</a> for real-time updates.
</p>
</td>
</tr>
{% endif %}
<!-- Divider -->
<tr>
<td style="padding: 0 40px;">
<div style="height: 2px; background-color: #e0e0e0;"></div>
</td>
</tr>
<!-- Category Sections -->
{% for section in category_sections %}
<tr>
@@ -195,39 +290,6 @@
</td>
</tr>
<!-- TL;DR Section -->
<tr>
<td style="padding: 30px 40px;">
<h2 style="margin: 0 0 20px 0; font-size: 22px; font-weight: 700; color: #1a1a1a;">
📋 TL;DR - Quick Summary
</h2>
<p style="margin: 0 0 15px 0; font-size: 14px; color: #666666;">
Here's everything in one sentence each:
</p>
{% for section in category_sections %}
{% for article in section.articles %}
<div style="margin-bottom: 12px; padding-left: 20px; position: relative;">
<span style="position: absolute; left: 0; top: 0; color: #667eea; font-weight: 700;">{{ loop.index }}.</span>
<p style="margin: 0; font-size: 14px; line-height: 1.5; color: #333333;">
<strong>{{ article.title_en if article.title_en else article.title }}</strong> —
{{ article.summary.split('.')[0] }}.
</p>
</div>
{% endfor %}
{% endfor %}
</td>
</tr>
<!-- Divider -->
<tr>
<td style="padding: 0 40px;">
<div style="height: 1px; background-color: #e0e0e0;"></div>
</td>
</tr>
<!-- Summary Box -->
<tr>
<td style="padding: 30px 40px;">
<table role="presentation" width="100%" cellpadding="0" cellspacing="0" border="0" style="background-color: #f8f8f8; border-radius: 8px;">
<tr>
@@ -245,72 +307,6 @@
</tr>
</table>
</td>
</tr>
{% if transport_disruptions and transport_disruptions|length > 0 %}
<!-- Divider -->
<tr>
<td style="padding: 0 40px;">
<div style="height: 2px; background-color: #e0e0e0;"></div>
</td>
</tr>
<!-- Transport Disruptions Section -->
<tr>
<td style="padding: 30px 40px;">
<h2 style="margin: 0 0 20px 0; font-size: 22px; font-weight: 700; color: #1a1a1a;">
🚆 S-Bahn Disruptions Today
</h2>
<p style="margin: 0 0 20px 0; font-size: 14px; color: #666666;">
Current service disruptions affecting Munich S-Bahn:
</p>
{% for disruption in transport_disruptions %}
<!-- Disruption Card -->
<table role="presentation" width="100%" cellpadding="0" cellspacing="0" border="0" style="margin-bottom: 15px; background-color: #fff8f0; border-left: 4px solid #ff9800; border-radius: 4px;">
<tr>
<td style="padding: 15px 20px;">
<!-- Severity and Lines -->
<p style="margin: 0 0 8px 0; font-size: 13px; color: #666666;">
{{ disruption.severity_icon }} <strong style="color: #000000;">{{ disruption.lines_str }}</strong>
</p>
<!-- Title -->
<p style="margin: 0 0 8px 0; font-size: 15px; font-weight: 700; color: #1a1a1a; line-height: 1.4;">
{{ disruption.title }}
</p>
<!-- Description -->
{% if disruption.description %}
<p style="margin: 0 0 8px 0; font-size: 14px; color: #333333; line-height: 1.5;">
{{ disruption.description }}
</p>
{% endif %}
<!-- Time -->
{% if disruption.start_time_str or disruption.end_time_str %}
<p style="margin: 0; font-size: 13px; color: #666666;">
⏰
{% if disruption.start_time_str %}
From {{ disruption.start_time_str }}
{% endif %}
{% if disruption.end_time_str %}
until {{ disruption.end_time_str }}
{% endif %}
</p>
{% endif %}
</td>
</tr>
</table>
{% endfor %}
<p style="margin: 15px 0 0 0; font-size: 12px; color: #999999; font-style: italic;">
💡 Plan your commute accordingly. Check <a href="https://www.mvg.de" style="color: #667eea; text-decoration: none;">MVG.de</a> for real-time updates.
</p>
</td>
</tr>
{% endif %}
<!-- Footer -->
<tr>
<td style="background-color: #1a1a1a; padding: 30px 40px; text-align: center;">

32
scripts/README.md Normal file
View File

@@ -0,0 +1,32 @@
# Utility Scripts
This folder contains utility scripts for testing and managing the Munich News Daily system.
## Available Scripts
### Ollama / GPU
- `setup-ollama-model.sh` - Pull and setup the Ollama model (used by Docker)
- `configure-ollama.sh` - Configure Ollama settings
- `pull-ollama-model.sh` - Manually pull Ollama model
- `start-with-gpu.sh` - Start services with GPU support
- `check-gpu.sh` - Check GPU availability
- `check-gpu-api.sh` - Check GPU via API
- `diagnose-gpu.sh` - Diagnose GPU issues
- `test-ollama-setup.sh` - Test Ollama configuration
### Testing
- `test-mongodb-connectivity.sh` - Test MongoDB connection
- `test-newsletter-api.sh` - Test newsletter API endpoints
- `check-articles.sh` - Check articles in database
## Usage
Make scripts executable:
```bash
chmod +x scripts/*.sh
```
Run a script:
```bash
./scripts/script-name.sh
```

0
check-articles.sh → scripts/check-articles.sh
View File

0
check-gpu-api.sh → scripts/check-gpu-api.sh
View File

0
check-gpu.sh → scripts/check-gpu.sh
View File

0
configure-ollama.sh → scripts/configure-ollama.sh
View File

0
diagnose-gpu.sh → scripts/diagnose-gpu.sh
View File

0
pull-ollama-model.sh → scripts/pull-ollama-model.sh
View File

0
start-with-gpu.sh → scripts/start-with-gpu.sh
View File

0
test-mongodb-connectivity.sh → scripts/test-mongodb-connectivity.sh
View File

0
test-newsletter-api.sh → scripts/test-newsletter-api.sh
View File

0
test-ollama-setup.sh → scripts/test-ollama-setup.sh
View File