Transitioning a Flask Blog to a Hierarchical Directory Structure

In this conversation, I explored how to improve the directory structure of a Flask-based blog using Flask-FlatPages, moving from a flat structure (e.g., pages/my-first-post.md) to a hierarchical one (e.g., pages/YYYY/MM/DD/slug.md). The goal was to enhance organization and scalability while addressing challenges like view count preservation, legacy URL redirects, and a hybrid approach for gradual transition. Below is a summary of the key points and solutions.

Initial Inquiry: Adopting a Hierarchical Structure

I asked how to transition my blog’s flat pages directory to a hierarchical structure based on publication dates. The proposed structure organizes posts as pages/YYYY/MM/DD/slug.md, a common practice in blogging platforms like Jekyll. The existing blog.py, blog.html, and post.html were found to support this structure without changes, as Flask-FlatPages handles nested paths natively. Key steps included:

Reorganizing Files: Move existing Markdown files to date-based directories matching their date metadata (e.g., date: 2025-08-25 moves my-first-post.md to 2025/08/25/my-first-post.md).
Updating View Counts: Ensure the view_counter module’s SQLite database (views.db) reflects new paths (e.g., from my-first-post to 2025/08/25/my-first-post).
SEO Considerations: Implement redirects for legacy URLs to preserve search engine rankings.

Preserving View Counts in SQLite

I sought clarification on updating the SQLite database to handle new paths without losing view counts. A script (migrate_view_counts.py) was provided to:

Parse each Markdown file’s date metadata without using the frontmatter package.
Map old paths (e.g., my-first-post) to new paths (e.g., 2025/08/25/my-first-post).

Update the views table’s path column while preserving the count column.

import os
import sqlite3
from datetime import datetime

# Configuration - adjust these to match your setup
PAGES_DIR = 'pages'          # Directory with flat .md files
DB_FILE = 'views.db'         # SQLite database file
TABLE_NAME = 'post_views'    # Table name in the database
PATH_COLUMN = 'path'         # Column for the path
COUNT_COLUMN = 'view_count'  # Column for the view count

def extract_date_from_md(file_path):
    """
    Manually parse the front matter to extract the 'date' field.
    Assumes format: date: YYYY-MM-DD
    """
    try:
        with open(file_path, 'r', encoding='utf-8') as f:
            lines = f.readlines()

        in_frontmatter = False
        date = None
        for line in lines:
            stripped = line.strip()
            if stripped == '---':
                if not in_frontmatter:
                    in_frontmatter = True
                else:
                    break  # End of front matter
            elif in_frontmatter and stripped.startswith('date:'):
                date_str = stripped.split(':', 1)[1].strip()
                try:
                    date = datetime.strptime(date_str, '%Y-%m-%d')
                except ValueError:
                    print(f"Warning: Invalid date format in {file_path}: {date_str}")
                    date = None
                break  # Found date, no need to continue
        return date
    except Exception as e:
        print(f"Error reading {file_path}: {e}")
        return None

def get_new_path(slug, date):
    """
    Compute new hierarchical path based on date.
    """
    if date:
        return date.strftime('%Y/%m/%d') + '/' + slug
    else:
        print(f"Warning: No date for {slug}, using old path as fallback.")
        return slug

# Step 1: Build old_path -> new_path mappings
mappings = {}
for filename in os.listdir(PAGES_DIR):
    if filename.endswith('.md'):
        file_path = os.path.join(PAGES_DIR, filename)
        slug = os.path.splitext(filename)[0]  # Old path is the slug
        date = extract_date_from_md(file_path)
        new_path = get_new_path(slug, date)
        mappings[slug] = new_path

print("Generated mappings:")
for old, new in mappings.items():
    print(f"  {old} -> {new}")

# Step 2: Update the database
try:
    conn = sqlite3.connect(DB_FILE)
    cursor = conn.cursor()

    # Verify table exists (optional, for safety)
    cursor.execute(f"SELECT name FROM sqlite_master WHERE type='table' AND name='{TABLE_NAME}'")
    if not cursor.fetchone():
        raise ValueError(f"Table '{TABLE_NAME}' not found in {DB_FILE}")

    for old_path, new_path in mappings.items():
        # Check if old_path exists
        cursor.execute(f"SELECT {COUNT_COLUMN} FROM {TABLE_NAME} WHERE {PATH_COLUMN} = ?", (old_path,))
        row = cursor.fetchone()
        if row:
            # Update path
            cursor.execute(
                f"UPDATE {TABLE_NAME} SET {PATH_COLUMN} = ? WHERE {PATH_COLUMN} = ?",
                (new_path, old_path)
            )
            print(f"Updated {old_path} to {new_path}")
        else:
            print(f"Skipping {old_path}: No view count entry found")

    conn.commit()
    print("Database update complete.")
except sqlite3.Error as e:
    print(f"SQLite error: {e}")
finally:
    if conn:
        conn.close()

This script ensures view counts are preserved during the transition.

Rewriting the Reorganization Script

I requested a version of the reorganize_posts script without the frontmatter package. The updated script (reorganize_posts.py) manually parses Markdown front matter to extract date and moves files to their corresponding hierarchical directories.

import os
from datetime import datetime

def extract_date_from_md(file_path):
    """
    Manually parse the front matter to extract the 'date' field.
    Assumes format: date: YYYY-MM-DD
    """
    try:
        with open(file_path, 'r', encoding='utf-8') as f:
            lines = f.readlines()

        in_frontmatter = False
        date = None
        for line in lines:
            stripped = line.strip()
            if stripped == '---':
                if not in_frontmatter:
                    in_frontmatter = True
                else:
                    break  # End of front matter
            elif in_frontmatter and stripped.startswith('date:'):
                date_str = stripped.split(':', 1)[1].strip()
                try:
                    date = datetime.strptime(date_str, '%Y-%m-%d')
                except ValueError:
                    print(f"Warning: Invalid date format in {file_path}: {date_str}")
                    date = None
                break  # Found date, no need to continue
        return date
    except Exception as e:
        print(f"Error reading {file_path}: {e}")
        return None

def reorganize_posts(pages_dir="pages"):
    for root, _, files in os.walk(pages_dir):
        for file in files:
            if file.endswith(".md"):
                file_path = os.path.join(root, file)
                date = extract_date_from_md(file_path)
                if date:
                    new_dir = os.path.join(pages_dir, date.strftime("%Y/%m/%d"))
                    os.makedirs(new_dir, exist_ok=True)
                    new_path = os.path.join(new_dir, file)
                    os.rename(file_path, new_path)
                    print(f"Moved {file_path} to {new_path}")

# Move md files into a hierarchical directory
reorganize_posts()

Per-File Redirect Functions

I asked if per-file redirect functions (e.g., redirect_legacy_my_first_post) would be effective for a small number of posts. This approach was deemed viable for 5–10 posts, adding routes like:

@app.route('/my-first-post')
def redirect_legacy_my_first_post():
    logger.debug("Redirecting legacy URL: /my-first-post")
    return redirect("/2025/08/25/my-first-post", code=301)

However, it’s less scalable than the SQLite solution and requires manual updates.

Hybrid Directory Structure

Finally, I explored keeping existing posts in a flat structure while using a hierarchical structure for new posts. This hybrid approach was confirmed to work without code changes, as Flask-FlatPages handles both structures.

Key Takeaways

The hybrid approach (flat for existing posts, hierarchical for new ones) is ideal for gradual transition, requiring no changes to blog.py, blog.html, post.html, or 404.html. The SQLite legacy_mappings solution is recommended for scalable redirects if existing posts are later migrated. Always validate metadata and back up files before changes.

References

Flask-FlatPages Documentation: https://flask-flatpages.readthedocs.io/
Flask Documentation: https://flask.palletsprojects.com/
SQLite Documentation: https://www.sqlite.org/

Disclaimer: This blog post was created with assistance from Grok 3, an AI developed by xAI, under my direct supervision and guidance to ensure accuracy and alignment with my vision for the content.