sharey/docs/STORAGE.md

Storage Configuration Guide

Sharey supports multiple storage backends for maximum flexibility. You can choose between cloud storage (Backblaze B2) or local filesystem storage.

🔄 How to Choose Storage Type

Quick Method: Edit config.json

1. Open your config.json file
2. Find the "storage" section
3. Change "backend" to either "local" or "b2"

{
  "storage": {
    "backend": "local"    // Change this to "local" or "b2"
  }
}

Interactive Method

python src/config_util.py set       # Interactive setup
python src/config_util.py validate  # Verify configuration  
python test_storage.py              # Test the backend

Environment Variable Method

export STORAGE_BACKEND=local  # or "b2"

📊 Storage Backend Comparison

Feature	Local Storage	B2 Cloud Storage
Setup Difficulty	✅ Easy	⚠️ Requires B2 account
Cost	✅ Free	💰 ~$0.005/GB/month
Speed	✅ Very fast	⚠️ Network dependent
Reliability	⚠️ Single machine	✅ Cloud redundancy
Scalability	⚠️ Disk space limited	✅ Unlimited
Best for	Development, self-hosting	Production, scaling

Storage Backends

1. Backblaze B2 (Cloud Storage)

• ✅ Best for production deployments
• ✅ Scalable and reliable
• ✅ No local disk space requirements
• ❌ Requires B2 account and credentials

2. Local Filesystem

• ✅ No external dependencies
• ✅ Fast access
• ✅ No cloud costs
• ❌ Limited by local disk space
• ❌ Not suitable for distributed deployments

Configuration Details

Configuration Details

Option 1: config.json (Recommended)

{
  "storage": {
    "backend": "b2",           // "b2" or "local"
    "local_path": "storage"    // Path for local storage (only used if backend is "local")
  },
  "b2": {
    "application_key_id": "your_key_id_here",
    "application_key": "your_application_key_here",
    "bucket_name": "your_bucket_name_here"
  }
}

Option 2: Environment Variables

# Storage configuration
export STORAGE_BACKEND=local
export STORAGE_LOCAL_PATH=./my_storage

# B2 configuration (only needed if using B2)
export B2_APPLICATION_KEY_ID=your_key_id_here
export B2_APPLICATION_KEY=your_application_key_here
export B2_BUCKET_NAME=your_bucket_name_here

🚀 Quick Start Guide

For Development/Testing (Local Storage)

# 1. Edit config.json
{
  "storage": {
    "backend": "local"
  }
}

# 2. Test it
python test_storage.py

# 3. Start the app
python run.py

Files will be stored in the storage/ directory.

For Production (B2 Cloud Storage)

# 1. Get B2 credentials from backblaze.com
# 2. Edit config.json
{
  "storage": {
    "backend": "b2"
  },
  "b2": {
    "application_key_id": "your_key_id",
    "application_key": "your_key",
    "bucket_name": "your_bucket"
  }
}

# 3. Test it
python test_storage.py

# 4. Start the app
python run.py

Setup Examples

Using Local Storage

1. Edit config.json:

{
  "storage": {
    "backend": "local",
    "local_path": "storage"
  }
}

2. Start the application:

python run.py

The application will automatically create the storage directory and subdirectories as needed.

Using Backblaze B2

1. Create a B2 account at backblaze.com

2. Create a bucket and application key with read/write permissions

3. Edit config.json:

{
  "storage": {
    "backend": "b2"
  },
  "b2": {
    "application_key_id": "your_actual_key_id",
    "application_key": "your_actual_key",
    "bucket_name": "your_bucket_name"
  }
}

4. Start the application:

python run.py

Interactive Configuration

Use the configuration utility for easy setup:

# Configure storage interactively
python src/config_util.py set

# Validate your configuration
python src/config_util.py validate

# Show current configuration
python src/config_util.py show

Testing Storage

Test your storage configuration:

# Test storage backend
python test_storage.py

This will:

• ✅ Initialize the storage backend
• ✅ Upload a test file
• ✅ Download and verify the file
• ✅ Test file operations (exists, size, list)
• ✅ Clean up test files

Migration Between Backends

From B2 to Local Storage

1. Download your data from B2 (optional - keep as backup)
2. Update config.json:

{
  "storage": {
    "backend": "local",
    "local_path": "storage"
  }
}

3. Restart the application

Note: Existing files in B2 won't be automatically transferred. URLs will break unless you migrate the data.

From Local to B2 Storage

1. Set up B2 bucket and credentials
2. Update config.json:

{
  "storage": {
    "backend": "b2"
  },
  "b2": {
    "application_key_id": "your_key_id",
    "application_key": "your_key",
    "bucket_name": "your_bucket"
  }
}

3. Restart the application

Migration Script (Optional): If you want to migrate existing local files to B2, create a simple script:

#!/usr/bin/env python3
import os
from pathlib import Path
from src.storage import LocalStorageBackend, B2StorageBackend

# Initialize both backends
local = LocalStorageBackend("storage")
b2 = B2StorageBackend("key_id", "key", "bucket_name")

# Migrate files
for file_path in local.list_files():
    print(f"Migrating: {file_path}")
    content = local.download_file(file_path)
    b2.upload_file(content, file_path)
    print(f"✅ Migrated: {file_path}")

Directory Structure

Local Storage Structure

storage/
├── files/
│   ├── abc123.jpg
│   ├── def456.pdf
│   └── ...
└── pastes/
    ├── xyz789.txt
    ├── uvw012.txt
    └── ...

B2 Storage Structure

bucket/
├── files/
│   ├── abc123.jpg
│   ├── def456.pdf
│   └── ...
└── pastes/
    ├── xyz789.txt
    ├── uvw012.txt
    └── ...

Performance Considerations

Local Storage

• Pros: Very fast access, no network latency
• Cons: Limited by disk I/O, single point of failure

B2 Storage

• Pros: Unlimited capacity, built-in redundancy, CDN capabilities
• Cons: Network latency, dependent on internet connection

Security

Local Storage

• Files are stored with filesystem permissions
• Secure as your server's filesystem
• Access controlled by Sharey application

B2 Storage

• Files stored with B2's encryption at rest
• Access controlled by application keys
• Private bucket - files not publicly accessible
• All access proxied through Sharey application

Troubleshooting

Common Issues

"Storage backend not initialized"

• Check your configuration syntax
• Verify credentials (for B2)
• Run python test_storage.py

"B2 authentication failed"

• Verify your application key ID and key
• Check bucket name spelling
• Ensure key has read/write permissions

"Permission denied" (Local storage)

• Check filesystem permissions on storage directory
• Ensure Sharey has write access to the path

"File not found"

• Check if storage backend was changed without migration
• Verify file was uploaded successfully

Debug Commands

# Test storage backend
python test_storage.py

# Validate configuration
python src/config_util.py validate

# Check application logs
tail -f logs/sharey.log

# Test B2 connection (if using B2)
python tests/test_b2.py

Best Practices

1. Production: Use B2 for scalability and reliability
2. Development: Local storage is fine for testing
3. Backup: Consider periodic backups regardless of backend
4. Monitoring: Monitor disk space (local) or B2 costs (cloud)
5. Security: Use environment variables for sensitive credentials in production

Cost Considerations

Local Storage

• Cost: Server disk space and bandwidth
• Scaling: Limited by hardware

B2 Storage

• Storage: $0.005/GB/month
• Download: $0.01/GB
• API calls: Mostly free (10,000 free per day)
• Scaling: Pay as you grow

For most small to medium deployments, B2 costs are minimal (under $5/month for moderate usage).