Uploaded code
This commit is contained in:
147
docs/CONFIG_SYSTEM.md
Normal file
147
docs/CONFIG_SYSTEM.md
Normal file
@@ -0,0 +1,147 @@
|
||||
# Sharey Configuration System
|
||||
|
||||
## Overview
|
||||
|
||||
Sharey now features a comprehensive JSON-based configuration system that replaces environment variables with a more structured and feature-rich approach.
|
||||
|
||||
## Configuration Files
|
||||
|
||||
### Primary Configuration: `config.json`
|
||||
```json
|
||||
{
|
||||
"b2": {
|
||||
"application_key_id": "your_key_id_here",
|
||||
"application_key": "your_application_key_here",
|
||||
"bucket_name": "your_bucket_name_here"
|
||||
},
|
||||
"flask": {
|
||||
"host": "127.0.0.1",
|
||||
"port": 8866,
|
||||
"debug": true
|
||||
},
|
||||
"upload": {
|
||||
"max_file_size_mb": 100,
|
||||
"allowed_extensions": [".jpg", ".jpeg", ".png", ".gif", ".pdf", ".txt", ".doc", ".docx", ".zip", ".mp4", ".mp3"]
|
||||
},
|
||||
"paste": {
|
||||
"max_length": 1000000
|
||||
},
|
||||
"security": {
|
||||
"rate_limit_enabled": false,
|
||||
"max_uploads_per_hour": 50
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Fallback: Environment Variables
|
||||
For backwards compatibility, Sharey still supports `.env` files and environment variables.
|
||||
|
||||
## Configuration Management Tools
|
||||
|
||||
### Setup Script: `setup.py`
|
||||
- Creates `config.json` from template
|
||||
- Sets up virtual environment
|
||||
- Installs dependencies
|
||||
- Validates configuration
|
||||
|
||||
### Configuration Utility: `config_util.py`
|
||||
```bash
|
||||
python config_util.py show # Display current config
|
||||
python config_util.py validate # Validate configuration
|
||||
python config_util.py set # Interactive setup
|
||||
python config_util.py reset # Reset to defaults
|
||||
```
|
||||
|
||||
### Test Utility: `test_b2.py`
|
||||
- Tests B2 connection
|
||||
- Validates credentials
|
||||
- Shows configuration summary
|
||||
|
||||
## New Features
|
||||
|
||||
### File Upload Controls
|
||||
- **File size limits**: Configurable max file size in MB
|
||||
- **File type restrictions**: Whitelist of allowed extensions
|
||||
- **Validation**: Automatic file type checking
|
||||
|
||||
### Paste Controls
|
||||
- **Length limits**: Configurable maximum paste length
|
||||
- **Validation**: Automatic length checking
|
||||
|
||||
### Flask Configuration
|
||||
- **Host/Port**: Configurable server settings
|
||||
- **Debug mode**: Environment-specific settings
|
||||
|
||||
### Security Features (Future)
|
||||
- **Rate limiting**: Configurable upload limits
|
||||
- **Extensible**: Ready for additional security features
|
||||
|
||||
## Configuration Loading Priority
|
||||
|
||||
1. **config.json** (if exists)
|
||||
2. **Environment variables** (fallback)
|
||||
3. **Built-in defaults** (last resort)
|
||||
|
||||
## Benefits of JSON Configuration
|
||||
|
||||
### ✅ **Structured Data**
|
||||
- Hierarchical organization
|
||||
- Type safety (strings, numbers, booleans, arrays)
|
||||
- Better validation
|
||||
|
||||
### ✅ **Feature Rich**
|
||||
- Upload restrictions
|
||||
- File type filtering
|
||||
- Paste length limits
|
||||
- Server configuration
|
||||
|
||||
### ✅ **User Friendly**
|
||||
- Interactive setup utility
|
||||
- Configuration validation
|
||||
- Clear error messages
|
||||
- Comprehensive documentation
|
||||
|
||||
### ✅ **Developer Friendly**
|
||||
- Easy to extend
|
||||
- Version controllable (with secrets excluded)
|
||||
- IDE support with syntax highlighting
|
||||
- Better tooling support
|
||||
|
||||
### ✅ **Backwards Compatible**
|
||||
- Still supports .env files
|
||||
- Smooth migration path
|
||||
- No breaking changes
|
||||
|
||||
## Migration from .env
|
||||
|
||||
Old `.env` approach:
|
||||
```env
|
||||
B2_APPLICATION_KEY_ID=key123
|
||||
B2_APPLICATION_KEY=secret456
|
||||
B2_BUCKET_NAME=mybucket
|
||||
```
|
||||
|
||||
New `config.json` approach:
|
||||
```json
|
||||
{
|
||||
"b2": {
|
||||
"application_key_id": "key123",
|
||||
"application_key": "secret456",
|
||||
"bucket_name": "mybucket"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Quick Start
|
||||
|
||||
1. **Run setup**: `python setup.py`
|
||||
2. **Configure**: `python config_util.py set`
|
||||
3. **Test**: `python test_b2.py`
|
||||
4. **Run**: `python app.py`
|
||||
|
||||
## Security Considerations
|
||||
|
||||
- `config.json` is excluded from git by default
|
||||
- Sensitive data can be hidden in display utilities
|
||||
- Environment variable fallback for production deployments
|
||||
- Configuration validation prevents common mistakes
|
||||
79
docs/DEPLOYMENT.md
Normal file
79
docs/DEPLOYMENT.md
Normal file
@@ -0,0 +1,79 @@
|
||||
# Deployment Configuration
|
||||
|
||||
## Environment Variables for Production
|
||||
|
||||
When deploying Sharey to production, make sure to set these environment variables:
|
||||
|
||||
```bash
|
||||
# Required B2 Configuration
|
||||
export B2_APPLICATION_KEY_ID="your_key_id"
|
||||
export B2_APPLICATION_KEY="your_key"
|
||||
export B2_BUCKET_NAME="your_bucket_name"
|
||||
|
||||
# Flask Configuration
|
||||
export FLASK_ENV="production"
|
||||
export FLASK_DEBUG="False"
|
||||
```
|
||||
|
||||
## Docker Deployment
|
||||
|
||||
### Dockerfile
|
||||
```dockerfile
|
||||
FROM python:3.9-slim
|
||||
|
||||
WORKDIR /app
|
||||
|
||||
COPY requirements.txt .
|
||||
RUN pip install --no-cache-dir -r requirements.txt
|
||||
|
||||
COPY . .
|
||||
|
||||
EXPOSE 8866
|
||||
|
||||
CMD ["python", "app.py"]
|
||||
```
|
||||
|
||||
### Docker Compose
|
||||
```yaml
|
||||
version: '3.8'
|
||||
services:
|
||||
sharey:
|
||||
build: .
|
||||
ports:
|
||||
- "8866:8866"
|
||||
environment:
|
||||
- B2_APPLICATION_KEY_ID=${B2_APPLICATION_KEY_ID}
|
||||
- B2_APPLICATION_KEY=${B2_APPLICATION_KEY}
|
||||
- B2_BUCKET_NAME=${B2_BUCKET_NAME}
|
||||
volumes:
|
||||
- .env:/app/.env
|
||||
```
|
||||
|
||||
## Nginx Reverse Proxy
|
||||
|
||||
```nginx
|
||||
server {
|
||||
listen 80;
|
||||
server_name your-domain.com;
|
||||
|
||||
location / {
|
||||
proxy_pass http://127.0.0.1:8866;
|
||||
proxy_set_header Host $host;
|
||||
proxy_set_header X-Real-IP $remote_addr;
|
||||
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||||
proxy_set_header X-Forwarded-Proto $scheme;
|
||||
|
||||
# Increase client max body size for file uploads
|
||||
client_max_body_size 100M;
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Production Considerations
|
||||
|
||||
1. **File Size Limits**: Configure your web server to handle large file uploads
|
||||
2. **HTTPS**: Use SSL/TLS certificates for secure file transfers
|
||||
3. **Rate Limiting**: Implement rate limiting to prevent abuse
|
||||
4. **Monitoring**: Set up logging and monitoring for the application
|
||||
5. **Backup**: B2 provides versioning, but consider backup strategies
|
||||
6. **Security**: Restrict B2 bucket access and use environment variables for secrets
|
||||
67
docs/MAINTENANCE.md
Normal file
67
docs/MAINTENANCE.md
Normal file
@@ -0,0 +1,67 @@
|
||||
# Maintenance Mode
|
||||
|
||||
Sharey includes a maintenance mode feature that allows you to temporarily disable the service and show a maintenance page to users.
|
||||
|
||||
## How to Enable Maintenance Mode
|
||||
|
||||
```bash
|
||||
python config_util.py maintenance enable
|
||||
```
|
||||
|
||||
This will:
|
||||
- Ask for a custom maintenance message (optional)
|
||||
- Ask for an estimated return time (optional)
|
||||
- Enable maintenance mode immediately
|
||||
|
||||
## How to Disable Maintenance Mode
|
||||
|
||||
```bash
|
||||
python config_util.py maintenance disable
|
||||
```
|
||||
|
||||
## Check Maintenance Status
|
||||
|
||||
```bash
|
||||
python config_util.py maintenance status
|
||||
```
|
||||
|
||||
## Manual Configuration
|
||||
|
||||
You can also manually edit `config.json` to enable/disable maintenance mode:
|
||||
|
||||
```json
|
||||
{
|
||||
"maintenance": {
|
||||
"enabled": true,
|
||||
"message": "Sharey is currently under maintenance. Please check back later!",
|
||||
"estimated_return": "2025-08-22 15:00 UTC"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## What Happens During Maintenance
|
||||
|
||||
When maintenance mode is enabled:
|
||||
|
||||
- ✅ **Health check endpoint** (`/health`) still works (for monitoring)
|
||||
- ❌ **All other routes** show the maintenance page
|
||||
- ❌ **API endpoints** return maintenance page with 503 status
|
||||
- ❌ **File uploads** are disabled
|
||||
- ❌ **File downloads** are disabled
|
||||
- ❌ **Paste creation** is disabled
|
||||
- ❌ **Paste viewing** is disabled
|
||||
|
||||
## Maintenance Page Features
|
||||
|
||||
- 🎨 **Themed**: Respects user's light/dark theme preference
|
||||
- 📱 **Responsive**: Works on mobile and desktop
|
||||
- 🔄 **Refresh button**: Users can easily check if maintenance is over
|
||||
- ⏰ **Estimated return time**: Shows when service is expected to return (optional)
|
||||
- 💬 **Custom message**: Display custom maintenance information
|
||||
|
||||
## Use Cases
|
||||
|
||||
- **Server updates**: When updating the application
|
||||
- **Database maintenance**: When performing B2 bucket operations
|
||||
- **Security issues**: When temporarily disabling service for security reasons
|
||||
- **Planned downtime**: When performing infrastructure maintenance
|
||||
118
docs/MIGRATION_SUMMARY.md
Normal file
118
docs/MIGRATION_SUMMARY.md
Normal file
@@ -0,0 +1,118 @@
|
||||
# Sharey B2 Migration Summary
|
||||
|
||||
## What Changed
|
||||
|
||||
Your Sharey application has been successfully modified to use Backblaze B2 cloud storage instead of local file storage. Here are the key changes:
|
||||
|
||||
### 🔄 Core Changes
|
||||
|
||||
1. **Storage Backend**:
|
||||
- Removed local file storage (`uploads/` and `pastes/` folders)
|
||||
- Added Backblaze B2 cloud storage integration
|
||||
- Files and pastes now stored in B2 bucket
|
||||
|
||||
2. **Dependencies Added**:
|
||||
- `b2sdk`: Official Backblaze B2 SDK
|
||||
- `python-dotenv`: Environment variable management
|
||||
|
||||
3. **Configuration**:
|
||||
- Added `.env` file support for B2 credentials
|
||||
- Added connection validation and error handling
|
||||
- Added health check endpoint
|
||||
|
||||
### 📁 New Files Created
|
||||
|
||||
- `requirements.txt` - Python dependencies
|
||||
- `.env.example` - Environment template
|
||||
- `setup.py` - Automated setup script (Python-based)
|
||||
- `test_b2.py` - B2 connection testing utility
|
||||
- `DEPLOYMENT.md` - Production deployment guide
|
||||
- Updated `.gitignore` - Improved git ignore rules
|
||||
- Updated `README.md` - Comprehensive setup instructions
|
||||
|
||||
### 🔧 Modified Functions
|
||||
|
||||
1. **File Upload** (`/api/upload`):
|
||||
- Now uploads files directly to B2 bucket under `files/` prefix
|
||||
- Returns B2 direct download URLs
|
||||
- Better error handling
|
||||
|
||||
2. **File Serving** (`/files/<file_id>`):
|
||||
- Now redirects to B2 download URLs
|
||||
- No longer serves files locally
|
||||
|
||||
3. **Paste Creation** (`/api/paste`):
|
||||
- Stores paste content in B2 under `pastes/` prefix
|
||||
- UTF-8 encoding support
|
||||
|
||||
4. **Paste Viewing** (`/pastes/<paste_id>` and `/pastes/raw/<paste_id>`):
|
||||
- Downloads paste content from B2
|
||||
- Maintains same user interface
|
||||
|
||||
### 🏗️ Bucket Organization
|
||||
|
||||
Your B2 bucket will be organized as:
|
||||
```
|
||||
your-bucket/
|
||||
├── files/ # Uploaded files (images, documents, etc.)
|
||||
│ ├── abc123.jpg
|
||||
│ ├── def456.pdf
|
||||
│ └── ...
|
||||
└── pastes/ # Text pastes
|
||||
├── ghi789.txt
|
||||
├── jkl012.txt
|
||||
└── ...
|
||||
```
|
||||
|
||||
## Next Steps
|
||||
|
||||
1. **Get B2 Credentials**:
|
||||
- Sign up at [Backblaze B2](https://www.backblaze.com/b2)
|
||||
- Create application key and bucket
|
||||
- Note down: Key ID, Application Key, Bucket Name
|
||||
|
||||
2. **Configure Environment**:
|
||||
```bash
|
||||
cp .env.example .env
|
||||
# Edit .env with your B2 credentials
|
||||
```
|
||||
|
||||
3. **Install Dependencies**:
|
||||
```bash
|
||||
python setup.py # or manually: pip install -r requirements.txt
|
||||
```
|
||||
|
||||
4. **Test Configuration**:
|
||||
```bash
|
||||
python test_b2.py
|
||||
```
|
||||
|
||||
5. **Run Application**:
|
||||
```bash
|
||||
python app.py
|
||||
```
|
||||
|
||||
## Benefits of B2 Storage
|
||||
|
||||
- ✅ **Scalable**: No local disk space limitations
|
||||
- ✅ **Reliable**: Built-in redundancy and backups
|
||||
- ✅ **Cost-effective**: Pay only for what you use
|
||||
- ✅ **Global CDN**: Fast downloads worldwide
|
||||
- ✅ **Secure**: Encrypted storage with access controls
|
||||
- ✅ **Maintenance-free**: No local file management needed
|
||||
|
||||
## Migration Notes
|
||||
|
||||
- Existing local files/pastes will remain in local folders
|
||||
- New uploads will go to B2
|
||||
- Old URLs will break (files are now served from B2)
|
||||
- No data migration script provided (manual migration needed if desired)
|
||||
- B2 bucket should be set to "Public" for file sharing to work properly
|
||||
|
||||
## Support
|
||||
|
||||
If you encounter issues:
|
||||
1. Run `python test_b2.py` to test your configuration
|
||||
2. Check the health endpoint: `http://localhost:8866/health`
|
||||
3. Verify B2 bucket permissions and settings
|
||||
4. Ensure your .env file has correct credentials
|
||||
99
docs/ORGANIZATION.md
Normal file
99
docs/ORGANIZATION.md
Normal file
@@ -0,0 +1,99 @@
|
||||
# Project Organization Summary
|
||||
|
||||
## 📁 Reorganized Structure
|
||||
|
||||
The Sharey project has been reorganized into a clean, professional structure:
|
||||
|
||||
### 🏗️ Directory Structure
|
||||
```
|
||||
sharey/
|
||||
├── src/ # Main application code
|
||||
│ ├── app.py # Flask application
|
||||
│ ├── config.py # Configuration management
|
||||
│ ├── config_util.py # Config utilities
|
||||
│ ├── static/ # CSS, JS, assets
|
||||
│ │ ├── script.js # Main JavaScript
|
||||
│ │ ├── style.css # Stylesheets
|
||||
│ │ └── script_backup.js
|
||||
│ └── templates/ # HTML templates
|
||||
│ ├── index.html # Main page
|
||||
│ ├── admin.html # Admin panel
|
||||
│ ├── admin_login.html
|
||||
│ ├── maintenance.html
|
||||
│ ├── view_file.html # File viewer
|
||||
│ └── view_paste.html # Paste viewer
|
||||
├── tests/ # Test files
|
||||
│ ├── test_b2.py
|
||||
│ ├── test_bucket_contents.py
|
||||
│ └── test_paste.py
|
||||
├── scripts/ # Utility scripts
|
||||
│ ├── clean.sh # Cleanup script
|
||||
│ ├── migrate.py # Database migration
|
||||
│ ├── set_admin_password.py
|
||||
│ ├── setup.py
|
||||
│ └── setup.sh
|
||||
├── docs/ # Documentation
|
||||
│ ├── CONFIG_SYSTEM.md
|
||||
│ ├── DEPLOYMENT.md
|
||||
│ ├── MAINTENANCE.md
|
||||
│ ├── MIGRATION_SUMMARY.md
|
||||
│ └── README.md (old)
|
||||
├── logs/ # Application logs
|
||||
│ ├── app.log
|
||||
│ └── migration_log_20250815_121855.txt
|
||||
├── config.json # Main configuration
|
||||
├── config.json.example # Configuration template
|
||||
├── requirements.txt # Python dependencies
|
||||
├── run.py # Application entry point
|
||||
├── dev-setup.sh # Development setup
|
||||
└── README.md # Project documentation
|
||||
```
|
||||
|
||||
## 🚀 Running the Application
|
||||
|
||||
### New Entry Point
|
||||
```bash
|
||||
python run.py
|
||||
```
|
||||
|
||||
### Development Setup
|
||||
```bash
|
||||
./dev-setup.sh
|
||||
```
|
||||
|
||||
### Cleanup
|
||||
```bash
|
||||
./scripts/clean.sh
|
||||
```
|
||||
|
||||
## ✅ Changes Made
|
||||
|
||||
1. **Moved core application** (`app.py`, `config.py`) to `src/`
|
||||
2. **Organized static assets** (`static/`, `templates/`) under `src/`
|
||||
3. **Collected tests** in dedicated `tests/` directory
|
||||
4. **Grouped scripts** in `scripts/` directory
|
||||
5. **Centralized documentation** in `docs/` directory
|
||||
6. **Created logs directory** for log files
|
||||
7. **Added entry point** (`run.py`) for easy execution
|
||||
8. **Created development setup** script for easy onboarding
|
||||
9. **Added cleanup script** for maintenance
|
||||
10. **Removed temporary files** and debug artifacts
|
||||
11. **Updated README.md** with new structure
|
||||
|
||||
## 🎯 Benefits
|
||||
|
||||
- **Cleaner root directory** - Only essential files at project root
|
||||
- **Logical grouping** - Related files organized together
|
||||
- **Professional structure** - Follows Python project best practices
|
||||
- **Easy navigation** - Clear separation of concerns
|
||||
- **Better maintainability** - Easier to find and modify files
|
||||
- **Development friendly** - Scripts for common tasks
|
||||
|
||||
## 🔧 Next Steps
|
||||
|
||||
1. Update any deployment scripts to use new structure
|
||||
2. Test the new entry point thoroughly
|
||||
3. Update CI/CD pipelines if applicable
|
||||
4. Consider adding more development tools (linting, formatting)
|
||||
|
||||
The project is now much cleaner and follows modern Python project conventions!
|
||||
196
docs/README.md
Normal file
196
docs/README.md
Normal file
@@ -0,0 +1,196 @@
|
||||
# Sharey
|
||||
|
||||
A simple file sharing and pastebin service using Backblaze B2 cloud storage.
|
||||
|
||||
## Features
|
||||
|
||||
- 📁 **File Sharing**: Upload files and get shareable links
|
||||
- 📝 **Pastebin**: Share text snippets with syntax highlighting support
|
||||
- ☁️ **Cloud Storage**: Files stored securely in Backblaze B2
|
||||
- 🔗 **Short URLs**: Clean, easy-to-share links
|
||||
- 🎨 **Clean UI**: Simple, responsive web interface
|
||||
|
||||
## Setup
|
||||
|
||||
### Prerequisites
|
||||
|
||||
- Python 3.7+
|
||||
- Backblaze B2 account with:
|
||||
- Application Key ID
|
||||
- Application Key
|
||||
- Bucket name
|
||||
|
||||
### Installation
|
||||
|
||||
1. **Clone the repository**
|
||||
```bash
|
||||
git clone <your-repo-url>
|
||||
cd sharey
|
||||
```
|
||||
|
||||
2. **Run the setup script**
|
||||
```bash
|
||||
python setup.py
|
||||
```
|
||||
|
||||
This will automatically create a virtual environment and install dependencies.
|
||||
|
||||
3. **Configure B2 credentials**
|
||||
|
||||
Sharey supports two configuration methods:
|
||||
|
||||
**Option 1: JSON Configuration (Recommended)**
|
||||
```bash
|
||||
# Edit config.json with your credentials
|
||||
nano config.json
|
||||
```
|
||||
|
||||
**Option 2: Environment Variables**
|
||||
```bash
|
||||
# Edit .env file with your credentials
|
||||
nano .env
|
||||
```
|
||||
|
||||
**Interactive Configuration**
|
||||
```bash
|
||||
python config_util.py set
|
||||
```
|
||||
|
||||
4. **Test B2 connection**
|
||||
```bash
|
||||
python test_b2.py
|
||||
```
|
||||
|
||||
5. **Run the application**
|
||||
```bash
|
||||
# If using virtual environment (recommended)
|
||||
source venv/bin/activate
|
||||
python app.py
|
||||
|
||||
# Or directly
|
||||
venv/bin/python app.py
|
||||
```
|
||||
|
||||
The application will be available at `http://127.0.0.1:8866`
|
||||
|
||||
## Backblaze B2 Setup
|
||||
|
||||
1. **Create a Backblaze B2 account** at [backblaze.com](https://www.backblaze.com/b2/cloud-storage.html)
|
||||
|
||||
2. **Create an application key**:
|
||||
- Go to [App Keys](https://secure.backblaze.com/app_keys.htm)
|
||||
- Click "Add a New Application Key"
|
||||
- Name it "Sharey" or similar
|
||||
- Choose appropriate permissions (read/write access to your bucket)
|
||||
- Save the Key ID and Application Key
|
||||
|
||||
3. **Create a bucket**:
|
||||
- Go to [Buckets](https://secure.backblaze.com/b2_buckets.htm)
|
||||
- Click "Create a Bucket"
|
||||
- Choose "Public" if you want files to be publicly accessible
|
||||
- Note the bucket name
|
||||
|
||||
4. **Configure bucket for public access** (if desired):
|
||||
- Set bucket type to "Public"
|
||||
- Configure CORS settings if needed for web access
|
||||
|
||||
## File Organization in B2
|
||||
|
||||
Files are organized in your B2 bucket as follows:
|
||||
```
|
||||
your-bucket/
|
||||
├── files/
|
||||
│ ├── abc123.jpg
|
||||
│ ├── def456.pdf
|
||||
│ └── ...
|
||||
└── pastes/
|
||||
├── ghi789.txt
|
||||
├── jkl012.txt
|
||||
└── ...
|
||||
```
|
||||
|
||||
## Development
|
||||
|
||||
### Manual Installation
|
||||
|
||||
If you prefer not to use the setup script:
|
||||
|
||||
```bash
|
||||
# Install dependencies
|
||||
pip install -r requirements.txt
|
||||
|
||||
# Copy environment template
|
||||
cp .env.example .env
|
||||
|
||||
# Edit .env with your credentials
|
||||
nano .env
|
||||
|
||||
# Test configuration
|
||||
python test_b2.py
|
||||
|
||||
# Run the app
|
||||
python app.py
|
||||
```
|
||||
|
||||
## Configuration
|
||||
|
||||
Sharey uses a flexible configuration system that supports both JSON files and environment variables.
|
||||
|
||||
### Configuration Files
|
||||
|
||||
- `config.json` - Main configuration file (recommended)
|
||||
- `.env` - Environment variables (fallback/legacy support)
|
||||
|
||||
### Configuration Management
|
||||
|
||||
**View current configuration:**
|
||||
```bash
|
||||
python config_util.py show
|
||||
```
|
||||
|
||||
**Interactive setup:**
|
||||
```bash
|
||||
python config_util.py set
|
||||
```
|
||||
|
||||
**Validate configuration:**
|
||||
```bash
|
||||
python config_util.py validate
|
||||
```
|
||||
|
||||
**Reset to defaults:**
|
||||
```bash
|
||||
python config_util.py reset
|
||||
```
|
||||
|
||||
### Configuration Options
|
||||
|
||||
```json
|
||||
{
|
||||
"b2": {
|
||||
"application_key_id": "your_key_id_here",
|
||||
"application_key": "your_application_key_here",
|
||||
"bucket_name": "your_bucket_name_here"
|
||||
},
|
||||
"flask": {
|
||||
"host": "127.0.0.1",
|
||||
"port": 8866,
|
||||
"debug": true
|
||||
},
|
||||
"upload": {
|
||||
"max_file_size_mb": 100,
|
||||
"allowed_extensions": [".jpg", ".jpeg", ".png", ".gif", ".pdf", ".txt", ".doc", ".docx", ".zip", ".mp4", ".mp3"]
|
||||
},
|
||||
"paste": {
|
||||
"max_length": 1000000
|
||||
},
|
||||
"security": {
|
||||
"rate_limit_enabled": false,
|
||||
"max_uploads_per_hour": 50
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## License
|
||||
|
||||
MIT License - see [LICENSE](LICENSE) file for details.
|
||||
375
docs/STORAGE.md
Normal file
375
docs/STORAGE.md
Normal file
@@ -0,0 +1,375 @@
|
||||
# Storage Configuration Guide
|
||||
|
||||
Sharey supports multiple storage backends for maximum flexibility. You can choose between cloud storage (Backblaze B2) or local filesystem storage.
|
||||
|
||||
## 🔄 How to Choose Storage Type
|
||||
|
||||
### **Quick Method: Edit config.json**
|
||||
1. Open your `config.json` file
|
||||
2. Find the `"storage"` section
|
||||
3. Change `"backend"` to either `"local"` or `"b2"`
|
||||
|
||||
```json
|
||||
{
|
||||
"storage": {
|
||||
"backend": "local" // Change this to "local" or "b2"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### **Interactive Method**
|
||||
```bash
|
||||
python src/config_util.py set # Interactive setup
|
||||
python src/config_util.py validate # Verify configuration
|
||||
python test_storage.py # Test the backend
|
||||
```
|
||||
|
||||
### **Environment Variable Method**
|
||||
```bash
|
||||
export STORAGE_BACKEND=local # or "b2"
|
||||
```
|
||||
|
||||
## 📊 Storage Backend Comparison
|
||||
|
||||
| Feature | **Local Storage** | **B2 Cloud Storage** |
|
||||
|---------|------------------|-------------------|
|
||||
| **Setup Difficulty** | ✅ Easy | ⚠️ Requires B2 account |
|
||||
| **Cost** | ✅ Free | 💰 ~$0.005/GB/month |
|
||||
| **Speed** | ✅ Very fast | ⚠️ Network dependent |
|
||||
| **Reliability** | ⚠️ Single machine | ✅ Cloud redundancy |
|
||||
| **Scalability** | ⚠️ Disk space limited | ✅ Unlimited |
|
||||
| **Best for** | Development, self-hosting | Production, scaling |
|
||||
|
||||
## Storage Backends
|
||||
|
||||
### 1. Backblaze B2 (Cloud Storage)
|
||||
- ✅ **Best for production deployments**
|
||||
- ✅ **Scalable and reliable**
|
||||
- ✅ **No local disk space requirements**
|
||||
- ❌ **Requires B2 account and credentials**
|
||||
|
||||
### 2. Local Filesystem
|
||||
- ✅ **No external dependencies**
|
||||
- ✅ **Fast access**
|
||||
- ✅ **No cloud costs**
|
||||
- ❌ **Limited by local disk space**
|
||||
- ❌ **Not suitable for distributed deployments**
|
||||
|
||||
## Configuration Details
|
||||
|
||||
## Configuration Details
|
||||
|
||||
### Option 1: config.json (Recommended)
|
||||
|
||||
```json
|
||||
{
|
||||
"storage": {
|
||||
"backend": "b2", // "b2" or "local"
|
||||
"local_path": "storage" // Path for local storage (only used if backend is "local")
|
||||
},
|
||||
"b2": {
|
||||
"application_key_id": "your_key_id_here",
|
||||
"application_key": "your_application_key_here",
|
||||
"bucket_name": "your_bucket_name_here"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Option 2: Environment Variables
|
||||
|
||||
```bash
|
||||
# Storage configuration
|
||||
export STORAGE_BACKEND=local
|
||||
export STORAGE_LOCAL_PATH=./my_storage
|
||||
|
||||
# B2 configuration (only needed if using B2)
|
||||
export B2_APPLICATION_KEY_ID=your_key_id_here
|
||||
export B2_APPLICATION_KEY=your_application_key_here
|
||||
export B2_BUCKET_NAME=your_bucket_name_here
|
||||
```
|
||||
|
||||
## 🚀 Quick Start Guide
|
||||
|
||||
### **For Development/Testing (Local Storage)**
|
||||
```bash
|
||||
# 1. Edit config.json
|
||||
{
|
||||
"storage": {
|
||||
"backend": "local"
|
||||
}
|
||||
}
|
||||
|
||||
# 2. Test it
|
||||
python test_storage.py
|
||||
|
||||
# 3. Start the app
|
||||
python run.py
|
||||
```
|
||||
Files will be stored in the `storage/` directory.
|
||||
|
||||
### **For Production (B2 Cloud Storage)**
|
||||
```bash
|
||||
# 1. Get B2 credentials from backblaze.com
|
||||
# 2. Edit config.json
|
||||
{
|
||||
"storage": {
|
||||
"backend": "b2"
|
||||
},
|
||||
"b2": {
|
||||
"application_key_id": "your_key_id",
|
||||
"application_key": "your_key",
|
||||
"bucket_name": "your_bucket"
|
||||
}
|
||||
}
|
||||
|
||||
# 3. Test it
|
||||
python test_storage.py
|
||||
|
||||
# 4. Start the app
|
||||
python run.py
|
||||
```
|
||||
|
||||
## Setup Examples
|
||||
|
||||
### Using Local Storage
|
||||
|
||||
1. **Edit config.json:**
|
||||
```json
|
||||
{
|
||||
"storage": {
|
||||
"backend": "local",
|
||||
"local_path": "storage"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
2. **Start the application:**
|
||||
```bash
|
||||
python run.py
|
||||
```
|
||||
|
||||
The application will automatically create the `storage` directory and subdirectories as needed.
|
||||
|
||||
### Using Backblaze B2
|
||||
|
||||
1. **Create a B2 account** at [backblaze.com](https://www.backblaze.com/b2/)
|
||||
|
||||
2. **Create a bucket** and application key with read/write permissions
|
||||
|
||||
3. **Edit config.json:**
|
||||
```json
|
||||
{
|
||||
"storage": {
|
||||
"backend": "b2"
|
||||
},
|
||||
"b2": {
|
||||
"application_key_id": "your_actual_key_id",
|
||||
"application_key": "your_actual_key",
|
||||
"bucket_name": "your_bucket_name"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
4. **Start the application:**
|
||||
```bash
|
||||
python run.py
|
||||
```
|
||||
|
||||
## Interactive Configuration
|
||||
|
||||
Use the configuration utility for easy setup:
|
||||
|
||||
```bash
|
||||
# Configure storage interactively
|
||||
python src/config_util.py set
|
||||
|
||||
# Validate your configuration
|
||||
python src/config_util.py validate
|
||||
|
||||
# Show current configuration
|
||||
python src/config_util.py show
|
||||
```
|
||||
|
||||
## Testing Storage
|
||||
|
||||
Test your storage configuration:
|
||||
|
||||
```bash
|
||||
# Test storage backend
|
||||
python test_storage.py
|
||||
```
|
||||
|
||||
This will:
|
||||
- ✅ Initialize the storage backend
|
||||
- ✅ Upload a test file
|
||||
- ✅ Download and verify the file
|
||||
- ✅ Test file operations (exists, size, list)
|
||||
- ✅ Clean up test files
|
||||
|
||||
## Migration Between Backends
|
||||
|
||||
### From B2 to Local Storage
|
||||
|
||||
1. **Download your data** from B2 (optional - keep as backup)
|
||||
2. **Update config.json:**
|
||||
```json
|
||||
{
|
||||
"storage": {
|
||||
"backend": "local",
|
||||
"local_path": "storage"
|
||||
}
|
||||
}
|
||||
```
|
||||
3. **Restart the application**
|
||||
|
||||
**Note:** Existing files in B2 won't be automatically transferred. URLs will break unless you migrate the data.
|
||||
|
||||
### From Local to B2 Storage
|
||||
|
||||
1. **Set up B2 bucket and credentials**
|
||||
2. **Update config.json:**
|
||||
```json
|
||||
{
|
||||
"storage": {
|
||||
"backend": "b2"
|
||||
},
|
||||
"b2": {
|
||||
"application_key_id": "your_key_id",
|
||||
"application_key": "your_key",
|
||||
"bucket_name": "your_bucket"
|
||||
}
|
||||
}
|
||||
```
|
||||
3. **Restart the application**
|
||||
|
||||
**Migration Script (Optional):**
|
||||
If you want to migrate existing local files to B2, create a simple script:
|
||||
|
||||
```python
|
||||
#!/usr/bin/env python3
|
||||
import os
|
||||
from pathlib import Path
|
||||
from src.storage import LocalStorageBackend, B2StorageBackend
|
||||
|
||||
# Initialize both backends
|
||||
local = LocalStorageBackend("storage")
|
||||
b2 = B2StorageBackend("key_id", "key", "bucket_name")
|
||||
|
||||
# Migrate files
|
||||
for file_path in local.list_files():
|
||||
print(f"Migrating: {file_path}")
|
||||
content = local.download_file(file_path)
|
||||
b2.upload_file(content, file_path)
|
||||
print(f"✅ Migrated: {file_path}")
|
||||
```
|
||||
|
||||
## Directory Structure
|
||||
|
||||
### Local Storage Structure
|
||||
```
|
||||
storage/
|
||||
├── files/
|
||||
│ ├── abc123.jpg
|
||||
│ ├── def456.pdf
|
||||
│ └── ...
|
||||
└── pastes/
|
||||
├── xyz789.txt
|
||||
├── uvw012.txt
|
||||
└── ...
|
||||
```
|
||||
|
||||
### B2 Storage Structure
|
||||
```
|
||||
bucket/
|
||||
├── files/
|
||||
│ ├── abc123.jpg
|
||||
│ ├── def456.pdf
|
||||
│ └── ...
|
||||
└── pastes/
|
||||
├── xyz789.txt
|
||||
├── uvw012.txt
|
||||
└── ...
|
||||
```
|
||||
|
||||
## Performance Considerations
|
||||
|
||||
### Local Storage
|
||||
- **Pros:** Very fast access, no network latency
|
||||
- **Cons:** Limited by disk I/O, single point of failure
|
||||
|
||||
### B2 Storage
|
||||
- **Pros:** Unlimited capacity, built-in redundancy, CDN capabilities
|
||||
- **Cons:** Network latency, dependent on internet connection
|
||||
|
||||
## Security
|
||||
|
||||
### Local Storage
|
||||
- Files are stored with filesystem permissions
|
||||
- Secure as your server's filesystem
|
||||
- Access controlled by Sharey application
|
||||
|
||||
### B2 Storage
|
||||
- Files stored with B2's encryption at rest
|
||||
- Access controlled by application keys
|
||||
- Private bucket - files not publicly accessible
|
||||
- All access proxied through Sharey application
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Common Issues
|
||||
|
||||
**"Storage backend not initialized"**
|
||||
- Check your configuration syntax
|
||||
- Verify credentials (for B2)
|
||||
- Run `python test_storage.py`
|
||||
|
||||
**"B2 authentication failed"**
|
||||
- Verify your application key ID and key
|
||||
- Check bucket name spelling
|
||||
- Ensure key has read/write permissions
|
||||
|
||||
**"Permission denied" (Local storage)**
|
||||
- Check filesystem permissions on storage directory
|
||||
- Ensure Sharey has write access to the path
|
||||
|
||||
**"File not found"**
|
||||
- Check if storage backend was changed without migration
|
||||
- Verify file was uploaded successfully
|
||||
|
||||
### Debug Commands
|
||||
|
||||
```bash
|
||||
# Test storage backend
|
||||
python test_storage.py
|
||||
|
||||
# Validate configuration
|
||||
python src/config_util.py validate
|
||||
|
||||
# Check application logs
|
||||
tail -f logs/sharey.log
|
||||
|
||||
# Test B2 connection (if using B2)
|
||||
python tests/test_b2.py
|
||||
```
|
||||
|
||||
## Best Practices
|
||||
|
||||
1. **Production:** Use B2 for scalability and reliability
|
||||
2. **Development:** Local storage is fine for testing
|
||||
3. **Backup:** Consider periodic backups regardless of backend
|
||||
4. **Monitoring:** Monitor disk space (local) or B2 costs (cloud)
|
||||
5. **Security:** Use environment variables for sensitive credentials in production
|
||||
|
||||
## Cost Considerations
|
||||
|
||||
### Local Storage
|
||||
- **Cost:** Server disk space and bandwidth
|
||||
- **Scaling:** Limited by hardware
|
||||
|
||||
### B2 Storage
|
||||
- **Storage:** $0.005/GB/month
|
||||
- **Download:** $0.01/GB
|
||||
- **API calls:** Mostly free (10,000 free per day)
|
||||
- **Scaling:** Pay as you grow
|
||||
|
||||
For most small to medium deployments, B2 costs are minimal (under $5/month for moderate usage).
|
||||
Reference in New Issue
Block a user