1117 lines
No EOL
27 KiB
Text
1117 lines
No EOL
27 KiB
Text
{
|
|
"cells": [
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"# Running Your Trading Bot 24/7 with Systemd\n",
|
|
"\n",
|
|
"## Option 3: Systemd Services - Production-Ready Deployment\n",
|
|
"\n",
|
|
"**Best for:** Production environments, automatic restarts, professional deployments\n",
|
|
"\n",
|
|
"**What Systemd does:**\n",
|
|
"- Manages your bot as a proper Linux service\n",
|
|
"- Automatically starts on system boot\n",
|
|
"- Restarts your bot if it crashes\n",
|
|
"- Provides structured logging\n",
|
|
"- Allows easy start/stop/restart operations\n",
|
|
"- Professional service management"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Step 1: Understanding Systemd\n",
|
|
"\n",
|
|
"### What is Systemd?\n",
|
|
"- **Service manager** for Linux systems\n",
|
|
"- **Handles** starting, stopping, and monitoring services\n",
|
|
"- **Manages dependencies** between services\n",
|
|
"- **Provides logging** through journald\n",
|
|
"- **Standard** on modern Linux distributions"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Step 2: Create Your First Service File\n",
|
|
"\n",
|
|
"### 2.1 Basic Service File"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"source": [
|
|
"# Create the service file\n",
|
|
"sudo nano /etc/systemd/system/trading-bot.service"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"**Basic service file content:**"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"source": [
|
|
"[Unit]\n",
|
|
"Description=Crypto Trading Bot\n",
|
|
"After=network.target\n",
|
|
"\n",
|
|
"[Service]\n",
|
|
"Type=simple\n",
|
|
"User=root\n",
|
|
"WorkingDirectory=/root/trading-bot\n",
|
|
"ExecStart=/usr/bin/python3 /root/trading-bot/main.py\n",
|
|
"Restart=always\n",
|
|
"RestartSec=10\n",
|
|
"\n",
|
|
"[Install]\n",
|
|
"WantedBy=multi-user.target"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"### 2.2 Enable and Start the Service"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"source": [
|
|
"# Reload systemd to recognize new service\n",
|
|
"sudo systemctl daemon-reload\n",
|
|
"\n",
|
|
"# Enable service (start automatically on boot)\n",
|
|
"sudo systemctl enable trading-bot\n",
|
|
"\n",
|
|
"# Start the service now\n",
|
|
"sudo systemctl start trading-bot\n",
|
|
"\n",
|
|
"# Check service status\n",
|
|
"sudo systemctl status trading-bot"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Step 3: Service Management Commands"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"source": [
|
|
"# Start service\n",
|
|
"sudo systemctl start trading-bot\n",
|
|
"\n",
|
|
"# Stop service\n",
|
|
"sudo systemctl stop trading-bot\n",
|
|
"\n",
|
|
"# Restart service\n",
|
|
"sudo systemctl restart trading-bot\n",
|
|
"\n",
|
|
"# Reload service configuration\n",
|
|
"sudo systemctl reload trading-bot\n",
|
|
"\n",
|
|
"# Check service status\n",
|
|
"sudo systemctl status trading-bot\n",
|
|
"\n",
|
|
"# Enable auto-start on boot\n",
|
|
"sudo systemctl enable trading-bot\n",
|
|
"\n",
|
|
"# Disable auto-start on boot\n",
|
|
"sudo systemctl disable trading-bot\n",
|
|
"\n",
|
|
"# Check if service is enabled\n",
|
|
"sudo systemctl is-enabled trading-bot\n",
|
|
"\n",
|
|
"# Check if service is active\n",
|
|
"sudo systemctl is-active trading-bot"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Step 4: Advanced Service Configuration\n",
|
|
"\n",
|
|
"### 4.1 Enhanced Service File with Environment Variables"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"source": [
|
|
"# Create enhanced service file\n",
|
|
"sudo nano /etc/systemd/system/trading-bot-advanced.service"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"**Advanced service file content:**"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"source": [
|
|
"[Unit]\n",
|
|
"Description=Advanced Crypto Trading Bot\n",
|
|
"Documentation=https://your-docs.com\n",
|
|
"After=network-online.target\n",
|
|
"Wants=network-online.target\n",
|
|
"RequiresMountsFor=/root/trading-bot\n",
|
|
"\n",
|
|
"[Service]\n",
|
|
"Type=simple\n",
|
|
"User=root\n",
|
|
"Group=root\n",
|
|
"WorkingDirectory=/root/trading-bot\n",
|
|
"\n",
|
|
"# Environment variables\n",
|
|
"Environment=PYTHONPATH=/root/trading-bot\n",
|
|
"Environment=PYTHON_ENV=production\n",
|
|
"EnvironmentFile=-/root/trading-bot/.env\n",
|
|
"\n",
|
|
"# Main execution\n",
|
|
"ExecStartPre=/bin/mkdir -p /var/log/trading-bot\n",
|
|
"ExecStart=/usr/bin/python3 /root/trading-bot/main.py\n",
|
|
"ExecReload=/bin/kill -HUP $MAINPID\n",
|
|
"ExecStop=/bin/kill -TERM $MAINPID\n",
|
|
"\n",
|
|
"# Restart configuration\n",
|
|
"Restart=always\n",
|
|
"RestartSec=10\n",
|
|
"StartLimitInterval=60\n",
|
|
"StartLimitBurst=3\n",
|
|
"\n",
|
|
"# Security settings\n",
|
|
"NoNewPrivileges=yes\n",
|
|
"PrivateTmp=yes\n",
|
|
"ProtectSystem=strict\n",
|
|
"ProtectHome=yes\n",
|
|
"ReadWritePaths=/root/trading-bot /var/log/trading-bot\n",
|
|
"\n",
|
|
"# Resource limits\n",
|
|
"LimitNOFILE=65536\n",
|
|
"MemoryHigh=512M\n",
|
|
"MemoryMax=1G\n",
|
|
"\n",
|
|
"# Logging\n",
|
|
"StandardOutput=journal\n",
|
|
"StandardError=journal\n",
|
|
"SyslogIdentifier=trading-bot\n",
|
|
"\n",
|
|
"[Install]\n",
|
|
"WantedBy=multi-user.target"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"### 4.2 Service File Sections Explained\n",
|
|
"\n",
|
|
"#### [Unit] Section\n",
|
|
"- **Description**: Human-readable description\n",
|
|
"- **After**: Start after these services\n",
|
|
"- **Wants**: Weak dependency (nice to have)\n",
|
|
"- **Requires**: Strong dependency (must have)\n",
|
|
"\n",
|
|
"#### [Service] Section\n",
|
|
"- **Type**: How systemd should manage the process\n",
|
|
"- **User/Group**: Which user runs the service\n",
|
|
"- **WorkingDirectory**: Where to run the command\n",
|
|
"- **ExecStart**: Main command to run\n",
|
|
"- **Restart**: When to restart (always, on-failure, etc.)\n",
|
|
"\n",
|
|
"#### [Install] Section\n",
|
|
"- **WantedBy**: Which target should include this service"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Step 5: Service Types and Restart Policies\n",
|
|
"\n",
|
|
"### 5.1 Service Types"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"source": [
|
|
"# Type=simple (default)\n",
|
|
"# Systemd considers service started immediately\n",
|
|
"Type=simple\n",
|
|
"\n",
|
|
"# Type=forking\n",
|
|
"# Service forks and parent exits\n",
|
|
"Type=forking\n",
|
|
"PIDFile=/var/run/trading-bot.pid\n",
|
|
"\n",
|
|
"# Type=oneshot\n",
|
|
"# Service runs once and exits\n",
|
|
"Type=oneshot\n",
|
|
"RemainAfterExit=yes\n",
|
|
"\n",
|
|
"# Type=notify\n",
|
|
"# Service sends readiness notification\n",
|
|
"Type=notify\n",
|
|
"NotifyAccess=main"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"### 5.2 Restart Policies"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"source": [
|
|
"# Restart=always (recommended for trading bots)\n",
|
|
"# Always restart, regardless of exit status\n",
|
|
"Restart=always\n",
|
|
"\n",
|
|
"# Restart=on-failure\n",
|
|
"# Restart only on failure (non-zero exit)\n",
|
|
"Restart=on-failure\n",
|
|
"\n",
|
|
"# Restart=on-abnormal\n",
|
|
"# Restart on abnormal exit (signals, timeouts)\n",
|
|
"Restart=on-abnormal\n",
|
|
"\n",
|
|
"# Restart=on-abort\n",
|
|
"# Restart only on abort signals\n",
|
|
"Restart=on-abort\n",
|
|
"\n",
|
|
"# Additional restart settings\n",
|
|
"RestartSec=10 # Wait 10 seconds before restart\n",
|
|
"StartLimitBurst=3 # Max 3 restarts\n",
|
|
"StartLimitInterval=60 # Within 60 seconds"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Step 6: Logging and Monitoring\n",
|
|
"\n",
|
|
"### 6.1 Viewing Service Logs"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"source": [
|
|
"# View recent logs\n",
|
|
"sudo journalctl -u trading-bot\n",
|
|
"\n",
|
|
"# Follow logs in real-time\n",
|
|
"sudo journalctl -u trading-bot -f\n",
|
|
"\n",
|
|
"# View logs from today\n",
|
|
"sudo journalctl -u trading-bot --since today\n",
|
|
"\n",
|
|
"# View logs from specific time\n",
|
|
"sudo journalctl -u trading-bot --since \"2024-01-01 00:00:00\"\n",
|
|
"\n",
|
|
"# View last 100 lines\n",
|
|
"sudo journalctl -u trading-bot -n 100\n",
|
|
"\n",
|
|
"# View logs with priority (error, warning, info)\n",
|
|
"sudo journalctl -u trading-bot -p err\n",
|
|
"\n",
|
|
"# Export logs to file\n",
|
|
"sudo journalctl -u trading-bot > /tmp/trading-bot.log"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"### 6.2 Log Configuration"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"source": [
|
|
"# Configure journal log retention\n",
|
|
"sudo nano /etc/systemd/journald.conf"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"**Add to journald.conf:**"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"source": [
|
|
"[Journal]\n",
|
|
"# Keep logs for 30 days\n",
|
|
"MaxRetentionSec=30d\n",
|
|
"# Limit journal size to 1GB\n",
|
|
"SystemMaxUse=1G\n",
|
|
"# Limit individual log files to 100MB\n",
|
|
"SystemMaxFileSize=100M"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Step 7: Multiple Bot Services\n",
|
|
"\n",
|
|
"### 7.1 Template Service Files"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"source": [
|
|
"# Create template service\n",
|
|
"sudo nano /etc/systemd/system/trading-bot@.service"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"**Template service content:**"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"source": [
|
|
"[Unit]\n",
|
|
"Description=Trading Bot - %i\n",
|
|
"After=network.target\n",
|
|
"\n",
|
|
"[Service]\n",
|
|
"Type=simple\n",
|
|
"User=root\n",
|
|
"WorkingDirectory=/root/trading-bots/%i\n",
|
|
"ExecStart=/usr/bin/python3 /root/trading-bots/%i/main.py\n",
|
|
"Restart=always\n",
|
|
"RestartSec=10\n",
|
|
"Environment=BOT_NAME=%i\n",
|
|
"\n",
|
|
"[Install]\n",
|
|
"WantedBy=multi-user.target"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"### 7.2 Managing Multiple Bots"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"source": [
|
|
"# Create bot directories\n",
|
|
"mkdir -p /root/trading-bots/{btc-bot,eth-bot,doge-bot}\n",
|
|
"\n",
|
|
"# Start specific bot instances\n",
|
|
"sudo systemctl start trading-bot@btc-bot\n",
|
|
"sudo systemctl start trading-bot@eth-bot\n",
|
|
"sudo systemctl start trading-bot@doge-bot\n",
|
|
"\n",
|
|
"# Enable auto-start for specific bots\n",
|
|
"sudo systemctl enable trading-bot@btc-bot\n",
|
|
"sudo systemctl enable trading-bot@eth-bot\n",
|
|
"\n",
|
|
"# Check status of all bot instances\n",
|
|
"sudo systemctl status 'trading-bot@*'\n",
|
|
"\n",
|
|
"# View logs for specific bot\n",
|
|
"sudo journalctl -u trading-bot@btc-bot -f"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Step 8: Service Dependencies and Ordering\n",
|
|
"\n",
|
|
"### 8.1 Creating Dependencies"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"source": [
|
|
"# Example: Trading bot that depends on database\n",
|
|
"sudo nano /etc/systemd/system/trading-bot-with-deps.service"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"**Service with dependencies:**"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"source": [
|
|
"[Unit]\n",
|
|
"Description=Trading Bot with Database Dependency\n",
|
|
"After=network-online.target postgresql.service\n",
|
|
"Wants=network-online.target\n",
|
|
"Requires=postgresql.service\n",
|
|
"BindsTo=postgresql.service\n",
|
|
"\n",
|
|
"[Service]\n",
|
|
"Type=simple\n",
|
|
"User=root\n",
|
|
"WorkingDirectory=/root/trading-bot\n",
|
|
"ExecStart=/usr/bin/python3 /root/trading-bot/main.py\n",
|
|
"Restart=always\n",
|
|
"RestartSec=10\n",
|
|
"\n",
|
|
"# Stop if database stops\n",
|
|
"PartOf=postgresql.service\n",
|
|
"\n",
|
|
"[Install]\n",
|
|
"WantedBy=multi-user.target"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Step 9: Health Checks and Monitoring\n",
|
|
"\n",
|
|
"### 9.1 Service with Health Checks"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"source": [
|
|
"# Create health check script\n",
|
|
"nano /root/trading-bot/health_check.py"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"**Health check script:**"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"source": [
|
|
"#!/usr/bin/env python3\n",
|
|
"import sys\n",
|
|
"import os\n",
|
|
"import time\n",
|
|
"from datetime import datetime, timedelta\n",
|
|
"\n",
|
|
"def check_bot_health():\n",
|
|
" \"\"\"Check if bot is healthy\"\"\"\n",
|
|
" \n",
|
|
" # Check if heartbeat file exists and is recent\n",
|
|
" heartbeat_file = '/tmp/trading-bot-heartbeat'\n",
|
|
" \n",
|
|
" if not os.path.exists(heartbeat_file):\n",
|
|
" print(\"ERROR: Heartbeat file not found\")\n",
|
|
" return False\n",
|
|
" \n",
|
|
" # Check if heartbeat is recent (within last 5 minutes)\n",
|
|
" file_time = datetime.fromtimestamp(os.path.getmtime(heartbeat_file))\n",
|
|
" if datetime.now() - file_time > timedelta(minutes=5):\n",
|
|
" print(\"ERROR: Heartbeat file is too old\")\n",
|
|
" return False\n",
|
|
" \n",
|
|
" # Check log file for errors\n",
|
|
" log_file = '/root/trading-bot/bot.log'\n",
|
|
" if os.path.exists(log_file):\n",
|
|
" with open(log_file, 'r') as f:\n",
|
|
" lines = f.readlines()[-10:] # Last 10 lines\n",
|
|
" for line in lines:\n",
|
|
" if 'CRITICAL' in line or 'FATAL' in line:\n",
|
|
" print(\"ERROR: Critical error found in logs\")\n",
|
|
" return False\n",
|
|
" \n",
|
|
" print(\"OK: Bot is healthy\")\n",
|
|
" return True\n",
|
|
"\n",
|
|
"if __name__ == \"__main__\":\n",
|
|
" if check_bot_health():\n",
|
|
" sys.exit(0)\n",
|
|
" else:\n",
|
|
" sys.exit(1)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"### 9.2 Service with Health Monitoring"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"source": [
|
|
"# Create service with health checks\n",
|
|
"sudo nano /etc/systemd/system/trading-bot-monitored.service"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"**Service with monitoring:**"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"source": [
|
|
"[Unit]\n",
|
|
"Description=Monitored Trading Bot\n",
|
|
"After=network.target\n",
|
|
"\n",
|
|
"[Service]\n",
|
|
"Type=simple\n",
|
|
"User=root\n",
|
|
"WorkingDirectory=/root/trading-bot\n",
|
|
"ExecStart=/usr/bin/python3 /root/trading-bot/main.py\n",
|
|
"ExecStartPost=/bin/sleep 30\n",
|
|
"ExecReload=/root/trading-bot/health_check.py\n",
|
|
"\n",
|
|
"# Health monitoring\n",
|
|
"Restart=always\n",
|
|
"RestartSec=10\n",
|
|
"WatchdogSec=300\n",
|
|
"NotifyAccess=all\n",
|
|
"\n",
|
|
"# Failure detection\n",
|
|
"StartLimitBurst=5\n",
|
|
"StartLimitInterval=300\n",
|
|
"\n",
|
|
"[Install]\n",
|
|
"WantedBy=multi-user.target"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Step 10: Notification and Alerting\n",
|
|
"\n",
|
|
"### 10.1 Email Alerts on Failure"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"source": [
|
|
"# Create alert script\n",
|
|
"nano /root/scripts/trading-bot-alert.sh"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"**Alert script:**"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"source": [
|
|
"#!/bin/bash\n",
|
|
"\n",
|
|
"SERVICE=$1\n",
|
|
"STATUS=$2\n",
|
|
"TIMESTAMP=$(date)\n",
|
|
"HOSTNAME=$(hostname)\n",
|
|
"\n",
|
|
"# Email configuration\n",
|
|
"TO_EMAIL=\"your-email@example.com\"\n",
|
|
"SUBJECT=\"Trading Bot Alert: $SERVICE $STATUS\"\n",
|
|
"\n",
|
|
"# Create email body\n",
|
|
"EMAIL_BODY=\"\n",
|
|
"Trading Bot Alert\n",
|
|
"================\n",
|
|
"\n",
|
|
"Service: $SERVICE\n",
|
|
"Status: $STATUS\n",
|
|
"Timestamp: $TIMESTAMP\n",
|
|
"Hostname: $HOSTNAME\n",
|
|
"\n",
|
|
"Recent logs:\n",
|
|
"$(sudo journalctl -u $SERVICE -n 20 --no-pager)\n",
|
|
"\n",
|
|
"System status:\n",
|
|
"$(df -h)\n",
|
|
"$(free -h)\n",
|
|
"\"\n",
|
|
"\n",
|
|
"# Send email (requires mail command)\n",
|
|
"echo \"$EMAIL_BODY\" | mail -s \"$SUBJECT\" \"$TO_EMAIL\"\n",
|
|
"\n",
|
|
"# Send Telegram notification (optional)\n",
|
|
"if [ ! -z \"$TELEGRAM_BOT_TOKEN\" ] && [ ! -z \"$TELEGRAM_CHAT_ID\" ]; then\n",
|
|
" curl -s -X POST \"https://api.telegram.org/bot$TELEGRAM_BOT_TOKEN/sendMessage\" \\\n",
|
|
" -d chat_id=\"$TELEGRAM_CHAT_ID\" \\\n",
|
|
" -d text=\"🚨 Trading Bot Alert: $SERVICE $STATUS on $HOSTNAME at $TIMESTAMP\"\n",
|
|
"fi"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"### 10.2 Service with Failure Notifications"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"source": [
|
|
"# Create service with failure hooks\n",
|
|
"sudo nano /etc/systemd/system/trading-bot-alerts.service"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"**Service with alerts:**"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"source": [
|
|
"[Unit]\n",
|
|
"Description=Trading Bot with Alerts\n",
|
|
"After=network.target\n",
|
|
"OnFailure=trading-bot-failure@%n.service\n",
|
|
"\n",
|
|
"[Service]\n",
|
|
"Type=simple\n",
|
|
"User=root\n",
|
|
"WorkingDirectory=/root/trading-bot\n",
|
|
"ExecStart=/usr/bin/python3 /root/trading-bot/main.py\n",
|
|
"ExecStartPost=/root/scripts/trading-bot-alert.sh %n started\n",
|
|
"ExecStopPost=/root/scripts/trading-bot-alert.sh %n stopped\n",
|
|
"Restart=always\n",
|
|
"RestartSec=10\n",
|
|
"\n",
|
|
"[Install]\n",
|
|
"WantedBy=multi-user.target"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"### 10.3 Failure Handler Service"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"source": [
|
|
"# Create failure handler\n",
|
|
"sudo nano /etc/systemd/system/trading-bot-failure@.service"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"**Failure handler service:**"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"source": [
|
|
"[Unit]\n",
|
|
"Description=Trading Bot Failure Handler for %i\n",
|
|
"\n",
|
|
"[Service]\n",
|
|
"Type=oneshot\n",
|
|
"ExecStart=/root/scripts/trading-bot-alert.sh %i failed\n",
|
|
"User=root"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Step 11: Backup and Recovery\n",
|
|
"\n",
|
|
"### 11.1 Service State Backup"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"source": [
|
|
"# Create backup script\n",
|
|
"nano /root/scripts/backup-trading-bot.sh"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"**Backup script:**"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"source": [
|
|
"#!/bin/bash\n",
|
|
"\n",
|
|
"BACKUP_DIR=\"/root/backups/$(date +%Y%m%d_%H%M%S)\"\n",
|
|
"BOT_DIR=\"/root/trading-bot\"\n",
|
|
"\n",
|
|
"# Create backup directory\n",
|
|
"mkdir -p \"$BACKUP_DIR\"\n",
|
|
"\n",
|
|
"# Stop service\n",
|
|
"sudo systemctl stop trading-bot\n",
|
|
"\n",
|
|
"# Backup files\n",
|
|
"cp -r \"$BOT_DIR\" \"$BACKUP_DIR/\"\n",
|
|
"cp /etc/systemd/system/trading-bot*.service \"$BACKUP_DIR/\"\n",
|
|
"\n",
|
|
"# Backup service status\n",
|
|
"sudo systemctl status trading-bot > \"$BACKUP_DIR/service-status.txt\"\n",
|
|
"sudo journalctl -u trading-bot -n 1000 > \"$BACKUP_DIR/recent-logs.txt\"\n",
|
|
"\n",
|
|
"# Start service\n",
|
|
"sudo systemctl start trading-bot\n",
|
|
"\n",
|
|
"echo \"Backup completed: $BACKUP_DIR\""
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"### 11.2 Automated Backup Service"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"source": [
|
|
"# Create backup service\n",
|
|
"sudo nano /etc/systemd/system/trading-bot-backup.service"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"**Backup service:**"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"source": [
|
|
"[Unit]\n",
|
|
"Description=Trading Bot Backup\n",
|
|
"Requires=trading-bot.service\n",
|
|
"\n",
|
|
"[Service]\n",
|
|
"Type=oneshot\n",
|
|
"ExecStart=/root/scripts/backup-trading-bot.sh\n",
|
|
"User=root"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"### 11.3 Backup Timer"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"source": [
|
|
"# Create backup timer\n",
|
|
"sudo nano /etc/systemd/system/trading-bot-backup.timer"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"**Backup timer:**"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"source": [
|
|
"[Unit]\n",
|
|
"Description=Trading Bot Daily Backup\n",
|
|
"Requires=trading-bot-backup.service\n",
|
|
"\n",
|
|
"[Timer]\n",
|
|
"OnCalendar=daily\n",
|
|
"Persistent=true\n",
|
|
"\n",
|
|
"[Install]\n",
|
|
"WantedBy=timers.target"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"source": [
|
|
"# Enable and start backup timer\n",
|
|
"sudo systemctl enable trading-bot-backup.timer\n",
|
|
"sudo systemctl start trading-bot-backup.timer\n",
|
|
"\n",
|
|
"# Check timer status\n",
|
|
"sudo systemctl status trading-bot-backup.timer\n",
|
|
"\n",
|
|
"# List all timers\n",
|
|
"sudo systemctl list-timers"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Step 12: Troubleshooting and Debugging\n",
|
|
"\n",
|
|
"### 12.1 Common Issues"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"source": [
|
|
"# Service won't start\n",
|
|
"sudo systemctl status trading-bot\n",
|
|
"sudo journalctl -u trading-bot -n 50\n",
|
|
"\n",
|
|
"# Check service file syntax\n",
|
|
"sudo systemd-analyze verify /etc/systemd/system/trading-bot.service\n",
|
|
"\n",
|
|
"# Service starts but immediately fails\n",
|
|
"sudo journalctl -u trading-bot --since \"5 minutes ago\"\n",
|
|
"\n",
|
|
"# Permission issues\n",
|
|
"sudo ls -la /root/trading-bot/\n",
|
|
"sudo chmod +x /root/trading-bot/main.py\n",
|
|
"\n",
|
|
"# Environment variable issues\n",
|
|
"sudo systemctl show trading-bot --property=Environment\n",
|
|
"\n",
|
|
"# Service dependency issues\n",
|
|
"sudo systemctl list-dependencies trading-bot"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"### 12.2 Debug Mode Service"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"source": [
|
|
"# Create debug version of service\n",
|
|
"sudo nano /etc/systemd/system/trading-bot-debug.service"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"**Debug service:**"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"source": [
|
|
"[Unit]\n",
|
|
"Description=Trading Bot (Debug Mode)\n",
|
|
"After=network.target\n",
|
|
"\n",
|
|
"[Service]\n",
|
|
"Type=simple\n",
|
|
"User=root\n",
|
|
"WorkingDirectory=/root/trading-bot\n",
|
|
"ExecStart=/usr/bin/python3 -u -v /root/trading-bot/main.py\n",
|
|
"Environment=PYTHONUNBUFFERED=1\n",
|
|
"Environment=DEBUG=1\n",
|
|
"StandardOutput=journal+console\n",
|
|
"StandardError=journal+console\n",
|
|
"Restart=no\n",
|
|
"\n",
|
|
"[Install]\n",
|
|
"WantedBy=multi-user.target"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Step 13: Performance Monitoring\n",
|
|
"\n",
|
|
"### 13.1 Resource Usage Monitoring"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"source": [
|
|
"# Show service resource usage\n",
|
|
"sudo systemctl status trading-bot\n",
|
|
"\n",
|
|
"# Detailed resource usage\n",
|
|
"sudo systemd-cgtop\n",
|
|
"\n",
|
|
"# Memory usage of service\n",
|
|
"sudo systemctl show trading-bot --property=MemoryCurrent\n",
|
|
"\n",
|
|
"# CPU usage\n",
|
|
"sudo systemctl show trading-bot --property=CPUUsageNSec\n",
|
|
"\n",
|
|
"# All service properties\n",
|
|
"sudo systemctl show trading-bot"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Pros and Cons Summary\n",
|
|
"\n",
|
|
"### ✅ Pros:\n",
|
|
"- **Professional deployment** - industry standard approach\n",
|
|
"- **Automatic restart** on crash or system reboot\n",
|
|
"- **Structured logging** - integrated with system logs\n",
|
|
"- **Resource management** - memory and CPU limits\n",
|
|
"- **Dependency management** - service ordering and requirements\n",
|
|
"- **Security features** - sandboxing and privilege restrictions\n",
|
|
"- **Monitoring integration** - works with system monitoring tools\n",
|
|
"- **Scalable** - easy to manage multiple services\n",
|
|
"\n",
|
|
"### ❌ Cons:\n",
|
|
"- **Complex setup** - requires understanding of systemd\n",
|
|
"- **Learning curve** - many configuration options\n",
|
|
"- **Root access required** - need sudo for service management\n",
|
|
"- **Less interactive** - harder to debug running processes\n",
|
|
"- **System-specific** - tied to systemd-based Linux distributions\n",
|
|
"\n",
|
|
"### 🎯 Best Use Cases:\n",
|
|
"- Production trading bot deployments\n",
|
|
"- Multiple trading strategies running simultaneously\n",
|
|
"- When you need reliable 24/7 operation\n",
|
|
"- Professional or commercial trading operations\n",
|
|
"- When automatic restart and monitoring are critical\n",
|
|
"- Server environments with multiple services"
|
|
]
|
|
}
|
|
],
|
|
"metadata": {
|
|
"kernelspec": {
|
|
"display_name": "Python 3",
|
|
"language": "python",
|
|
"name": "python3"
|
|
},
|
|
"language_info": {
|
|
"codemirror_mode": {
|
|
"name": "ipython",
|
|
"version": 3
|
|
},
|
|
"file_extension": ".py",
|
|
"mimetype": "text/x-python",
|
|
"name": "python",
|
|
"nbconvert_exporter": "python",
|
|
"pygments_lexer": "ipython3",
|
|
"version": "3.8.5"
|
|
}
|
|
},
|
|
"nbformat": 4,
|
|
"nbformat_minor": 4
|
|
} |