v1.91.6
Cookbook

API Metrics

Async API performance and usage analytics with batching, daily aggregates, rate limit visibility, and system health endpoints.

This comprehensive guide covers Glueful's enterprise-grade API metrics and monitoring system, which provides real-time performance tracking, comprehensive analytics, rate limiting monitoring, and detailed endpoint statistics for production API management.

Overview

Glueful's API Metrics system provides comprehensive monitoring and analytics for API performance, usage patterns, error tracking, and system health. The system is designed for production environments with asynchronous processing, intelligent caching, and minimal performance impact.

Key Features

  • Asynchronous Metrics Collection: Non-blocking metrics recording with cache-based queuing
  • Dual-Table Storage: Raw metrics and daily aggregates for optimal query performance
  • Rate Limiting Integration: Built-in rate limiting monitoring and threshold alerts
  • Endpoint Categorization: Automatic categorization based on URL patterns
  • Comprehensive Analytics: Response times, error rates, usage patterns, and system health
  • Permission-Based Access: Role-based data filtering and access control
  • Production-Ready: Automatic data retention, bulk processing, and memory optimization

Architecture Components

  1. ApiMetricsService: Core metrics collection and aggregation engine
  2. MetricsMiddleware: RouteMiddleware for request interception
  3. MetricsController: REST API endpoints for metrics retrieval
  4. Database Schema: Optimized tables for raw and aggregated data storage

ApiMetricsService Core Features

Basic Usage

use Glueful\Services\ApiMetricsService;

// Resolve from the container (autowired with the cache store and DB connection)
$metricsService = app($context, ApiMetricsService::class);

// Record metric asynchronously (non-blocking)
$metricsService->recordMetricAsync([
    'endpoint' => '/api/users',
    'method' => 'GET',
    'response_time' => 156.4,
    'status_code' => 200,
    'is_error' => false,
    'timestamp' => time(),
    'ip' => '192.168.1.100'
]);

// Get comprehensive metrics
$metrics = $metricsService->getApiMetrics();

// Reset all metrics
$success = $metricsService->resetApiMetrics();

Asynchronous Processing

The metrics system uses cache-based queuing to avoid impacting API performance:

// Metrics are queued in cache for batch processing
$metric = [
    'endpoint' => $request->getPathInfo(),
    'method' => $request->getMethod(),
    'response_time' => 234.5,
    'status_code' => 200,
    'is_error' => false,
    'timestamp' => time(),
    'ip' => $request->getClientIp()
];

// Queue metric (non-blocking)
$metricsService->recordMetricAsync($metric);

// Automatic batch flush when threshold reached (default: 50 metrics)
// Or manual flush
$metricsService->flushMetrics();

Comprehensive Metrics Report

The getApiMetrics() method returns detailed analytics:

[
    'endpoints' => [
        [
            'endpoint' => '/api/users',
            'method' => 'GET',
            'route' => '/api/users',
            'calls' => 1247,
            'avgResponseTime' => 145.6,
            'errorRate' => 2.1,
            'lastCalled' => '2025-01-02 14:30:25',
            'category' => 'Users'
        ],
        [
            'endpoint' => '/api/auth/login',
            'method' => 'POST',
            'route' => '/api/auth/login',
            'calls' => 892,
            'avgResponseTime' => 89.3,
            'errorRate' => 0.8,
            'lastCalled' => '2025-01-02 14:28:45',
            'category' => 'Auth'
        ]
    ],
    'total_requests' => 15847,
    'avg_response_time' => 156.4,
    'total_errors' => 234,
    'error_rate' => 1.48,
    'rate_limits' => [
        [
            'ip' => '192.168.1.100',
            'endpoint' => '/api/users',
            'remaining' => 15,
            'limit' => 100,
            'reset_time' => '2025-01-02 15:00:00',
            'usage_percentage' => 85.0
        ]
    ],
    'requests_over_time' => [
        ['date' => '2025-01-01', 'count' => 5234],
        ['date' => '2025-01-02', 'count' => 6789]
    ],
    'categories' => ['Users', 'Auth', 'Products', 'Orders'],
    'category_distribution' => [
        ['category' => 'Users', 'count' => 4567],
        ['category' => 'Auth', 'count' => 2345],
        ['category' => 'Products', 'count' => 3456],
        ['category' => 'Orders', 'count' => 1234]
    ],
    'top_endpoints' => [
        // Top 5 endpoints by call volume
    ]
]

MetricsMiddleware Integration

The middleware automatically captures metrics for all API requests:

Setup

// Apply the built-in metrics middleware (alias registered as 'metrics')
$router->get('/api/users', [UserController::class, 'index'])
    ->middleware(['metrics']);

// Or reference the class directly
use Glueful\Routing\Middleware\MetricsMiddleware;

$router->post('/api/orders', [OrderController::class, 'create'])
    ->middleware([MetricsMiddleware::class]);

Features

  • Glueful RouteMiddleware: Native Glueful middleware (RouteMiddleware)
  • Automatic Request Tracking: Captures endpoint, method, response time, status codes
  • Error Handling: Graceful failure - metrics issues don't break API functionality
  • IP Address Tracking: Client IP recording for rate limiting and analytics

Implementation Details

Note: Metrics recording is handled by Glueful\Routing\Middleware\MetricsMiddleware, which measures request duration, status code, and enqueues metrics via ApiMetricsService::recordMetricAsync(...) without impacting the request lifecycle.

MetricsController API Endpoints

Glueful\Controllers\MetricsController is registered in the container, but core does not register any /metrics routes. The endpoints below exist only once your app maps them. A typical registration in an app route file (for example routes/api.php):

use Glueful\Controllers\MetricsController;
use Glueful\Routing\Router;

$router->group(['prefix' => '/metrics', 'middleware' => ['auth']], function (Router $router) {
    $router->get('/api', [MetricsController::class, 'getApiMetrics']);
    $router->post('/api/reset', [MetricsController::class, 'resetApiMetrics']);
    $router->get('/system', [MetricsController::class, 'systemHealth']);
});

The paths are your choice; the examples below assume the ones above. Each action calls requirePermission(): unauthenticated requests are refused, and the permission is decided by the installed permission provider (for example glueful/aegis) or, without one, by the Gate (config roles and voters in config/permissions.php). See Permissions and Authorization. Before 1.91, Gate-only apps refused every call with "Permission system unavailable".

API Endpoints

Get API Metrics

GET /metrics/api
Authorization: Bearer <token>

Response:

{
    "success": true,
    "message": "API metrics retrieved successfully",
    "data": {
        "endpoints": [...],
        "total_requests": 15847,
        "avg_response_time": 156.4,
        "error_rate": 1.48,
        "rate_limits": [...],
        "requests_over_time": [...]
    }
}

Reset API Metrics

POST /metrics/api/reset
Authorization: Bearer <token>

Response:

{
    "success": true,
    "message": "API metrics reset successfully"
}

System Health Metrics

GET /metrics/system
Authorization: Bearer <token>

Response:

{
    "success": true,
    "message": "System health metrics retrieved successfully",
    "data": {
        "php": {
            "version": "8.2.15",
            "memory_limit": "256M",
            "max_execution_time": "30"
        },
        "memory": {
            "current_usage": "45.2 MB",
            "peak_usage": "67.8 MB"
        },
        "database": {
            "status": "connected",
            "response_time_ms": 12.4,
            "table_count": 15,
            "total_size": "124.5 MB"
        },
        "cache": {
            "type": "Redis",
            "status": "enabled",
            "memory_usage": "89.3 MB",
            "hit_rate": "95.6%"
        }
    }
}

Permission-Based Access Control

// Different data levels based on user roles
if ($this->isAdmin()) {
    // Full access to all metrics and sensitive data
    $ttl = 60; // 1 minute cache for admins
} else {
    // Limited data for regular users
    $ttl = 300; // 5 minutes cache for users
    // Remove sensitive information
}

// Permission checks
$this->requirePermission('system.metrics.view', 'metrics:api');
$this->requirePermission('system.health.view', 'metrics:system');
$this->requirePermission('system.metrics.reset', 'metrics:api');

Database Schema

The metrics system uses three optimized tables:

api_metrics (Raw Metrics)

CREATE TABLE api_metrics (
    id BIGINT PRIMARY KEY AUTO_INCREMENT,
    uuid CHAR(12) NOT NULL,
    endpoint VARCHAR(255) NOT NULL,
    method VARCHAR(10) NOT NULL,
    response_time DECIMAL(10,2) NOT NULL,
    status_code INT NOT NULL,
    is_error TINYINT(1) NOT NULL DEFAULT 0,
    timestamp DATETIME NOT NULL,
    ip VARCHAR(45) NOT NULL,
    INDEX idx_api_metrics_timestamp (timestamp),
    INDEX idx_api_metrics_endpoint_method (endpoint, method)
);

api_metrics_daily (Aggregated Metrics)

CREATE TABLE api_metrics_daily (
    id BIGINT PRIMARY KEY AUTO_INCREMENT,
    uuid CHAR(12) NOT NULL,
    date DATE NOT NULL,
    endpoint VARCHAR(255) NOT NULL,
    method VARCHAR(10) NOT NULL,
    endpoint_key VARCHAR(266) NOT NULL,
    calls INT NOT NULL DEFAULT 0,
    total_response_time DECIMAL(15,2) NOT NULL DEFAULT 0,
    error_count INT NOT NULL DEFAULT 0,
    last_called DATETIME NULL,
    UNIQUE KEY idx_api_metrics_daily_date_endpoint_key (date, endpoint_key),
    INDEX idx_api_metrics_daily_date (date)
);

api_rate_limits (Rate Limiting)

CREATE TABLE api_rate_limits (
    id BIGINT PRIMARY KEY AUTO_INCREMENT,
    uuid CHAR(12) NOT NULL,
    ip VARCHAR(45) NOT NULL,
    endpoint VARCHAR(255) NOT NULL,
    remaining INT NOT NULL,
    limit INT NOT NULL,
    reset_time DATETIME NOT NULL,
    usage_percentage FLOAT NOT NULL,
    UNIQUE KEY idx_api_rate_limits_ip_endpoint (ip, endpoint)
);

Real-time Metrics Collection

Batch Processing

Metrics are collected asynchronously and processed in batches for optimal performance:

// Configuration
private int $metricsFlushThreshold = 50; // Flush after 50 metrics
private int $metricsTTL = 86400 * 30; // 30 days retention
private int $aggregatedMetricsTTL = 86400 * 365; // 1 year retention

// Automatic batch processing
public function recordMetricAsync(array $metric): void
{
    $pendingMetrics = $this->cache->get('api_metrics_pending') ?? [];
    $pendingMetrics[] = $metric;
    
    $this->cache->set('api_metrics_pending', $pendingMetrics, 86400);
    
    // Auto-flush when threshold reached
    if (count($pendingMetrics) >= $this->metricsFlushThreshold) {
        $this->flushMetrics();
    }
}

Daily Aggregation

Raw metrics are aggregated daily for efficient querying:

private function updateDailyAggregatesBulk(array $metrics): void
{
    $aggregates = [];
    
    foreach ($metrics as $metric) {
        $date = date('Y-m-d', $metric['timestamp']);
        $key = $metric['endpoint'] . '|' . $metric['method'];
        $combinedKey = $date . '|' . $key;
        
        if (!isset($aggregates[$combinedKey])) {
            $aggregates[$combinedKey] = [
                'date' => $date,
                'endpoint' => $metric['endpoint'],
                'method' => $metric['method'],
                'endpoint_key' => $key,
                'calls' => 0,
                'total_response_time' => 0,
                'error_count' => 0
            ];
        }
        
        $aggregates[$combinedKey]['calls']++;
        $aggregates[$combinedKey]['total_response_time'] += $metric['response_time'];
        $aggregates[$combinedKey]['error_count'] += $metric['is_error'] ? 1 : 0;
    }
    
    // Bulk insert/update operations
    $this->bulkUpdateAggregates($aggregates);
}

Rate Limiting Integration

Rate Limit Monitoring

The metrics system automatically tracks rate limiting:

private function updateRateLimit(string $ip, string $endpoint): void
{
    $key = 'rate_limit|' . $ip . '|' . $endpoint;
    $minute = floor(time() / 60) * 60;
    
    $rateLimit = $this->cache->get($key) ?? [
        'count' => 0,
        'minute' => $minute,
        'limit' => 100 // Default or endpoint-specific limit
    ];
    
    if ($rateLimit['minute'] != $minute) {
        // Reset for new minute
        $rateLimit = ['count' => 1, 'minute' => $minute, 'limit' => $rateLimit['limit']];
    } else {
        $rateLimit['count']++;
    }
    
    $this->cache->set($key, $rateLimit, 3600);
    
    // Store in database when approaching threshold (80%)
    if ($rateLimit['count'] >= $rateLimit['limit'] * 0.8) {
        $this->storeRateLimitAlert($ip, $endpoint, $rateLimit);
    }
}

Rate Limit Analytics

// Get rate limits approaching threshold
$rateLimits = $this->db->select('api_rate_limits')
    ->where(['usage_percentage' => ['>', 80]])
    ->orderBy(['usage_percentage' => 'DESC'])
    ->get();

// Example rate limit data
[
    'ip' => '192.168.1.100',
    'endpoint' => '/api/users',
    'remaining' => 15,
    'limit' => 100,
    'reset_time' => '2025-01-02 15:00:00',
    'usage_percentage' => 85.0
]

Performance Analytics

Response Time Analysis

// Endpoint performance metrics
$endpointMetrics = [
    'endpoint' => '/api/users',
    'method' => 'GET',
    'calls' => 1247,
    'avgResponseTime' => 145.6, // milliseconds
    'errorRate' => 2.1, // percentage
    'lastCalled' => '2025-01-02 14:30:25',
    'category' => 'Users'
];

// Time series data
$requestsOverTime = [
    ['date' => '2025-01-01', 'count' => 5234],
    ['date' => '2025-01-02', 'count' => 6789],
    ['date' => '2025-01-03', 'count' => 4567]
];

Endpoint Categorization

private function getCategoryFromEndpoint(string $endpoint): string
{
    $parts = explode('/', trim($endpoint, '/'));
    
    // Skip 'api' prefix if it exists
    $categoryIndex = ($parts[0] === 'api' && count($parts) > 1) ? 1 : 0;
    
    return isset($parts[$categoryIndex]) ? ucfirst($parts[$categoryIndex]) : 'Other';
}

// Example categories: Auth, Users, Products, Orders, Other

Performance Insights

// Performance analysis
function analyzeEndpointPerformance($metrics) {
    $insights = [];
    
    foreach ($metrics['endpoints'] as $endpoint) {
        if ($endpoint['avgResponseTime'] > 500) {
            $insights[] = "Slow endpoint: {$endpoint['endpoint']} ({$endpoint['avgResponseTime']}ms)";
        }
        
        if ($endpoint['errorRate'] > 5) {
            $insights[] = "High error rate: {$endpoint['endpoint']} ({$endpoint['errorRate']}%)";
        }
        
        if ($endpoint['calls'] > 10000) {
            $insights[] = "High traffic: {$endpoint['endpoint']} ({$endpoint['calls']} calls)";
        }
    }
    
    return $insights;
}

System Health Monitoring

Comprehensive Health Metrics

private function generateSystemHealthMetrics(): array
{
    return [
        'php' => [
            'version' => phpversion(),
            'memory_limit' => ini_get('memory_limit'),
            'max_execution_time' => ini_get('max_execution_time'),
            'extensions' => get_loaded_extensions() // Admin only
        ],
        'memory' => [
            'current_usage' => $this->formatBytes(memory_get_usage(true)),
            'peak_usage' => $this->formatBytes(memory_get_peak_usage(true))
        ],
        'database' => [
            'status' => 'connected',
            'response_time_ms' => 12.4,
            'table_count' => 15,
            'total_size' => '124.5 MB' // Admin only
        ],
        'file_system' => [
            'storage_free_space' => $this->formatBytes(disk_free_space($storagePath)),
            'storage_total_space' => $this->formatBytes(disk_total_space($storagePath)),
            'storage_usage_percent' => $this->calculateStoragePercentage($storagePath)
        ],
        'cache' => [
            'type' => 'Redis',
            'status' => 'enabled',
            'memory_usage' => '89.3 MB',
            'hit_rate' => '95.6%',
            'connected_clients' => 24
        ],
        'logs' => [
            'log_file_count' => 12,
            'recent_logs' => [
                'file' => 'app-2025-01-02.log',
                'last_modified' => '2025-01-02 14:30:25',
                'recent_entries' => [...] // Limited for non-admins
            ]
        ],
        'extensions' => [
            'total_count' => 8,
            'enabled_count' => 6,
            'extensions' => [...]
        ],
        'server_load' => [
            '1min' => 0.45,
            '5min' => 0.38,
            '15min' => 0.41
        ]
    ];
}

Permission-Based Filtering

private function filterMetricsByUserContext(array $metrics): array
{
    if ($this->isAdmin()) {
        return $metrics; // Full access
    }
    
    // Remove sensitive data for non-admins
    unset($metrics['file_system']['storage_path']);
    
    // Limit log entries
    if (isset($metrics['logs']['recent_logs']['recent_entries'])) {
        $metrics['logs']['recent_logs']['recent_entries'] = array_slice(
            $metrics['logs']['recent_logs']['recent_entries'], -3
        );
    }
    
    // Convert extensions list to count
    if (isset($metrics['php']['extensions'])) {
        $metrics['php']['extensions'] = count($metrics['php']['extensions']);
    }
    
    return $metrics;
}

Configuration

There is no config/api_metrics.php and no API_METRICS_* environment variables; the service does not read any. Its tuning values are fixed in ApiMetricsService:

SettingValue
Flush threshold50 pending metrics, then a batch write to the database
Pending-queue cache keyapi_metrics_pending (24-hour TTL)
Raw metrics retention30 days (older rows are purged on each flush)
Aggregated metrics retention1 year (purged on each flush)
Rate-limit counter default100 requests per IP + endpoint per minute (visibility only; enforcement is the rate_limit middleware, which API_RATE_LIMITING_ENABLED=false switches off)

The three tables (api_metrics, api_metrics_daily, api_rate_limits) are a core framework migration (glueful/framework:metrics), created by php glueful migrate:run like the other core tables. The service performs no runtime DDL.

Collection only happens on routes that carry the metrics middleware, so "turning metrics off" means not attaching it. (The former capabilities.metrics switch, METRICS_DATABASE_STORE, gated nothing and was removed in 1.91; drop it from published config copies.)

Production Notes

  • Attach metrics selectively. Every tracked request does a cache read/write for the pending queue plus a rate-limit counter update. Put it on the route groups you actually want to measure.
  • Use a shared cache store (Redis or Memcached) when you run several PHP workers, so the pending queue isn't split per process. With the file or array driver each worker buffers separately.
  • Pending metrics live in the cache for up to 24 hours. If the cache is flushed before the threshold is reached, those metrics are lost. Call flushMetrics() from a scheduled job if you need tighter guarantees:
use Glueful\Bootstrap\ApplicationContext;
use Glueful\Services\ApiMetricsService;

final class FlushApiMetrics
{
    public function __construct(private ApplicationContext $context)
    {
    }

    public function handle(): void
    {
        app($this->context, ApiMetricsService::class)->flushMetrics();
    }
}
  • Protect the read endpoints. The controller actions check system.metrics.view, system.metrics.reset, and system.health.view; grant them only to operators, through your permission provider or a role in config/permissions.php.
  • External monitoring. For Prometheus/Datadog-style monitoring, export from your own endpoint or job using getApiMetrics(); the framework ships no exporter.

Writing Middleware

Build custom RouteMiddleware — the interface, runtime parameters, registering aliases, and testing.

Performance Recipes

Short performance recipes — cache-then-invalidate, HTTP caching for public endpoints, and reverse-proxy caching. The full reference is Advanced → Performance.