Cookbook

Memory Management

Glueful's memory-management and observability tooling — the memory manager, alerting, efficient iterators, object pool, chunked DB processing, lazy container, and request tracing.

This guide consolidates Glueful's memory-management and observability tools into a single reference: real-time memory monitoring and alerting, memory-efficient iteration over large datasets, object pooling, chunked database processing, the lazy service container, and request tracing.

Memory Manager

The Memory Manager provides advanced memory monitoring, tracking, and management capabilities with configurable thresholds and automatic garbage collection.

Features

Real-time Memory Monitoring: Track current, peak, and limit usage
Automatic Garbage Collection: Trigger collection based on configurable thresholds
Memory State Tracking: Monitor allocation patterns and system health
Configurable Alerts: Set custom thresholds for different memory states

Basic Usage

use Glueful\Performance\MemoryManager;

// Resolve from the container (or construct with an optional logger/context)
$memoryManager = app($context, MemoryManager::class);

// Monitor memory usage (logs + runs GC when thresholds are exceeded)
$usage = $memoryManager->monitor();
echo "Current: {$usage['formatted']['current']}, Peak: {$usage['formatted']['peak']}";

// Check memory state
if ($memoryManager->isMemoryCritical()) {
    // Handle critical memory situation
    $memoryManager->forceGarbageCollection();
}

Configuration Options

Thresholds are read from configuration (defaults shown), not from the constructor:

// config/app.php → 'performance' (or the MEMORY_* environment overrides)
'performance' => [
    'memory' => [
        'monitoring' => [
            'enabled' => true,
            'alert_threshold' => 0.8,     // MEMORY_ALERT_THRESHOLD — high-usage alert at 80%
            'critical_threshold' => 0.9,  // MEMORY_CRITICAL_THRESHOLD — critical at 90%
        ],
    ],
],

Memory States

monitor() compares current usage (percentage = usage ÷ memory_limit) against the configured thresholds:

Normal: Usage below alert_threshold
High: Usage at/above alert_threshold (logged; optional GC) — see isMemoryHighUsage()
Critical: Usage at/above critical_threshold — see isMemoryCritical()

Advanced Features

// Inspect the active memory limit
$limit = $memoryManager->getMemoryLimit();              // bytes
$human = $memoryManager->getFormattedMemoryLimit();     // e.g. "512 MB"

// Force garbage collection
$memoryManager->forceGarbageCollection();

// Get current memory statistics (current/peak/limit/percentage + formatted values)
$stats = $memoryManager->getCurrentUsage();
print_r($stats);

// Quick threshold checks
$high     = $memoryManager->isMemoryHighUsage();
$critical = $memoryManager->isMemoryCritical();

Memory Alerting Service

The Memory Alerting Service provides intelligent memory monitoring with configurable thresholds, alert channels, and automatic escalation.

Features

Multi-Channel Alerting: Email, Slack, webhook notifications
Intelligent Throttling: Prevent alert spam with configurable intervals
Escalation Policies: Automatic escalation for critical situations
Historical Tracking: Maintain alert history and patterns

Basic Setup

use Glueful\Performance\MemoryAlertingService;

$alertService = new MemoryAlertingService([
    'channels' => [
        'email' => [
            'enabled' => true,
            'recipients' => ['[email protected]', '[email protected]'],
            'threshold' => 'warning'
        ],
        'slack' => [
            'enabled' => true,
            'webhook_url' => 'https://hooks.slack.com/...',
            'channel' => '#alerts',
            'threshold' => 'critical'
        ]
    ],
    'thresholds' => [
        'warning' => 75,   // 75% memory usage
        'critical' => 90,  // 90% memory usage
        'emergency' => 95  // 95% memory usage
    ]
]);

// Check memory and send alerts if needed
$alertService->checkAndAlert();

Alert Channels

Email Alerts

$emailConfig = [
    'enabled' => true,
    'recipients' => ['[email protected]'],
    'threshold' => 'warning',
    'throttle_minutes' => 15,
    'template' => 'memory_alert'
];

Slack Alerts

$slackConfig = [
    'enabled' => true,
    'webhook_url' => 'https://hooks.slack.com/services/...',
    'channel' => '#alerts',
    'username' => 'Glueful Monitor',
    'threshold' => 'critical',
    'throttle_minutes' => 5
];

Webhook Alerts

$webhookConfig = [
    'enabled' => true,
    'url' => 'https://your-monitoring-system.com/alerts',
    'threshold' => 'warning',
    'timeout' => 30,
    'retry_attempts' => 3
];

Escalation Policies

$escalationConfig = [
    'enabled' => true,
    'levels' => [
        1 => ['email' => ['[email protected]']],
        2 => ['email' => ['[email protected]'], 'slack' => true],
        3 => ['webhook' => 'https://pager-duty.com/...']
    ],
    'escalation_intervals' => [5, 15, 30] // minutes
];

Advanced Configuration

$advancedConfig = [
    'history_retention_days' => 30,
    'alert_cooldown_minutes' => 10,
    'batch_alerts' => true,
    'include_system_info' => true,
    'custom_metrics' => [
        'cpu_usage' => true,
        'disk_usage' => true,
        'active_connections' => true
    ]
];

Memory Efficient Iterators

Memory efficient iterators for processing large datasets without loading everything into memory.

StreamingIterator

Wrap any iterable, generator, or array and stream it through a bounded in-memory buffer:

use Glueful\Performance\StreamingIterator;

// Second argument is the buffer size (items held in memory at once); default 100.
$iterator = new StreamingIterator($dataSource, 1000);

foreach ($iterator as $item) {
    processItem($item);
}

Lazy database streaming

Stream a large result set row by row instead of fetching it all at once — pass an executed PDO statement to MemoryEfficientIterators::databaseResults(), which yields rows in batches:

use Glueful\Performance\MemoryEfficientIterators;

$stmt = $connection->getPDO()->prepare('SELECT * FROM events WHERE created_at >= ?');
$stmt->execute([$since]);

// Generator — fetches `$fetchSize` rows at a time (default 100), never the whole set.
foreach (MemoryEfficientIterators::databaseResults($stmt, 500) as $row) {
    processItem($row);
}

Mapping & filtering without buffering

map() and filter() return generators, so transforms and predicates apply lazily as you iterate — no intermediate array is built:

use Glueful\Performance\MemoryEfficientIterators;

$active = MemoryEfficientIterators::filter(
    $dataSource,
    fn ($item) => $item['status'] === 'active' && $item['score'] > 50,
);

foreach ($active as $item) {
    processActiveHighScoreItem($item);
}

// Compose helpers — stream a file line by line, then map each line:
foreach (MemoryEfficientIterators::map(
    MemoryEfficientIterators::fileLineByLine('/path/to/large.csv'),
    fn ($line) => str_getcsv($line),
) as $columns) {
    // ...
}

Chunked processing

For chunk-at-a-time work (e.g. bulk inserts), processInChunks() calls your callback once per chunk:

use Glueful\Performance\MemoryEfficientIterators;

MemoryEfficientIterators::processInChunks($dataSource, function (array $chunk) {
    // $chunk holds up to the chunk size (default 1000) items
    bulkInsert($chunk);
}, chunkSize: 1000);

Memory Pool

Object storage and reuse system to reduce memory allocation overhead.

Features

Object Pooling: Reuse expensive objects by key
Bounded Size: LRU eviction removes least-recently-used entries when the pool is full
Statistics: Monitor pool usage and efficiency

Basic Usage

MemoryPool is a bounded, keyed object store for reusing expensive-to-build objects within a process. Construct it with a maximum size, then add/get/has/remove objects by key:

use Glueful\Performance\MemoryPool;

$pool = new MemoryPool(maxSize: 100);

// Store an object under a key (evicts the least-recently-used entry if the pool is full)
$pool->add('report:' . $id, $expensiveReport);

// Reuse it later instead of rebuilding
if ($pool->has('report:' . $id)) {
    $report = $pool->get('report:' . $id);
}

$pool->remove('report:' . $id);   // drop one entry
$pool->clear();                    // drop everything

When the pool reaches maxSize, the least-recently-used entries are evicted to make room, so it bounds memory automatically.

Pool statistics

$stats = $pool->getStats();   // array of pool-usage metrics (size, capacity, hit/miss counters)
$keys  = $pool->getKeys();    // the keys currently held

Memory Monitor Command

CLI tool for monitoring and analyzing memory usage patterns.

Basic Usage

# Real-time memory monitoring
php glueful system:memory

# Monitor with custom interval
php glueful system:memory --interval=5

# Monitor an external command's memory usage
php glueful system:memory "php artisan-like-task"

# Log metrics to a CSV file
php glueful system:memory --log --csv-file=memory_report.csv

Command Options

# Monitoring options
--interval=N, -i    # Check interval in seconds (default: 1)
--duration=N, -d    # Monitor for N seconds (0 = unlimited, default)
--threshold=N, -t   # Alert threshold in MB (default: 128)
--watch, -w         # Watch mode with real-time updates

# Output options
--format=FORMAT, -f # Output format: table, json, plain (default: table)
--log, -l           # Log memory usage to a CSV file
--csv-file=FILE, -c # CSV file path (default: memory-usage.csv)

# Analysis options
--analysis, -a      # Perform memory analysis and leak detection
--profile, -p       # Enable detailed memory profiling
--trends            # Show memory usage trends and statistics
--summary, -s       # Show memory summary and recommendations
--alert-script=PATH # Script to run when the threshold is exceeded

Real-time Monitoring

# Display live memory usage
php glueful system:memory --interval=1 --format=table

┌─────────────────┬──────────────┬─────────────┬─────────────┐
│ Time            │ Current (MB) │ Peak (MB)   │ Limit (MB)  │
├─────────────────┼──────────────┼─────────────┼─────────────┤
│ 2023-10-15 14:30│ 245.7       │ 267.3       │ 512.0       │
│ 2023-10-15 14:31│ 248.2       │ 267.3       │ 512.0       │
│ 2023-10-15 14:32│ 251.8       │ 267.3       │ 512.0       │
└─────────────────┴──────────────┴─────────────┴─────────────┘

Memory Analysis

# Run for a fixed duration and log metrics to CSV
php glueful system:memory --duration=300 --log --csv-file=analysis.csv

# Perform analysis and leak detection
php glueful system:memory --analysis --summary

Memory Usage Analysis Report
============================
Average Usage: 245.7 MB
Peak Usage: 312.4 MB
Memory Efficiency: 87.3%
Potential Issues: 2 memory spikes detected

Integration with Monitoring Systems

# Run an external alert script when the threshold (in MB) is exceeded
php glueful system:memory \
    --threshold=256 \
    --alert-script=/usr/local/bin/notify-ops.sh

Chunked Database Processor

Process large database result sets in memory-efficient chunks.

Features

Chunked Processing: Process large datasets without memory issues
Configurable Chunk Sizes: Set a default chunk size on the constructor, or override per call
Multiple Sources: Process a raw SELECT query (PDO/mysqli) or iterate a table by id

Basic Usage

The constructor takes the database connection and an optional default chunk size. The per-call chunk size is the last argument of each processing method, overriding the default.

use Glueful\Performance\ChunkedDatabaseProcessor;

// Second argument is the default chunk size (default 1000)
$processor = new ChunkedDatabaseProcessor($connection, 1000);

// Process a SELECT query in chunks. The processor callback receives each chunk
// (an array of rows) and its return values are collected into the result array.
$results = $processor->processSelectQuery(
    'SELECT * FROM large_table WHERE active = ?',
    function (array $rows) {
        foreach ($rows as $row) {
            updateUserRecord($row);
        }
        return count($rows);
    },
    [1],   // query parameters
    500    // chunk size for this call (overrides the default)
);

Processing a Table by Id

processTableInChunks() iterates a table in id-ordered chunks, applying optional equality conditions. Each chunk is passed to the processor callback.

$results = $processor->processTableInChunks(
    'users',
    function (array $rows) {
        foreach ($rows as $user) {
            sendNotification($user);
        }
        return count($rows);
    },
    'id',                    // id column to paginate on
    ['status' => 'active'],  // equality conditions
    500                      // chunk size (overrides the default)
);

Chunk Size

The chunk size is the only tunable. Set a default on the constructor and override it per call via the last argument of processSelectQuery() / processTableInChunks():

// Default of 2000 rows per chunk for every call on this instance
$processor = new ChunkedDatabaseProcessor($connection, 2000);

// Override to 250 for a single, heavier-per-row job
$processor->processSelectQuery($query, $callback, $params, 250);

Lazy Container

Deferred object creation container for improved memory efficiency and performance. It exposes just two methods: register() to bind a factory closure under an id, and get() to resolve it. The factory runs the first time the id is requested, and the resulting instance is cached and returned on every subsequent get().

Basic Usage

use Glueful\Performance\LazyContainer;

$container = new LazyContainer();

// Register factory closures
$container->register('database', function () use ($config) {
    return new DatabaseConnection($config);
});

$container->register('userService', function () use ($container) {
    return new UserService($container->get('database'));
});

// Objects are created only when first accessed, then cached
$userService = $container->get('userService'); // 'database' is built here, on first use
$sameService = $container->get('userService'); // returns the cached instance

Notes

There is no auto-wiring, aliasing, tagging, or statistics API — the container simply instantiates each registered factory lazily on the first get() and memoizes the result.
Express dependencies by resolving them from the container inside the factory closure (as shown with $container->get('database') above).

Best Practices

Memory Management Guidelines

Monitor Continuously: Use real-time monitoring for production systems
Set Appropriate Thresholds: Configure warnings before critical situations
Use Object Pooling: Reuse expensive objects when possible
Process in Chunks: Handle large datasets with chunked processing
Lazy Load Resources: Create objects only when needed

Performance Optimization

Choose Right Tools: Select appropriate iterator for your use case
Configure Limits: Set memory limits for all processing tasks
Enable Alerting: Get notified before problems occur
Track Metrics: Monitor memory patterns and trends
Regular Cleanup: Implement automated cleanup processes

Common Patterns

// Combine multiple tools for optimal performance
$container = new LazyContainer();
$pool = new MemoryPool(maxSize: 10);
$processor = new ChunkedDatabaseProcessor($connection, 1000);

// Reuse an expensive-to-build worker across chunks via the keyed pool
$processor->processSelectQuery($query, function (array $rows) use ($pool) {
    if (!$pool->has('worker')) {
        $pool->add('worker', new BatchWorker());
    }

    $worker = $pool->get('worker');
    $worker->processBatch($rows);

    return count($rows);
}, $params);

This comprehensive memory management system provides all the tools needed to build memory-efficient, scalable applications with Glueful.

Tracing and Observability

Request Tracing

Glueful includes built-in request tracing capabilities for monitoring request flows and performance bottlenecks.

Tracing Middleware

use Glueful\Http\Middleware\TracingMiddleware;

// Add tracing middleware to your application
$app->add(new TracingMiddleware([
    'enabled' => true,
    'sample_rate' => 1.0,  // Trace 100% of requests
    'trace_queries' => true,
    'trace_cache' => true,
    'trace_external_calls' => true
]));

Custom Tracing

use Glueful\Tracing\Tracer;

// Start a trace span
$span = Tracer::startSpan('user_lookup');

try {
    $user = $userService->findById($userId);
    $span->setTag('user_id', $userId);
    $span->setTag('user_status', $user->status);
} catch (Exception $e) {
    $span->setTag('error', true);
    $span->setTag('error_message', $e->getMessage());
    throw $e;
} finally {
    $span->finish();
}

Distributed Tracing

// For microservices, bind a tracer adapter
use Glueful\Tracing\Adapters\JaegerAdapter;

$tracer = new JaegerAdapter([
    'service_name' => 'glueful-api',
    'jaeger_endpoint' => 'http://localhost:14268/api/traces'
]);

Tracer::setAdapter($tracer);

Monitoring Dashboards

Glueful provides guidance and examples for setting up monitoring dashboards using popular tools.

Metrics Export

// Export metrics for external monitoring systems
use Glueful\Monitoring\MetricsExporter;

$exporter = new MetricsExporter([
    'format' => 'prometheus',  // prometheus, grafana, datadog
    'endpoint' => '/metrics',
    'include_system_metrics' => true
]);

// Register metrics endpoint
Router::get('/metrics', [$exporter, 'export']);

Dashboard Configuration Examples

Grafana Dashboard JSON:

{
  "dashboard": {
    "title": "Glueful API Metrics",
    "panels": [
      {
        "title": "Request Rate",
        "type": "graph",
        "targets": [
          {
            "expr": "rate(http_requests_total[5m])",
            "legendFormat": "Requests/sec"
          }
        ]
      },
      {
        "title": "Response Time",
        "type": "graph", 
        "targets": [
          {
            "expr": "histogram_quantile(0.95, rate(http_request_duration_seconds_bucket[5m]))",
            "legendFormat": "95th percentile"
          }
        ]
      }
    ]
  }
}

Prometheus Configuration:

# prometheus.yml
scrape_configs:
  - job_name: 'glueful-api'
    static_configs:
      - targets: ['localhost:8000']
    metrics_path: '/metrics'
    scrape_interval: 15s

Key Metrics to Monitor

Request Metrics
- Request rate (requests/second)
- Response time percentiles (50th, 95th, 99th)
- Error rate by status code
- Endpoint-specific performance
System Metrics
- Memory usage and trends
- CPU utilization
- Database connection pool status
- Cache hit/miss ratios
Business Metrics
- User registration rate
- API endpoint usage patterns
- Feature adoption metrics
- Performance budget compliance

Alert Configuration

// Configure alerts for critical metrics
use Glueful\Monitoring\AlertManager;

$alertManager = new AlertManager([
    'channels' => ['email', 'slack', 'webhook'],
    'rules' => [
        [
            'name' => 'High Response Time',
            'condition' => 'avg_response_time > 500',  // ms
            'duration' => '5m',
            'severity' => 'warning'
        ],
        [
            'name' => 'High Error Rate', 
            'condition' => 'error_rate > 5',           // %
            'duration' => '2m',
            'severity' => 'critical'
        ]
    ]
]);

This comprehensive observability system provides complete visibility into your Glueful application's performance and behavior in production environments.

Performance Deep Dive

Glueful's performance surface — query optimization, query caching, profiling/analysis tools, the query logger, and response caching.

Writing Console Commands

Build your own Glueful CLI commands on Symfony Console — the BaseCommand base class, DI-enabled services, registration, and authoring best practices.