Merge pull request #46 from iazaran/Scalibility-improvementsd

Scalibility improvementsd

Author: iazaran

CHANGELOG.md

@@ -5,6 +5,23 @@ All notable changes to the `iazaran/smart-cache` package will be documented in t
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [1.13.0] - 2026-06-16
+### Added
+- Declarative model invalidation rules via a protected `cacheInvalidation(): array` method. Existing fluent setters still work and are merged with declared rules.
+- `Model::flushCacheTags()` for explicit invalidation after `saveQuietly()`, query-builder updates/deletes, upserts, mass inserts, raw SQL, or any other path that bypasses Eloquent events.
+- `TagFlushed` event, including the tag name, live key count, and source (`manual`, `model`, or `model_helper`), when cache events are enabled.
+- Config options for metadata locks (`smart-cache.metadata_lock.*`) and transaction-aware model invalidation (`smart-cache.model_invalidation.after_commit`).
+
+### Changed
+- Model auto-invalidation now defers cache flushing until the active database transaction commits by default. Rollbacks no longer flush cache, and nested transactions wait for the outer commit. Set `smart-cache.model_invalidation.after_commit` to `false` to restore immediate invalidation.
+- Tag metadata writes now register before the cache value is written and use a short Laravel cache lock when the store supports `LockProvider`, reducing lost tag-index updates under concurrent writers while preserving best-effort behavior for stores without locks.
+- Tag reads lazily prune expired or missing key references, and tag flushes correctly handle keys written under an active namespace.
+
+### Fixed
+- `SmartCache::add()` no longer leaks active tags into the next write when the atomic add fails because the key already exists.
+- Cache DNA deduplicated writes now still refresh tag/managed-key metadata and only skip the value write when the cached value is still present.
+- Dependency invalidation now refreshes dependency metadata before traversal, so long-running workers do not miss relationships added by another process after the local map was loaded.
+
 ## [1.12.2] - 2026-05-29
 ### Security
 - Upgraded every remaining `symfony/*` lockfile entry from `v8.0.8` to `v8.1.0` (>= patched lines `8.0.12` / `8.0.13`) to clear the rest of the open Dependabot advisories plus two pending CVEs surfaced by `composer audit`. Runtime: `symfony/mailer` (CVE-2026-45068, `SendmailTransport` argument injection via dash-prefixed recipient), `symfony/routing` (CVE-2026-45065, `UrlGenerator` route-requirement bypass via unanchored regex alternation; CVE-2026-48784, dot-segment encoding skip), `symfony/http-foundation` (CVE-2026-48736), `symfony/http-kernel` (CVE-2026-45075, `#[IsGranted(methods: ['GET'])]` filter bypass via `HEAD`). Dev: `symfony/yaml` (CVE-2026-45133 uncontrolled recursion, CVE-2026-45304 collection-alias "Billion Laughs", CVE-2026-45305 `Parser::cleanup()` ReDoS). `composer audit` is now clean across runtime and dev scopes. `composer.json` is unchanged — the existing ranges already permitted these versions; Dependabot was failing because of a stale resolver state on its side.

CONTRIBUTING.md

@@ -34,7 +34,7 @@ composer test
 composer test-coverage
 ```
 
-All pull requests must pass the existing test suite (425+ tests) before merging. If you add a new feature, include corresponding test cases.
+All pull requests must pass the existing test suite (485 tests) before merging. If you add a new feature, include corresponding test cases.
 
 ## Coding Style
 
@@ -52,4 +52,4 @@ In short, when you submit code changes, your submissions are understood to be un
 
 ## License
 
-By contributing, you agree that your contributions will be licensed under its MIT License.
+By contributing, you agree that your contributions will be licensed under its MIT License.

README.md

@@ -221,13 +221,30 @@ class User extends Model
 {
     use CacheInvalidation;
 
-    public function getCacheKeysToInvalidate(): array
+    protected function cacheInvalidation(): array
     {
-        return ["user_{$this->id}_profile", "user_{$this->id}_posts", 'users_list_*'];
+        return [
+            'keys' => ['user_{id}_profile', 'user_{id}_posts'],
+            'tags' => ['users', 'user_{id}', 'team_{team_id}'],
+            'patterns' => ['users_list_*'],
+            'dependencies' => ['team_{team_id}_summary'],
+        ];
     }
 }
+
+// Event-blind writes can flush explicitly:
+User::where('status', 'inactive')->update(['archived' => true]);
+User::flushCacheTags(['users']);
 ```
 
+Model invalidation is deferred until the current database transaction commits by default (`smart-cache.model_invalidation.after_commit = true`). This prevents another request from re-caching pre-commit data between an Eloquent event and the final commit. Set the flag to `false` if you need the historical immediate behavior.
+
+Eloquent events do not fire for `saveQuietly()`, query-builder `update()` / `delete()`, `upsert()`, mass `insert()`, or raw SQL. For those paths, call `flushCacheTags()` or `SmartCache::flushTags()` explicitly.
+
+### Choosing Cache Tags
+
+Tags should describe the data used to build a response, not the controller that built it. Start with the tables or models read by the endpoint: list endpoints usually use coarse tags such as `products`, while item endpoints can add instance tags such as `product_123`. Over-tagging causes extra refreshes; under-tagging leaves stale data behind. For hard-to-map endpoints, enable Laravel's query log in a test and compare the tables read during response generation with the tags declared for that cache entry.
+
 ### Encryption at Rest
 
 ```php
@@ -268,6 +285,10 @@ SmartCache::deleteMultiple(['key1', 'key2', 'key3']);
 config(['smart-cache.events.enabled' => true]);
 Event::listen(CacheHit::class, fn($e) => Log::info("Hit: {$e->key}"));
 Event::listen(CacheMissed::class, fn($e) => Log::warning("Miss: {$e->key}"));
+Event::listen(TagFlushed::class, fn($e) => Log::notice("Flushed {$e->tag}", [
+    'keys' => $e->keyCount,
+    'source' => $e->source, // manual, model, or model_helper
+]));
 ```
 
 ### Monitoring & Dashboard
@@ -355,6 +376,8 @@ return [
     'self_healing'    => ['enabled' => true],   // Auto-evict corrupted entries
     'swr'             => ['single_flight' => false], // v1.12.0: opt-in single-flight refresh
     'managed_keys'    => ['max_tracked' => 0],       // v1.12.0: 0 = unlimited (default)
+    'metadata_lock'   => ['enabled' => true, 'ttl' => 5, 'wait' => 1],
+    'model_invalidation' => ['after_commit' => true],
     'dashboard'       => ['enabled' => false, 'prefix' => 'smart-cache', 'middleware' => ['web']],
     'warmers'         => [],                    // Cache warmer classes for smart-cache:warm
 ];
@@ -383,7 +406,7 @@ $users = SmartCache::get('users');
 ## Testing
 
 ```bash
-composer test            # 471 tests, 1,936 assertions
+composer test            # 485 tests, 1,972 assertions
 composer test-coverage   # with code coverage
 ```
 

TESTING.md

@@ -29,7 +29,7 @@ vendor/bin/phpunit
 Current suite size:
 
 ```bash
-# 452 tests, 1,895 assertions
+# 485 tests, 1,972 assertions
 ```
 
 ### Run Specific Test Suites

config/smart-cache.php

@@ -214,6 +214,39 @@
         'max_tracked' => null,
     ],
 
+    /*
+    |--------------------------------------------------------------------------
+    | Metadata Locks
+    |--------------------------------------------------------------------------
+    |
+    | Tag indexes and dependency metadata are small shared cache entries. When
+    | the selected cache store supports Laravel atomic locks, SmartCache wraps
+    | metadata mutations in a short lock to reduce lost updates during concurrent
+    | writes. Stores without lock support keep the historical best-effort
+    | behavior.
+    |
+    */
+    'metadata_lock' => [
+        'enabled' => true,
+        'ttl' => 5,
+        'wait' => 1,
+    ],
+
+    /*
+    |--------------------------------------------------------------------------
+    | Model Invalidation
+    |--------------------------------------------------------------------------
+    |
+    | Eloquent model events fire while a database transaction is still open.
+    | Deferring invalidation until commit prevents another request from
+    | re-caching pre-commit data between the flush and the commit. Set
+    | `after_commit` to false to restore immediate invalidation.
+    |
+    */
+    'model_invalidation' => [
+        'after_commit' => true,
+    ],
+
     /*
     |--------------------------------------------------------------------------
     | Chunk Registry
@@ -297,6 +330,7 @@
             'key_written' => true,
             'key_forgotten' => true,
             'optimization_applied' => true,
+            'tag_flushed' => true,
         ],
     ],
 
@@ -331,4 +365,4 @@
         // 'users'    => App\CacheWarmers\UserCacheWarmer::class,
         // 'products' => App\CacheWarmers\ProductCacheWarmer::class,
     ],
-];
+];

docs/index.html

@@ -1026,6 +1026,19 @@ <h2>Configuration Options</h2>
 
     'events' => [
         'enabled' => false,          // Enable for monitoring
+        'dispatch' => [
+            'tag_flushed' => true,
+        ],
+    ],
+
+    'metadata_lock' => [
+        'enabled' => true,           // Lock tag/dependency metadata when supported
+        'ttl' => 5,
+        'wait' => 1,
+    ],
+
+    'model_invalidation' => [
+        'after_commit' => true,      // Flush after DB commit when inside a transaction
     ],
 
     'monitoring' => [
@@ -1281,7 +1294,7 @@ <h2>📡 Cache Events</h2>
 config(['smart-cache.events.enabled' => true]);
 
 // Listen to cache operations
-use SmartCache\Events\{CacheHit, CacheMissed, KeyWritten, KeyForgotten, OptimizationApplied};
+use SmartCache\Events\{CacheHit, CacheMissed, KeyWritten, KeyForgotten, OptimizationApplied, TagFlushed};
 
 Event::listen(CacheHit::class, function ($event) {
     Log::info("Cache hit: {$event->key}", ['tags' => $event->tags]);
@@ -1293,6 +1306,13 @@ <h2>📡 Cache Events</h2>
 
 Event::listen(OptimizationApplied::class, function ($event) {
     Log::info("Optimized {$event->key}: {$event->strategy} - {$event->ratio}% reduction");
+});
+
+Event::listen(TagFlushed::class, function ($event) {
+    Log::notice("Flushed {$event->tag}", [
+        'keys' => $event->keyCount,
+        'source' => $event->source, // manual, model, or model_helper
+    ]);
 });</code></pre>
 
                     <div class="alert alert-warning">
@@ -1402,19 +1422,33 @@ <h3>Model Auto-Invalidation</h3>
 {
     use CacheInvalidation;
 
-    public function getCacheKeysToInvalidate(): array
+    protected function cacheInvalidation(): array
     {
         return [
-            "user_{$this->id}_profile",
-            "user_{$this->id}_posts",
-            'users_list_*'
+            'keys' => ['user_{id}_profile', 'user_{id}_posts'],
+            'tags' => ['users', 'user_{id}', 'team_{team_id}'],
+            'patterns' => ['users_list_*'],
+            'dependencies' => ['team_{team_id}_summary'],
         ];
     }
 }
 
 // Cache automatically cleared when user changes!
 $user = User::find(1);
-$user->update(['name' => 'New Name']); // Cache cleared automatically</code></pre>
+$user->update(['name' => 'New Name']); // Flushes after the transaction commits
+
+// Event-blind writes should flush explicitly.
+User::where('status', 'inactive')->update(['archived' => true]);
+User::flushCacheTags(['users']);</code></pre>
+
+                    <div class="alert alert-warning">
+                        <strong>Eloquent event blind spots.</strong> Model auto-invalidation depends on Eloquent events. It does not run for <code>saveQuietly()</code>, query-builder <code>update()</code> / <code>delete()</code>, <code>upsert()</code>, mass <code>insert()</code>, or raw SQL. Use <code>Model::flushCacheTags()</code> or <code>SmartCache::flushTags()</code> in those paths.
+                    </div>
+
+                    <p>When an Eloquent event fires inside a database transaction, SmartCache defers invalidation until the outer transaction commits by default. This avoids flushing cache and then letting another request re-cache pre-commit data. Disable with <code>smart-cache.model_invalidation.after_commit = false</code> if your application needs immediate invalidation.</p>
+
+                    <h3>Dependency Map Cookbook</h3>
+                    <p>Declare tags for the data used to build the response, not for the controller or route name. Use coarse tags for list endpoints such as <code>products</code>, add instance tags for detail endpoints such as <code>product_123</code>, and prefer over-tagging to under-tagging. For opaque endpoints, capture Laravel's query log in a test and compare the selected tables with the cache tags you declared.</p>
 
                     <h2>Custom Optimization Strategies</h2>
                     <p>Create custom optimization strategies for your specific needs:</p>
@@ -1837,6 +1871,16 @@ <h2>Configuration Reference</h2>
         'max_tracked' => 0,             // v1.12.0: cap the in-memory managed-keys index (0 = unlimited)
     ],
 
+    'metadata_lock' => [
+        'enabled' => true,              // Lock tag/dependency metadata when the store supports locks
+        'ttl' => 5,
+        'wait' => 1,
+    ],
+
+    'model_invalidation' => [
+        'after_commit' => true,         // Defer model invalidation until DB commit
+    ],
+
     'dashboard' => [
         'enabled' => false,
         'prefix' => 'smart-cache',
@@ -2177,11 +2221,14 @@ <h2 id="invalidation-api">🔗 Cache Invalidation & Dependencies</h2>
 
                     <div class="method">
                         <div class="method-name">SmartCache::flushTags()</div>
-                        <div class="method-signature">SmartCache::flushTags(string|array $tags): bool</div>
-                        <p>Flush all cache entries associated with given tags.</p>
+                        <div class="method-signature">SmartCache::flushTags(string|array $tags, string $source = 'manual'): bool</div>
+                        <p>Flush all cache entries associated with given tags. Tag metadata is pruned lazily when tags are read, so expired entries do not build up indefinitely in rarely flushed tags.</p>
                         <div class="parameter">
                             <span class="parameter-name">$tags</span> <span class="parameter-type">(string|array)</span> - Tag name(s)
                         </div>
+                        <div class="parameter">
+                            <span class="parameter-name">$source</span> <span class="parameter-type">(string)</span> - Optional source label exposed on the TagFlushed event
+                        </div>
                         <div class="parameter">
                             <span class="return-type">Returns:</span> bool - True on success
                         </div>
@@ -2191,6 +2238,17 @@ <h2 id="invalidation-api">🔗 Cache Invalidation & Dependencies</h2>
                         </div>
                     </div>
 
+                    <div class="method">
+                        <div class="method-name">Model::flushCacheTags()</div>
+                        <div class="method-signature">User::flushCacheTags(string|array|null $tags = null): bool</div>
+                        <p>Flush model cache tags explicitly after writes that bypass Eloquent events, such as query-builder updates, upserts, mass inserts, quiet saves, or raw SQL.</p>
+                        <div class="example">
+                            <strong>Example:</strong>
+                            <pre><code class="language-php">User::where('active', false)->delete();
+User::flushCacheTags(['users']);</code></pre>
+                        </div>
+                    </div>
+
                     <div class="method">
                         <div class="method-name">SmartCache::dependsOn()</div>
                         <div class="method-signature">SmartCache::dependsOn(string $key, string|array $dependencies): static</div>
@@ -2903,7 +2961,7 @@ <h2>Package Test Suite</h2>
                     <p>SmartCache ships with automated coverage for the Laravel-compatible API surface, optimization strategies, SWR patterns, invalidation, memoization, dashboard command execution, and large-data recovery behavior.</p>
 
                     <pre><code class="language-bash">composer test
-# 471 tests, 1,936 assertions
+# 485 tests, 1,972 assertions
 
 vendor/bin/phpunit tests/Unit/Strategies/ChunkingStrategyTest.php --testdox
 vendor/bin/phpunit tests/Unit/SmartCacheTest.php --filter "missing_chunk|self_healing" --testdox</code></pre>

src/Contracts/SmartCache.php

@@ -434,4 +434,4 @@ public function rememberWithStampedeProtection(string $key, int $ttl, \Closure $
      * @return mixed
      */
     public function rememberIf(string $key, mixed $ttl, \Closure $callback, callable $condition): mixed;
-}
+}

src/Events/TagFlushed.php

@@ -0,0 +1,41 @@
+<?php
+
+namespace SmartCache\Events;
+
+class TagFlushed
+{
+    /**
+     * The tag that was flushed.
+     *
+     * @var string
+     */
+    public string $tag;
+
+    /**
+     * Number of live keys associated with the tag when it was flushed.
+     *
+     * @var int
+     */
+    public int $keyCount;
+
+    /**
+     * Source of the flush operation, such as manual, model, or model_helper.
+     *
+     * @var string
+     */
+    public string $source;
+
+    /**
+     * Create a new event instance.
+     *
+     * @param string $tag
+     * @param int $keyCount
+     * @param string $source
+     */
+    public function __construct(string $tag, int $keyCount, string $source = 'manual')
+    {
+        $this->tag = $tag;
+        $this->keyCount = $keyCount;
+        $this->source = $source;
+    }
+}

src/Facades/SmartCache.php

@@ -47,7 +47,7 @@
  * @method static mixed withFallback(callable $callback, mixed $fallback = null)
  * @method static mixed throttle(string $key, int $maxAttempts, int $decaySeconds, callable $callback)
  * @method static static tags(string|array $tags)
- * @method static bool flushTags(string|array $tags)
+ * @method static bool flushTags(string|array $tags, string $source = 'manual')
  * @method static static dependsOn(string $key, string|array $dependencies)
  * @method static bool invalidate(string $key)
  * @method static int flushPatterns(array $patterns)
@@ -100,4 +100,4 @@ protected static function getFacadeAccessor()
     {
         return 'smart-cache';
     }
-} 
+}