Skip to content

Conversation

@michalsn
Copy link
Member

@michalsn michalsn commented Oct 28, 2025

Description
This PR introduces deferred writes for both the DatabaseHandler and FileHandler. This feature is optional and can be enabled or disabled via the config setting.

Closes #8

DatabaseHandler
Multiple calls to set() or forget() within a single request are now deferred and executed in a single database transaction at the end of the request.

  • set() calls are grouped into one upsertBatch updateBatch and/or insertBatch query.
  • forget() calls are grouped into one DELETE query.

This significantly reduces the number of database operations per request.

FileHandler
Multiple set() or forget() calls are now batched into a single file write, performed once at the end of the request.
Each file will be touched only once, substantially reducing I/O operations.

Implementation details
Deferred writes are currently executed during the post_system event.

This approach has some drawbacks - database queries (upsert insert/update and delete) will not be visible in the Debug Toolbar.

I'm open to switching this to a Filter-based implementation, though this would require users to manually register the filter in the $required array - just before the toolbar filter.

Version and migration notes

  • The minimum required CodeIgniter version has been increased to 4.3, as this version introduces support for:
    • Adding indexes to existing table schemas
    • The upsertBatch() method
  • Since this feature requires a database schema change (adding a unique index to support upserts), it would be appropriate to release this as v3 of the library once (if) merged.

EDIT:

As it turned out, the null value (used for context) is not treated as unique by most database engines, so the entire idea of using upsert fell through.

Some good and bad things came out of this.

Let's start with the good ones:

  • We don’t need to make any changes to the table schema (adding a unique index would be misleading).
  • We don’t have to require a higher version of the framework.

And the bad part:

  • We need to handle inserts and updates separately, but it's still an improvement when performing multiple operations.

I've also added more tests to ensure that deferred writes don’t create any duplicate entries.


Checklist:

  • Securely signed commits
  • Component(s) with PHPDoc blocks, only if necessary or adds value
  • Unit testing, with >80% coverage
  • User guide updated
  • Conforms to style guide

Copy link
Member

@lonnieezell lonnieezell left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is a fantastic addition. Well done!

@michalsn
Copy link
Member Author

Well, crap... I've been working a lot with SQL Server lately, and it turns out unique keys with NULL values behave a bit differently there compared to other databases. As a result, upsertBatch() isn't working correctly. The unique index isn't triggered when the context is NULL. I didn't check for this in the tests 😢 I'll need to come up with a different approach.

@michalsn
Copy link
Member Author

Ok, everything is working fine now. I've updated some descriptions in the docs and added an additional summary to the first post.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Feature: Bulk handling

4 participants