Skip to content

Conversation

lennartzellmer
Copy link

@lennartzellmer lennartzellmer commented Oct 16, 2025

Loosen URL version_id Path Validation to Match S3 Specification

Purpose

The current validation for version_id is too strict. It assumes that a version ID consists only of lowercase characters, numbers, and hyphens. This holds true for MINIO, which uses UUIDs as version IDs. However, according to the S3 documentation, version IDs can be more opaque strings:

Version IDs are Unicode, UTF-8 encoded, URL-ready, opaque strings that are no more than 1,024 bytes long. The following is an example: 3sL4kqtJlcpXroDTDmJ+rmSpXd3dIbrHY+MTRCxf3vjVBH40Nr8X8gdRQBpUMLUo

-> More details: https://docs.aws.amazon.com/AmazonS3/latest/userguide/versioning-workflows.html#version-ids

This discrepancy causes issues with Hetzner's S3-compatible storage, which uses opaque strings as version IDs. These IDs can contain uppercase and lowercase letters as well as special characters.

When running tests against a Hetzner S3 bucket with versioning enabled, the current validation fails:

image

Here’s an example of a Hetzner version_id that doesn’t conform to the UUID format:

{
  ...
  "versions": [
    {
      "etag": "\"d153d2e52bbf6f443d6eb47439079b2f\"",
      "is_latest": false,
      "last_modified": "2025-10-16T14:55:33.471000Z",
      "version_id": "jH7VgK5xKEY-yWGxRpcDSRw30.p0r0m"
    },
    ...
  ]
}

Solution

Loosen the version_id validation logic to accept any opaque string up to 1,024 characters, aligning with the official S3 specification.

Why Tests Are Not Adjusted

The generation of version_id values is handled by the S3 implementation and is outside the scope of the test suite. The underlying issue is the assumption that all S3-compatible systems use UUID-like values. Since this is not a guaranteed or testable property across providers, no test adjustments are necessary.

All existing tests were executed successfully against both Hetzner S3 and MINIO after applying the fix:
SCR-20251016-tsor


External contribution

Thank you for your contribution! 🎉

Please ensure the following items are checked before submitting your pull request:

  • I have read and followed the contributing guidelines
  • I have read and agreed to the Code of Conduct
  • I have signed off my commits with git commit --signoff (DCO compliance)
  • I have signed my commits with my SSH or GPG key (git commit -S)
  • My commit messages follow the required format: <gitmoji>(type) title description
  • I have added a changelog entry under ## [Unreleased] section (if noticeable change)
  • I have added corresponding tests for new features or bug fixes (if applicable)

Loosen url version_id path validation to match s3 specification

Signed-off-by: lennartzellmer <2241624+lennartzellmer@users.noreply.github.com>
@lunika lunika self-requested a review October 17, 2025 07:57
@lunika lunika self-assigned this Oct 17, 2025
@lunika lunika added the bug Something isn't working label Oct 17, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants