Ubuntu image runs out of disk space

[liyaka]

Description

Ubuntu image runs out of disk space since yesterday ( 17.10.225) on some of our test workflows

Runner details:

Current runner version: '2.329.0'
Runner Image Provisioner
  Hosted Compute Agent
  Version: 20251013.424
  Commit: cfdd8bfed34d71a55b72df4d2e82343c3fc2bab3
  Build Date: 2025-10-13T20:22:23Z
Operating System
  Ubuntu
  24.04.3
  LTS
Runner Image
  Image: ubuntu-24.04
  Version: 20251014.76.1
  Included Software: https://github.com/actions/runner-images/blob/ubuntu24/20251014.76/images/ubuntu/Ubuntu2404-Readme.md
  Image Release: https://github.com/actions/runner-images/releases/tag/ubuntu24%2F20251014.76

Platforms affected

• Azure DevOps
• GitHub Actions - Standard Runners
• GitHub Actions - Larger Runners

Runner images affected

• Ubuntu 22.04
• Ubuntu 24.04
• macOS 13
• macOS 13 Arm64
• macOS 14
• macOS 14 Arm64
• macOS 15
• macOS 15 Arm64
• macOS 26 Arm64
• Windows Server 2019
• Windows Server 2022
• Windows Server 2025

Image version and build link

Runner details:

Current runner version: '2.329.0'
Runner Image Provisioner
  Hosted Compute Agent
  Version: 20251013.424
  Commit: cfdd8bfed34d71a55b72df4d2e82343c3fc2bab3
  Build Date: 2025-10-13T20:22:23Z
Operating System
  Ubuntu
  24.04.3
  LTS
Runner Image
  Image: ubuntu-24.04
  Version: 20251014.76.1
  Included Software: https://github.com/actions/runner-images/blob/ubuntu24/20251014.76/images/ubuntu/Ubuntu2404-Readme.md
  Image Release: https://github.com/actions/runner-images/releases/tag/ubuntu24%2F20251014.76

Link to the build:
https://github.com/comet-ml/opik/actions/runs/18615255723/job/53078889510?pr=3630

Is it regression?

20250912.392

Expected behavior

runs successfully...

Actual behavior

Fails with error:

ERROR: Could not install packages due to an OSError: [Errno 28] No space left on device

Repro steps

Our project is an open source public project, so you can fork it and try to run the same workflow that fails https://github.com/comet-ml/opik/actions/workflows/sdk-e2e-tests.yaml

[erik-bershel]

Hi!

I've checked that the image contains sufficient space. My immediate question is, how much space do your dependencies consume during installation? If the usage exceeds the available space, I can only recommend deleting some files from the image before running the installation. The easiest way is to delete unused caches.

Filesystem      Size  Used Avail Use% Mounted on
/dev/root        72G   53G   19G  74% /
...

[jsoref]

@erik-bershel since we're using containers, I don't think we can do anything to the host.

If I could mount /mnt to /tmp in the container (rw), then we'd have plenty of space, but there's no support for this (I'd file a bug, but I'm not quite sure what to say).

[erik-bershel]

By the way. While poking around I should say that we rolling that image back to a previous version due to a kernel bug.

[erik-bershel]

@jsoref we don't own .container functionality. Right repo to report your issue would be actions/runner.