Skip to content

Conversation

@jenskeiner
Copy link
Contributor

@jenskeiner jenskeiner commented Sep 22, 2025

This adds Linux builds on arm64 architecture to broaden the platforms we test on.

@codspeed-hq
Copy link

codspeed-hq bot commented Sep 22, 2025

CodSpeed Performance Report

Merging #169 will create unknown performance changes

Comparing feature/linux-arm (f73b22b) with develop (b4dae23)

Summary

🆕 704 new
⏩ 88 skipped1

Benchmarks breakdown

Benchmark BASE HEAD Change
🆕 ubuntu-24.04-arm_clang_kaiserbessel_double_openmp/nfft_adjoint_direct_1d[128/400] N/A 3.3 ms N/A
🆕 ubuntu-24.04-arm_clang_kaiserbessel_double_openmp/nfft_adjoint_direct_1d[256/800] N/A 13.1 ms N/A
🆕 ubuntu-24.04-arm_clang_kaiserbessel_double_openmp/nfft_adjoint_direct_1d[32/100] N/A 203.6 µs N/A
🆕 ubuntu-24.04-arm_clang_kaiserbessel_double_openmp/nfft_adjoint_direct_1d[512/1600] N/A 52.4 ms N/A
🆕 ubuntu-24.04-arm_clang_kaiserbessel_double_openmp/nfft_adjoint_direct_1d[64/200] N/A 811.2 µs N/A
🆕 ubuntu-24.04-arm_clang_kaiserbessel_double_openmp/nfft_adjoint_direct_1d_omp[128/400] N/A 14.3 ms N/A
🆕 ubuntu-24.04-arm_clang_kaiserbessel_double_openmp/nfft_adjoint_direct_1d_omp[256/800] N/A 27.8 ms N/A
🆕 ubuntu-24.04-arm_clang_kaiserbessel_double_openmp/nfft_adjoint_direct_1d_omp[32/100] N/A 10.7 ms N/A
🆕 ubuntu-24.04-arm_clang_kaiserbessel_double_openmp/nfft_adjoint_direct_1d_omp[512/1600] N/A 72.3 ms N/A
🆕 ubuntu-24.04-arm_clang_kaiserbessel_double_openmp/nfft_adjoint_direct_1d_omp[64/200] N/A 11.2 ms N/A
🆕 ubuntu-24.04-arm_clang_kaiserbessel_double_openmp/nfft_adjoint_direct_2d[16/16/500] N/A 9.4 ms N/A
🆕 ubuntu-24.04-arm_clang_kaiserbessel_double_openmp/nfft_adjoint_direct_2d[32/32/1000] N/A 75.5 ms N/A
🆕 ubuntu-24.04-arm_clang_kaiserbessel_double_openmp/nfft_adjoint_direct_2d[64/64/2000] N/A 610.1 ms N/A
🆕 ubuntu-24.04-arm_clang_kaiserbessel_double_openmp/nfft_adjoint_direct_2d_omp[16/16/500] N/A 27.4 ms N/A
🆕 ubuntu-24.04-arm_clang_kaiserbessel_double_openmp/nfft_adjoint_direct_2d_omp[32/32/1000] N/A 89.9 ms N/A
🆕 ubuntu-24.04-arm_clang_kaiserbessel_double_openmp/nfft_adjoint_direct_2d_omp[64/64/2000] N/A 621.6 ms N/A
🆕 ubuntu-24.04-arm_clang_kaiserbessel_double_openmp/nfft_adjoint_direct_3d[16/16/16/1000] N/A 303.4 ms N/A
🆕 ubuntu-24.04-arm_clang_kaiserbessel_double_openmp/nfft_adjoint_direct_3d[4/4/4/250] N/A 1.2 ms N/A
🆕 ubuntu-24.04-arm_clang_kaiserbessel_double_openmp/nfft_adjoint_direct_3d[8/8/8/500] N/A 18.8 ms N/A
🆕 ubuntu-24.04-arm_clang_kaiserbessel_double_openmp/nfft_adjoint_direct_3d_omp[16/16/16/1000] N/A 325.2 ms N/A
... ... ... ... ...

ℹ️ Only the first 20 benchmarks are displayed. Go to the app to view all benchmarks.

Footnotes

  1. 88 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

@jenskeiner jenskeiner force-pushed the feature/linux-arm branch 4 times, most recently from f690b00 to 8943ac0 Compare September 24, 2025 09:24
@jenskeiner jenskeiner added the chore Repository maintenance and tooling changes label Sep 24, 2025
@fargito
Copy link

fargito commented Sep 25, 2025

Hello, CodSpeed team member here,

This error message is misleading

Too many benchmarks in a single upload

The performance report could not be generated because there were too many benchmarks in a single upload to CodSpeed. We recommend sharding your benchmarks into smaller uploads, see the documentation for more information.

Actually, the issue in this case is the number of jobs in your workflow run. We put fixing this issue on our roadmap. In the meantime, maybe you can try and reduce the number of jobs in the matrix, or run CodSpeed only on a subset to make it work.

@jenskeiner
Copy link
Contributor Author

jenskeiner commented Sep 27, 2025

Hello, CodSpeed team member here,

This error message is misleading

Too many benchmarks in a single upload

The performance report could not be generated because there were too many benchmarks in a single upload to CodSpeed. We recommend sharding your benchmarks into smaller uploads, see the documentation for more information.

Actually, the issue in this case is the number of jobs in your workflow run. We put fixing this issue on our roadmap. In the meantime, maybe you can try and reduce the number of jobs in the matrix, or run CodSpeed only on a subset to make it work.

Thanks for the heads-up @fargito. We have quite a few different configurations to test, but the overall number of jobs can probably be reduced. We're working on it. Otherwise, this has been working quite well so far, so good job.

I was also testing running benchmarks on macOS recently. After some patches to instrument-hooks I got codspeed-cpp to compile. I could also link against the library, but running the benchmarks inside the provided GitHub Action failed with a runtime error that I could not yet get to the bottom of.

@jenskeiner jenskeiner force-pushed the feature/linux-arm branch 2 times, most recently from ff9ef66 to 609cff3 Compare September 29, 2025 16:25
@jenskeiner jenskeiner added test Test-related changes with no production code impact and removed chore Repository maintenance and tooling changes labels Sep 29, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

test Test-related changes with no production code impact

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants