Skip to content

uniq: optimize memory usage for ignore-case comparison - #10050

Merged
cakebaker merged 1 commit into
uutils:mainfrom
CrazyRoka:uniq-optimize-case-sensitive
Jan 5, 2026
Merged

uniq: optimize memory usage for ignore-case comparison#10050
cakebaker merged 1 commit into
uutils:mainfrom
CrazyRoka:uniq-optimize-case-sensitive

Conversation

@CrazyRoka

Copy link
Copy Markdown
Contributor

Previously, when running uniq -i, the implementation eagerly allocated a new vector and copied the lowercased line into it for every single line read. Profiling indicated that build_meta accounted for a significant portion of execution time (approx 27%) due to this allocation.

This change removes the lowercase buffer from LineMeta entirely. Instead, it implements a lazy, iterator-based comparison in keys_differ. This eliminates the per-line allocation overhead and enables short-circuiting during comparison.

@github-actions

github-actions Bot commented Jan 4, 2026

Copy link
Copy Markdown

GNU testsuite comparison:

Congrats! The gnu test tests/printf/printf-surprise is now passing!

@codspeed-hq

codspeed-hq Bot commented Jan 4, 2026

Copy link
Copy Markdown

CodSpeed Performance Report

Merging #10050 will improve performance by 7.04%

Comparing CrazyRoka:uniq-optimize-case-sensitive (28deca2) with main (666c6df)

Summary

⚡ 1 improvement
✅ 138 untouched
⏩ 37 skipped1

Benchmarks breakdown

Benchmark BASE HEAD Efficiency
uniq_case_insensitive[10000] 1.8 ms 1.6 ms +7.04%

Footnotes

  1. 37 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

@CrazyRoka
CrazyRoka force-pushed the uniq-optimize-case-sensitive branch from 12f3e7f to 28deca2 Compare January 4, 2026 19:39
@github-actions

github-actions Bot commented Jan 4, 2026

Copy link
Copy Markdown

GNU testsuite comparison:

GNU test failed: tests/tty/tty-eof. tests/tty/tty-eof is passing on 'main'. Maybe you have to rebase?
Skip an intermittent issue tests/timeout/timeout (fails in this run but passes in the 'main' branch)

@CrazyRoka
CrazyRoka marked this pull request as ready for review January 4, 2026 21:40
@cakebaker
cakebaker merged commit 5ce558c into uutils:main Jan 5, 2026
130 of 131 checks passed
@cakebaker

Copy link
Copy Markdown
Contributor

Nice refactoring, thanks!

@CrazyRoka
CrazyRoka deleted the uniq-optimize-case-sensitive branch January 5, 2026 13:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants