Skip to content

gh-154594: Fix deepcopy memo lookup when __deepcopy__ returns None - #154595

Open
tonghuaroot wants to merge 3 commits into
python:mainfrom
tonghuaroot:fix-deepcopy-none-sentinel
Open

gh-154594: Fix deepcopy memo lookup when __deepcopy__ returns None#154595
tonghuaroot wants to merge 3 commits into
python:mainfrom
tonghuaroot:fix-deepcopy-none-sentinel

Conversation

@tonghuaroot

@tonghuaroot tonghuaroot commented Jul 24, 2026

Copy link
Copy Markdown
Contributor

PR #138429 (gh-132657) replaced the memo-miss sentinel from _nil = [] to None for free-threading performance. But None is a valid deepcopy result, so memo.get(d, None) can't distinguish "not found" from "found None". This skips memoization for None-valued results, causing redundant __deepcopy__ calls.

Fix: use a private _MEMO_MISS = object() sentinel. Immortalized objects have no refcount contention, so the FT benefit is preserved.

@eendebakpt eendebakpt left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two small comments, but I am leaning more towards the approach suggested in the corresponding issue.

Comment thread Lib/copy.py Outdated
Comment thread Lib/test/test_copy.py Outdated
Addresses review: clearer, no sentinel, and the test treats memo as opaque.
@tonghuaroot

Copy link
Copy Markdown
Contributor Author

Switched to if d in memo: return memo[d] as suggested. Drops the sentinel, and the test no longer touches memo.

Microbenchmark (5M lookups): d in memo is ~6 ns faster on a hit and ~22 ns on a miss than memo.get(d, None); full deepcopy is within noise.

@pochmann

Copy link
Copy Markdown
Contributor

d in memo is ~6 ns faster on a hit and ~22 ns on a miss than memo.get(d, None)

Faster in both cases? Did you measure purely d in memo and memo.get(d, None)? Or did you include the other differences as well?

@tonghuaroot

Copy link
Copy Markdown
Contributor Author

The earlier numbers included the surrounding statements. Measuring just the lookup, each returning the value (best of 5 runs of 10M):

                                  hit (present)   miss (absent)
memo.get(d, None)                    51 ns           60 ns
memo[d] if d in memo else None       55 ns           49 ns

So d in memo is about 4 ns slower on a hit and 11 ns faster on a miss. In deepcopy each object is looked up once, so misses are the common case, and full deepcopy timings are within noise either way.

@pochmann

Copy link
Copy Markdown
Contributor

Ok if the surrounding stuff was properly included, then that's good. This new comparison for "just the lookup" is not, it's missing the is not None check in an if-context and the store/load of y.

@pochmann

Copy link
Copy Markdown
Contributor

Anyway, I agree that misses are probably the common case, and I'm not surprised if full deepcopy timings are within noise either way, as deepcopy is rather slow.

@eendebakpt eendebakpt left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two small nits, but this looks good to me.

Comment thread Misc/NEWS.d/next/Library/2026-07-24-00-02-00.gh-issue-154594.Xk7mQ2.rst Outdated
Comment thread Lib/test/test_copy.py Outdated
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants