[rust] add support of iceberg format key encoder - #3934
Conversation
|
could you help review this? @fresh-borzoni |
fresh-borzoni
left a comment
fresh-borzoni
left a comment
There was a problem hiding this comment.
@zuston Thank you for the PR, LGTM over, some minor comments and suggestions, PTAL
| } | ||
|
|
||
| #[test] | ||
| fn encodes_values_with_iceberg_binary_layout() { |
There was a problem hiding this comment.
These check the bytes against values copied from our Java code, not from Iceberg — and the IT can't help since write and read share the encoder.
Could you assert Iceberg's Appendix B hash vectors instead? Run them through IcebergBucketingFunction with num_buckets = i32::MAX so the modulo cancels and you get the raw hash back. Negative ones need hash & i32::MAX accordingly.
Let me know if you need more clarifications.
There was a problem hiding this comment.
really insightful comment! updated
fresh-borzoni
left a comment
fresh-borzoni
left a comment
There was a problem hiding this comment.
@zuston Thank you, LGTM 👍
2 participants
Purpose
add support of iceberg format key encoder
Brief change log
Tests
API and Format
Documentation