Fix binary date/time conversion; add Date32 #115
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Add tests for Postgres aggregate function pushdown. Eventually it will cover all of the aggregate functions; for now it tests
avg()andarray_agg()withmin()started but commented-out, because Postgres doesn't push it down, instead sending a anORDER BYandLIMIT 1, which works great for some databases, but as of now pg_clickhouse doesn't push down thatLIMIT 1. And it would be better to let it execute its own plan. More investigation on that in future commits.Meanwhile, fix a few bugs discovered while testing
array_agg()and a failing ClickBench query.First, ClickHouse UUID arrays fetched via the http engine failed to parse into Postgres Arrays because ClickHouse single-quotes each UUID and Postgres does not. Update the parser to replace those single quotes with spaces, which Postgres happily ignores.
Next, dates were improperly displayed as
10529827-09-17instead of2025-12-05when loaded in arrays (as bydate_agg()). Tom Lane suggested usingtimestamptz_dateinstead of thetime_t_to_timestamptz(val)call used previously, but it kept returning the wrong values: off by a day in the aggregation tests and2000-01-01in the binary insert tests.Turns out there were two issues: 1).
timestamp_dateis the right function, nottimestamptz_date; and 2) there was some conversion code just for Dates after selection! Switching totimestamp_dateand eliminating the conversion code fixed those issues.Finally, dates from ClickBench weren't showing up at all. I figured out that the dates I was looking at were all
1970-01-01, and that the binary conversion code handled a "special case" where an epoch value of0was null. It's not!So add tests for epoch zero
DateandDateTimevalues and remove those "special cases".This revealed an issue where a DateTime64 epoch
0returned2000-01-01on macOS, though not on Linux. Switching totime_t_to_timestamptzfixes the issue, but requires separate handling of sub-second precision, done with integers rather than floats, fixing imprecision in the output of some timestamps.While at it, add support for
Date32, which behaves exactly likeDateas far as Postgres is concerned.