Skip to content

Disable pull-up when correlated inner columns are not a subset of the PARTITION BY columns - #2274

Merged
Alena0704 merged 4 commits into
apache:mainfrom
andr-sokolov:CXformLeftSemiApply2LeftSemiJoin
Nov 18, 2026
Merged

Alena0704 merged 4 commits into
apache:mainfrom
andr-sokolov:CXformLeftSemiApply2LeftSemiJoin

Conversation

@andr-sokolov

@andr-sokolov andr-sokolov commented Nov 9, 2026 •

Copy link
Copy Markdown
Collaborator

Orca built invalid plan because it pulled up subquery which contains a window
function during the Apply to Join transformations. Window function could return
incorrect value after the pull-up because tuples number which this function was
calculated over was changed after the optimization.

The pull-up is safe when the correlated inner columns are a subset of
the PARTITION BY columns. Otherwise the pull-up is disabled.

FProcessProject processes CLogicalProject too. CLogicalProject processing is
not changed.

The queries to reproduce the bug are added to tests.

@andr-sokolov

Copy link
Copy Markdown
Collaborator Author

@my-ship-it , this PR fixes bug from your comment #2233 (comment)

@andr-sokolov andr-sokolov changed the title Disable CXformLeftSemiApply2LeftSemiJoin when subquery contains a win… Disable CXformLeftSemiApply2LeftSemiJoin when subquery contains a window function Nov 9, 2026
@Alena0704
Alena0704 force-pushed the CXformLeftSemiApply2LeftSemiJoin branch from 6891a29 to fa68005 Compare November 9, 2026 14:39
@yjhjstz

yjhjstz commented Nov 9, 2026

Copy link
Copy Markdown
Member

Thanks for working on this! I simulated the patch by disabling CXformLeftSemiApply2LeftSemiJoin / CXformLeftSemiApplyIn2LeftSemiJoin, which is what the new Exfp check amounts to for these queries. The query in the test is fixed. However, the same subquery still returns wrong results when it is planned through other Apply→Join xforms:

CREATE TABLE t_out AS SELECT 1 a, 2 b DISTRIBUTED BY (b);
CREATE TABLE t_in AS SELECT v.a FROM (VALUES (1),(1),(2)) v(a) DISTRIBUTED BY (a);
ANALYZE t_out, t_in;

-- expected: 1 | 2    got: 0 rows
SELECT * FROM t_out
WHERE t_out.b IN (SELECT count(*) OVER () FROM t_in WHERE t_in.a = t_out.a)
   OR t_out.a = 5;

-- expected: t        got: f
SELECT t_out.b IN (SELECT count(*) OVER () FROM t_in WHERE t_in.a = t_out.a)
FROM t_out;

-- expected: 1 | 2    got: 0 rows
SELECT * FROM t_out
WHERE EXISTS (SELECT 1 FROM (SELECT count(*) OVER () c FROM t_in
                             WHERE t_in.a = t_out.a) s
              WHERE s.c = t_out.b);

-- expected: 0 rows   got: 1 | 2
SELECT * FROM t_out
WHERE NOT EXISTS (SELECT 1 FROM (SELECT count(*) OVER () c FROM t_in
                                 WHERE t_in.a = t_out.a) s
                  WHERE s.c = t_out.b);

-- expected: 1 | 2    got: 0 rows
SELECT * FROM t_out
WHERE t_out.b = (SELECT max(c) FROM (SELECT count(*) OVER () c FROM t_in
                                     WHERE t_in.a = t_out.a) s);

-- expected: 2        got: 3
SELECT (SELECT max(c) FROM (SELECT count(*) OVER () c FROM t_in
                            WHERE t_in.a = t_out.a) s)
FROM t_out;

All of these go through CDecorrelator::FProcessProject. It pulls the correlated predicate above the SequenceProject without checking that the correlated inner columns are PARTITION BY columns, so the window is computed over the whole inner table. Maybe the check belongs there instead: only allow the pullup when the correlated inner columns are a subset of the PARTITION BY columns. That would also cover LeftAntiSemiApply, InnerApply and LeftOuterApply.

The new check also blocks a valid case. For example, ... IN (SELECT count(*) OVER (PARTITION BY t_in.a) FROM t_in WHERE t_in.a = t_out.a) is a Hash Join on main but becomes a per-row SubPlan with this patch. CorrelatedSubqueryWithAggWindowFunc.mdp (run by CSubqueryHandlerTest) covers this shape and expects the semi join.

It would also help to add EXPLAIN / optimizer_trace_fallback to the test, so that a fallback to the planner doesn't make it pass silently.

@my-ship-it

Copy link
Copy Markdown
Contributor

@my-ship-it , this PR fixes bug from your comment #2233 (comment)

Thanks for following up on this. I will take a look.

@andr-sokolov andr-sokolov changed the title Disable CXformLeftSemiApply2LeftSemiJoin when subquery contains a window function Disable pull-up when correlated inner columns are not a subset of the PARTITION BY columns Nov 11, 2026
@andr-sokolov

Copy link
Copy Markdown
Collaborator Author

Thanks for working on this! I simulated the patch by disabling CXformLeftSemiApply2LeftSemiJoin / CXformLeftSemiApplyIn2LeftSemiJoin, which is what the new Exfp check amounts to for these queries. The query in the test is fixed. However, the same subquery still returns wrong results when it is planned through other Apply→Join xforms:

CREATE TABLE t_out AS SELECT 1 a, 2 b DISTRIBUTED BY (b);
CREATE TABLE t_in AS SELECT v.a FROM (VALUES (1),(1),(2)) v(a) DISTRIBUTED BY (a);
ANALYZE t_out, t_in;

-- expected: 1 | 2    got: 0 rows
SELECT * FROM t_out
WHERE t_out.b IN (SELECT count(*) OVER () FROM t_in WHERE t_in.a = t_out.a)
   OR t_out.a = 5;

-- expected: t        got: f
SELECT t_out.b IN (SELECT count(*) OVER () FROM t_in WHERE t_in.a = t_out.a)
FROM t_out;

-- expected: 1 | 2    got: 0 rows
SELECT * FROM t_out
WHERE EXISTS (SELECT 1 FROM (SELECT count(*) OVER () c FROM t_in
                             WHERE t_in.a = t_out.a) s
              WHERE s.c = t_out.b);

-- expected: 0 rows   got: 1 | 2
SELECT * FROM t_out
WHERE NOT EXISTS (SELECT 1 FROM (SELECT count(*) OVER () c FROM t_in
                                 WHERE t_in.a = t_out.a) s
                  WHERE s.c = t_out.b);

-- expected: 1 | 2    got: 0 rows
SELECT * FROM t_out
WHERE t_out.b = (SELECT max(c) FROM (SELECT count(*) OVER () c FROM t_in
                                     WHERE t_in.a = t_out.a) s);

-- expected: 2        got: 3
SELECT (SELECT max(c) FROM (SELECT count(*) OVER () c FROM t_in
                            WHERE t_in.a = t_out.a) s)
FROM t_out;

All of these go through CDecorrelator::FProcessProject. It pulls the correlated predicate above the SequenceProject without checking that the correlated inner columns are PARTITION BY columns, so the window is computed over the whole inner table. Maybe the check belongs there instead: only allow the pullup when the correlated inner columns are a subset of the PARTITION BY columns. That would also cover LeftAntiSemiApply, InnerApply and LeftOuterApply.

The new check also blocks a valid case. For example, ... IN (SELECT count(*) OVER (PARTITION BY t_in.a) FROM t_in WHERE t_in.a = t_out.a) is a Hash Join on main but becomes a per-row SubPlan with this patch. CorrelatedSubqueryWithAggWindowFunc.mdp (run by CSubqueryHandlerTest) covers this shape and expects the semi join.

It would also help to add EXPLAIN / optimizer_trace_fallback to the test, so that a fallback to the planner doesn't make it pass silently.

@yjhjstz, fixed

@Alena0704
Alena0704 force-pushed the CXformLeftSemiApply2LeftSemiJoin branch from 7b6a212 to 7d39952 Compare November 16, 2026 12:11

@leborchuk leborchuk left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks

@andr-sokolov
andr-sokolov force-pushed the CXformLeftSemiApply2LeftSemiJoin branch from 56c8bd9 to 8f091f5 Compare November 17, 2026 07:29
@Alena0704
Alena0704 enabled auto-merge (squash) November 17, 2026 08:30
@Alena0704
Alena0704 force-pushed the CXformLeftSemiApply2LeftSemiJoin branch 2 times, most recently from e250604 to 0e141eb Compare November 18, 2026 09:39
@tuhaihe

tuhaihe commented Nov 18, 2026

Copy link
Copy Markdown
Member

Hi @Alena0704 the PR #2261 has been in the merge pipeline and will be merged once the tests are done. Maybe you need to rebase this PR again. 😢

Before rebasing, welcome to check the status: https://github.com/apache/cloudberry/pulls?q=is%3Apr+state%3Aopen+review%3Aapproved

While waiting, we also need to resolve the PR's comments. Then the auto-merge can be run properly. FYI.

@Alena0704
Alena0704 disabled auto-merge November 18, 2026 10:26
@Alena0704

Copy link
Copy Markdown
Collaborator

Hi @Alena0704 the PR #2261 has been in the merge pipeline and will be merged once the tests are done. Maybe you need to rebase this PR again. 😢

Before rebasing, welcome to check the status: https://github.com/apache/cloudberry/pulls?q=is%3Apr+state%3Aopen+review%3Aapproved

While waiting, we also need to resolve the PR's comments. Then the auto-merge can be run properly. FYI.

Okay, understood. Resolved it. Thank you!

…dow function

Orca built invalid plan because it pulled up subquery which contains a window
function during the CXformLeftSemiApply2LeftSemiJoin transformation. Window
function could return incorrect value after the pullup because tuples number
which this function was calculated over was changed after the optimization.

Make CXformLeftSemiApply2LeftSemiJoin inapplicable when subquery tree contains
the EopScalarWindowFunc operator.

Groups are shared between alternatives in Memo. We should visit each group only
once to keep the traversal linear in the number of groups. The pbsVisited
variable is added to keep track of visited groups.

The queries to reproduce the bug are added to tests.
@Alena0704
Alena0704 force-pushed the CXformLeftSemiApply2LeftSemiJoin branch from 0e141eb to 3eac5d9 Compare November 18, 2026 10:33
@Alena0704
Alena0704 enabled auto-merge (squash) November 18, 2026 10:36
@Alena0704
Alena0704 merged commit b98a1e5 into apache:main Nov 18, 2026
130 checks passed
@andr-sokolov
andr-sokolov deleted the CXformLeftSemiApply2LeftSemiJoin branch November 18, 2026 12:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants