Fix Iceberg view test failures in CICD

[res-life]

Fixes #14075.

Description

why pipelines failed

For pipelines: rapids_it-iceberg-s3tables-dev and rapids_it-iceberg-dev
pipeline 1.6.x passed, but 1.9.x failed.

From Iceberg 1.9.x, using hadoop type with SparkSessionCatalog will fail with:
"Creating a view is not supported by catalog: spark_catalog"
Iceberg 1.9.x changed the behavior.

For S3TableCatalog, it also does not support view, tested, trying to create a view in s3_catalog.namespace also fails with: "Creating a view is not supported by catalog"

how to fix

Only test rest catalog, skip hadoop and s3table catalog.
rest catalog supports view.

Checklists

• This PR has added documentation for new or modified features or behaviors.
• This PR has added new tests or modified existing tests to cover new code paths.
  (Please explain in the PR description how the new code paths are tested, such as names of the new/existing tests that cover them.)
• Performance testing has been performed and its results are added in the PR description. Or, an issue has been filed with a link in the PR description.

[res-life]

build

[greptile-apps]

Greptile Summary

This PR fixes Iceberg view test failures that occurred with Iceberg 1.9.x by restricting view tests to only run with REST catalog and ensuring test tables have data before creating views.

Key Changes:

• Skips view tests when not using REST catalog, as Iceberg 1.9.x removed view support for Hadoop and S3Tables catalogs
• Adds missing data insertion step after table creation, following the pattern used in other Iceberg test files (e.g., iceberg_merge_test.py)
• Sets ICEBERG_TEST_CATALOG_TYPE="rest" environment variable in CI to enable proper catalog detection

The fix is well-structured and aligns with existing patterns in the codebase.

Confidence Score: 5/5

• This PR is safe to merge with minimal risk
• The changes are focused, follow established patterns in the codebase, and directly address the root cause identified in issue [BUG] Iceberg integration tests fail with pyspark.errors.exceptions.captured.UnsupportedOperationException: Creating a view is not supported by catalog: spark_catalog #14075. The fix properly handles the breaking change in Iceberg 1.9.x where view support was removed from non-REST catalogs.
• No files require special attention

Important Files Changed

Filename	Overview
integration_tests/src/main/python/conftest.py	Added is_iceberg_rest_catalog() helper function to check if REST catalog is enabled via environment variable
integration_tests/src/main/python/iceberg/iceberg_view_test.py	Added REST catalog skip condition and data insertion step to fix view test failures in Iceberg 1.9.x
jenkins/spark-tests.sh	Added ICEBERG_TEST_CATALOG_TYPE="rest" environment variable to REST catalog test configuration

Sequence Diagram

sequenceDiagram
    participant Test as test_iceberg_view
    participant Factory as spark_tmp_table_factory
    participant Helper as create_iceberg_table
    participant Session as with_cpu_session
    participant Spark as SparkSession
    
    Test->>Factory: get_full_table_name()
    Factory-->>Test: table_name
    
    Test->>Helper: create_iceberg_table(table_name)
    Helper->>Session: with_cpu_session(set_iceberg_table)
    Session->>Spark: Generate schema from df_gen
    Session->>Spark: CREATE TABLE ... USING ICEBERG
    Spark-->>Session: Table created (empty)
    Session-->>Helper: Done
    Helper-->>Test: table_name
    
    Note over Test: NEW: Insert data step added in PR
    Test->>Session: with_cpu_session(insert_data)
    Session->>Spark: gen_df() with iceberg_gens_list
    Session->>Spark: df.writeTo(table_name).append()
    Spark-->>Session: Data inserted
    Session-->>Test: Done
    
    Test->>Session: with_cpu_session(setup_iceberg_view)
    Session->>Spark: CREATE VIEW view_name AS view_sql
    Spark-->>Session: View created
    Session-->>Test: Done
    
    Test->>Test: assert_gpu_and_cpu_are_equal_collect()
    Test->>Spark: SELECT * FROM view_name
    Spark-->>Test: Results verified

Loading

[greptile-apps]

Additional Comments (1)

1. integration_tests/src/main/python/iceberg/iceberg_view_test.py, line 73-74 (link)

   style: The temporary Spark view created here is never dropped and could accumulate across test runs. Consider adding cleanup after table creation.

_{1 file reviewed, 1 comment}

_{Edit Code Review Agent Settings | Greptile}

[liurenjie1024]

I'm thinkg that maybe another solution would be to run it on rest catalog only?

[res-life]

Now rapids_it-iceberg-rest-catalog-dev is testing Iceberg 1.6.1, it will not expose this problem.
So another solution is to update rapids_it-iceberg-rest-catalog-dev to test 1.9.2 and add skip for non-rest catalogs.

[res-life]

build

[greptile-apps]

Greptile's behavior is changing!

From now on, if a review finishes with no comments, we will not post an additional "statistics" comment to confirm that our review found nothing to comment on. However, you can confirm that we reviewed your changes in the status check section.

_{This feature can be toggled off in your Code Review Settings by deselecting "Create a status check for each PR".}

[res-life]

@yinqingh @pxLi Help review jenkins/spark-tests.sh

[abellina]

LGTM

[sameerz]

build

[pxLi]

Going to merge this to ensure today's run can cover this change, thanks