Skip to content

Conversation

@weixiuli
Copy link

Currently, the velox BloomFilterAggregate checks the input row and throws an exception if there are some null values in the row. So we need to be consistent with spark's behavior and ignore null values.

The spark BloomFilterAggregate will Ignore null values. https://github.com/apache/spark/blob/6cdca10f148433664b3e2be6f655b0ddba817537/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/BloomFilterAggregate.scala#L180-L188

 override def update(buffer: BloomFilter, inputRow: InternalRow): BloomFilter = {
    val value = child.eval(inputRow)
    // Ignore null values.
    if (value == null) {
      return buffer
    }
    updater.update(buffer, value)
    buffer
  }

weixiuli pushed a commit to weixiuli/gluten that referenced this pull request Dec 12, 2023
@zhztheplayer zhztheplayer force-pushed the update branch 2 times, most recently from 13e79b6 to 8a6ef2b Compare December 13, 2023 07:11
@weixiuli weixiuli changed the title Support the null values in bloom_filter Spark aggregate Support the null value in bloom_filter_agg Spark aggregate function Dec 14, 2023
@prestodb-ci prestodb-ci force-pushed the update branch 9 times, most recently from c5d9aac to 9f0ef60 Compare October 26, 2025 04:46
@jinchengchenghh jinchengchenghh force-pushed the update branch 2 times, most recently from 704b7d6 to 9f74a76 Compare October 29, 2025 10:18
@prestodb-ci prestodb-ci force-pushed the update branch 9 times, most recently from 07f098d to 537e695 Compare November 4, 2025 04:45
@prestodb-ci prestodb-ci force-pushed the update branch 3 times, most recently from e963905 to b228025 Compare November 7, 2025 11:21
@prestodb-ci prestodb-ci force-pushed the update branch 5 times, most recently from cc6eb06 to 7dfc0d2 Compare November 15, 2025 04:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants