Skip to content

Conversation

@BulkBeing
Copy link
Contributor

@BulkBeing BulkBeing commented Oct 20, 2025

numaproj/numaflow#2811

Tested by modifying the existing examples and running below pipeline. The simple-source generates some user metadata, then the map vertex adds additional groups to this and forwards to sink. Verified all user metadata is available at the sink vertex.

apiVersion: numaflow.numaproj.io/v1alpha1
kind: Pipeline
metadata:
  name: simple-source
spec:
  vertices:
    - name: in
      scale:
        min: 1
        max: 1
      source:
        udsource:
          container:
            # A simple user-defined async source
            image: simple-source-py:v5
            imagePullPolicy: Never
      limits:
        readBatchSize: 2
    - name: flatmap
      scale:
        min: 1
        max: 1
      udf:
        container:
          image: map-flatmap-py:v5
          imagePullPolicy: Never
          # image: quay.io/numaio/numaflow-go/map-forward-message:stable
          # imagePullPolicy: Always
          env:
            - name: PYTHONDEBUG
              value: "true"
      containerTemplate:
        env:
          - name: NUMAFLOW_DEBUG
            value: "true" # DO NOT forget the double quotes!!!
    - name: inbuilt
      sink:
        log: {}
    - name: out
      scale:
        min: 1
        max: 1
      sink:
        udsink:
          container:
            args:
            - python
            - example.py
            image: sink-log-py:v2
            imagePullPolicy: Never
            env:
              - name: PYTHONDEBUG
                value: "true"
              - name: INVOKE
                value: "func_handler"
  edges:
    - from: in
      to: flatmap
    - from: flatmap
      to: inbuilt
    - from: flatmap
      to: out

Signed-off-by: Sreekanth <prsreekanth920@gmail.com>
Signed-off-by: Sreekanth <prsreekanth920@gmail.com>
Signed-off-by: Sreekanth <prsreekanth920@gmail.com>
Signed-off-by: Sreekanth <prsreekanth920@gmail.com>
Signed-off-by: Sreekanth <prsreekanth920@gmail.com>
Signed-off-by: Sreekanth <prsreekanth920@gmail.com>
Signed-off-by: Sreekanth <prsreekanth920@gmail.com>
Signed-off-by: Sreekanth <prsreekanth920@gmail.com>
@codecov
Copy link

codecov bot commented Oct 20, 2025

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 93.89%. Comparing base (2990279) to head (b0c7e28).

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #274      +/-   ##
==========================================
+ Coverage   93.63%   93.89%   +0.25%     
==========================================
  Files          65       66       +1     
  Lines        2907     3013     +106     
  Branches      152      156       +4     
==========================================
+ Hits         2722     2829     +107     
+ Misses        135      134       -1     
  Partials       50       50              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Signed-off-by: Sreekanth <prsreekanth920@gmail.com>
Signed-off-by: Sreekanth <prsreekanth920@gmail.com>
Signed-off-by: Sreekanth <prsreekanth920@gmail.com>
Signed-off-by: Sreekanth <prsreekanth920@gmail.com>
Signed-off-by: Sreekanth <prsreekanth920@gmail.com>
Signed-off-by: Sreekanth <prsreekanth920@gmail.com>
Signed-off-by: Sreekanth <prsreekanth920@gmail.com>
Signed-off-by: Sreekanth <prsreekanth920@gmail.com>
Signed-off-by: Sreekanth <prsreekanth920@gmail.com>
Signed-off-by: Sreekanth <prsreekanth920@gmail.com>
Signed-off-by: Sreekanth <prsreekanth920@gmail.com>
Signed-off-by: Sreekanth <prsreekanth920@gmail.com>
@BulkBeing BulkBeing marked this pull request as ready for review October 21, 2025 10:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant