stepfunctions: Add support for ItemsPointer for DistributedMap's items reader

[TomasChmelik]

Describe the feature

It is no longer necessary to S3 item to be an JSON array of items which should be processed. Now it is possible to specify path in the S3 file which points to a JSON array which contains items which should be iterated.

https://docs.aws.amazon.com/step-functions/latest/dg/input-output-itemreader.html#itemsource-example-json-data shows ItemsPointer which adds support for this.

Use Case

This feature is supported by AWS, I would like to use it in my project

Proposed Solution

No response

Other Information

No response

Acknowledgements

• I may be able to implement this feature request
• This feature might incur a breaking change

AWS CDK Library version (aws-cdk-lib)

AWS CDK CLI version

Environment details (OS name and version, etc.)

[pahud]

Hi @TomasChmelik,

Thank you for this feature request! The ItemsPointer parameter is indeed a useful AWS Step Functions feature that allows you to specify a JSONPath to a nested array within your S3 JSON file, rather than requiring the array to be at the root level.

Currently, the CDK's S3JsonItemReader class doesn't support the ItemsPointer parameter. To add this feature, we would need to:

1. Add an optional itemsPointer property to the S3FileItemReaderProps interface
2. Update the S3FileItemReader.render() method to include ItemsPointer in the ReaderConfig when specified

This would be a non-breaking change since the parameter would be optional. Would you be interested in contributing a pull request for this feature? We'd be happy to guide you through the process. The implementation should be relatively straightforward based on the existing pattern for other optional parameters like maxItems.

As a temporary workaround, you could use an escape hatch to add the ItemsPointer directly to the CloudFormation template, though a proper CDK implementation would be the better long-term solution.