Added tests for the cwl.output.json feature
#81
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This commit adds three conformance tests for the `cwl.output.json` feature: - The first test confirms that if the `cwl.output.json` file contains a field that has not been declared in the `output` object of a CWL `CommandLineTool`, that field is discarded (fix #80) - The second test confirms that outputs from `cwl.output.json` are type-checked according to the type declared in the `output` object of the `CommandLineTool` - The third test confirms that if an output is not present in the `cwl.output.json` file but has been declared as required in the `output` object of a CWL `CommandLineTool`, the execution fails
GlassOfWhiskey
force-pushed
the
test-cwl-output-json
branch
2 times, most recently
from
3df8bac to
48a4989
Compare
GlassOfWhiskey
force-pushed
the
test-cwl-output-json
branch
from
48a4989 to
a4c7f89
Compare
GlassOfWhiskey
force-pushed
the
test-cwl-output-json
branch
from
a4c7f89 to
4de5e29
Compare
mr-c
requested changes
Nov 11, 2025
Member
mr-c
left a comment
mr-c
left a comment
There was a problem hiding this comment.
Thank you! I added some change requests
Comment on lines
+162
to
+165
| section, but `outputBinding` is ignored. If an output name is present | ||
| in the "cwl.output.json" file, but has not been declared in the | ||
| `outputs` section, it must be ignored and not propagated to the next | ||
| steps. |
Member
There was a problem hiding this comment.
Suggested change
| section, but `outputBinding` is ignored. If an output name is present | |
| in the "cwl.output.json" file, but has not been declared in the | |
| `outputs` section, it must be ignored and not propagated to the next | |
| steps. | |
| section, but `outputBinding` is ignored. Any output is present in the | |
| "cwl.output.json" file but not declared in the `outputs` section must | |
| be discarded from the output object. |
Comment on lines
+54
to
+57
| * If an output name is present in the "cwl.output.json" file, but has | ||
| not been declared in the `outputs` section, it must be ignored and | ||
| not propagated to the next steps. Previous versions of CWL exhibited | ||
| undefined behaviour for this occurrence. |
Member
There was a problem hiding this comment.
Suggested change
| * If an output name is present in the "cwl.output.json" file, but has | |
| not been declared in the `outputs` section, it must be ignored and | |
| not propagated to the next steps. Previous versions of CWL exhibited | |
| undefined behaviour for this occurrence. | |
| * [Output Binding](#Output_binding): Any output present in the | |
| "cwl.output.json" file but not declared in the `outputs` section must | |
| be discarded from the output object. Previous versions of CWL did not | |
| define if undeclared outputs in "cwl.output.json" should be present | |
| in the output object. |
Comment on lines
+3
to
+17
| requirements: | ||
| - class: ShellCommandRequirement | ||
| hints: | ||
| DockerRequirement: | ||
| dockerPull: docker.io/debian:stable-slim | ||
|
|
||
| inputs: [] | ||
|
|
||
| baseCommand: sh | ||
|
|
||
| arguments: | ||
| - -c | ||
| - | | ||
| echo foo > foo && echo '{"foo": {"location": "foo", "class": "File"} }' | ||
|
|
Member
There was a problem hiding this comment.
Please refactor this test to not use ShellCommandRequirement so that the conformance test doesn't have to be tagged shell_command and can keep its required tag.
Comment on lines
+4
to
+16
| - class: ShellCommandRequirement | ||
| hints: | ||
| DockerRequirement: | ||
| dockerPull: docker.io/debian:stable-slim | ||
|
|
||
| inputs: [] | ||
|
|
||
| baseCommand: sh | ||
|
|
||
| arguments: | ||
| - -c | ||
| - | | ||
| echo foo > foo && echo '{"foo": {"location": "foo", "class": "File"} }' |
Member
There was a problem hiding this comment.
Please refactor this test to not use ShellCommandRequirement so that the conformance test doesn't have to be tagged shell_command and can keep its required tag.
Comment on lines
+3
to
+16
| requirements: | ||
| - class: ShellCommandRequirement | ||
| hints: | ||
| DockerRequirement: | ||
| dockerPull: docker.io/debian:stable-slim | ||
|
|
||
| inputs: [] | ||
|
|
||
| baseCommand: sh | ||
|
|
||
| arguments: | ||
| - -c | ||
| - | | ||
| echo foo > foo && echo '{"foo": {"location": "foo", "class": "File"} }' |
Member
There was a problem hiding this comment.
Please refactor this test to not use ShellCommandRequirement so that the conformance test doesn't have to be tagged shell_command and can keep its required tag.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This commit adds three conformance tests for the
cwl.output.jsonfeature:cwl.output.jsonfile contains a field that has not been declared in theoutputssection of a CWLCommandLineTool, that field is discarded (fix [Proposal] If cwl.output.json is created by the tool, outputs not listed in the CWL tool description must be removed #80)cwl.output.jsonare type-checked according to the type declared in theoutputssection of theCommandLineToolcwl.output.jsonfile but has been declared as required in theoutputssection of a CWLCommandLineTool, the execution fails