Submission Format¶
For submission, a JSON file needs to be provided with the predicted rupture status, but also with information on used hardware and processing time. The JSON file needs to be named "results-task3.json".
JSON Submission Format Example¶
The following is just an abbreviated example illustrating the submission format. Always make sure to validate your submission using the JSON schema provided at the page bottom!
{`` ``"grand_challenge_username"``: ``"my_user_name_on_grand_challenge"``,`` ``"used_hardware_specification"``: {`` ``"CPU"``: ``"Intel Core i9 9900K 8x 3.60GHz"``,`` ``"GPU"``: ``"NVIDIA RTX 2080 Ti"``,`` ``"#GPUs"``: 1,`` ``"RAM_in_GB"``: 4,`` ``"additional_remarks"``: ``"special hardware requirements, other comments"`` ``},`` ``"task_3_results"``: [`` ``{`` ``"dataset_id"``: ``"A035"``,`` ``"processing_time_in_seconds"``: 22.7,`` ``"rupture_probability"``: 0.4, ``// optional`` ``"rupture_status"``: 0`` ``},`` ``{`` ``"dataset_id": ``"A038_R_A"``,`` ``"processing_time_in_seconds"``: 43,`` ``"rupture_probability"``: 0.99, ``// optional`` ``"rupture_status"``: 1`` ``},`` ``{`` ``"dataset_id"``: ``"A038_R_B"``,`` ``"processing_time_in_seconds"``: 13,`` ``"rupture_probability"``: 0.89, ``// optional`` ``"rupture_status"``: 1`` ``}`` ``]``}
JSON Schema for Submission Validation¶
In order for you to check if your JSON file is syntactically correct and complete, please check your file against this JSON schema:
Submissions that cannot be validated with the above schema cannot be evaluated and thus not be considered for the challenge!
To check your submission against the schema, you can use online validators like https://www.jsonschemavalidator.net, or your favourite programming language (e.g. python has a jsonschema package).
Paper submissions¶
We expect the participants to write either a full paper or a short paper (4 pages), describing their approach in detail. The paper needs to be submitted with the solution to grand-challenge. For docker submissions, the paper should be included in the output folder, named paper.pdf. A template for the paper format can be found here: LNCS
Docker submissions¶
We will provide you with an account to the Fraunhofer GitLab, where you will get a docker registry accessible only to you and the challenge organizers for the duration of the challenge. For access to the docker registry, please send an email to cada.challenge(at)gmail.com. When submitting a new image matching the criteria, it will be run on the test data and submitted to grand-challenge (this might take some time to appear in the leaderboard).
Just like for pre-processed results submissions, we will evaluate one per day. For the docker images, we will pull any new images matching the naming criteria (see 'Docker build' below) at CEST 00:00 for processing.
Docker build¶
We ask you to build your image accordingly to the exemplary build command:
docker build -t task3:\$USERNAME-YYYYMMDD
where \$USERNAME is your grand-challenge username and YYYYMMDD the date of your push to the docker registry (e.g. 20200528)
Docker run¶
Your containers will be run on our processing server with CUDA-based GPU support, but without internet connection. Here is an exemplary docker run command:
docker run -d --rm --network none --mount type=bind,source=data/input,target=/input,readonly --mount type=bind,source=results/\$DOCKER-IMAGE-HASH,target=/output task3:testuser-20200528
where the /input folder contains the test and mask images in compressed NIFTI format, as well as the aneurysm and vessel geometries in STL format (like the training data).
- A001.nii.gz
- A001_labeledMasks.nii.gz
- A001.stl
- A001_vessel.stl
- A003.nii.gz
- A003_labeledMasks.nii.gz
- A003.stl
- A003_vessel.stl
- ...
The /output folder is expected to contain the results-task3.json file and the paper.pdf, as specified above.
- results-task3.json
- paper.pdf