I tried to plug this file into Jenkins junit("FILE.xml") but Jenkins could not recognize it (null pointer exception), I opened the XML file and I saw that it lacks many tag attributes that are normally found on other JUnit XML report files.
So how this file should be interpreted?
Hi @andresvia,
The goal is indeed to be parsed with typical software like you tried, but it sounds like the Jenkins parser isn’t very robust and so it crashes when given a file that lacks some element that it assumes will always be present.
Since the Terraform language isn’t structured the same way as Java JUnit the shape of the test results isn’t quite the same either, but we did our best to map to the JUnit model as closely as we could.
This format is not properly specified and so unfortunately the only path here is to gradually learn about the expectations of each individual parsing implementation and keep tweaking until it contains enough information to please all of them. Clearly we will need to learn more about what the Jenkins parser is assuming here and adjust the XML format in the next iteration of the experiment to meet those assumptions.
In the meantime, it would be helpful if you would open a bug report issue in the main Terraform GitHub repository with as much detail as you have about exactly what error message you saw in Jenkins, which version of Jenkins (and any relevant plugins) you are using, and what parts of the XML format you noticed seem to be obviously missing. We can then try to reproduce with a similar version of Jenkins and plugins and hopefully then iterate until we find an input structure that it will accept.
Thanks!
Could there be something wrong with the JUnit XML that the Terraform test command is producing? OK. I read your post above, which answers this.
Its a bit hard to tell because I can’t seem to lock down an official standard for JUnit XML test result schema. May be my bad Googling though. That said, IBM, Stack Overflow, etc.,
seem to point to this format: https://github.com/windyroad/JUnit-Schema/blob/master/JUnit.xsd
I tried to have CircleCI consume the file and it also choked on it, siting the follow issues with the file:
Archiving the following test results
- /root/project/terraform-test-results.xml
Failed uploading test results
Error File: root/project/terraform-test-results.xml had the following problems:
- invalid testsuites element: errors
- invalid top level element: failures
- invalid top level element: tests
- invalid testsuite element: name
- invalid top level element: tests
- invalid top level element: skipped
- invalid top level element: errors
(there were a further 15 problems.
Its a public repo, so you can view the results from CircleCI and there is also the artifact:
CircleCI Terraform Test Results Comsuption
I am uncertain, but I think I remember someone, maybe even on this forum, commenting that there is no official specification, it’s just a case of people having to try to mimic the original implementation.
That may account for Terraform and CircleCI having differing opinions on the format.
EDIT: Hah, I do remember that - I should have scrolled up further - the post I remember apparentlymart’s earlier in this topic.