Hi we are unable to sync repository into spacewalk
got these logs
2023-03-27 14:37:12,842 [Thread-64] ERROR com.redhat.rhn.taskomatic.task.RepoSyncTask - Could not retrieve mirrorlist https://rpm.releases.hashicorp.com/RHEL/8/x86_64/stable error was
14: HTTPS Error 404 - Not Found
URL is obviously right as the same provides repofile
ANy ideas how to proceed to sync repo ?
THnx
Ok this has evolve a bit…
i`ve tried to sync this into even fresh installed spacewalk ( actually oracle linux manager(OLM) 2.10) and it fails the very same way as our production OLM…665 packages success to sync and 457 fails to sync.
Now oracle support suggests that OLM is metadata compliant ( whatever that means…i am waiting for details ), so I have compared packer that IS imported/download/synced successfully and one that is failing to sync
query command
rpm -qp packer-1.8.6-1.x86_64.rpm --qf “$(rpm --querytags | sed -nr ‘s/(.*)/\1:%{\1}/p’ | tr ‘[:space:]’ ‘\n’)”
results are quite different…
which suggest the build process has changed and now required metadata either missing or are wrong…
Any idea how to get in touch with anyone who is responsible for that?
Or any other idea to move this issue on?
THnx
D.
HI, again,
Thank you Mr. Atkins for response over github isssue.
Finally i`ve got some output from oracle support so
Packages ae required to have beside NEVRA metadata ( Name Epoch Version Revison Architecture)
also
package checksum
signing checksum
Now comparing successfully syncable packer ( packer-1.7.4-1.x86_64.rpm ) with a packer version failed to sync ( packer-1.8.1-1.x86_64.rpm ) shows differences on various metadata example below
[root@itssklvolm01 1]# rpm --nosignature -qp ./dce/packer/1.7.4-1/x86_64/dce742528000f47841344a7a5c011b6a/packer-1.7.4-1.x86_64.rpm --qf “$(rpm --querytags | sed -nr ‘s/(.*)/\1:%{\1}/p’ | tr ‘[:space:]’ ‘\n’)” |grep -i sha
SHA1HEADER:daf6e8f45825c01911adfb2fc00c4d423c39cef7
[root@itssklvolm01 1]#
above sha1header exists
[root@itssklvolm01 stage]# rpm --nosignature -qp packer-1.8.1-1.x86_64.rpm --qf “$(rpm --querytags | sed -nr ‘s/(.*)/\1:%{\1}/p’ | tr ‘[:space:]’ ‘\n’)” |grep -i sha
SHA1HEADER:(none)
[root@itssklvolm01 stage]#
on example above sha1header is missing
So overall seems that build process has changed because half of packages im yum RHEL8 repo is syncable, other half isn`t… and various metadata are missing…
THnx
Dusan
It looks like whatever software HashiCorp are using to make the RPMs now, has discontinued including legacy SHA-1 checksums in RPM headers.
That’s not too crazy, considering SHA-1 is considered a broken algorithm nowadays … but seems to pose a problem for this legacy Oracle software.
Hopefully someone from HashiCorp sees this thread and can consider re-adding the legacy hashes for compatibility … though really Oracle should fix their software too … good luck! 
HI Again
these are ( by oracle ) required package metadata that are missing
Mandatory:
PKGID: (none)
PLATFORM: (none)
SHA1HEADER: (none)
SIGMD5: (none)
Used to determine which fields to read:
RPMVERSION: (none)
SO pretty much yes…i hope someone from HashiCorp responsible for packaging sees this thread and act or at least info me/us
THnx
Dusan
.small update
(none) means they are not there… and should be
packer version 1.7.4 got them all …1.8+ not
D.
I’m not personally familiar with the intimate details of how the RPM repository is set up, but your hypothesis here seems plausible because all of HashiCorp’s open source projects are in the process of moving to a new release pipeline that handles these derived packages (everything except the .zip files on releases.hashicorp.com) differently.
With that said though: I believe the software which is constructing these indices is not HashiCorp-specific and is instead using some typical standard tools for managing RPM repositories. The removal of SHA1 support is probably therefore an upstream decision, which has appeared for us now only because this new pipeline is using a newer version of that software.
The priority for these repositories is being compatible with the official installation tools that ship with each of the supported operating systems, so we cannot promise compatibility with third-party software which has different requirements and expectations than the main system package manager. Given that SHA1 is obsolete and no longer an effective checksum algorithm, I agree that contacting your vendor for updated software is the most appropriate next step.
It sounds like @drunkez 's issues stem not from the repository indices, but the actual .rpm files themselves.
Are you able to tell us about the software constructing the .rpm files?
HI
Thnx for a reply
Here are the list of a metadata package could have
https://rpm-software-management.github.io/rpm/manual/tags.html
Problem is that half of a hashicorp repo( in this case RHEL8) is valid and second half is invalid
so something has changed, and the build pipeline for rpm packages does not fill metadata available before.
I know this might be a problem that is quite on the edge of an interest…but this prevents us from using hashicorp repo in an enterprise manner ( local repository sync then deliver to servers when needed, auditable, and so on…)
Still very thanx for a valuable discussion here and interest on moving this issue forward.
drunkez
The build step that produces the .rpm and .deb files is open source, over here:
This is outside of my direct scope of influence so I cannot promise anything about how that could potentially evolve in future, but if you have ideas for how it could be improved to be more compatible with third-party software then you could open an issue in that repository so the release engineering team can consider it. Any changes made there will be adopted into the build process for Terraform and anything else using the new release pipeline.
thank you…i have overlooked that in github so i`ll open issue there
HI just for a reference
if anyone would come here and had to follow…here is an issue
THnx
D
I’m not sure filing an issue in that repo is all that useful… the packages are being built by the open source tool GitHub - goreleaser/nfpm: nFPM is Not FPM - a simple deb, rpm and apk packager written in Go, so it’s that which would need to change to address this. (And then hashicorp/actions-packaging-linux could pick up the fixed version.)
If someone did want to directly propose a fix to this upstream then indeed the HashiCorp-specific GitHub Action could adopt it.
I was assuming that the goal here was to share the use-case for HashiCorp’s release engineering team to consider, since nFPM is even further out of my area of influence 
but if the upstream nFPM maintainers did want to address this then I assume HashiCorp release engineering team would indeed just adopt the updated version, as long as the new version produces something that’s backward-compatible to the old.
Well further i am digging into it …stranger it gets
PKGID metadata flag is calculated out of Name Version Release Architecture and optional Epoch ( NVRAE )
so a punch question is, how is it that pkgID is missing since packer 1.8.1
SO pretty much I am on a way to create my own test environment and i`ll try to reproduce an issue…
i have no other option just to reverse engineer whole process I suppose:)
Thnx
D
Are you sure that is true? According to rpm.org - RPM Tags, PKGID is an alias for SIGMD5, weirdly.