RPM Repository not available

We try to mirror the rpm repository: https://rpm.releases.hashicorp.com/RHEL/7/x86_64/stable/, but it fails.
Our company’s security does not allow to contact an rpm mirror directly, we need to mirror the full repo internally.

Could you please fix this repository, so we can sync again?

Thank you very much

Hi @zicklam,

Would you mind sharing additional info as to how you are attempting to mirror this repository, and the specific error you are encountering?

Thanks.

Hi @blake ,

we are using Pulp (v2.15.3) and feeding our repository from the URL:

Description:          upstream-shared7-hashicorp feed by
                      https://rpm.releases.hashicorp.com/RHEL/7/x86_64/stable

We are getting following error message on sync:

+----------------------------------------------------------------------+
         Synchronizing Repository [upstream-shared7-hashicorp]
+----------------------------------------------------------------------+

This command may be exited via ctrl+c without affecting the request.


Downloading metadata...
[\]
... completed

Downloading repository content...
[-]

Task Failed

Malformed repository: metadata is missing for some packages in filelists.xml and
in other.xml

When I visit the feed URL with an standard browser, i’ll get 404 Not Found message:

Thanks for the info, @zicklam.

I tried reproducing this issue using Pulp 3.17.2. I ran into a different error, but was able to resolve it and begin mirroring the repository.

I followed the instructions at https://pulpproject.org/pulp-in-one-container/ to set up Pulp. The only change I made was adding the following ALLOWED_CONTENT_CHECKSUMS line to settings.py.

ALLOWED_CONTENT_CHECKSUMS = ['md5', 'sha1', 'sha224', 'sha256', 'sha384', 'sha512']

After starting Pulp and setting an admin password (per the setup instructions), and installing the pulp CLI, I proceeded with creating the config to mirror the repo.

$ pulp rpm remote create \
    --name hashicorp \
    --url https://rpm.releases.hashicorp.com/RHEL/7/x86_64/stable \
    --policy on_demand

$ pulp rpm repository create \
    --name upstream-shared7-hashicorp \
    --remote hashicorp \
    --retain-package-versions 1 \ 
    --gpgcheck 1

I then tried to sync the repo using pulp rpm repository sync --name upstream-shared7-hashicorp, but received an error.

Started background task /pulp/api/v3/tasks/cc1a4c2a-902e-4711-84e1-bca95b79c224/
…Error: Task /pulp/api/v3/tasks/cc1a4c2a-902e-4711-84e1-bca95b79c224/ failed: ‘Content at the url https://rpm.releases.hashicorp.com/RHEL/7/x86_64/stable/repodata/148a0b63ab1f25fbcf16638c9462126b9cc17da7-other.xml.gz does not contain at least one trusted hasher which is specified in ‘ALLOWED_CONTENT_CHECKSUMS’ setting.’

This error is notably different than what you received, but the end result was the same; I could not synchronize the repo.

I came across the bug report Issue #9362: Mirroring rpm repositories with sha1 metatdata fails - RPM Support - Pulp where the reporter mentions that…

The metadata files of those repositories are mentioning sha instead of sha1 , pulp tries to mirror the repositories by setting this to sha, but that’s not a valid option in the repository object.

The solution was to explicitly set metadata_checksum_type and package_checksum_type on the repository to sha1.

$ pulp rpm repository update \
    --name upstream-shared7-hashicorp \
    --package-checksum-type sha1 \
    --metadata-checksum-type sha1

After this, the sync was successfully able to run.

$ pulp rpm repository sync --name upstream-shared7-hashicorp                                                                 
Started background task /pulp/api/v3/tasks/9a57e994-606f-4ab8-aa16-651c13c2ef9b/
.Done.

I’m not sure whether the same will work for you since you are using an older version of Pulp, but its worth a try to see if explicitly setting the hash type will resolve your issue.

You can find repo’s metadata file at https://rpm.releases.hashicorp.com/RHEL/7/x86_64/stable/repodata/repomd.xml. That file contains references to additional files under the repodata/ directory that contain the list of packages available from the repo, their versions, etc.

I hope this info helps. Let me know if you have any other questions and I will try to assist.

Hi, thank you for your help.

I’ve tried with a clean pulp3 instance and could sync the repository with the sha1 checksum settings.

But the packages can not be installed without deltarpm:

Delta RPMs disabled because /usr/bin/applydeltarpm not installed.
consul-1.11.2.x86_64.rpm       FAILED                                          
https://pulp3/pulp/content/upstream-shared7-hashicorp/Packages/c/consul-1.11.2.x86_64.rpm: [Errno -1] Package does not match intended download. Suggestion: run yum --enablerepo=upstream-shared7-hashicorp clean metadata

What did changed on your repo since Consul Version 1.10.3? This is the last version we synced successfully with our pulp.