Tips on how to Allow GitLab’s Dependency Proxy for Docker Photos – CloudSavvy IT

Posted on


Graphic showing the GitLab logo, a stylised fox head

GitLab has an built-in Dependency Proxy which caches upstream Docker photos. Previously a premium function, Dependency Proxy was open-sourced and made accessible to all GitLab variations in November 2020 as a part of GitLab 13.6.

The Dependency Proxy behaves as a pull-through cache for Docker photos saved on Docker Hub. Establishing the Dependency Proxy can speed up your pipelines and helps you keep inside Docker’s rate limits.

Enabling The Dependency Proxy

Dependency Proxy’s availability is managed by an instance-level setting. Enabling the Dependency Proxy requires GitLab to be reconfigured. This may trigger a quick interval of downtime.

To allow the function, add the next line to your set up’s /and many others/gitlab/gitlab.rb file:

gitlab_rails["dependency_proxy_enabled"] = true

Save the file and run the next command in your terminal:

sudo gitlab-ctl reconfigure

The directions above are for GitLab Omnibus installations. In the event you put in from supply, the dependency proxy must be enabled within your config/gitlab.yml file.

Utilizing the Dependency Proxy

Dependency Proxy solely works with GitLab teams. You’ll be able to’t at present use it with standalone private tasks.

The function is generally used inside CI pipeline scripts. When referencing a picture inside a pipeline, prefix the image’s Docker Hub identify with the CI_DEPENDENCY_PROXY_GROUP_IMAGE_PREFIX variable. This variable robotically resolves to the Dependency Proxy URL on your energetic GitLab group.

picture: ${CI_DEPENDENCY_PROXY_GROUP_IMAGE_PREFIX}/nodejs:newest

This pipeline will run its job inside a nodejs:newest container. The picture can be pulled by the Dependency Proxy. Subsequent pipeline runs gained’t must hit Docker Hub until the upstream picture truly modifications.

You can even entry the Dependency Proxy manually, exterior of GitLab CI. You should authenticate with docker login first. You’ll want to make use of your GitLab username and password, or your username and a private entry token.

docker login gitlab.instance.com --username username --password password

As soon as authenticated, you possibly can docker pull utilizing the GitLab Dependency Proxy. Exchange example-group within the URL beneath with the identify of the group you need to use. The pulled picture can be cached into that group’s Dependency Proxy.

docker pull gitlab.instance.com/example-group/dependency_proxy/containers/nodejs:newest

In the event you additionally use GitLab’s Container Registry (to retailer photos you construct), take word that Dependency Proxy is solely separate and has a unique URL. Whereas Container Registry is generally uncovered by itself subdomain (e.g. registry.instance.com), Dependency Proxy is accessed by way of the identical hostname because the GitLab internet UI.

How The Dependency Proxy Works

The Dependency Proxy presents itself as one other Docker registry. If you need to use the proxy, you docker login to it after which docker pull as regular.

If the Dependency Proxy has already cached the picture, it’ll return it straight with out utilizing Docker Hub. In any other case, the picture is pulled from Docker Hub, cached within the proxy and returned to your Docker CLI.

GitLab will attempt to contact Docker Hub for each docker pull, even when a cached picture is on the market. It’s because the proxy should verify whether or not the picture has been up to date on Docker Hub.

This process doesn’t have an effect on Docker’s charge limiting. Docker permits free HEAD requests to check picture manifest variations. If Docker signifies the cached picture is outdated, GitLab will pull the contemporary model (incurring a charge restrict hit). In any other case, the cached picture can be returned, with out including to your Docker Hub charge restrict tally.

These traits make the Dependency Proxy preferrred for CI pipelines. By logging into the proxy, you possibly can safely docker pull on each pipeline run, with out hitting the Docker Hub charge restrict.

Configuring Dependency Proxy Settings

Dependency Proxy can use a considerable quantity of storage over time. You’re caching photos from Docker Hub; these photos may be fairly giant relying on what you’re utilizing.

GitLab enables you to customise the storage location. Set the dependency_proxy_storage_path setting in /and many others/gitlab/gitlab.rb if you wish to use a devoted storage drive.

gitlab_rails["dependency_proxy_storage_path"] = "/mnt/my-storage-drive"

Supply installations ought to set the storage_path property inside the dependency_proxy part of config/gitlab.yml as an alternative.

You can even retailer your cached photos on an object storage service akin to Amazon S3. Right here’s an instance Omnibus configuration in /and many others/gitlab/gitlab.rb:

gitlab_rails["dependency_proxy_object_store_enabled"] = true
 
# That is the S3 bucket identify
gitlab_rails["dependency_proxy_object_store_remote_directory"] = "gitlab-dependency-proxy"
 
gitlab_rails["dependency_proxy_object_store_connection"] = {
    "supplier" => "AWS",
    "area" => "eu-west-1",
    "aws_access_key_id" => "AWS_ACCESS_KEY_ID",
    "aws_secret_access_key" => "AWS_SECRET_ACCESS_KEY"
}

To enhance efficiency, GitLab will cache photos regionally after which add them to S3 within the background. In the event you’d reasonably add on to S3, set the dependency_proxy_object_store_direct_upload setting to true.

You should reconfigure GitLab (sudo gitlab-ctl reconfigure) after making modifications to the storage settings. The Dependency Proxy will then retailer cached photos utilizing your new configuration.

Releasing Up Storage

GitLab never deletes cached Dependency Proxy information. You’ll be able to view the contents of a teams cache by choosing Packages & Registries > Dependency Proxy from its sidebar. This display screen enables you to allow or disable the Dependency Proxy for the group and see the entire dimension of the saved information. Nonetheless, you possibly can’t use the UI to clear previous blobs.

If it’s essential liberate storage, you will need to use the GitLab API. There’s a single endpoint which helps you to clear all of the Dependency Proxy information saved for a particular group.

Create a private entry token by clicking your profile within the top-right, clicking “Entry Tokens” within the left sidebar and including a brand new entry token with the api scope.

Subsequent, use curl to delete a gaggle’s Dependency Proxy cache:

curl --request DELETE --header "PRIVATE-TOKEN: <Entry-Token>" https://gitlab.instance.com/api/v4/teams/<Group-Id>/dependency_proxy/cache

To search out your group ID, go to the homepage of the group you need to cleanup. The group’s ID can be proven subsequent to its identify.

Conclusion

Enabling the Dependency Proxy is an easy step which improves the resiliency of your pipelines. If Docker Hub goes down, the proxy will nonetheless present your pipeline with cached picture variations.

The Dependency Proxy additionally helps you keep inside Docker Hub’s charge limits. You’ll solely want to drag photos from Docker Hub after they truly change. For an energetic crew working many pipelines every day, this may help you keep away from having to improve to a premium Docker Hub plan.



Source link

Gravatar Image
I love to share everything with you

Leave a Reply

Your email address will not be published. Required fields are marked *