GitLab Omnibus package upgrade error: It seems you are upgrading from X to Y

Written by Claudio Kuenzler - 0 comments

Published on November 14th 2023 - Listed in Git Linux

---

The last time I logged into our on-premise GitLab Community Edition (CE), I was greeted by a security notice with a red alert icon:

Yikes! Looks like a critical security fix was released and we need to upgrade.

As the notice states, the currently used version is 16.2.1, which is meanwhile outdated and no security fix is available for the 16.2 release.

Upgrade to latest version: Fail

As this installation uses the GitLab omnibus package, the package upgrade can be executed through a handy apt-get dist-upgrade. At least this works fine for one minor version jump, e.g. from 16.2 to 16.3. But APT by default chooses the latest available version, which is 16.5.1 while I'm writing this.

And this results in the following error at the end of the APT output:

Preparing to unpack .../27-gitlab-ce_16.5.1-ce.0_amd64.deb ...
gitlab preinstall: It seems you are upgrading from 16.2 to 16.5.
gitlab preinstall: It is required to upgrade to the latest 16.3.x version first before proceeding.
gitlab preinstall: Please follow the upgrade documentation at https://docs.gitlab.com/ee/update/index.html#upgrade-paths
dpkg: error processing archive /tmp/apt-dpkg-install-mGP23F/27-gitlab-ce_16.5.1-ce.0_amd64.deb (--unpack):
 new gitlab-ce package pre-installation script subprocess returned error exit status 1

The package's preinstall script detected a jump in the version number, not just one minor number but 3.

Yep, you're right, GitLab package. Let's do minor upgrades first.

Note: Big thanks and kudos to the package maintainer(s) at GitLab! Not all deb packages detect such situations and this can sometimes lead to major outages by breaking an upgrade.

Do one upgrade at a time (minor version hopping)

The output from the package nicely showed us we should first upgrade to 16.3.x. Let's search for all 16.x versions currently available in the GitLab repository:

root@gitlab:~# apt-cache show gitlab-ce | egrep "^Version: 16"
Version: 16.5.1-ce.0
Version: 16.5.0-ce.0
Version: 16.4.2-ce.0
Version: 16.4.1-ce.0
Version: 16.4.0-ce.0
Version: 16.3.6-ce.0
Version: 16.3.5-ce.0
Version: 16.3.4-ce.0
Version: 16.3.3-ce.0
Version: 16.3.2-ce.0
Version: 16.3.1-ce.0
Version: 16.3.0-ce.0
Version: 16.2.8-ce.0
Version: 16.2.7-ce.0
Version: 16.2.6-ce.0
Version: 16.2.5-ce.0
Version: 16.2.4-ce.0
Version: 16.2.3-ce.0
Version: 16.2.2-ce.0
Version: 16.2.1-ce.0
Version: 16.2.0-ce.0
Version: 16.1.5-ce.0
Version: 16.1.4-ce.0
Version: 16.1.3-ce.0
Version: 16.1.2-ce.0
Version: 16.1.1-ce.0
Version: 16.1.0-ce.0
Version: 16.0.8-ce.0
Version: 16.0.7-ce.0
Version: 16.0.6-ce.0
Version: 16.0.5-ce.0
Version: 16.0.4-ce.0
Version: 16.0.3-ce.0
Version: 16.0.2-ce.0
Version: 16.0.1-ce.0
Version: 16.0.0-ce.0

The latest available 16.3.x version is therefore 16.3.6.

And what about the gitlab-runner package? Interestingly the upgrade of that package supported the jump over multiple minor versions after the apt-get dist-upgrade:

root@gitlab:~# dpkg -l|grep gitlab
ii  gitlab-ce        16.2.1-ce.0  amd64   GitLab Community Edition (including NGINX, Postgres, Redis)
iU  gitlab-runner    16.5.0       amd64   GitLab Runner

To make the versions consistent, I decided to use the same major/minor release (16.3.x) for both packages. For gitlab-runner, this is currently the 16.3.3 release.

To install a specific version, the release/version tag can be added behind the package name:

root@gitlab:~# apt-get install gitlab-ce=16.3.6-ce.0 gitlab-runner=16.3.3
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following package was automatically installed and is no longer required:
  libreadline7
Use 'apt autoremove' to remove it.
Suggested packages:
  docker-engine
The following packages will be upgraded:
  gitlab-ce
The following packages will be DOWNGRADED:
  gitlab-runner
1 upgraded, 0 newly installed, 1 downgraded, 0 to remove and 0 not upgraded.
37 not fully installed or removed.
Need to get 1,774 MB of archives.
After this operation, 156 MB disk space will be freed.
Do you want to continue? [Y/n] y

The installation of the package(s) ran through nicely:

     _______ __  __          __
    / ____(_) /_/ /   ____ _/ /_
   / / __/ / __/ /   / __ `/ __ \
  / /_/ / / /_/ /___/ /_/ / /_/ /
  \____/_/\__/_____/\__,_/_.___/


Upgrade complete! If your GitLab server is misbehaving try running
  sudo gitlab-ctl restart
before anything else.
If you need to roll back to the previous version you can use the database
backup made during the upgrade (scroll up for the filename).

And now the next version (16.4.x) could be installed. Should, actually, because GitLab 16.3.x is EOL in just 2 days (November 16th 2023).

TL;DR: When upgrading GitLab, make sure to not jump over a release and do one minor version upgrade after another.

Add a comment

Show form to leave a comment

Comments (newest first)

No comments yet.