Upgrade Chef Infra Server
Each new release of Chef Infra Server improves reliability and updates 3rd party components to ensure the security of the server. It is important to keep Chef Infra Server up to date to ensure the secure and reliable operation of Chef Infra in your organization.
Warning
Upgrade Matrix
If running a Chef Infra Server 12.17.15 or later you can upgrade directly to the latest releases of Chef Infra Server 14. If you are running a release before 12.17.15 you must perform a stepped upgrade as outlined below.
Running Version | Upgrade To Version | Requires License | Supported Version |
---|---|---|---|
13 | 14 | Yes | Yes |
12.17.15 | 14 | Yes | No |
12.3.0 | 12.17.15 | No | No |
11 | 12.3.0 | No | No |
- Requires License
- Chef Infra Server 13 and later are governed by the Chef EULA. You are required to accept these terms when using Chef Infra Server for the first time by entering
Yes
when prompted. - Supported Release
- Chef Infra Server 14 and later are supported Chef Software releases. Earlier releases are not supported. For more information about supported Chef Software see the Supported Versions documentation.
Release-Specific Steps
Upgrading to 14.x
Chef Infra Server 14.0 moved from Solr to Elasticsearch as its search index. The Chef Infra Server 14 upgrade process requires downtime for stopping the server, installing the new package, and then upgrading the server, which will include an automatic Elasticsearch reindexing operation for existing Solr users. We estimate the reindexing operation will take 2 minutes for each 1000 nodes, but the it could take more time, depending on your server hardware and the complexity of your Chef data.
The Chef Infra Server 14 upgrade does not automatically reindex existing external Elasticsearch installations.
Upgrading to 14.8
Chef Infra Server 14.8 upgrades PostgreSQL from 9.6 to 13.3. The 14.8 upgrade process requires a one-time downtime to vacuum, upgrade, and re-index the database. The entire upgrade operation takes about one minute for each 1000 nodes (1000 nodes is approximately 286MB). This process may take longer depending on your server hardware and the size of the node objects on your Chef Infra Server.
Note
postgresql['pg_upgrade_timeout']
attribute in chef-server.rb to the timeout value for the upgrade. Set this value based on the size of your data, where it take about one minute per 1,000 nodes which is approximately 286MB.
Database Preparation
-
Run
VACUUM FULL
on the PostgreSQL database if you don’t have automatic vacuuming set up. This process will reduce the size of the database by deleting unnecessary data and speeds up the migration. TheVACUUM FULL
operation takes around 1 to 2 minutes per gigabyte of data depending on the complexity of the data, and requires free disk space at least as large as the size of your database.sudo su - opscode-pgsql /opt/opscode/embedded/bin/vacuumdb --all --full
You should then see output like:
vacuumdb: vacuuming database "bifrost" vacuumdb: vacuuming database "oc_id" vacuumdb: vacuuming database "opscode-pgsql" vacuumdb: vacuuming database "opscode_chef" vacuumdb: vacuuming database "postgres" vacuumdb: vacuuming database "template1"
-
Back up the PostgreSQL database before upgrading so you can restore the full database to a previous release in the event of a failure. See Backup and Restore for more information.
Upgrade Steps
Follow the Chef Infra Server upgrade instructions below.
Note
Upgrading to 12.17.15
Warning
insecure_addon_compat
to false
.
As of version 12.14, Chef Infra Server renders passwords inside of the /etc/opscode
directory by default.
If you are using Chef Infra Server without add-ons, or if you are using the latest add-ons versions, you can set insecure_addon_compat
to false
in `/etc/opscode/chef-server.rb.
and Chef Infra Server will write all credentials to a single location.
For more information on password generation, including a list of supported add-on versions, see Chef Infra Server Credentials Management.
Upgrading to 12.3.0
If you are running a Chef Infra Server release before 12.3.0, please contact Chef Support for guidance on upgrading your Chef Infra Server installation.
Chef Infra Server 14 Upgrade Process
Standalone Server
The Chef Infra Server 14 upgrade process requires downtime for stopping the server, installing the new package, and then upgrading the server, which will include an automatic Elasticsearch reindexing operation for existing Solr users. We estimate the reindexing operation will take 2 minutes for each 1000 nodes, but the it could take more time, depending on your server hardware and the complexity of your Chef data.
-
Run
vacuumdb
before starting the upgrade:sudo su - opscode-pgsql /opt/opscode/embedded/bin/vacuumdb --all --full exit
You should see output like:
vacuumdb: vacuuming database "bifrost" vacuumdb: vacuuming database "oc_id" vacuumdb: vacuuming database "opscode-pgsql" vacuumdb: vacuuming database "opscode_chef" vacuumdb: vacuuming database "postgres" vacuumdb: vacuuming database "template1"
-
Back up your Chef Infra Server data before starting the upgrade process using chef-server-ctl-backup. Make a note of where the backup is located (The default is
/var/opt/chef-backup
). Please note that Chef Infra Server will go offline to perform the backup:sudo chef-server-ctl backup
-
Confirm that the Chef Infra Server services are operational:
chef-server-ctl reconfigure
-
If you are already running 12.17.15 or greater, proceed to the next step. Otherwise consult the upgrade matrix and perform a stepped upgrade.
- If you are running Chef Infra Server 12.3.0, upgrade to 12.17.15.
- If you are running Chef Infra Server 11, first upgrade to 12.3.0, and then to 12.17.15.
After performing the stepped upgrade to 12.17.15, continue with the next step.
-
Download the desired Chef Infra Server version from the Chef Infra Server Downloads.
-
Stop the Chef Infra Server:
chef-server-ctl stop || chef-server-ctl kill rabbitmq
-
Install the Chef Infra Server package:
To install with
dpkg
:dpkg -iEG /path/to/chef-server-core-VERSION.deb
To install with the RPM Package Manager:
rpm -Uvh --nopostun /path/to/chef-server-core-VERSION.rpm
-
Upgrade the server and accept the Chef Software license by entering
Yes
at the prompt:chef-server-ctl upgrade
To accept the license and upgrade in one command:
CHEF_LICENSE='accept' chef-server-ctl upgrade
-
If the upgrade failed, see the section below on how to handle an upgrade failure.
If the upgrade was successful, start Chef Infra Server:
chef-server-ctl start
-
Upgrade any Chef Infra Server add-ons.
-
After the upgrade process is complete, test and verify that the server works.
-
Clean up the Chef Infra Server by removing the old data:
chef-server-ctl cleanup
-
Reindex the database:
sudo su - opscode-pgsql /opt/opscode/embedded/bin/reindexdb --all
You should see output like:
reindexdb: reindexing database "bifrost" reindexdb: reindexing database "oc_id" reindexdb: reindexing database "opscode-pgsql" reindexdb: reindexing database "opscode_chef" reindexdb: reindexing database "postgres" reindexdb: reindexing database "template1"
You are now finished with the upgrade.
Note
Upgrade Failure Troubleshooting
-
If the upgrade failed and you have a corrupted Chef Infra Server and/or a corrupted database, DO NOT RISK YOUR BACKUP OF THE DATABASE. Take all steps necessary to preserve the backup, including copying it to another disk. Consult with a professional sysadmin for instructions and best practices.
-
Contact customer support.
-
Reinstall the original version of Chef Infra Server you were using before attempting the upgrade process (if you had to perform a stepped upgrade, install your original version of Chef Infra Server before the stepped upgrade, not any versions you upgraded to in the stepped upgrade process). Again, DO NOT RISK YOUR BACKUP OF THE DATABASE. For example, consider using a separate disk from your backup for the new installation.
-
Consult the restore documentation and restore the database from the path to where it was saved:
chef-server-ctl restore /path/to/tar/archive.tar.gz
External PostgreSQL
The following External PostgreSQL upgrade steps are provided as a courtesy only. It is the responsibility of the user to upgrade and maintain any External PostgreSQL configurations.
Upgrade Chef Infra Server
-
Log into the external PostgreSQL machine.
-
Run
vacuumdb
before starting the upgrade:sudo su postgres /usr/bin/vacuumdb --all --full exit
-
Log into the Chef Infra Server machine.
-
Consult the documentation on
knife-ec-backup
.Install
knife-ec-backup
, if it not already installed. A sample session follows (note that your steps could differ, depending on the versions of your software, the topology of your setup, your OS and distribution, and a range of other factors).For example:
apt-get update apt install ruby apt install make curl -L https://chef.io/chef/install.sh | sudo bash -s -- -P chefdk export PATH=$PATH:/root/.chefdk/gem/ruby/2.6.0/bin apt-get -y install gcc postgresql libpq-dev /opt/chefdk/embedded/bin/gem install knife-ec-backup -- --with-pg-config=/opt/opscode/embedded/postgresql/9.6/bin/pg_config
-
Configure
knife
if it is not already configured. A sample session follows (again, note that your steps could differ, depending on a range of factors).For Example:
$ chef-server-ctl org-create 4thcafe 'Fourth Cafe, Inc.' --association_user janedoe --filename /tmp/4thcafe-validator.pem $ chef generate repo chef-repo $ cd chef-repo/ $ mkdir -p .chef $ echo '.chef' >> .gitignore $ cp /tmp/4thcafe-validator.pem .chef $ cp /home/ubuntu/janedoe.pem .chef $ cd .chef $ knife configure WARNING: No knife configuration file found. See https://docs.chef.io/config_rb/ for details. Please enter the chef server URL: [https://1.2.3.4/organizations/myorg] https://1.2.3.4/organizations/4thcafe Please enter an existing username or clientname for the API: [ubuntu] janedoe $ knife ssl fetch
-
Backup the database. For Example:
mkdir /backup /opt/chefdk/embedded/bin/knife ec backup /backup
-
If you are running Chef Infra Server version 12.17.15 or greater, proceed to the next step below. Otherwise consult the upgrade matrix and perform a stepped upgrade.
If you are running a Chef Infra Server release before 12.17.15, you cannot upgrade directly to 14.8.X. You must perform a stepped upgrade first.
- If you are running Chef Infra Server 12.3.0, upgrade to 12.17.15.
- If you are running Chef Infra Server 11, you must first upgrade to 12.3.0, and then to 12.17.15.
After performing the stepped upgrade, return here and continue with the next step below.
-
Download the desired version of Chef Infra Server.
-
Stop the Chef Infra Server:
chef-server-ctl stop || chef-server-ctl kill rabbitmq
-
Install the Chef Infra Server package:
To install with
dpkg
:dpkg -iEG /path/to/chef-server-core-VERSION.deb
To install with the RPM Package Manager:
rpm -Uvh --nopostun --force /path/to/chef-server-core-VERSION.rpm
-
Upgrade Chef Infra Server and accept the Chef Software license by entering
Yes
at the prompt:chef-server-ctl upgrade
To accept the license and upgrade in one command:
CHEF_LICENSE='accept' chef-server-ctl upgrade
-
If the upgrade failed, see the section on upgrade failure troubleshooting.
If the upgrade was successful, start the Chef Infra Server services and cleanup.
sudo chef-server-ctl start sudo chef-server-ctl cleanup
You are now finished with the Chef Infra Server upgrade. Proceed directly to the Upgrade PostgreSQL section.
Upgrade PostgreSQL
-
Log into the external PostgreSQL machine.
-
Update packages and install your selected PostgreSQL version. Example (Ubuntu/PostgreSQL 13.3):
sudo apt-get update sudo apt-get install postgresql-13
-
Check if there are any differences in the config files. Make sure to update the new config files if required. Example (PostgreSQL 13.3):
diff /etc/postgresql/OLD_POSTGRESQL_VERSION/main/postgresql.conf /etc/postgresql/13/main/postgresql.conf diff /etc/postgresql/OLD_POSTGRESQL_VERSION/main/pg_hba.conf /etc/postgresql/13/main/pg_hba.conf
-
Stop the PostgreSQL service.
sudo systemctl stop postgresql.service
-
Log in as the
postgres
user.su postgres
-
Ensure that you are in a directory where you can run the
pg_upgrade
command.Example:
cd /tmp
-
Check clusters (notice the
--check
argument, this will not change any data). Example (PostgreSQL 13.3):/usr/lib/postgresql/13/bin/pg_upgrade \ --old-datadir=/var/lib/postgresql/9.6/main \ --new-datadir=/var/lib/postgresql/13/main \ --old-bindir=/usr/lib/postgresql/9.6/bin \ --new-bindir=/usr/lib/postgresql/13/bin \ --old-options '-c config_file=/etc/postgresql/9.6/main/postgresql.conf' \ --new-options '-c config_file=/etc/postgresql/13/main/postgresql.conf' \ --check
-
Migrate the data (without the
--check
argument). Example (PostgreSQL 13.3):/usr/lib/postgresql/13/bin/pg_upgrade \ --old-datadir=/var/lib/postgresql/9.6/main \ --new-datadir=/var/lib/postgresql/13/main \ --old-bindir=/usr/lib/postgresql/9.6/bin \ --new-bindir=/usr/lib/postgresql/13/bin \ --old-options '-c config_file=/etc/postgresql/9.6/main/postgresql.conf' \ --new-options '-c config_file=/etc/postgresql/13/main/postgresql.conf'
-
Log out of the
postgres
user.exit
-
Swap the ports for the old and new PostgreSQL versions. Example (PostgreSQL 13.3):
$ sudo vim /etc/postgresql/13/main/postgresql.conf # change "port = 5433" to "port = 5432" $ sudo vim /etc/postgresql/9.6/main/postgresql.conf # change "port = 5432" to "port = 5433"
-
Start the PostgreSQL service.
sudo systemctl start postgresql.service
-
Log in as the
postgres
user and confirm that the new PostgreSQL version is correct. Example (PostgreSQL 13.3):$ sudo su - postgres $ psql -c "SELECT version();" version --------------------------------------------------------------------------------------------------------------------------------------------- PostgreSQL 13.3 (Ubuntu 13.3-1.pgdg16.04+1) on x86_64-pc-linux-gnu, compiled by gcc (Ubuntu 5.4.0-6ubuntu1~16.04.12) 5.4.0 20160609, 64-bit (1 row)
-
Run
reindexdb
. Example:$ /usr/bin/reindexdb --all reindexdb: reindexing database "bifrost" reindexdb: reindexing database "oc_id" reindexdb: reindexing database "opscode_chef" reindexdb: reindexing database "postgres" reindexdb: reindexing database "template1"
-
Log into the Chef Infra Server machine.
-
Check the status of Chef Infra Server. PostgreSQL should be connected.
$ sudo chef-server-ctl status ------------------- Internal Services ------------------- run: bookshelf: (pid 15763) 219163s; run: log: (pid 16559) 228464s run: elasticsearch: (pid 15797) 219162s; run: log: (pid 16345) 228507s run: nginx: (pid 15901) 219162s; run: log: (pid 16745) 228452s run: oc_bifrost: (pid 15909) 219161s; run: log: (pid 16237) 228519s run: oc_id: (pid 15915) 219161s; run: log: (pid 16255) 228513s run: opscode-erchef: (pid 15948) 219160s; run: log: (pid 16673) 228458s run: redis_lb: (pid 15952) 219160s; run: log: (pid 16779) 228445s ------------------- External Services ------------------- run: postgresql: connected OK to 10.0.11.0:5432
Upgrade Failure Troubleshooting
-
If the upgrade failed and you are left with a corrupted Chef Infra Server and/or a corrupted database, DO NOT RISK YOUR BACKUP OF THE DATABASE. Take all steps necessary to preserve the backup, including copying it to another disk. Consult with a professional sysadmin for instructions and best practices.
-
Contact customer support.
-
Reinstall the original version of Chef Infra Server you were using before attempting the upgrade process (if you had to perform a stepped upgrade, install your original version of Chef Infra Server before the stepped upgrade, not any versions you upgraded to in the stepped upgrade process). Again, DO NOT RISK YOUR BACKUP OF THE DATABASE. For example, consider using a separate disk from your backup for the new installation.
-
Consult the documentation on knife-ec-restore, and restore the database from the path where it was saved.
Example:
/opt/chefdk/embedded/bin/knife ec restore /backup/
-
Do not continue upgrading PostgreSQL until you have an uncorrupted Chef Infra Server and an uncorrupted PostgreSQL database.
Chef Backend Install
Warning
Chef Backend is deprecated and no longer under active development. Contact your Chef account representative for information about migrating to Chef Automate HA.
This document is no longer maintained.
The Chef Infra Server can operate in a high availability configuration that provides automated load balancing and failover for stateful components in the system architecture.
To upgrade your Chef Backend installation, see High Availability: Upgrade to Chef Backend 2.
Tiered Install
This section describes the upgrade process from a tiered server configuration.
Note
tier
topology.
For the latest information on setting up a highly available server cluster, see High Availability: Backend Cluster.
Tiered Upgrade Steps
To upgrade to Chef Infra Server on a tiered Chef Infra Server configuration, do the following:
-
Back up the Chef Infra Server data before starting the upgrade process using knife-ec-backup.
-
Confirm that the Chef Infra Server services are operational:
chef-server-ctl reconfigure
-
Download the desired Chef Infra Server version from the Chef Infra Server Downloads page, then copy it to each server.
-
Stop all front end servers:
chef-server-ctl stop
-
Install the Chef Infra Server package on all servers:
To install with
dpkg
:dpkg -i /path/to/chef-server-core-<version>.deb
To install with the RPM Package Manager:
rpm -Uvh --nopostun /path/to/chef-server-core-<version>.rpm
-
Stop the back end server:
chef-server-ctl stop
-
Upgrade the server and accept the Chef Software license by entering
Yes
at the prompt:chef-server-ctl upgrade
To accept the license and upgrade in one command:
CHEF_LICENSE='accept' chef-server-ctl upgrade
-
Copy the entire
/etc/opscode
directory from the back end server to all front end servers:scp -r /etc/opscode <each server's IP>:/etc
-
Upgrade each of the front end servers:
chef-server-ctl upgrade
-
Run the following command on both the front end, and back end servers:
chef-server-ctl start
-
Upgrade any Chef Infra Server add-ons.
-
After the upgrade process is complete, test and verify that the server works.
-
Clean up the server by removing the old data:
chef-server-ctl cleanup
Was this page helpful?