Applying system upgrades
Automated package update policy
Cloud 66 aims to make it easier to build immutable infrastructure. Building servers and applications from scratch is much better than modifying existing server configurations and tinkering with settings until things start to work.
When a new server is provisioned we automatically install the latest versions of packages. After the server is created we only auto-install packages that are marked as
security updates. Cloud 66 doesn’t (typically) automatically update other packages because it doesn’t want to risk disrupting your existing application. However, depending on your configuration, we may update your packages each time you redeploy your app (see below).
Do not configure servers manually
Cloud 66 has a number of features that systematize and automate server configuration. Manual changes to configuration files and settings on servers and components are very likely to be either overwritten or cause conflicts.
Controlling package updates
You can add custom Linux (Ubuntu) packages to your application using Deploy Hooks (read our full guide here.) Depending on how you’ve set them up, Deploy Hooks can either install packages only when a new server is created, or on every (re)deploy (or both).
Managing high-risk updates
If you’re concerned about a major update to one of the packages you depend on, our first recommendation is to build a new application and redirect your traffic to the new application using your Failover Address.
If rebuilding is impractical or impossible, there are three approaches to dealing with the issue:
- Ignore the package update — this is the safest bet if you’re concerned about app stability, but can create issues in the longer term. Be sure to check and update your Deploy Hooks so that the package doesn’t get automatically installed the next time you deploy.
- Manually update the packages by logging into the servers and using
sudo apt-get -y upgradeor
- Update the packages indirectly through scaling up a new server, and then dropping the old one. New servers will have the latest packages installed on them unless you have explicitly locked package versions using Deploy Hooks.
Some package updates require server-reboot. When scaling up we restart your new servers automatically to ensure everything works correctly. Alternatively, you can reboot your servers manually or via the Toolbelt.
Upgrade package types
In the event of a security vulnerability on any of the components we deploy on the servers, we aim to be as quick as possible to roll out the recommended patches.
To manually trigger security upgrades:
- Open the Application Overview from your Dashboard
- Clicking on the Build / Deploy button and choose Deploy with Options
- Click on the Options tab
- Check Apply Security Upgrades and Yes, reboot my servers if required
- Click Run Now
This will perform operating system security package upgrades and also set up unattended upgrades for your application. Unattended upgrades will automatically check for and install the latest Ubuntu security packages on a daily basis.
Note that some security packages may require a server restart. We don’t automatically restart your server, and it is at your discretion to do so.
If the file
/var/run/reboot-required exists, that means your server requires a restart. To see which packages contributed to the requirement for a restart, read
There are multiple options and considerations when upgrading your base version of Ruby. Please read our detailed guide on the subject.
A quick summary:
- Ruby version should be defined in your
manifest.ymland not in a Gemfile (or
- A safe way to roll out the upgraded version is to use our “scale up” feature to create a new server that uses the new version of Ruby
We automatically install the latest available version of Bundler whenever either:
- a new server is created
- the version of Ruby is upgraded on an existing server
If an existing server is throwing errors related to your Bundler version, the best option is usually to upgrade Ruby to a more recent version. Be sure to follow our best practice (linked above) when doing so to ensure the upgrade runs smoothly.
Rails should be upgraded in the same way as Ruby. See above for details.
The recommended way to upgrade your passenger to the latest one is to deploy to a new web server and drop the old one, so the scaled up one will automatically have the latest version supported by Cloud 66.
Switching from malloc to jemalloc
If you’d prefer to use jemalloc as your memory allocator, you can change this via your manifest file. However, when you deploy (with upgrades) this will cause downtime as it is applied to each server. There are two solutions for avoiding downtime:
- Use a rolling deployment strategy (requires a load balancer & at least four servers)
- Scale up new servers, upgrade them, and then switch them with the current servers (see above)
About manual upgrades
If you need to upgrade any part of your application the best course of action is always to build a new server. However, if that is not desirable or possible, you can always perform manual upgrades.
We detect the version of all the components we have configured or deployed on your servers on a regular basis and after each deployment. If you upgrade any part of your application manually, the upgrade will be detected by Cloud 66. This helps with the future automated upgrades.
Troubleshooting package upgrades
Although there are an almost infinite number of issues that could occur with updating different packages, we have summarised some common issues:
Deployment failed: Unable to install package(s)
The most common reason for this error is that the version of one or more components specified in your
Docker file conflicts with other components - particularly with the version of the underlying operating system (Ubuntu).
If you are trying to upgrade an existing application - particularly one that had been running for 12 months or longer - be sure to check both your Manifest file and (where appropriate) your Docker file. You can view and edit these files via your Dashboard or directly in your git repo.
If you have explicitly specified a version for any component in either of these files, Cloud 66 will attempt to use that version on the assumption that you have specified it for a good reason (such as backward compatibility).
“Failed to Fetch … 404 Not Found” errors
The most common cause of these kinds of errors is some kind of interruption in network connectivity - typically between your cloud provider and the package repository. Such interruptions are not uncommon, and typically resolve themselves quickly. Wait a few minutes and try again. If the problem persists:
- Check you can reach the server yourself (via SSH)
- If you can access the machine, you should attempt to run the package upgrade manually via ssh (using the standard
apt-get install <package>command)
- If you’re still getting 404s you should check that your cloud provider’s firewall isn’t blocking access to the package repo (and/or Cloud 66’s IP range).
“Unable to find expected entry” errors
If your application depends on older or legacy packages, these packages (or the repos that support them) can sometimes cease to exist, particularly when an older version of an operating system reaches “end of life” (EOL). There are sometimes manual workarounds for this kind of things, but in general we recommend upgrading to packages that are being actively maintained.
“…has no installation candidate” errors
This typically only occurs if you’re using your own (“registered”) servers and running an unsupported operating system and/or distribution. We only support the two most recent major releases of Ubuntu. You can check what we currently support here. The best plan here is to reinstall a supported version of Ubuntu.