Elastic Agent Upgrade Command Allows Downgrade to Insecure version

[allamiro]

Elastic Agent Upgrade Command Allows Downgrade to Insecure or older version.

Reproduction steps:

Currently we using 8.9.0

1. Upgrade the elastic version using the upgrade command

elastic-agent upgrade 8.11.3 --skip-verify

2. Run the Elastic Agent upgrade command with an older version number.```

elastic-agent upgrade 8.9.0 --skip-verify

3. Observe that the agent successfully downgrades to the older version.

elastic-agent status 

Recommendations:

• Add a version comparison check to prevent downgrading to an older version.
• Display a warning message or prompt the user for confirmation before allowing the downgrade.
• Consider adding a --force or --allow-downgrade flag to allow users to bypass the version check (with caution).

Example use case:

Running the command elastic-agent upgrade <old_version> should either:

• Refuse to downgrade and display a warning message.
• Prompt the user to confirm the downgrade.
• Require an additional flag (e.g., --force) to force the downgrade.

[elasticmachine]

Pinging @elastic/elastic-agent-control-plane (Team:Elastic-Agent-Control-Plane)

[cmacknz]

I don't think we'll ever forbid going backwards as it is a useful escape hatch when the unexpected happens (although we are likely going to change the way this is implemented see discussion in elastic/kibana#172745).

Up until the --unprivileged option was added, you had to be root on the local machine to do this which was reasonable protection. With unprivileged perhaps we could consider requiring root to upgrade as install and uninstall do since it is a similarly powerful operation. Root isn't naturally required for upgrade the way it is when installing because we aren't changing any of the OS service definitions.

A potentially less disruptive change in the same vein would be to guard use of --skip-verify behind being root since that is the dangerous part of this. We don't allow use of --skip-verify for Fleet because allowing mass installation of unverified binaries is not a feature we want in the product.

[allamiro]

I don't think we'll ever forbid going backwards as it is a useful escape hatch when the unexpected happens (although we are likely going to change the way this is implemented see discussion in elastic/kibana#172745).

Up until the --unprivileged option was added, you had to be root on the local machine to do this which was reasonable protection. With unprivileged perhaps we could consider requiring root to upgrade as install and uninstall do since it is a similarly powerful operation. Root isn't naturally required for upgrade the way it is when installing because we aren't changing any of the OS service definitions.

A potentially less disruptive change in the same vein would be to guard use of --skip-verify behind being root since that is the dangerous part of this. We don't allow use of --skip-verify for Fleet because allowing mass installation of unverified binaries is not a feature we want in the product.

I believe the main concern here is the lack of restrictions on which versions you can downgrade to which is also discussed in the issue you referred to #172745 . While the suggestion to require root access for using --skip-verify is a good step, it may not provide enough security. A more robust approach would include:

1. In case of any problems during the upgrade automatically rolling back or restricting downgrades to previously installed, working versions by tracking the current version state.
2. Requiring a mandatory confirmation flag, particularly when downgrading involves bypassing critical security updates or may be displaying explicit warnings about potential security risks when downgrading to known vulnerable versions.

While downgrades can be valuable in certain situations, they should be tightly controlled in environments where security is critical, especially when reverting to older versions with potentially known vulnerabilities.
For more details, see the security update

https://discuss.elastic.co/t/beats-and-elastic-agent-8-11-3-7-17-16-security-update-esa-2023-30/349180
@cmacknz

[allamiro]

I think the reasoning behind using --skip-verify In an air-gapped environment, the agent won’t be able to download the remote key and therefore cannot be upgraded. Please check the release notes for 8.9.0

https://www.elastic.co/guide/en/fleet/8.9/release-notes-8.9.0.html

@cmacknz

[cmacknz]

In case of any problems during the upgrade automatically rolling back or restricting downgrades to previously installed, working versions by tracking the current version state.

The end implementation in elastic/kibana#172745 will be similar to this, we will only allow rolling back to the previously installed version and can disallow downgrades to arbitrary versions. I link that issue back to this one.