Incident update: Protocol paused out of an abundance of caution; unpaused after investigation determined no ETH at risk.

Information on the background, impact, and nature of the incident, plus its remediation.

Incident update: Protocol paused out of an abundance of caution; unpaused after investigation determined no ETH at risk.

  • Severity: 0
  • Status: Resolved, with monitoring in place to ensure ongoing stability
  • Risks:
    • No risk to user ETH
    • Reduced protocol efficiency for a limited scope of time while the risk was determined, during which time a percentage of staked ETH was not actively receiving ETH network rewards


  • During an investigation of unexpected validator exits (related to unexpected exit daemon behavior) on January 30, 2024, the Liquid Collective protocol was paused out of an abundance of caution.
  • While assessing the cause of the incident, the protection of user funds was prioritized over protocol liveness in a conservative approach.
  • Following an investigation, it was determined there was no security vulnerability, slashing risk, or user ETH at risk of loss. The protocol's infrastructure and Node Operator infrastructure was not compromised.
  • The protocol was then unpaused.


Incident

A Liquid Collective Node Operator unexpectedly exited validators from Ethereum, triggering an investigation. It was determined that the cause was due to a bug in Liquid Collective’s exit daemon software.

Out of an abundance of caution, the protocol was paused while investigation determined that no user ETH was at risk, and that no malicious activity had taken place. The protocol was then unpaused via a coordinated action of The Liquid Foundation’s executive governor multisig.


Severity classification

This protocol incident was classified with a severity of 0 as no ETH was at risk of loss, and there was no slashing incident.


Next steps

  • A temporary increase of the protocol’s variation safeguard will be put in place to ensure the exited ETH can be seamlessly absorbed by the Liquid Collective protocol and programmatically restaked. The protocol variation safeguard will then be put back in place.
  • The bug in the exit daemon is being investigated in collaboration with Liquid Collective’s Node Operators and Service Providers to ensure there will be no further impact to the protocol’s expected function.
  • More updates will be provided here as the exit daemon software is hardened and improved.

Liquid Collective is dedicated to maintaining high security standards. We recognize the importance of protecting participants' security when using the protocol, and understand that security is primordial to maintaining participants' trust.

You can view the protocol’s security resources on Liquid Collective’s Diligence page.


Update – Jan 31, 2024

After further investigation, the cause of the incident appears to be isolated to offchain operations of the exit daemon.

Yesterday, a temporary increase of the protocol’s variation safeguard was prepared as a cautionary step due to the randomness of the Ethereum exit process. After confirming new onchain data this morning, and running simulations, the temporary increase of the protocol’s variation safeguard may not be necessary to ensure the exited ETH can be absorbed by the protocol and programmatically restaked.

We will continue to monitor the situation and only implement the temporary increase if it becomes necessary.


Update – Feb 9, 2024

All Liquid Collective Node Operators have now successfully deployed v0.17.0 of the exit daemon software, including two patches to remediate the cause of this incident. The protocol is fully functioning as expected at this time.

We will continue to actively monitor the protocol to ensure ongoing stability, and will issue an incident report.

Your subscription could not be saved. Please try again.
Thank you for subscribing!

Sign up for the newsletter

Follow @liquid_col for updates