INTRODUCTION TO ERROR BUDGETS

31 May INTRODUCTION TO ERROR BUDGETS

We need to think of error budgets as a balance we have in our pocket for the amount of risk that we can take in a given period. This balance arises out of not setting up the SLOs to straight 100% due to high operational costs and little to no benefit of increased reliability.

Error budgets are proportionally allocated as per the organisation’s need. If the error budget is exhausted before the allotted time period, the organisation will have to prioritise increasing the reliability and provide uninterrupted service.

On the other hand, if the organisation has an unutilised error budget more often than not, they have the capability of taking more risk to get in new features and bring in changes.

Coming to the calculation of Error Budgets, they are defined by the service level objective (SLO) itself. They imply how unreliable a particular service is allowed to be. For example, if the SLO target is 99.9% in a year, our error budget is equal to 100%-99.9% which comes out to be 0.1%.

So, as per our error budget, we can have a downtime of 525 minutes in a single year. Keeping track of the consumption of the error budget is important as it helps us not to overspend when planning on new releases. 

Moreover, it helps us determine a balance between risk and availability as if a service incurs too much downtime the provider will have to reduce the risk and make the service more available which will halt the new deployments. On the other hand, if the users want to release new features regularly they will need to accept a much lenient SLO which will significantly increase our error budget and compromise on the availability. It is based on the assumption that making a service 100% available is not feasible and also unnecessary. 

It is important to understand that having the right SLO and error budget is directly linked with the customer’s happiness so this index must be considered while preparing the SLOs and error budget.

It may be possible that the provider fails to meet the SLO because of external factors, so all these terms must be defined in the error budget and not limited only to the provider’s end. Error budgets may also provide a special provision to the users to reduce the availability if they want to launch a really important new feature.

There is an important factor to consider while using error budgets which is known as error budget burn rate.

Error budget burn rate is calculated by taking three factors into account-

1. Error rate
2. Error budget balance
3. Time until error budget period completion.

Simply deploying an error budget does not serve any purpose to any team or an organisation, so it will be really helpful for the teams to be mindful of the burn rate. By this, the teams can predict whether they will be exhausting up their budget before the completion of the period or they are on track and will meet their SLOs. This can serve as an alarm to the teams to get alerted if the burn rate shows they are not going to meet their SLOs and do the needful.

Conclusion
SLOs become more useful for an organisation when they are complemented with error budgets and help the organisation create the right balance between risk and availability keeping user’s happiness as a key index. Also, when an error budget is defined the senior management mustn’t try to change the priorities and disturb the budget consumption to try to launch a new feature on an urgent basis when a team is trying to achieve reliability. It needs to be a collective effort between different teams and management in an organisation to take the error budget seriously as that is what will make customers happy.