Consistency Models

Services that store files in the cloud adhere to one or a mix of a number of different consistency models. Consistency assertions reflect which assumptions can be made over the availability of data after a transaction. The most common consistency models are (variations of) read-after-write consistency and eventual consistency.

Read-after-write consistency means that data is available for reading immediately after it was written. For example, if you upload a file to a service with read-after-write consistency you will be able to move or download the file immediately afterwards.

Eventual consistency on the other hand means that data will be available for reading after it was written but makes no assertion as to when. This can lead to stale and inconsistent views of the data. For example, if you upload a file to a service with eventual consistency you might not be able to move or download it immediately afterwards. In fact, listing the contents of a folder you just uploaded a file to might not show the file. If you try to delete it, this might fail with the service claiming that the file does not exist.

Read-after-write consistency is generally easier to work with but also more expensive to maintain in terms of money and availability. If the cloud data is distributed over multiple locations, maintaining read-after-write consistency can be hard. It means letting read requests wait until the system has assured that previous writes have successfully and completely propagated. In case one data node fails, the system might have to stall the reads until the node comes back up.

Service support

This is why both consistency models can be found and the following table shows which of the CloudRail supported services uses which model:

Service Name Measured Consistency Official Consistency Remarks
Dropbox Read-after-write ?
GoogleDrive Read-after-write ?
OneDrive Eventual ?
OneDrive Business Eventual ?
Box Read-after-write ?
Egnyte Read-after-write Read-after-write
Amazon S3 Eventual Eventual Read-after-write consistency for a few special cases
Azure Read-after-write Read-after-write
Google Cloud Mixed Mixed List operations for objects and buckets are eventually consistent, the rest is read-after-write consistent
Backblaze Read-after-write ?
Rackspace Eventual Eventual

'Measured Consistency' describes what kind of consistency we have experienced in our tests and 'Official Consistency' is what the respective provider officially claims. There are still a couple of question marks in the table, especially with the consumer oriented cloud storage services. We will update it regularly if things change. You should not rely on read-after-write consistency if it is not officially confirmed. Our tests may have missed eventual consistency that was too fast to detect with the data used in the test.

How to deal with it

Dealing with read-after-write consistency is easy, it works like one would intuitively expect it to. Call a CloudRail function to write a file and make sure that no error/exception has been raised by the SDK. Then, call any kind of reading CloudRail function on that file and it will work just fine.

Dealing with eventual consistency is harder. You need to program more defensively, so for example you will want to (repeatedly) check if a change has been made on the cloud storage after writing before you perform any kind of read operation. Alternatively, you can expect the read operation to fail and retry after a while if it does. This requires more implementation effort but works for both eventual and read-after-write consistent storages.