Two Decades of Linux Expertise 400-9696-513 24/7 technical support

Cloudflare discloses cross-tenant data exposure in its container platform: the root cause was the storage allocator, not the VM boundary

Cloudflare discloses cross-tenant data exposure in its container platform: the root cause was the storage allocator, not the VM boundary

A cloud provider's promise of VM isolation does not automatically cover the storage layer. InfoQ reports that Cloudflare recently disclosed and fixed a cross-tenant data exposure vulnerability in its Containers platform, which also underpins the Sandboxes product: customers on paid Workers accounts had a way to read disk blocks left behind by other customers' containers on the same host.

The technical root cause is worth reading closely. Every Cloudflare container runs inside a Firecracker micro-VM, and the root disk is allocated through the Linux device mapper's thin provisioning. The affected resource pool used 64 KiB thin blocks with the skip_block_zeroing option enabled, meaning newly allocated blocks were not zeroed. When a container wrote a 4 KiB block, the layer beneath allocated a 64 KiB physical block, and the remaining up to 60 KiB could still hold data written by the previous container, recoverable through a raw read of the disk. In testing, security researchers identified thousands of directory blocks belonging to other users and even restored structurally intact SQLite databases.

Cloudflare's response was quick: the fix was merged on Sept 4, the same day the report arrived, followed by removal of the problematic option, retirement of existing container disks, and clearing of host data and image caches, with all cleanup completed on Sept 19. Retrospective analysis of historical telemetry found no sign of malicious exploitation. As industry commentary put it, the Firecracker VM boundary itself held up; the leak happened in the block-allocation layer beneath the boundary.

The lessons for enterprise cloud native teams are concrete: when evaluating a multi-tenant platform, do not only ask whether VM isolation is in place, but also confirm that ephemeral disks are zeroed at allocation. Teams running their own container platforms should write zero-on-allocate or erase-on-release into the isolation baseline and back it with repeatable verification tests.

Source: InfoQ

← Back to list Enquire about related services

Related reading

Call us Message
Home Services Consult Message