How Privacy-Preserving are Line Clouds? Recovering Scene Details from 3D Lines

TL;DR

This paper investigates the privacy-preserving properties of line clouds used in 3D scene representations, demonstrating that scene details can still be approximately recovered, raising privacy concerns.

Contribution

It reveals that line clouds, proposed for privacy, still retain significant scene information, enabling approximate scene reconstruction.

Findings

Line clouds preserve substantial scene geometry information.

Approximate 3D point recovery from line clouds is feasible.

Privacy protection in line clouds is less effective than previously thought.

Abstract

Visual localization is the problem of estimating the camera pose of a given image with respect to a known scene. Visual localization algorithms are a fundamental building block in advanced computer vision applications, including Mixed and Virtual Reality systems. Many algorithms used in practice represent the scene through a Structure-from-Motion (SfM) point cloud and use 2D-3D matches between a query image and the 3D points for camera pose estimation. As recently shown, image details can be accurately recovered from SfM point clouds by translating renderings of the sparse point clouds to images. To address the resulting potential privacy risks for user-generated content, it was recently proposed to lift point clouds to line clouds by replacing 3D points by randomly oriented 3D lines passing through these points. The resulting representation is unintelligible to humans and effectively prevents point cloud-to-image translation. This paper shows that a significant amount of information about the 3D scene geometry is preserved in these line clouds, allowing us to (approximately) recover the 3D point positions and thus to (approximately) recover image content. Our approach is based on the observation that the closest points between lines can yield a good approximation to the original 3D points. Code is available at https://github.com/kunalchelani/Line2Point.