Position Tracking for Virtual Reality Using Commodity WiFi

TL;DR

WiCapture is a novel WiFi-based system that enables precise, range-resistant, and ubiquitous position tracking for virtual reality using commodity WiFi hardware, eliminating the need for specialized infrastructure.

Contribution

This paper introduces WiCapture, the first WiFi-based VR position tracking system that achieves high accuracy with off-the-shelf hardware, surpassing existing 3-DoF solutions.

Findings

Achieves 0.88 cm accuracy in position tracking

Uses commodity WiFi radios for tracking

Offers high range and resistance to occlusion

Abstract

Today, experiencing virtual reality (VR) is a cumbersome experience which either requires dedicated infrastructure like infrared cameras to track the headset and hand-motion controllers (e.g., Oculus Rift, HTC Vive), or provides only 3-DoF (Degrees of Freedom) tracking which severely limits the user experience (e.g., Samsung Gear). To truly enable VR everywhere, we need position tracking to be available as a ubiquitous service. This paper presents WiCapture, a novel approach which leverages commodity WiFi infrastructure, which is ubiquitous today, for tracking purposes. We prototype WiCapture using off-the-shelf WiFi radios and show that it achieves an accuracy of 0.88 cm compared to sophisticated infrared based tracking systems like the Oculus, while providing much higher range, resistance to occlusion, ubiquity and ease of deployment.

Equations

$${\hat{h}}_q=\frac{1}{T}{\int }_0^T{h}_q{\mathrm{e}}^{j2\pi ft}{\mathrm{e}}^{-j2\pi ft+j\nu }dt=h_q{\mathrm{e}}^{j\nu },$$

$$\vec{a}({\theta }_k)=[1{\mathrm{e}}^{-j2\pi d\cos ({\theta }_k)/\lambda }{\mathrm{e}}^{-j4\pi d\cos ({\theta }_k)/\lambda }{]}^{\top }.$$

$$\text{H}=[\vec{a}({\theta }_1)\dots \vec{a}({\theta }_L)]\text{F}=\text{A}\text{F},$$

$${\text{H}}_2=\text{A}\text{D}\text{F},$$

$${\hat{\text{H}}}_p={\text{H}}_p{\mathrm{e}}^{j{\nu }_p}.$$

$${\hat{\text{H}}}_1=\text{A}\text{F}{\mathrm{e}}^{j{\nu }_1},{\hat{\text{H}}}_2=\text{A}\text{D}\text{F}{\mathrm{e}}^{j{\nu }_2}.$$

$$\text{X}=[{\hat{\text{H}}}_1{\hat{\text{H}}}_2\dots {\hat{\text{H}}}_P]=\text{A}\text{G},$$

$${\hat{\text{F}}}_p={\text{A}}^{†}{\hat{\text{H}}}_p.$$

$${\hat{\text{F}}}_1=\text{F}{\mathrm{e}}^{j{\nu }_1},{\hat{\text{F}}}_2=\text{D}\text{F}{\mathrm{e}}^{j{\nu }_2}$$

$$\underset{\text{D}}{\mathrm{minimize}}$$

$$\text{D}\text{ is diagonal}.$$

$${\vec{s}}_k=\text{phase of }({\text{D}}_{k+1,k+1}/{\text{D}}_{1,1}).$$

$$\frac{-2\pi }{\lambda }[(\text{cos}({\theta }_{k+1})-\text{cos}({\theta }_1))(\text{sin}({\theta }_{k+1})-\text{sin}({\theta }_1))].$$

$$\vec{\delta }=\arg \min \Vert \text{R}\vec{\delta }-\vec{s}\Vert .$$

Full text

1 Introduction

Immersive experiences like virtual reality (VR) require accurate tracking of the headset and other accessories like hand-motion controllers. Current commercial tracking systems like Oculus Rift [4] and HTC Vive [1] are outside-in where the tracking is performed using infrastructure external to the VR accessories. The external infrastructure is specialized and typically uses infrared (IR) cameras along with sensors on the headset to perform the tracking. These systems are very accurate but have the following limitations:

• •

They require installing specialized hardware and dedicated infrastructure wherever user wants to experience VR. So if a user wishes to use VR headsets anywhere in her home, one would need IR cameras everywhere.

• •

These systems are not occlusion resistant. For example, if the camera is blocked by furniture or if the user turns away from the camera, then the tracking fails.

• •

These systems have limited range, typically around $2$ m in front of the camera [8].

A competing technology to provide position tracking is inside-out position tracking found in systems like the Microsoft Hololens [3]. These systems use cameras (both RGB and depth sensing) and implement vision based tracking algorithms on the headset. These systems are both accurate and infrastructure-free, however they come with certain limitations. Specifically, they significantly increase the complexity of the headset since they need to have several cameras as well as complex algorithms running on the headset to provide tracking. Further they are not robust, tracking fails in environments with transparent or texture-less objects (e.g. a white wall) [63]. Finally and most importantly, these systems cannot be used for tracking peripherals such as hand-motion controllers; the complexity of inside-out tracking is too high to be implemented on such peripherals which are meant to be lightweight and cheap.

In this paper, we present WiCapture, a novel VR position tracking system which addresses the above limitations of existing systems. WiCapture is a WiFi based position tracking system. Headsets transmit standard WiFi packets which are received by standard WiFi access points (APs). The WiFi APs receive metadata from each WiFi packet reception called Channel State Information (CSI) which encodes the transformation the environment has induced upon the transmitted WiFi signals. WiCapture invents novel algorithms that mine the CSI metadata to recover the position of the headset accurately. It has the following properties:

• •

WiCapture does not require special hardware, it uses commodity APs that can be bought in retail stores.

• •

WiCapture is occlusion resistant, it continues to work even when the APs and headsets are occluded due to furniture or other objects in between.

• •

WiCapture has larger range and operates across rooms.

• •

It is insensitive to room illumination or texture and it can work in the dark. Further, headset complexity is minimal, all the headset needs is a standard WiFi chip.

At a high level, as illustrated in Fig. 2, WiCapture obtains the change in the position of the transmitter by using the change in the phase of the CSI between packets. As with the ToF (Time of Flight) cameras, the phase is distorted due to the signal received from reflectors; this phenomenon is called multipath propagation. However, unlike ToF cameras where the light transmitter and camera are time-synchronized, the phase of WiFi signal is also distorted due to the lack of synchronization of clocks at the WiFi transmitter and receiver. WiCapture tackles these challenges using novel algorithms that compensate for these distortions and provides accurate phase measurement which in turn enables accurate position tracking.

1.1 Contributions

• •

WiCapture is the first commodity WiFi-based sub-centimeter level accurate tracking system.

• •

We developed a novel technique to overcome the distortion due to clock differences by exploiting the multipath. This is surprising as multipath is traditionally viewed as a complication in localization systems [35].

• •

WiCapture is the first system that accurately disentangles signal from different paths by using CSI from multiple packets. The key observation is that the direction of the paths remain stationary over small intervals of time and CSI of all the packets obtained within this time can be used to resolve multipath accurately.

• •

We built WiCapture using commodity Intel 5300 WiFi chips  [28] which demonstrated a precision of $0.25$ mm and a position tracking error of $0.88$ cm.

1.2 Limitations

WiCapture’s current prototype however has two limitations compared to existing systems. It has higher latency since the tracking is computed in the network and then provided as an update to the headset. Second, it is less accurate than current outside-in position tracking systems. We believe that WiCapture’s accuracy is acceptable for VR given the significant benefits that WiCapture provides around deployment, coverage and occlusion resistance.

2 Related work

Motion tracking has been of great interest with applications in 3D object reconstruction, virtual/augmented reality, motion capture and motion control [58, 41]. Systems requiring infrared LEDs or photodiodes on the tracked object have short-range, limited field of view, difficulty in tracking multiple objects, and require line of sight between tracked object and sensor [36, 4, 63, 30, 49]. Pulsed laser light based systems, in addition, require time synchronization of multiple laser light sources [30]. WiCapture does not share these limitations as it has long range and typical obstructions like walls and humans are transparent to WiFi signals.

Magnetic signal-based systems [48, 5] are occlusion-resistant but have a small range and are affected by distortions due to ferromagnetic materials  [56]. Radio frequency signal based techniques using RFIDs (Radio Frequency IDentification) [55, 62] and ultrawideband [25] signals demonstrated centimeter-level accuracy but have limited range and require specialized infrastructure that is not as ubiquitous as WiFi. Tracking systems using other modalities like ultrasound [23, 52] and IR [7, 18] achieve high accuracy but require infrastructure dedicated for tracking.

Infrastructure-free approaches using inertial measurement units (IMUs) can only track the orientation of a device but not the position [22]. Visual-inertial navigation systems [21, 45, 16] which use cameras and IMUs track the motion of a camera using natural image features unlike earlier systems [57] which required instrumentation of the environment with markers. However, visual systems have problems in environments with transparent or textureless objects [63] and are not applicable when the camera is occluded; for example, a phone cannot track itself when placed in a user’s pocket. However, when they are applicable, we view WiCapture and infrastructure-free systems as complementary solutions that together can potentially form a robust and easy-to-deploy motion tracking system.

Disentangling multipath is a widely studied problem in Time of Flight (ToF) cameras and wireless literature as it enables several important applications like transient light imaging  [46, 38, 42], wireless imaging [10, 11] and localization  [35, 54].  [26, 24, 34, 20, 29, 19, 13, 38, 32] explored transmitting signals at multiple frequencies to resolve multipath in ToF cameras. Similarly, wireless localization systems  [35, 59] explored using multiple frequencies to resolve multipath. Unlike all the previous systems, we use signal received from multiple packets to improve the accuracy of estimated multipath parameters. This hinges on the fact that the multipath parameters like direction of the propagation paths are stationary over small time periods.

WiFi-based localization systems can be broadly classified into RSSI (Received Signal Strength Indicator) based, ToF based, and AoA (Angle of Arrival at the receiver) based systems. Signal strength (RSSI) based systems have 2-4 m localization error [12, 17, 27] because of distortions in RSSI due to multipath. RSSI fingerprint-based approaches [64, 43] achieve $0.6$ m error but require an expensive, recursive fingerprinting operation. ToF based approaches achieve meter-level localization error  [40, 61, 54]. AoA based approaches achieved state-of-the-art decimeter-level localization error [35, 60, 37, 51, 31]. WiCapture on the other hand is targeted towards estimating relative trajectory rather than the absolute position. WiFi-based tracking systems  [65] can only classify the trajectory of the target WiFi device into limited number (four) of known gestures.

Techniques-wise,  [53] theoretically observed that stationarity of multipath parameters improves multipath estimation accuracy. [47] removed the effect of time offset between laser source and camera receiver on the ToF measurements by considering differential ToF between different rays.  WiCapture builds on these techniques and compensates not only for time offset but also frequency offset between the source and the receiver.

3 Preliminaries

In WiFi systems, data is chunked into units of efficient size for communication and each unit is called as a packet [9]. At the start of each packet, a WiFi transmitter sends a reference signal. A WiFi receiver correlates the received signal with the reference signal and samples at time zero to calculate Channel State Information (CSI). This is similar to how image is formed in a ToF camera by correlating the received signal at each pixel with a reference signal [13]. So, each antenna is a sensor element which performs job similar to a pixel in a ToF camera and CSI calculated at WiFi receiver is similar to an image obtained at a ToF camera. We now provide a brief primer on CSI (or image) formation model. We included a glossary at the end of the paper for WiFi-specific terms.

3.1 CSI calculation

Consider the reference complex sinusoid signal $\mathrm{e}^{j2\pi ft}$ of frequency $f$ emitted by the $q^{\mathrm{th}}$ antenna on the transmitter; here $j$ is the complex root of $-1$ and $t$ is time. The signal passes through wireless channel $h_{q}$ resulting in the received signal $h_{q}\mathrm{e}^{j2\pi ft}$. CSI corresponding to the $q^{\mathrm{th}}$ transmit antenna, $\hat{h}_{q}$, is obtained as

[TABLE]

where $T$ is time for which sinusoid is transmitted and $\nu$ is the phase of the receiver’s local sinusoidal signal relative to the phase of the transmitter’s sinusoidal signal (see Fig. 4).

In indoor environments, similar to ToF camera systems, transmitted signal travels along multiple paths and the signals from all the paths superpose to form the received signal. Each path is associated with an AoD (Angle of Departure) from the transmitter and attenuation of the signal along the path. We now describe the relation between the CSI and these path parameters. For this description, we consider a linear $3$-antenna array (see Fig. 3) although the model can be extended to any arbitrary antenna array geometry.

3.2 Wireless channel modeling

Consider an environment with $L$ paths; e.g., the setup in Fig. 5(b) has $2$ paths. Let the spacing between two consecutive antennas at the transmitter be $d$ (see Fig. 3). Let the AP/receiver be in the same plane as the transmitter antenna array. Let $\theta_{k}$ denote the angle of departure (AoD) from the transmitter for the $k^{\mathrm{th}}$ path using the convention shown in Fig. 3. Let $\gamma_{k}$ denote the signal received along $k^{\mathrm{th}}$ path from the first transmitter antenna to the receiver.

The signal along $k^{\mathrm{th}}$ path travels different distances from different transmit antennas to the receiver. These differences result in different phases for CSI corresponding to different antennas. As described in  [60] and illustrated in Fig. 3, the vector of signals received from the transmit antennas along $k^{\mathrm{th}}$ path can be written as $\vec{a}(\theta_{k})\gamma_{k}$, where

[TABLE]

This vector $\vec{a}(\theta_{k})$ is also known as the steering vector. We have as many steering vectors as the number of paths.

The signals from different paths superpose and result in the wireless channel $h_{q}$. So, wireless channel,

[TABLE]

where $\textbf{H}\in\mathbb{C}^{3\times 1}$ is the wireless channel whose $q^{\mathrm{th}}$ element is $h_{q}$, $\textbf{F}\in\mathbb{C}^{L\times 1}$ is the matrix of complex attenuations whose $k^{\mathrm{th}}$ element is $\gamma_{k}$, $\textbf{A}\in\mathbb{C}^{3\times L}$ is the steering matrix whose $k^{\mathrm{th}}$ column is the steering vector for $k^{\mathrm{th}}$ path.

Let $\widehat{\textbf{H}}\in\mathbb{C}^{3\times 1}$ be the observed CSI (or image) whose $q^{\mathrm{th}}$ element is $\hat{h}_{q}$. Wireless channel H would be related to the observed CSI matrix using the relation, $\widehat{\textbf{H}}=\textbf{H}\mathrm{e}^{j\nu}$.

3.3 Change in channel due to displacement

We will now describe how the transmitter’s motion affects the wireless channel. For the rest of the paper, we will index the CSI with corresponding packet, i.e., $\widehat{\textbf{H}}_{p}\in\mathbb{C}^{3\times 1}$ is CSI for $p^{\mathrm{th}}$ packet and ${\textbf{H}_{p}}\in\mathbb{C}^{3\times 1}$ represents the corresponding wireless channel. Let’s say that the transmitter moved by a small displacement $\vec{\delta}$ between the times two consecutive reference signals are transmitted. Then the path-length for the $k^{\mathrm{th}}$ path changes by $(\vec{r}_{\theta_{k}}^{\top}\cdot\vec{\delta})$ where $\vec{r}_{\theta_{k}}$ is the unit vector along the direction of departure for the particular path with AoD $\theta_{k}$. This induces a phase shift, $2\pi(\vec{r}_{\theta_{k}}^{\top}\cdot\vec{\delta})/\lambda$. So, attenuation of the $k^{\mathrm{th}}$ path gets multiplied by $\mathrm{e}^{-j2\pi(\vec{r}_{\theta_{k}}^{\top}\cdot\vec{\delta})/\lambda}$. Mathematically, if $\textbf{H}_{1}=\textbf{A}\textbf{F}$ as in Eq. 3, then the wireless channel for the second packet is

[TABLE]

where ${\textbf{H}_{2}}\in\mathbb{C}^{3\times 1}$, $\textbf{A}\in\mathbb{C}^{3\times L}$, ${\textbf{F}}\in\mathbb{C}^{L\times 1}$, and $\textbf{D}\in\mathbb{C}^{L\times L}$ is a diagonal matrix with entries $\mathrm{e}^{-j2\pi(\vec{r}_{\theta_{1}}^{\top}\cdot\vec{\delta})/\lambda}$, $\ldots$, $\mathrm{e}^{-j2\pi(\vec{r}_{\theta_{L-1}}^{\top}\cdot\vec{\delta})/\lambda}$, and $\mathrm{e}^{-j2\pi(\vec{r}_{\theta_{L}}^{\top}\cdot\vec{\delta})/\lambda}$.

3.4 Phase distortion due to frequency offset

The transmitter and the receiver WiFi chips have different clocks and this creates an offset between the frequencies of the local reference sinusoids used at the transmitter and the receiver. Moreover, the offset is not constant with time as the clocks drift [66, 15]. As illustrated in Fig. 4, the frequency offset results in a change in the relative phase between the sinudoids at the transmitter and the receiver (see Equation 1) from packet to packet. Let $\nu_{p}$ be the relative phase between the two sinusoids for $p^{\mathrm{th}}$ packet. Then,

[TABLE]

So, if the transmitter moved by $\vec{\delta}$ between consecutive packets, then using Equations 3, 4 and  5, the CSI reported by the WiFi chip for the two packets can be written as

[TABLE]

Eq. 6 relates the observed CSI to the transmitter’s displacement. It is important to note that frequency offset of $20$ kHz is typical and WiFi standard allows it to be as high as $200$ kHz [9]. This implies that the distortion in CSI due to frequency offset is orders of magnitude greater than the change caused by transmitter’s motion. For example, if two consecutive WiFi packets are sent with a time-gap of $10$ ms (which is typical), then the relative phase between the transmitter and receiver sinusoids can change by $630$ radians and is observed in practice [44]. To compare, if the transmitter moved by $5$ mm during the same $10$ ms, then the change in the phase of any path is less than $0.6$ radians.

4 Design

Fig. 5 shows how WiCapture would be deployed and summarizes the solution procedure. The target transmits normal WiFi signals. Each WiFi AP/receiver calculates CSI (or image) and sends it to a central server. The server uses CSI from all the access points (APs) to determine the position of the target by performing the following two steps:

Estimate AoD and complex attenuation of all the paths. 2. 2.

Use the attenuation from consecutive CSIs along with AoD of the paths to estimate the displacement of the transmitter during the time between consecutive CSIs.

4.1 Estimating AoD of all the paths

Let us rewrite the CSI for the first packet from Eq. 6 as $\widehat{\textbf{H}}_{1}=\textbf{A}(\textbf{F}\mathrm{e}^{j\nu_{1}})$. We can observe that CSI is nothing but a linear combination of steering vectors in A. If one obtains the steering vectors, then finding the AoD of all the paths is trivial using Eq. 2. However, the problem of obtaining steering vectors from a single linear combination of theirs is ill-posed. Prior work [53] has theoretically investigated that the more number of linear combinations of steering vectors with independent weights are provided, the higher the accuracy of the estimated steering vectors.

Note that the AoDs of paths are relatively stationary and do not change rapidly from packet to packet. For example, if the receiver is $3$ m away from the transmitter, a tiny motion of $1$ mm changes the AoD of any path by atmost $0.02$ degrees, which does not cause any measurable changes in the elements of the steering matrix. However, the weights in the linear combination change significantly as even a small motion creates significant change in the phase of complex attenuation along a path.

Using this insight, CSI from $P$ packets where AoD has not changed much are concatenated using Eq. 7. CSI for any of these packets is linear combination of the same steering vectors. So,

[TABLE]

where $\textbf{X}\in\mathbb{C}^{3\times P}$, $\textbf{A}\in\mathbb{C}^{3\times L}$ is steering matrix which is common for all the $P$ packets, $\textbf{G}\in\mathbb{C}^{L\times P}$ is a matrix of steering vector weights. This is standard form for applying well-known MUSIC algorithm to compute AoD [50, 35, 59]. The goal of the MUSIC algorithm is to find all the steering vectors given multiple linear combinations of the steering vectors and it provides an accurate estimate of A due to diverse linear combinations.

4.2 Estimating the displacement of the transmitter between consecutive packets

We will now show how the displacement of the WiFi transmitter between two consecutive packets can be estimated. Consider the CSI from the first two packets.

**Estimate the steering vectors’ weights: ** CSI is a linear combination of steering vectors estimated using procedure in Sec. 4.1. For $p^{\mathrm{th}}$ packet, these weights in the linear combination, $\widehat{\textbf{F}}_{p}\in\mathbb{C}^{L\times 1}$, can be obtained using

[TABLE]

Substituting Eq. 6 into Eq. 8, we can observe that

[TABLE]

**Estimate the change in the complex attenuation for each of the paths: ** Note that $\textbf{D}\in\mathbb{C}^{L\times L}$ and it’s entries contain information about target’s displacement (see Eq. 4). We obtain an estimate D by solving the convex optimization problem 10 using standard procedures [14].

[TABLE]

From Equations 9, 10 and 4, the $k^{\mathrm{th}}$ diagonal element of D, ${\textbf{D}}_{k,k}$, is an estimate of $\mathrm{e}^{-j2\pi(\vec{r}_{\theta_{k}}^{\top}\cdot\vec{\delta})/\lambda+j\nu_{2}-j\nu_{1}}$. Note that unit vector in the direction of the departure can be obtained using $\vec{r}_{{\theta}_{k}}=[\cos({\theta}_{k})\;\sin({\theta}_{k})]^{\top}$; here ${\theta}_{k}$ is the AoD for the particular path estimated using procedure in Sec. 4.1. So, if the term $(\nu_{2}-\nu_{1})$ is removed, one can estimate $\vec{\delta}$ from the phase of elements of D.

However, as discussed in Sec. 3.4, the term $\nu_{2}-\nu_{1}$ due to frequency offset is orders of magnitude larger than the term $-2\pi(\vec{r}_{{\theta}_{k}}^{\top}\cdot\vec{\delta})/\lambda$ due to displacement. This change in phase due to frequency offset is not only unavailable but also extremely hard to predict [15] making it hard to estimate and remove the term $\nu_{2}-\nu_{1}$. This is precisely the phenomenon that led to long-held notion that phase information across packets is uncorrelated in commodity WiFi systems [44] and is unusable for tracking.

**Use attenuation-change between paths: **Our unique insight is that one can get rid of the effect of frequency offset by using the phase of the signal from multiple paths. This is surprising as multipath is traditionally viewed as a complicating distortion that needs to be compensated in many ToF depth sensor and WiFi localization systems [35, 33]. Notice that the effect of frequency offset is the same for all the paths, i.e., the term, $\nu_{2}-\nu_{1}$, is present in the change of phase for all the paths. So, effect of the clock differences is removed by considering the change in the phase of signal along a path with respect to change in the phase of signal from another path.

Specifically, consider the phase of the complex number ${\textbf{D}}_{k+1,k+1}/{\textbf{D}}_{1,1}$. It is an estimate of $(-2\pi\vec{r}_{{\theta}_{k+1}}^{\top}\cdot\vec{\delta}/\lambda+\nu_{2}-\nu_{1})-(-2\pi\vec{r}_{{\theta}_{1}}^{\top}\cdot\vec{\delta}/\lambda+\nu_{2}-\nu_{1})=(-2\pi(\vec{r}_{{\theta}_{k+1}}-\vec{r}_{{\theta}_{1}})^{\top}\cdot\vec{\delta}/\lambda)$. Notice that the term, $\nu_{2}-\nu_{1}$, canceled out by performing this operation. So, a $(L-1)$-dimensional vector $\vec{s}$ is calculated whose $k^{\mathrm{th}}$ element is

[TABLE]

Then $\textbf{R}\in\mathbb{R}^{(L-1)\times 2}$ is calculated whose $k^{\mathrm{th}}$ row is

[TABLE]

One can then obtain an estimate of the displacement by solving the simple linear least squares problem 13.

[TABLE]

If there are multiple APs, matrix R and vector $\vec{s}$ obtained from multiple access points are concatenated vertically. If there are $L$ paths from the target to each of the $U$ APs, then the concatenated $\textbf{R}\in\mathbb{R}^{U*(L-1)\times 2}$ and the concatenated $\vec{s}\in\mathbb{R}^{U*(L-1)}$. Since the target displacement is same irrespective of the AP, one can estimate of the displacement by solving Eq. 13 using these concatenated matrices. We summarize the overall algorithm in Algorithm 1.

5 Evaluation

We implemented WiCapture using off-the-shelf Intel 5300 WiFi cards which support three antennas. We employed Linux CSI tool [28] to obtain the CSI. The WiFi cards operate in 5 GHz WiFi spectrum ($f$ in Sec. 3). Also, the CSI information is quantized, i.e., each of real and imaginary parts of CSI is represented using 8 bits.

The system used for evaluation consists of APs and a target device equipped with WiFi cards. The target/transmitter has a $3-$antenna circular array with distance between any two antennas equal to $2.6$ cm. The APs operate in monitor mode. CSI is calculated once every $6$ ms. So, the trajectory is estimated at an update rate of $167$ Hz for our evaluation experiments. An implementation is provided in [2]. We use $U=4$ APs, set $V=10$ s and $L=2$ paths in Algorithm 1.

5.1 Stationary experiments

We start by examining the jitter/precision of WiCapture, i.e., how stationary the estimated position is when the target is stationary. This is important for VR because the scene displayed on the VR headset is not expected to change when the user is not moving. In our experiment, the target remains stationary and transmits $1000$ packets. The access points are placed at the corners of $5$ m $\times$ $6$ m space. The standard deviation of the trajectory obtained by using Algorithm 1 is used as the measure of jitter. The median jitter observed over 21 experiments at different target positions is $0.25$ mm.

We measured the jitter of Oculus DK2 system by placing it at the same positions. The Oculus camera is placed at $0.75$ m away from the headset. The median jitter observed for Oculus DK2 is $0.10$ mm. So, WiCapture’s jitter in position estimation is comparable to that of a commercial position tracking system which requires dedicated infrastructure.

5.2 Controlled tracking experiments

Next we evaluate the resolution of WiCapture. We mount the target on a mechanical stage which has a least count of $0.005$ cm. The target is moved in increments of $0.1$ cm and then in decrements of $0.1$ cm so that it reaches the initial position as shown in Fig. 6(a). The target transmits a WiFi packet at each position. Trajectory estimated by WiCapture is translated so that the initial position is origin. The maximum error in estimation of position of any point in the trajectory is $0.11$ cm.

We conducted another experiment where the target is moved mechanically to different positions on a trajectory shown in Fig. 6(b). The maximum error in estimation of position of any point in the trajectory is $0.27$ cm. Note that WiCapture resolves even millimeter-level target motion.

5.3 Extensive tracking experiments

Motion tracking accuracy of WiCapture is dependent on the multipath environment, the material used in walls, the presence of metallic objects, the density of WiFi AP deployment and many other factors. In this evaluation, we test WiCapture’s accuracy in different deployment scenarios.

Compared Approaches: To the best of our knowledge, there exists no WiFi-based system that is geared towards tracking the motion of a commodity WiFi chip. We faithfully implemented SpotFi, a state-of-the-art WiFi localization system and compared WiCapture against it. We did not compare with other non-vision based systems like ultrasound [39] because of limitations detailed in section 2 such as extensive installation of dedicated infrastructure.

**Ground truth: **We use an Oculus DK2 [8] headset which is rigidly attached to the target to obtain ground truth trajectory. We placed the headset $0.9$ m away from the Oculus camera. Under these conditions, we calculated the accuracy of Oculus position tracking to be at sub-millimeter level using mechanical stage experiments (see Sec. 5.2).

**Metric (Trajectory error): **To focus on the shape of the trajectory alone, we translate the trajectory reported by each of the systems (Oculus DK2, SpotFi and WiCapture) by the initial position of the trajectory so that the initial position of all the trajectories is origin. Similarly, since the reference coordinate axes of different systems are not aligned, we rotate the trajectory reported by WiCapture (and SpotFi) so that the Root Mean Squared Error (RMSE) between the points of the rotated trajectory of WiCapture (and SpotFi) and the points on the Oculus trajectory is minimized. Note that we just shift and rotate the trajectories of WiCapture and SpotFi but do not scale the trajectories. As in other motion tracking systems [55], absolute point-by-point position difference between this shifted and rotated trajectories and the ground truth trajectory is reported as the trajectory error.

5.3.1 Indoor office deployment

Method: We deployed WiCapture in a $5$ m $\times$ $6$ m room with access points placed at the four corners and the target is in the same room. This is the typical access point deployment density used in state-of-the-art WiFi localization systems [35]. We traced $97$ trajectories with the target device. Fig. 1 shows a sample trajectory. The target was moved in a continuous manner on a table rather than point by point as done in Sec. 5.2. Fig. 7(a) shows the experimental setup.

**Analysis: **From Fig. 8(a), WiCapture achieves a median trajectory error of $0.88$ cm. SpotFi’s median trajectory error, $132$ cm, is more than two orders of magnitude larger than that of WiCapture. WiCapture thus achieves sub-centimeter-level motion tracking using commodity WiFi.

5.3.2 Occlusion deployment

Method: We evaluate WiCapture under challenging conditions where the target is one room and all the APs are occluded from the target either by furniture or by walls. Fig. 7(b) shows the experimental setup which shows couple of APs where one AP is placed outside the room where the target is placed and another AP is separated from the target through a cubicle. We traced $64$ different trajectories.

**Analysis: **Fig. 8(b) plots the CDF of the trajectory error of WiCapture and SpotFi when either one or two access points are outside the room where the target is placed. However, note that all the access points are occluded whether they are inside the target’s room or not. Under these conditions, WiCapture achieves a median trajectory error of $1.51$ cm. Thus, even in the challenging conditions where the target is occluded from the positioning system, WiCapture achieves accuracy acceptable for many applications.

We further stress-tested by placing three or more APs outside the room at which point the reconstructed trajectories accumulated large errors. So, WiCapture can provide accurate tracking only when atleast two access points are present which are not separated from the target by walls.

5.3.3 Outdoor deployment

Method: We evaluate WiCapture in a $5$ m $\times$ $6$ m outdoor space. The experiments are conducted in shade for accurate Oculus measurements. Fig. 7(c) shows the experimental setup. We traced $89$ trajectories.

**Analysis: **From Fig. 8(c), WiCapture achieves a median trajectory error of $0.85$ cm compared to SpotFi’s $57$ cm trajectory error. Thus, WiCapture can ubiquitously track a commodity WiFi device as long as there is WiFi infrastructure irrespective of whether the target is indoors or outdoors.

5.4 Deep dive into WiCapture

WiCapture achieves accurate motion tracking due to two novel techniques. First, WiCapture accurately resolves multipath by using multiple packets for estimation. Second, WiCapture accurately removes the phase distortion due to frequency offset between the transmitter and the receiver. We now test the significance of each of these factors individually. For the following experiments, we consider the data from the indoor office deployment scenario in Sec. 5.3.1.

5.4.1 Improved AoD estimation

Method: We compare WiCapture against an alternate method which uses single packet for AoD estimation (set $P$ equal to $1$ in Eq. 7). Error of a particular method (single packet or multiple packets) is measured by using the absolute difference between ground truth direct path AoD (which is measured manually) and AoD estimated using the particular method that is closest to this ground truth.

Analysis: From Fig. 9(a), $80^{\mathrm{th}}$ percentile AoD estimation error by using multiple packets is $11$ degrees and is $3\times$ smaller than the error obtained by using a single packet.

5.4.2 Removing phase distortion from frequency offset

Method: We compare WiCapture against an alternate method which solves a system of equations where each equation equates the change in the phase of complex attenuation of a path directly as a linear function of displacement without considering the offset from the clock differences. We call this alternate method as assume same clock.

Analysis: From Fig. 9(b), the trajectory estimation errors are $6\times$ worse when frequency offset is ignored.

6 Discussion and conclusion

All the major WiFi chip families (Atheros, Intel, and Marvell) expose CSI [28, 51]. Hence, we believe WiCapture can be added to any commodity WiFi infrastructure.

To enable high-speed communication, WiFi chips transmit reference signals on multiple frequencies and receivers, typically, have multiple antennas. Algorithm 1 can be extended to use these additional signals to improve AoD estimation. Also, Algorithm 1 tracks the target when it moves in a 2D plane. 3D tracking can be enabled by extending Algorithm 1 to find the direction of paths in a 3D space.

The paper focuses on positioning alone since that is the hardest to provide. Orientation can be relatively easily obtained via IMUs already available on many devices [6].

In conclusion, we developed and implemented a WiFi-based motion tracking system and demonstrated its performance in different deployment scenarios. A commodity WiFi-based position tracking system would potentially enable VR on mobile devices and revolutionize number of applications that can be enabled on top of motion tracking.

Acknowledgments

Manikanta Kotaru was supported by Thomas and Sarah Kailath Stanford Graduate Fellowship. The authors thank Prof. Gordon Wetzstein and Stanford Computational Imaging Group for valuable feedback on the paper and the anonymous reviewers for their insightful comments.

Glossary

• •

**AoD (Angle of Departure): **The angle of a path with respect to a line joining two antennas at the source.

• •

**AP: **A WiFi access point or router.

• •

**CSI (Channel State Information): **The received signal when the source transmits the reference signal.

• •

**Frequency offset: **The difference in frequency of the reference signal used at two WiFi chips.

• •

**Packet: **WiFi chunks data into units of efficient size for communication and each unit is called as a packet.