# End-to-end Lane Detection through Differentiable Least-Squares Fitting

## TL;DR

This paper introduces an end-to-end lane detection method that directly regresses lane parameters by integrating a differentiable least-squares fitting module into a deep learning architecture, optimizing the entire process for accurate lane modeling.

## Contribution

It presents a novel end-to-end trainable lane detection framework that backpropagates through a least-squares fitting process, improving accuracy and interpretability over traditional two-step methods.

## Key findings

- Achieves more accurate lane parameter estimation.
- Enables visualization of intermediate segmentation-like maps.
- Improves robustness by learning features that prevent fitting instabilities.

## Abstract

Lane detection is typically tackled with a two-step pipeline in which a segmentation mask of the lane markings is predicted first, and a lane line model (like a parabola or spline) is fitted to the post-processed mask next. The problem with such a two-step approach is that the parameters of the network are not optimized for the true task of interest (estimating the lane curvature parameters) but for a proxy task (segmenting the lane markings), resulting in sub-optimal performance. In this work, we propose a method to train a lane detector in an end-to-end manner, directly regressing the lane parameters. The architecture consists of two components: a deep network that predicts a segmentation-like weight map for each lane line, and a differentiable least-squares fitting module that returns for each map the parameters of the best-fitting curve in the weighted least-squares sense. These parameters can subsequently be supervised with a loss function of choice. Our method relies on the observation that it is possible to backpropagate through a least-squares fitting procedure. This leads to an end-to-end method where the features are optimized for the true task of interest: the network implicitly learns to generate features that prevent instabilities during the model fitting step, as opposed to two-step pipelines that need to handle outliers with heuristics. Additionally, the system is not just a black box but offers a degree of interpretability because the intermediately generated segmentation-like weight maps can be inspected and visualized. Code and a video is available at github.com/wvangansbeke/LaneDetection_End2End.