Applying Medical Imaging Tractography Techniques to Painterly Rendering of Images

TL;DR

This paper adapts diffusion tensor imaging and tractography techniques from medical imaging to generate painterly renderings of images, using fiber tracking analogies to guide brush stroke placement for artistic effects.

Contribution

It introduces a novel application of DTI-based tractography algorithms to painterly image rendering, leveraging structural tensors for improved local orientation guidance.

Findings

Effective mimicry of human painting styles using tractography-inspired algorithms

Structural tensor-based orientation provides better stroke placement than gradient alone

Code implementation available for reproducibility and further exploration

Abstract

Doctors and researchers routinely use diffusion tensor imaging (DTI) and tractography to visualize the fibrous structure of tissues in the human body. This paper explores the connection of these techniques to the painterly rendering of images. Using a tractography algorithm the presented method can place brush strokes that mimic the painting process of human artists, analogously to how fibres are tracked in DTI. The analogue to the diffusion tensor for image orientation is the structural tensor, which can provide better local orientation information than the gradient alone. I demonstrate this technique in portraits and general images, and discuss the parallels between fibre tracking and brush stroke placement, and frame it in the language of tractography. This work presents an exploratory investigation into the cross-domain application of diffusion tensor imaging techniques to painterly rendering of images. All the code is available at https://github.com/tito21/st-python