A high-resolution large-scale dataset for building segmentation from aerial imagery in northeastern Italy

TL;DR

This paper introduces SegFVG, a large-scale dataset for building segmentation from high-resolution aerial images in northeastern Italy, supporting AI-based urban and disaster management applications.

Contribution

SegFVG is a new, large-scale, high-resolution dataset with precise pixel-level annotations for building segmentation in diverse geographical settings.

Findings

SegFVG includes over 15,000 aerial image tiles with 0.1-meter resolution and pixel-level building masks.

The dataset covers 616 km2 across urban, suburban, and rural areas in varied landscapes.

Benchmark results with deep learning models demonstrate the dataset's utility for segmentation research.

Abstract

Accurate building segmentation from high-resolution aerial imagery is essential for numerous applications in remote sensing, urban planning, and disaster management. While AI-based methods enable fast, scalable, and cost-effective segmentation of building footprints, their development is often limited by the scarce availability of large-scale, geographically diverse datasets with reliable pixel-level annotations. In this work, we present SegFVG, a large-scale, high-resolution, and geographically diverse dataset for building segmentation, focused on the Friuli Venezia Giulia region in northeastern Italy. The dataset includes over 15,000 true orthophoto aerial image tiles, each of size 2000 × 2000 pixels with a ground sampling distance of 0.1 meters, paired with precise pixel-level building segmentation masks. Covering approximately 616 km2, SegFVG captures a broad spectrum of urban, suburban, and rural settings across varied landscapes, including mountainous, flat, and coastal areas. Alongside the dataset, we provide benchmark results using several deep learning models. These support the usability of SegFVG for the development of accurate segmentation models and serve as a baseline to accelerate future research in building segmentation.

Figures

Example of UNet segmentation results on mountains and plains.

Spatial distribution of SegFVG image tiles (left) across the Friuli Venezia Giulia region. Each black dot represents an image tile of 2000 × 2000 pixels (200 × 200 m, examples on the right) included in the dataset, illustrating the geographic coverage.Fig. 2Overview of the dataset composition based on altimetric zone, urbanization level, and coastal proximity classification. In the top row, the percentages refer to the tiles in each class, while in the bottom row, they refer to the number of buildings in each class.

Map of the Friuli Venezia Giulia region showing the number of annotated buildings in the SegFVG dataset. There are a total of 215 municipalities. Each area refers to a municipality and is color-coded according to the total count of buildings, showing the concentration of annotations across the region.

Map of the Friuli Venezia Giulia region showing the number of annotated buildings in the SegFVG dataset. There are a total of 215 municipalities. Each area refers to a municipality and is color-coded according to the total count of buildings, showing the concentration of annotations across the region.

The framework used for the generation of the SegFVG dataset.

Directory structure of the SegFVG dataset.

Examples of aerial image tiles (top) and their corresponding building segmentation mask (bottom). The masks highlight building footprints in white.

Example of the images obtained using the data cleaning process.

Examples of building segmentation masks obtained using UNet^[18](#CR18)^. In Fig. 8d, white pixels are true positives, red ones are false negatives, blue ones are false positives.

Example of UNet segmentation results on urban and rural areas.

Full text

Background & Summary

Building segmentation refers to the task of identifying and delineating building footprints at the pixel level from aerial or satellite imagery. It plays a crucial role in a wide range of geospatial applications, including remote sensing, urban planning, environmental monitoring, infrastructure management, and disaster response^1–3^. In these domains, AI-based tools for automatic building segmentation have become increasingly essential, offering fast, scalable, and cost-effective solutions for extracting building footprints. The availability of these tools provides an opportunity to automate traditionally manual and resource-intensive mapping workflows^4^.

In recent years, deep learning has emerged as a highly effective approach for building segmentation, with models based on convolutional neural networks (CNNs) achieving remarkable performance. These models can learn hierarchical and contextual features directly from the input RGB imagery, allowing them to accurately delineate buildings across diverse environments and architectural styles^5^. The effectiveness of these models is strongly dependent on the availability of large-scale and high-quality annotated training datasets^6^. Such datasets must have high spatial resolution to accurately capture fine architectural details, and geographic diversity to ensure robustness across different urbanization levels and landscapes. Additionally, they must provide a sufficient volume of images to enable effective training of modern models and to ensure strong generalization performance.

Various datasets for building segmentation from aerial and satellite imagery have been proposed to support the training and evaluation of deep learning-based segmentation models. Table 1 shows a comparison among existing datasets.Table 1. Comparison of existing datasets for building segmentation.Dataset nameImagesTile size (pixels)Resolution (m)Covered area (km^2^)Building footprintsUrbanization levelTypeGeographical regionINRIA^8^3605,0000.3810210,365UrbanAerialChicago, Kitsap County, Bellingham, San Francisco, Bloomington (the US) and Vienna, West Tyrol, Innsbruck, East Tyrol (Austria)Massachusetts^13^15115001340N/AUrban, suburban, and ruralAerialMassachusetts (the US)ISPRS Potsdam^7^386,0000.054N/AUrbanAerialPotsdam (Germany)WHU^29^8,1885120.3450~ 220,000N/AAerialChristchurch (New Zealand)LandCover.ai^9^41~ 4,000-9,0000.25-0.5216N/ARuralAerialPolandSpacenet 6^10^20,4089000.5120~ 48,000Urban, suburban, and ruralSatelliteRotterdam (The Netherlands)CBIS^12^7,2605000.312463,886Urban and suburbanSatelliteBeijing, Shanghai, Shenzhen, Wuhan (China)UBC^14^8006000.5-0.866~ 61,000UrbanSatelliteBeijing (China) and Munich (Germany)GF-7^11^5,1755120.65573170,015Urban, suburban, and ruralSatelliteTianjin, Lanzhou, Chongqing, Ningbo, Guangzhou, Shenzhen (China)SegFVG^15^15,4032,0000.1616356,923Urban, suburban, and ruralAerialFriuli Venezia Giulia region (Italy)

Although many datasets exist, none of them is simultaneously large-scale with a high spatial resolution and broad geographic diversity. For example, the ISPRS Potsdam dataset^7^ offers very high spatial resolution but is geographically limited, covering only a single urban area of Potsdam, in Germany. The INRIA dataset^8^ provides imagery from multiple locations around the world, but it is limited to urban areas and does not include rural or suburban regions, while the Landcover.ai dataset^9^ only focuses on rural areas across Poland. This geographic bias may limit the model generalization across diverse urbanization levels and building architectural styles. Datasets like Spacenet 6^10^, GF-7^11^, and CBIS^12^ provide broader geographic diversity, including many urban, suburban, and rural areas, but lack the spatial detail necessary for fine-grained segmentation. Similarly, the Massachusetts dataset^13^ offers a good geographic coverage and diversity, but its spatial resolution is relatively low (only 1 meter per pixel). This limited spatial detail may reduce the accuracy in capturing fine architectural features, such as small building footprints or complex roof structures. Other datasets, such as UBC^14^, are relatively small in scale (only 66 km^2^), limiting their usability for the training of modern deep learning models. Recently, the Microsoft Building Footprints dataset (https://planetarycomputer.microsoft.com/dataset/ms-buildings) has been proposed: it includes over 999 million buildings worldwide, obtained from Bing Maps imagery collected between 2014 and 2021. However, it may suffer from temporal misalignment between imagery and annotations, leading to inconsistent footprints. In addition, it is generated entirely through automated deep learning methods without validation from authoritative cartographic sources, which may affect its accuracy and reliability in certain regions. For all these reasons, there is the need for a large-scale, high-resolution, and geographically diverse dataset that provides accurate and reliable pixel-level building annotations.

Given the limitations of existing datasets for building segmentation, in this work we introduce Segmentation Friuli Venezia Giulia (SegFVG)^15^, a large-scale, high-resolution dataset containing 15,403 aerial image tiles, each of size 2000 × 2000 pixels, with a ground sampling distance of 0.1 meters. SegFVG^15^ includes precise pixel-level annotations of building footprints across 616 km^2^ of the Friuli Venezia Giulia region in northeastern Italy. The area of Friuli Venezia Giulia is particularly interesting for building segmentation as it encompasses a diverse range of environments, including alpine rural zones in the north, flat agricultural plains in the center, and densely populated coastal settlements along the Adriatic Sea. The geographical distribution of the tiles contained in SegFVG^15^, as well as tile examples, is shown in Fig. 1. Each tile is represented by a black dot on the map. According to the classifications provided by the National Institute of Statistics (Istat) (https://www.istat.it/classificazione/principali-statistiche-geografiche-sui-comuni/), as shown in Fig. 2 (top row), SegFVG^15^ captures a diverse distribution of altimetric zones: 9.5% of the tiles are located in mountainous areas, 29.1% in hilly areas, and the remaining 61.5% in flat plains. In terms of urbanization levels, 11.5% of the tiles correspond to urban contexts, 45.8% to suburban areas, and 42.7% to rural settings. Finally, with respect to coastal proximity, 17.2% of the tiles are situated near the Adriatic Sea (coastal), while the remaining 82.8% are inland.Fig. 1. Spatial distribution of SegFVG image tiles (left) across the Friuli Venezia Giulia region. Each black dot represents an image tile of 2000 × 2000 pixels (200 × 200 m, examples on the right) included in the dataset, illustrating the geographic coverage.Fig. 2. Overview of the dataset composition based on altimetric zone, urbanization level, and coastal proximity classification. In the top row, the percentages refer to the tiles in each class, while in the bottom row, they refer to the number of buildings in each class.

In total, SegFVG^15^ includes approximately 357,000 annotated building structures. Figure 2 (bottom row) shows the building distribution according to their altimetric zone, urbanization level, and coastal proximity, while Fig. 3 illustrates the distribution of these buildings grouped by municipality, highlighting the spatial variability in building density across the region, from dense urban centers to sparsely populated rural areas. The combination of varied landscapes, urbanization levels, and architectural styles creates a representative and challenging benchmark for analyzing buildings across different geographic and urban contexts.Fig. 3. Map of the Friuli Venezia Giulia region showing the number of annotated buildings in the SegFVG dataset. There are a total of 215 municipalities. Each area refers to a municipality and is color-coded according to the total count of buildings, showing the concentration of annotations across the region.

Overall, SegFVG^15^ is characterized by its large scale, its high spatial resolution, and its geographic diversity (i.e., it includes heterogeneous environments, such as coastal areas, plains, hills, and Alpine regions, which are associated with distinct settlement patterns and building typologies). These features make it particularly well-suited for the development of deep learning models for building segmentation.

In addition to the dataset, we provide benchmark results using multiple deep learning models, which demonstrate the usability of SegFVG^15^ for the development of accurate segmentation models and can be used as a baseline for future research in this field. To the best of our knowledge, SegFVG^15^ is the first publicly available building segmentation dataset focused on the Italian territory.

Methods

An overview of the framework we adopted to create SegFVG^15^ is shown in Fig. 4. It involves three main steps: (1) data collection, (2) data processing, and (3) data cleaning. In the following, we describe these steps more in detail.Fig. 4. The framework used for the generation of the SegFVG dataset.

Data collection

We constructed the SegFVG^15^ dataset by leveraging official geospatial services provided by the Friuli Venezia Giulia (FVG) region. The data collection process begins with retrieving building shapes from the regional Web Feature Service (WFS) (https://serviziogc.regione.fvg.it/geoserver/EDIFICI/wfs). These layers provide building volumetric units for the entire region derived from the Regional Technical Numerical Map (RTNM) (https://eaglefvg.regione.fvg.it/). A volumetric unit is defined as the smallest portion of a building that has a uniform elevation from the ground (https://geoportale.comune.milano.it/sit/tematiche/territorio/).

We first reprojected all vector layers to the EPSG:6708 coordinate reference system. We divided the FVG region into square tiles measuring 200 meters per side to organize the spatial data. We then discarded tiles containing fewer than 40 shapes (volumetric units) to prioritize areas with meaningful building density and to reduce uninformative background regions.

For each retained tile, we downloaded true orthophotos with a ground sampling distance of 0.1 meters per pixel from the regional Web Map Service (WMS) (http://irdat-ortofoto.regione.fvg.it/geoserver/ortofoto/ows). These images were captured during aerial surveys conducted between 2017 and 2020 using aircraft equipped with Vexcel UltraCam Eagle and UltraCam Xp digital large-format cameras. We adopted a sliding window approach to manage the retrieval of images spanning large shapes. All resulting images are saved as GeoTIFFs to preserve accurate georeferencing metadata.

We clipped shapes to match tile boundaries and filtered them by a minimum area threshold of 20 m^2^ to remove small and potentially noisy shapes. We converted each remaining shape into a polygon defined in image pixel coordinates using an affine transformation and applied a y-axis inversion to align with the image coordinate system. Finally, the polygon boundaries are serialized into JSON format.

For each tile, we generated a pair of files: a raster RGB image and a corresponding JSON annotation file containing the filtered volumetric units of each building represented in the image.

Data pre-processing

Following the data collection phase, the raw dataset consisted of raster RGB images paired with volumetric unit information provided as JSON files. As a first step, we discarded a subset of images that exhibited large black regions in the background. These areas, represented by zero-valued pixels, are likely the result of incomplete image tiles, which occur when the tile contains areas that fall outside the boundaries of the Friuli Venezia Giulia region. We then converted the vector polygons in the JSON files into rasterized binary masks, assigning a value of 1 to pixels within the polygons and 0 to all other pixels. During this process, all connected volumetric units are aggregated to produce a single unified footprint for each building. This step is essential because, in the RGB true orthoimages, these structures appear as continuous buildings without internal divisions, and treating each volumetric unit separately creates artificial splits that do not reflect the visual reality. The dataset obtained after these pre-processing operations consists of image/mask pairs. We refer to this preliminary dataset version as SegFVG_v0.

Data cleaning

After manually inspecting a random subset of images from SegFVG_v0, we identified recurring annotation errors, such as missing buildings (false negatives) and incorrectly labeled structures (false positives). These issues are likely due to temporal discrepancies between the imagery and the RTNM data, as they were captured at different times, leading to mismatches caused by construction, demolition, or changes in land use. An example is presented in Fig. 7a, which shows both missing building footprints and erroneous annotations.

These inaccuracies can negatively impact both the training of deep learning models and the reliability of performance evaluations^16^. We thus implemented a data cleaning pipeline to improve the overall annotation quality, and we established a manually corrected reference set to quantitatively assess the effectiveness of the data cleaning procedure.

We first randomly selected 50 images from the SegFVG_v0 dataset and manually refined their corresponding masks using the CVAT tool (https://www.cvat.ai/). The goal was to ensure that all building footprints were accurately annotated, removing both false positives and false negatives. This manually corrected subset, referred to as the gold standard, is used as a reference for evaluating the effectiveness of the data cleaning procedure.

We then implemented a semi-automatic data cleaning procedure to address the annotation inconsistencies. In short, we trained different deep learning models for building segmentation with different backbones using all the images in the SegFVG_v0 dataset, excluding the 50 images in the gold standard, and finally used the majority voting consensus of their predictions to generate the corrected segmentation masks. The use of multiple backbones allows leveraging the complementary strengths of different feature extractors, which can capture different semantic patterns from the training data, even if they contain some inconsistencies^17^. Specifically, we used four models based on the U-Net architecture^18^ with Resnet-50^19^, Efficientnet-B4^20^, Densenet-201^21^, and Xception^22^ as backbone encoders. Each encoder is initialized with pre-trained weights from ImageNet^23^. During training, the batch size is set to 32 and the learning rate to 1e-4. We use the Dice loss \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${{\mathcal{L}}}_{Dice}$$\end{document} , which is defined as:

\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${{\mathcal{L}}}_{Dice}=1-2\cdot \frac{\sum {y}_{pred}\cdot {y}_{true}}{\sum {y}_{pred}+\sum {y}_{true}+\epsilon },$$\end{document}

where ypre**d ∈ [0, 1] is the predicted probability after sigmoid activation, ytru**e ∈ {0, 1} is the reference label, and ϵ is a small value to ensure numerical stability. We trained the models for 12,000 iterations. For data augmentation, we randomly cropped images at 256 × 256 pixel resolution and used various strategies, including flip, rotation, and corrections applied to brightness, contrast, hue, and saturation.

After all the models were trained, we used each of them to generate segmentation masks for the entire dataset. In addition to these four predictions, we considered the masks of SegFVG_v0, resulting in five candidate masks per image. We then applied majority voting consensus across these five candidates to obtain a refined segmentation mask, where each pixel was labeled as building (i.e., 1) if at least three of the five masks agreed. Formally, let M = {m1, m2, m3, m4, m5} be the set of binary masks, where each mi ∈ {0, 1}^H×W^ is a binary mask of height H and width W, mi(x, y) denotes the pixel value at location (x,y) in the i-th mask, with 1 indicating a building pixel and 0 indicating the background. Then, \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\widehat{m}\in {\{0,1\}}^{H\times W}$$\end{document} is the refined mask and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\widehat{m}(x,y)$$\end{document} is defined as:

We finally post-processed the obtained mask by applying morphological opening (erosion followed by dilation) using a 13 × 13 kernel, which helped remove small spurious regions.

The dataset obtained after these data cleaning operations is called SegFVG_v1, which corresponds to the final version.

Data Record

The SegFVG^15^ dataset is available for download at 10.17632/9kbc6zdn7b. It is distributed as a compressed ZIP archive of approximately 24 GB. The dataset presented and peer reviewed in this article corresponds to Version 2. Figure 5 illustrates the hierarchical directory structure.Fig. 5. Directory structure of the SegFVG dataset.

The root folder, named SegFVG, contains three subdirectories: images, masks_v0, and masks_v1. They store the input RGB aerial images and their corresponding segmentation masks, where masks_v0 contains the preliminary annotations (SegFVG_v0), and masks_v1 contains the cleaned annotations produced by the data cleaning pipeline (SegFVG_v1). Each RGB image is associated with a segmentation mask that provides precise pixel-level annotations of building footprints. Some examples are shown in Fig. 6.Fig. 6. Examples of aerial image tiles (top) and their corresponding building segmentation mask (bottom). The masks highlight building footprints in white.

All files are stored in GeoTIFF format and named consistently so that each image corresponds directly to its reference mask. Each image filename encodes the spatial extent of the corresponding tile using projected coordinate values. Specifically, the filename follows the structure \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${{\rm{x}}}_{\min }\_{{\rm{y}}}_{\min }\_{{\rm{x}}}_{\max }\_{{\rm{y}}}_{\max }.{\rm{tif}}$$\end{document} , where each value represents the coordinates of the tile boundary box (upper left and bottom right corners) in the projected coordinate reference system EPSG:6708.

The dataset is split into training, validation, and test partitions, which are provided as TXT files in the root directory: train.txt, val.txt, and test.txt. Each TXT file lists the filenames corresponding to the images in that split. Specifically, the training, validation, and test sets contain 10,748, 1,535, and 3,070 images, respectively. In addition, the root directory contains a gold_standard.txt file, which includes the filenames of the 50 manually corrected images and segmentation masks used to evaluate the data cleaning procedure. The corresponding masks before and after manual correction are placed in masks_v0 and masks_v1, respectively. The description of the dataset splits is reported in Table 2.Table 2. Overview of the SegFVG splits, including the number of image tiles, total buildings annotated, and total geographic area covered (in km^2^) for each split.Split nameNum of TilesNum. of buildingsCovered area (km^2^)Train10,748249,666430Validation1,53535,30161Test3,07070,531123Gold standard501,4252Total15,403356,923616

Finally, following the classifications described in Section “Background & Summary”, we provide a CSV file named classification.csv, which contains information for each image tile, including the municipality it belongs, its associated altimetric zone (mountain, hill, or plain), urbanization level (urban, suburban, or rural), and coastal proximity (coastal or inland). Table 3 shows some examples of records in the classification.csv file.Table 3. Examples of the classification.csv file content.TileMunicipalityAltimetric zoneUrbanization levelCoastal proximity403223_5050937_403423_5051137Muggiahillsuburbancoastal302255_5129201_302455_5129401Cimolaismountainruralinland302783_5091192_302983_5091392Sacileplainsuburbaninland397623_5064937_397823_5065137TriestehillurbancoastalEach tile is associated with information about the municipality, altimetric zone, urbanization level, and coastal proximity.

Technical Validation

We validated the quality of the segmentation masks obtained using the data cleaning pipeline described in Section “Data cleaning”. In addition, we conducted a series of experiments using standard deep learning models to assess the quality and usability of SegFVG^15^ for building segmentation from aerial imagery. The results are evaluated using standard metrics for semantic segmentation, i.e., pixel-level Precision, Recall, F1-score, and Intersection over Union (IoU).

Quality assessment of data cleaning pipeline

With this evaluation, we aim to validate the data cleaning pipeline we used to correct the errors we identified in SegFVG_v0, as described in Section “Data cleaning”. Indeed, the annotations in SegFVG_v0 may contain errors that typically appear as false negatives, where buildings are missed, or false positives, where background regions are incorrectly labeled as buildings.

We validated the quality of the data cleaning pipeline using the gold standard. The performance of the models involved in this process is reported in Table 4. As shown, the direct comparison of the segmentation masks in SegFVG_v0 with the gold standard reports an F1-score and IoU of 0.856 and 0.749, respectively. This highlights the presence of annotation errors and motivates the need to clean the data. We can consider these results as the baseline. The performance obtained by the different deep learning models shows an improvement over the baseline, confirming that model-based refinement is an effective solution to identify and correct annotation errors. Finally, when the model predictions are combined and integrated with the annotations of SegFVG_v0, the resulting masks achieve a higher agreement with the gold standard. Specifically, the performance improves to 0.947 and 0.900 in F1-score and IoU, respectively, demonstrating a considerable gain in segmentation quality. Since the gold standard contains manually corrected and error-free annotations, higher performance on this set suggests that the cleaned annotations are more accurate and of better quality. This is also visually confirmed by Fig. 7, which shows an example of segmentation masks from SegFVG_v0 cleaned using the adopted pipeline. Here we polygonized the segmentation masks and displayed only the polygon boundaries for better clarity. We can see that the cleaned masks are more similar to the gold standard.Table 4. Performance of different data cleaning strategies evaluated on the gold standard.Data cleaning strategyPrecisionRecallF1-scoreIoU—0.8950.8210.8560.749Resnet-500.9430.9130.9280.865Efficientnet-B40.8420.9120.9270.863Densenet-2010.9080.9450.9260.863Xception0.9290.9410.9350.879Majority voting0.9490.9460.9470.900The first row corresponds to the SegFVG_v0 annotations (i.e., no data cleaning).Fig. 7. Example of the images obtained using the data cleaning process.

These results validate both numerically and visually the reliability of the proposed data cleaning pipeline, which significantly improved the quality of the annotations compared to those of SegFVG_v0.

Performance of deep learning models

The goal of this evaluation is to assess whether the SegFVG^15^ can support the training of deep learning models for building segmentation from aerial imagery.

We selected different architectures, including U-Net^18^, which is a well-established encoder-decoder model; Pyramid Scene Parsing Network (PSPNet)^24^, which aggregates context using pyramid pooling at different spatial resolutions; Feature Pyramid Network (FPN)^25^, which enhances semantic segmentation by combining high-level semantic features with low-level spatial details across scales; DeepLabV3^26^, which leverages atrous spatial pyramid pooling to capture multi-scale contextual information; Pyramid Attention Network (PAN)^27^, which integrates multi-scale contextual information through spatial attention mechanisms; and Segmentation Transformer (SegFormer)^28^, which is a transformer-based model known for its efficiency and accuracy. We followed the same training setup adopted for the models used for the data cleaning pipeline, as described in Section “Data cleaning”. The only difference is the number of training iterations, which is here set to 8,400. The hyperparameters we used for these experiments are summarized in Table 5.Table 5. Summary of the hyperparameters used during the training of deep learning models.HyperparameterValueBatch size32Learning rate1e-4OptimizerAdamInitial weightsImageNetIterations8400Patch size256 × 256Loss functionDiceGeometric augmentationFlip, rotationColor augmentationBrightness, contrast, hue, saturation

We report the experimental results obtained on the test set, using SegFVG_v1 masks as a reference, in Table 6. These results can be used as a baseline for future research. In addition to evaluating model performance, the table reports the total number of model parameters, the training time required to complete 8400 iterations using the previously discussed setup, and the inference time required to generate a segmentation map from an input image of size 2000 × 2000. Time is measured using a NVIDIA GeForce GTX 1080 GPU.Table 6. Comparative evaluation of different deep learning models for building segmentation on the test set.ModelBackbonePrecisionRecallF1-scoreIoUParam.Train. timeInfer. timeUNetResnet-1010.9430.9750.9590.92151M1h 27m417 msPSPNetResnet-1010.8850.7590.8170.69143M7h 50m142 msFPNResnet-1010.9340.9620.9480.90145M1h 13m348 msDeepLabV3Resnet-1010.9340.9560.9450.89646M2h 48m427 msPANResnet-1010.8600.9650.9100.83443M1h 34m384 msSegFormerMiT-B30.9550.9400.9470.90045M2h 22m943 msThe training time is the time required to complete model training. The inference time is the time required to process an image of 2000 × 2000 pixel resolution.

Looking at precision, SegFormer^28^ achieves the highest value (0.955), showing its effectiveness in minimizing false positives. In terms of recall, UNet^18^ achieves the highest value (0.975), indicating a stronger ability to detect most pixels corresponding to buildings. Among all models, UNet^18^ also achieves the highest F1-score of 0.959, indicating a strong balance between precision and recall. In contrast, PSPNet exhibits the lowest performance with an F1-score of 0.817. Regarding IoU, UNet^18^ again performs best with a score of 0.921, confirming its robustness across different metrics. Overall, UNet is the best-performing model. In contrast, PSPNet^24^ is the worst-performing model, achieving F1-score and IoU values that are 0.142 and 0.230 lower, respectively, compared to UNet^18^. Fig. 8 shows some examples of building segmentation masks obtained using the UNet^18^ model. Specifically, Fig. 8d highlights the differences between the predicted masks and the reference ones, illustrating common errors made by building segmentation models. These include: (1) missed buildings (false negatives), where small or shadowed structures are not detected; (2) spurious detections (false positives), where non-building elements are incorrectly classified as buildings; and (3) boundary inaccuracies, where mask borders are overly smooth or slightly misaligned.Fig. 8. Examples of building segmentation masks obtained using UNet^18^. In Fig. 8d, white pixels are true positives, red ones are false negatives, blue ones are false positives.

Using the UNet^18^ model, we conducted a more detailed performance analysis aiming to understand how segmentation performance varies across different geographical and demographic conditions present in the dataset. For reference, we also documented the performance of the other models.

As a first step, we evaluated model performance in regions with varying urbanization levels. Each image tile was categorized as urban, suburban, or rural according to the number of inhabitants per square kilometer. These categories often reflect different patterns in the distribution and structure of the building. Urban areas are typically characterized by compact layouts, where buildings are closely spaced and exhibit a wide variety of shapes and sizes.

In contrast, rural areas contain only a few isolated structures, often dispersed across large expanses of natural or agricultural land. The obtained results are summarized in Table 7.Table 7. Performance of the models for building segmentation across areas categorized by population density, i.e., densely, moderately, and sparsely populated regions within the test set.Urbanization levelNum. of tilesModelPrecisionRecallF1-scoreIoUUrban363UNet0.9310.9670.9490.902PSPNet0.8880.7130.7900.654FPN0.9280.9420.9350.878DeepLabV30.9380.9290.9330.875PAN0.8160.9660.8850.793SegFormer0.9550.9080.9310.871Suburban1,396UNet0.9440.9780.9600.924PSPNet0.8860.7800.8290.708FPN0.9340.9660.9500.905DeepLabV30.9350.9610.9480.901PAN0.8630.9670.9120.839SegFormer0.9560.9430.9500.904Rural1,311UNet0.9470.9760.9610.925PSPNet0.8830.7530.8130.685FPN0.9350.9660.9500.905DeepLabV30.9310.9620.9460.898PAN0.8770.9630.9180.849SegFormer0.9530.9510.9520.908

The model demonstrates consistent performance across all urbanization levels, with F1-scores above 0.940 and IoU values above 0.900. Slightly lower performance in urban areas suggests challenges related to building overlap and structural complexity. In contrast, suburban and rural areas yield slightly better results, likely due to simpler spatial arrangements and clearer separation between buildings. Figure 9 shows some examples of segmentation results obtained in urban and rural areas.Fig. 9. Example of UNet segmentation results on urban and rural areas.

We further analyzed the performance of the model in different types of landscapes by grouping image tiles into three categories according to their altimetric zone: mountains, hills, and plains. This geographic diversity often presents buildings with different characteristics: mountainous regions often contain scattered buildings reflecting the lower population density, hilly areas tend to feature moderately dense and varied structures, while plains typically exhibit more regular, densely arranged buildings. The results presented in Table 8 show that the model performs well in all types of landscapes.Table 8. Performance of the models for building segmentation in different altimetric zones, i.e., mountains, hills, and plains within the test set.Altimetric zoneNum. of tilesModelPrecisionRecallF1-scoreIoUMountains306UNet0.9280.9710.9490.903PSPNet0.8630.6020.7090.549FPN0.9240.9470.9350.878DeepLabV30.9040.9580.9300.869PAN0.8580.9570.9040.825SegFormer0.9390.9500.9440.895Hills885UNet0.9430.9730.9580.919PSPNet0.8850.7570.8160.689FPN0.9390.9550.9470.899DeepLabV30.9360.9480.9420.890PAN0.8460.9660.9020.821SegFormer0.9560.9360.9460.897Plains1,879UNet0.9450.9770.9600.924PSPNet0.8870.7780.8290.708FPN0.9320.9670.9490.904DeepLabV30.9370.9600.9480.901PAN0.8670.9660.9140.842SegFormer0.9560.9410.9480.902

The lowest performance is observed in mountainous areas, where the model achieves an F1-score of 0.949 and an IoU of 0.903. This is likely due to visual and topographic complexity in such regions, including steep slopes, shadows, and more irregular building patterns. In contrast, performance improves slightly in hilly regions, with an F1-score of 0.958 and an IoU of 0.919, and reaches its highest levels in plains, where the model achieves an F1-score of 0.960 and an IoU of 0.924. The more uniform terrain and regular building layouts in plain regions likely contribute to this improved performance. Figure 10 shows examples of segmentation results obtained in mountainous and plain areas.Fig. 10. Example of UNet segmentation results on mountains and plains.

Finally, we investigated the model performance by separating coastal from inland areas. The results in Table 9 show that the model performs well in both contexts, but with slightly better results in inland areas. Specifically, the model achieves an F1-score of 0.961 and an IoU of 0.925 for inland areas, compared to an F1-score of 0.949 and an IoU of 0.903 in coastal areas. The lower performance in coastal zones may be attributed to the greater variability in building styles and densities, as well as visual interference from the sea, beaches, and other shoreline features.Table 9. Performance of the models for building segmentation in coastal and inland areas within the test set.Coastal proximityNum. of tilesModelPrecisionRecallF1-scoreIoUCoastal501UNet0.9330.9660.9490.903PSPNet0.8920.6850.7750.633FPN0.9360.9410.9380.884DeepLabV30.9320.9330.9320.873PAN0.8170.9630.8840.792SegFormer0.9530.9170.9340.877Inland2,569UNet0.9450.9770.9610.925PSPNet0.8830.7760.8260.704FPN0.9330.9670.9500.904DeepLabV30.9340.9610.9480.901PAN0.8710.9660.9160.845SegFormer0.9550.9460.9500.905

These results demonstrate the capability of SegFVG^15^ to support high-performance building segmentation from aerial imagery across diverse geographic and demographic contexts. The consistent performance observed in varying population densities and geographical contexts validates the representativeness of the dataset and its utility for downstream applications and model development.