List of multi-view fusion learning models proposed for remote sensing (RS) multi-view data. 📡 🌎 📡
This is a complementary source used in the following paper:
Common Practices and Taxonomy in Deep Multi-view Fusion for Remote Sensing Applications
| Name | Reference | Description | Code |
|---|---|---|---|
| MV CNN | Xu et al. 2018 | Feature-level fusion with 2D CNN. | https://github.com/Hsuxu/Two-branch-CNN-Multisource-RS-classification << Not available |
| V-FuseNet | Audebert et al. 2018 | Dense fusion with 2D CNN and central model. | https://github.com/nshaud/DeepNetsForEO |
| Multi3Net | Rudner et al. 2019 | Feature-level fusion with 2D CNN. | https://github.com/FrontierDevelopmentLab/multi3net |
| UNet-CLSTM | Rustowicz et al. 2019 | Decision-level fusion with 2D CNN and convolutional-LSTM | https://github.com/roserustowicz/crop-type-mapping |
| HRWN | Zhao et al. 2020 | Input-level fusion with 2D CNN and pixel graph constraints. | https://github.com/xudongzhao461/HRWN |
| FusAtNet | Mohla et al. 2020 | Feature-level fusion with 2D CNN and cross attention. | https://github.com/ShivamP1993/FusAtNet |
| LFMC from SAR | Rao et al | Input-level fusion with LSTM | https://github.com/kkraoj/lfmc_from_sar |
| CCR-Net | Wu et al. 2021 | Feature-level fusion with 2D CNN and cross view-reconstruction. | https://github.com/danfenghong/IEEE_TGRS_CCR-Net |
| MV PSE-TAE | Ofori-Ampofo et al. 2021 | Multiple fusion strategies with PSE-TAE. | https://github.com/ellaampy/CropTypeMapping |
| MDL-RS | Hong et al. 2021 | Multiple fusion strategies with NN. | https://github.com/danfenghong/IEEE_TGRS_MDL-RS |
| CMGFNet | Hosseinpour et al. 2022 | Dense fusion with 2D CNN and gated attention. | https://github.com/hamidreza2015/CMGFNet-Building_Extraction |
| S2FL | Hong et al. 2021 | Feature-level fusion with feature contrains. | https://github.com/danfenghong/ISPRS_S2FL |
| CFCNN | He et al. 2021 | Feature-level fusion with 2D and 1D CNN. | https://github.com/SysuHe/MultiSourceData_CFCNN |
| MV NN | Danilevicz et al. 2021 | Feature-level fusion with tabular NN and 2D CNN. | https://github.com/mdanilevicz/maize_early_yield_prediction |
| SEnSeI | Francis et al. 2022 | Sensor invariant model based on 2D CNN | https://github.com/aliFrancis/SEnSeI |
| ASF2N | Gao et al. 2022 | Feature-level fusion with 2D CNN and attention. | https://github.com/zhonghaocheng/ELSEVIER_IJAEOG_AS2F2N << Empty code |
| IP-CNN | Zhang et al. 2022 | Feature-level fusion with 2D CNN and view-reconstruction. | https://github.com/HelloPiPi/IP-CNN-code |
| MV CNN | Lu et al. 2022 | Feature-level fusion with 2D CNN and adaptive attention. | https://github.com/GeoX-Lab/UnifiedDL-UFZ-extraction |
| SE$^2$Net | Fang et al. 2022 | Feature-level fusion with 2D CNN. | https://github.com/likyoo/Multimodal-Remote-Sensing-Toolkit |
| EndNet | Hong et al. 2022 | Feature-level fusion with 2D CNN and view-reconstruction. | https://github.com/danfenghong/IEEE_GRSL_EndNet |
| MAHiDFNet | Wang et al. 2022 | Dense feature fusion with 2D CNN. | https://github.com/SYFYN0317/-MAHiDFNet |
| AM$^3$Net | Wang et al. 2022 | Feature-level fusion with 2D CNN and cross attention. | https://github.com/Cimy-wang/AM3Net_Multimodal_Data_Fusion |
| AMM-FuseNet | Ma et al. 2022 | Feature-level fusion with 2D CNN and attention. | https://github.com/oktaykarakus/ReSIF/tree/main/AMM-FuseNet |
| MCANet | Li et al. 2022 | Dense fusion with 2D CNN and cross attention. | https://github.com/yisun98/SOLC |
| ChangeFormer | Bandara et al. 2022 | Dense fusion with transformer and attention. | https://github.com/wgcban/ChangeFormer |
| CMAFF | Qingyun et al. 2022 | Dense fusion with 2D CNN and cross attention. | https://github.com/DocF/CMAFF |
| OmbriaNet | Drakonakis et al. 2022 | Feature fusion with 2D CNN and skip-connections | https://github.com/geodrak/OMBRIA |
| DCSA-Net | Wang et al. 2022 | Hybrid fusion with 2D CNN and attention. | https://github.com/Julia90/DCSA-Net |
| Siamese U-Net | Cummings et al. 2022 | Dense fusion with 2D CNN and skip-connections | https://github.com/solcummings/earthvision2021-weakly-supervised |
| SatViT | Fuller et al. | Input-level fusion with ViT (with self-supervised training) | https://github.com/antofuller/SatViT |
| ELECTS | Russwurm et al. 2023 | Input-level fusion with LSTM. | https://github.com/marccoru/elects |
| MV CNN | Ferrari et al. 2023 | Multiple fusion strategies with 2D CNN (encoder-decoder) | https://github.com/felferrari/deforestation-from-data-fusion |
| AFCF3D-Net | Ye et al. 2023 | Input-level fusion with 3D CNN. | https://github.com/wm-Githuber/AFCF3D-Net |
| UnCRtainTS | Ebel et al 2023 | Input fusion with 2D CNN and attention. | https://github.com/PatrickTUM/UnCRtainTS |
| MFT | Roy et al. 2023 | Feature-level fusion with transformer modules (one source - LIDAR- is used as a query over the main source - optical) | https://github.com/AnkurDeria/MFT |
| OOD Fusion | Gawlikowski et al. 2023 | Input-level, Feature-level, and Decision-level fusion with CNN and weighted average aggregation | https://github.com/JakobCode/OOD_DataFusion |
| PRESTO | Tseng et al. 2023 | Input-level fusion with transformer modules (self-supervised pretraining) | https://github.com/nasaharvest/presto |
| Cross-HL | Roy et al. 2023 | Feature-level fusion with directed attention in transformer layers | https://github.com/AtriSukul1508/Cross-HL |
| SCT Fusion | Hoffman et al. 2023 | Dense fusion with tranformer layers and class tokens in all | https://git.tu-berlin.de/rsim/sct-fusion |
| MMST-ViT | Lin et al. 2023 | Feature-level fusion with transformer layers | https://github.com/fudong03/MMST-ViT |
| DiffusionSat | Khanna et al. 2023 | Multi-modal diffusion generative model | https://github.com/samar-khanna/DiffusionSat |
| SSL4EO-S12 | Wang et al. 2024 | Self-supervised model | https://github.com/zhu-xlab/SSL4EO-S12 |
| EarthGPT | Zhang et al. 2024 | Feature-level fusion with transformer layers | https://github.com/wivizhang/EarthGPT |
| ContextFormer | Benson et al. 2024 | Feature-level fusion with transformer | https://github.com/vitusbenson/greenearthnet |
| SEnSeIv2 | Francis 2024 | Sensor-invariant model | https://github.com/aliFrancis/SEnSeIv2 |
| OmniSat | Astruc et al. 2024 | Feature-level fusion with transformer layers and pre-training | https://github.com/gastruc/OmniSat |
| MMEarth | Nedungadi et al. 2024 | Single-view input to predict another views (pretext task) | https://github.com/vishalned/MMEarth-train |
| MambaDifussion | Du et al. 2024 | Feature level fusion with Mamba over different layers (skip connection) including difussion models | https://github.com/WenliangDu/MambaDiffusion |
| FusionMamba | Peng et al. 2024 | Dense fusion with Mamba layers | https://github.com/PSRben/FusionMamba |
Feel free to ask me to update some content!
Some abbrevations
| Abbrevation | name |
|---|---|
| CNN | convolutional neural network |
| LSTM | long-short term memory |
| NN | neural network |
| PSE-TAE | pixel set encoder - temporal attention encoder |