CoDeRNet: Selective Cross-Task Routing under Heterogeneous Supervision for Change Detection and Captioning

Cho, Eunki; Kim, Hyeon Bae; Kim, Seong Tae

CoDeRNet

Selective Cross-Task Routing under Heterogeneous Supervision for Change Detection and Captioning

Eunki Cho, Hyeon Bae Kim, Seong Tae Kim

Kyung Hee University, Republic of Korea
{cekkec, hyeonbae.kim, st.kim}@khu.ac.kr
NeurIPS 2026

Paper Code Weights BibTeX

CoDeRNet architecture: shared encoder and fusion block, change detection branch, change captioning branch, and the routing between the two branches — **Overview of CoDeRNet.** A shared encoder and fusion block produce multi-scale change features, which are projected into a change-detection branch (mask decoder) and a change-captioning branch (multi-scale token memory + Transformer decoder). The two branches exchange information through Confidence-Decomposed Routing (CoDeR).

Abstract

Change Detection & Captioning (CDC) aims to jointly localize changed regions and describe their semantic content from bi-temporal remote sensing images. While recent approaches have demonstrated the effectiveness of unified modeling, they primarily rely on shared representations or jointly optimized architectures, leaving task-specific communication under heterogeneous supervision less explored. However, detection and captioning differ fundamentally in their supervision granularity, leading to heterogeneous representations with uneven reliability across tasks and spatial locations. In this paper, we reformulate CDC as a selective information routing problem between heterogeneous task branches, where the key challenge is to determine when and where information should be transferred. To this end, we propose CoDeRNet, a unified framework that preserves task-specific representations while enabling bidirectional exchange of complementary information. CoDeRNet introduces a Confidence-Decomposed Routing (CoDeR) mechanism, which decomposes the routing condition into receiver-side need for support and sender-side reliability cues to guide selective transfer. Experiments on LEVIR-MCI and WHU-CDC show that CoDeRNet achieves strong overall CDC performance, particularly improving caption generation while preserving competitive change localization. Extensive analyses further show that the benefits arise not merely from joint learning or generic cross-task communication, but from explicitly modeling selective cross-task information routing. These results highlight the importance of conditional information transfer for unified CDC under heterogeneous supervision.

Confidence-Decomposed Routing

CoDeR selectively regulates cross-task information transfer based on receiver-side need and sender-side reliability.

CoDeR selectively modulates cross-task information transfer at each spatial location. Rather than relying on a single interaction weight, it decomposes the routing condition into two learned cues: receiver-side need for support and sender-side reliability. Their multiplicative combination produces a soft routing mask that controls the strength of residual information transfer.

Receiver side

Where is support needed?

A need-for-support map is estimated from the receiving branch. Regions where the receiver is already confident are protected from being overwritten.

Sender side

Where is the source reliable?

A reliability map is estimated from the sending branch. Only evidence the sender is confident about is allowed to travel.

Conditional transfer

Should transfer occur?

The routing mask combines need and reliability, and it gates a lightweight residual transfer of the source feature into the receiving branch.

Routing is applied in both directions, caption → detection and detection → caption, at designated feature levels before the task decoders. Complementarity is therefore defined operationally by the sender–receiver state rather than as an intrinsic property of a feature, and the detection and captioning representations, together with the routing cues, are optimized jointly through the two task losses without a separate post-hoc integration model.

Quantitative Results

We evaluate CoDeRNet against existing methods on LEVIR-MCI and WHU-CDC.

Comparison with change detection methods, change captioning methods, and unified detection-and-captioning frameworks on the test splits of LEVIR-MCI and WHU-CDC. Bold marks the best score and underline the second best.

Table 1: comparison on LEVIR-MCI with change detection, change captioning, and unified methods; CoDeRNet reaches 87.79 mIoU and the best captioning scores — **Table 1. LEVIR-MCI** 3-class mIoU (background / road / building) and captioning metrics. CoDeRNet obtains the best score on every metric among the compared methods.

Table 2: comparison on WHU-CDC with change detection, change captioning, and unified methods; CoDeRNet reaches 90.25 IoU and the best METEOR, ROUGE-L, and CIDEr-D — **Table 2. WHU-CDC** Change-class IoU and captioning metrics. CoDeRNet achieves the best IoU, METEOR, ROUGE-L and CIDEr-D, and the second-best BLEU scores.

Ablation Study

We investigate the contributions of joint training and CoDeR, and compare CoDeR with alternative interaction operators.

Ablation of joint training and CoDeR: segmentation only, captioning only, joint training, and joint training with CoDeR, with mIoU, BLEU-4, METEOR, ROUGE-L and CIDEr-D — **Table 3(a). Joint Training and CoDeR** We examine the individual contributions of joint training and CoDeR to both tasks.

Comparison of cross-task interaction operators: Co-Attn, Cross-Attn, Exchange, Gating, and CoDeR, with mIoU, BLEU-4, METEOR, ROUGE-L and CIDEr-D — **Table 4(b). Interaction Operators** We compare CoDeR with alternative cross-task interaction operators to assess the effectiveness of selective routing.

Qualitative Results

We present visual comparisons of change localization and caption generation on both benchmarks.

Qualitative comparison on LEVIR-MCI test images: inputs, ground-truth mask, MCINet, CoDeRNet, and the corresponding captions — **LEVIR-MCI test examples compared with MCINet.** background road building Below each row, the ground-truth caption and the captions predicted by the two models are shown; road and building words are colored accordingly.

Qualitative comparison on WHU-CDC test images: inputs, ground-truth mask, MCINet, CoDeRNet, and the corresponding captions — **WHU-CDC test examples compared with MCINet.** background change Below each row, the ground-truth caption and the captions predicted by the two models are shown.

BibTeX

If you find our work useful, please consider citing our paper.

@inproceedings{cho2026codernet,
  title     = {CoDeRNet: Selective Cross-Task Routing under Heterogeneous Supervision
               for Change Detection and Captioning},
  author    = {Cho, Eunki and Kim, Hyeon Bae and Kim, Seong Tae},
  booktitle = {Advances in Neural Information Processing Systems (NeurIPS)},
  year      = {2026}
}

Acknowledgements

The data loading, the LEVIR-MCI preprocessing and the evaluation code are adapted from Change-Agent; the captioning scorers come from coco-caption. LEVIR-MCI is available on Hugging Face and WHU-CDC on Kaggle. The code is released under the MIT License.