Semantic Communication for Efficient Image Transmission Tasks based on Masked Autoencoders

Jiale Wu; Celimuge Wu; Yangfei Lin; Jingjing Bao; Zhaoyang Du; Lei Zhong; Xianfu Chen; Yusheng Ji

doi:10.1109/VTC2023-Fall60731.2023.10333576

Semantic Communication for Efficient Image Transmission Tasks based on Masked Autoencoders

Jiale Wu, Celimuge Wu, Yangfei Lin, Jingjing Bao, Zhaoyang Du, Lei Zhong, Xianfu Chen, Yusheng Ji

BA6401 Connectivity solutions

Research output: Chapter in Book/Report/Conference proceeding › Conference article in proceedings › Scientific › peer-review

Abstract

Semantic communication, a promising candidate for 6G technology, has become a research hot spot. However, existing studies tend to focus more on image reconstruction rather than accurately transmitting semantic information at the pixel level. This paper introduces a novel approach using codec-based Masked AutoEncoders (MAE) for efficient image transmission. The proposed system compresses local information into low-dimensional latent vectors, improving system efficiency. We also design a selective module for enhanced image reconstruction and implement Noise Adversarial Training (NAT) to increase the system's resilience to channel noise. Experimental results show that our method effectively improves downstream tasks while preserving image quality.

Original language	English
Title of host publication	2023 IEEE 98th Vehicular Technology Conference, VTC 2023-Fall - Proceedings
Publisher	IEEE Institute of Electrical and Electronic Engineers
ISBN (Electronic)	9798350329285
DOIs	https://doi.org/10.1109/VTC2023-Fall60731.2023.10333576
Publication status	Published - 2023
MoE publication type	A4 Article in a conference publication
Event	98th IEEE Vehicular Technology Conference, VTC 2023-Fall - Hong Kong, Hong Kong, China Duration: 10 Oct 2023 → 13 Oct 2023

Conference

Conference	98th IEEE Vehicular Technology Conference, VTC 2023-Fall
Country/Territory	China
City	Hong Kong
Period	10/10/23 → 13/10/23

Keywords

deep learning
generative models
masked image modeling
semantic communication

Access to Document

10.1109/VTC2023-Fall60731.2023.10333576

Cite this

@inproceedings{87bd31f94f1d4db1b562c3cc9ddec27a,

title = "Semantic Communication for Efficient Image Transmission Tasks based on Masked Autoencoders",

abstract = "Semantic communication, a promising candidate for 6G technology, has become a research hot spot. However, existing studies tend to focus more on image reconstruction rather than accurately transmitting semantic information at the pixel level. This paper introduces a novel approach using codec-based Masked AutoEncoders (MAE) for efficient image transmission. The proposed system compresses local information into low-dimensional latent vectors, improving system efficiency. We also design a selective module for enhanced image reconstruction and implement Noise Adversarial Training (NAT) to increase the system's resilience to channel noise. Experimental results show that our method effectively improves downstream tasks while preserving image quality.",

keywords = "deep learning, generative models, masked image modeling, semantic communication",

author = "Jiale Wu and Celimuge Wu and Yangfei Lin and Jingjing Bao and Zhaoyang Du and Lei Zhong and Xianfu Chen and Yusheng Ji",

year = "2023",

doi = "10.1109/VTC2023-Fall60731.2023.10333576",

language = "English",

booktitle = "2023 IEEE 98th Vehicular Technology Conference, VTC 2023-Fall - Proceedings",

publisher = "IEEE Institute of Electrical and Electronic Engineers",

address = "United States",

note = "98th IEEE Vehicular Technology Conference, VTC 2023-Fall ; Conference date: 10-10-2023 Through 13-10-2023",

}

Wu, J, Wu, C, Lin, Y, Bao, J, Du, Z, Zhong, L, Chen, X & Ji, Y 2023, Semantic Communication for Efficient Image Transmission Tasks based on Masked Autoencoders. in 2023 IEEE 98th Vehicular Technology Conference, VTC 2023-Fall - Proceedings. IEEE Institute of Electrical and Electronic Engineers, 98th IEEE Vehicular Technology Conference, VTC 2023-Fall, Hong Kong, China, 10/10/23. https://doi.org/10.1109/VTC2023-Fall60731.2023.10333576

Semantic Communication for Efficient Image Transmission Tasks based on Masked Autoencoders. / Wu, Jiale; Wu, Celimuge; Lin, Yangfei et al.
2023 IEEE 98th Vehicular Technology Conference, VTC 2023-Fall - Proceedings. IEEE Institute of Electrical and Electronic Engineers, 2023.

Research output: Chapter in Book/Report/Conference proceeding › Conference article in proceedings › Scientific › peer-review

TY - GEN

T1 - Semantic Communication for Efficient Image Transmission Tasks based on Masked Autoencoders

AU - Wu, Jiale

AU - Wu, Celimuge

AU - Lin, Yangfei

AU - Bao, Jingjing

AU - Du, Zhaoyang

AU - Zhong, Lei

AU - Chen, Xianfu

AU - Ji, Yusheng

PY - 2023

Y1 - 2023

N2 - Semantic communication, a promising candidate for 6G technology, has become a research hot spot. However, existing studies tend to focus more on image reconstruction rather than accurately transmitting semantic information at the pixel level. This paper introduces a novel approach using codec-based Masked AutoEncoders (MAE) for efficient image transmission. The proposed system compresses local information into low-dimensional latent vectors, improving system efficiency. We also design a selective module for enhanced image reconstruction and implement Noise Adversarial Training (NAT) to increase the system's resilience to channel noise. Experimental results show that our method effectively improves downstream tasks while preserving image quality.

AB - Semantic communication, a promising candidate for 6G technology, has become a research hot spot. However, existing studies tend to focus more on image reconstruction rather than accurately transmitting semantic information at the pixel level. This paper introduces a novel approach using codec-based Masked AutoEncoders (MAE) for efficient image transmission. The proposed system compresses local information into low-dimensional latent vectors, improving system efficiency. We also design a selective module for enhanced image reconstruction and implement Noise Adversarial Training (NAT) to increase the system's resilience to channel noise. Experimental results show that our method effectively improves downstream tasks while preserving image quality.

KW - deep learning

KW - generative models

KW - masked image modeling

KW - semantic communication

UR - http://www.scopus.com/inward/record.url?scp=85181165610&partnerID=8YFLogxK

U2 - 10.1109/VTC2023-Fall60731.2023.10333576

DO - 10.1109/VTC2023-Fall60731.2023.10333576

M3 - Conference article in proceedings

AN - SCOPUS:85181165610

BT - 2023 IEEE 98th Vehicular Technology Conference, VTC 2023-Fall - Proceedings

PB - IEEE Institute of Electrical and Electronic Engineers

T2 - 98th IEEE Vehicular Technology Conference, VTC 2023-Fall

Y2 - 10 October 2023 through 13 October 2023

ER -

Semantic Communication for Efficient Image Transmission Tasks based on Masked Autoencoders

Abstract

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Cite this