publications

2026

1. TCSVT

   Compression in 3d gaussian splatting: A survey of methods, trends, and future directions

   Ali, Muhammad Salman, Zhang, Chaoning, Cagnazzo, Marco, Valenzise, Giuseppe, Tartaglione, Enzo, and Bae, Sung-Ho

   IEEE Transactions on Circuits and Systems for Video Technology 2026

   Abs PDF

   3D Gaussian Splatting (3DGS) has recently emerged as a pioneering approach in explicit scene rendering and computer graphics. Unlike traditional neural radiance field (NeRF) methods, which typically rely on implicit, coordinate-based models to map spatial coordinates to pixel values, 3DGS utilizes millions of learnable 3D Gaussians. Its differentiable rendering technique and inherent capability for explicit scene representation and manipulation positions 3DGS as a potential game-changer for the next generation of 3D reconstruction and representation technologies. This enables 3DGS to deliver real-time rendering speeds while offering unparalleled editability levels. However, despite its advantages, 3DGS suffers from substantial memory and storage requirements, posing challenges for deployment on resource-constrained devices. In this survey, we provide a comprehensive overview focusing on the scalability and compression of 3DGS. We begin with a detailed background overview of 3DGS, followed by a structured taxonomy of existing compression methods. Additionally, we analyze and compare current methods from the topological perspective, evaluating their strengths and limitations in terms of fidelity, compression ratios, and computational efficiency. Furthermore, we explore how advancements in efficient NeRF representations can inspire future developments in 3DGS optimization. Finally, we conclude with current research challenges and highlight key directions for future exploration.

2. AAAI

   I-INR: Iterative Implicit Neural Representations

   Haider, Ali, Ali, Muhammad Salman, Qamar, Maryam, Khalil, Tahir, Kim, Soo Ye, Oh, Jihyong, Tartaglione, Enzo, and Bae, Sung-Ho

   AAAI Conference on Artificial Intelligence 2026

   Abs PDF

   Implicit Neural Representations (INRs) have revolutionized signal processing and computer vision by modeling signals as continuous, differentiable functions parameterized by neural networks. However, their inherent formulation as a regression problem makes them prone to regression to the mean, limiting their ability to capture fine details, retain high-frequency information, and handle noise effectively. To address these challenges, we propose Iterative Implicit Neural Representations (I-INRs) a novel plug-and-play framework that enhances signal reconstruction through an iterative refinement process. I-INRs effectively recover high-frequency details, improve robustness to noise, and achieve superior reconstruction quality. Our framework seamlessly integrates with existing INR architectures, delivering substantial performance gains across various tasks. Extensive experiments show that I-INRs outperform baseline methods, including WIRE, SIREN, and Gauss, in diverse computer vision applications such as image restoration, image denoising, and object occupancy prediction.

3. CVPR

   HandVQA: Diagnosing and Improving Fine-Grained Spatial Reasoning about Hands in Vision-Language Models

   Sayem, MD Khalequzzaman Chowdhury, Chowdhury, Mubarrat Tajoar, Tiruneh, Yihalem Yimolal, Khan, Muneeb A., Ali, Muhammad Salman, Bhattarai, Binod, and Baek, Seungryul

   Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2026

   Abs PDF

   Understanding fine-grained articulation of human hands is critical in high-stakes settings such as robot-assisted surgery, chip manufacturing, and AR/VR-based human–AI interaction. Despite strong performance on general vision-language benchmarks, current vision-language models (VLMs) struggle with fine-grained spatial reasoning, especially for complex, articulated hand poses. We introduce HandVQA, a large-scale diagnostic benchmark that evaluates VLMs’ understanding of detailed hand anatomy via visual question answering. Built upon high-quality 3D hand datasets (FreiHAND, InterHand2.6M, FPHA), HandVQA provides 1.6M+ controlled multiple-choice questions probing spatial relationships between hand joints, including angles, distances, and relative positions. We evaluate state-of-the-art VLMs (e.g., LLaVA, DeepSeek, Qwen-VL) in both base and LoRA fine-tuned settings. Our results reveal systematic limitations such as hallucinated finger parts, incorrect geometric interpretations, and poor generalization. HandVQA not only exposes these reasoning gaps, but also provides a validated path to improvement: the 3D-grounded spatial knowledge learned from HandVQA transfers zero-shot to novel downstream tasks, improving hand gesture recognition (+10.33%) and hand–object interaction recognition (+2.63%).

2025

1. WACV

   ELMGS: Enhancing memory and computation scaLability through coMpression for 3D Gaussian Splatting

   Ali, Muhammad Salman, Bae, Sung-Ho, and Tartaglione, Enzo

   IEEE/CVF Winter Conference on Applications of Computer Vision 2025

   Abs PDF Code

   3D models have recently been popularized by the potentiality of end-to-end training offered first by Neural Radiance Fields and most recently by 3D Gaussian Splatting models. The latter has the big advantage of naturally providing fast training convergence and high editability. However as the research around these is still in its infancy there is still a gap in the literature regarding the model’s scalability. In this work we propose an approach enabling both memory and computation scalability of such models. More specifically we propose an iterative pruning strategy that removes redundant information encoded in the model. We also enhance compressibility for the model by including a differentiable quantization and entropy coding estimator in the optimization strategy. Our results on popular benchmarks showcase the effectiveness of the proposed approach and open the road to the broad deployability of such a solution even on resource-constrained devices.

2024

1. BMVC

   Trimming the Fat: Efficient Compression of 3D Gaussian Splats through Pruning

   Ali, Muhammad Salman, Qamar, Maryam, Bae, Sung-Ho, and Tartaglione, Enzo

   British Machine Vision Conference 2024, BMVC 2024 2024

   PDF Code Poster Video
2. KBME

   Mirroring Real Data Distribution for Better Dataset Condensation

   Qamar, Maryam, Ali, Muhammad Salman, and Bae, Sung Ho

   Proceedings of the Korean Broadcast and Media Engineers Conference 2024

   Abs PDF

   Big datasets fuel advancements in deep learning but also present challenges due to resource-intensive demands. Dataset condensation (DC) aims to address this performance-resource conflict by condensing large datasets into smaller surrogates, aiming to maintain training performance. This study proposes to optimize the condensed data to closely mirror the entire dataset’s distribution, enhancing condensation performance. A statistical metric, number of statistically different bins (NDBs), is utilized to assess the distribution similarity, and is integrated into the optimization loss. Our experiments demonstrate improved performance over the baseline method.

3. SIVP

   Give Me a Hint: An Explicit Prior Based Image Denoising

   Uddin, AFM Shahab, Ali, Muhammad Salman, Awais, Muhammad, Hwang, Jiwon, Chung, TaeChoong, and Bae, Sung-Ho

   Signal, Image and Video Processing 2024

   Abs PDF

   The performance of an image denoising method highly depends on its ability to produce smoothed homogeneous regions and preserve fine-grained texture details in the restored image. Existing denoising methods rely on implicit learning of Convolutional Neural Networks (CNNs) to restore an image. We show that the implicitly learned representations are limited in capacity, resulting in a sub-optimal pixel estimation of homogeneous regions and loss in texture details. In this work, we argue that explicitly introducing low and high frequency information enhances the representational capacity of denoising CNNs. To this end, we propose XPrior that provides the network with two explicit priors, i.e., a Gaussian smoothed image and an Edge map. This simple yet effective integration of priors with the noisy image enhances the model’s ability to restore images with high perceived quality. The proposed approach can easily be plugged into the existing denoising methods. Comprehensive experimental results show the superiority of the proposed approach that achieves a remarkable performance gain of 1.23 dB and 1.49 dB average PSNR on BSD68 dataset over DRUNet (Zhang in IEEE Trans Pattern Anal Mach Intell 10.1109/TPAMI.2021.3088914, 2021) and DnCNN (Zhang in IEEE Trans Image Process 26:3142–3155, 2017), respectively.

4. VCIP

   ALICE: Adapt your Learnable Image Compression modEl for variable bitrates

   Spadaro, Gabriele, Ali, Muhammad Salman, Presta, Alberto, Pilo, Giommaria, Bae, Sung-Ho, Giraldo, Jhony H, Fiandrotti, Attilio, Grangetto, Marco, and Tartaglione, Enzo

   IEEE International Conference on Visual Communications and Image Processing (VCIP) 2024

   Abs PDF

   When training a Learned Image Compression model, the loss function is minimized such that the encoder and the decoder attain a target Rate-Distorsion trade-off. Therefore, a distinct model shall be trained and stored at the transmitter and receiver for each target rate, fostering the quest for efficient variable bitrate compression schemes. This paper proposes plugging Low-Rank Adapters into a transformer-based pre-trained LIC model and training them to meet different target rates. With our method, encoding an image at a variable rate is as simple as training the corresponding adapters and plugging them into the frozen pre-trained model. Our experiments show performance comparable with state-of-the-art fixed-rate LIC models at a fraction of the training and deployment cost. We publicly released the code at https://github.com/EIDOSLAB/ALICE.

2023

1. IEEEAccess

   A Super-Resolution-Based Feature Map Compression for Machine-Oriented Video Coding

   Kang, Jung-Heum, Ali, Muhammad Salman, Jeong, Hye-Won, Choi, Chang-Kyun, Kim, Younhee, Jeong, Se Yoon, Bae, Sung-Ho, and Kim, Hui Yong

   IEEE Access 2023

   Abs PDF

   Recently, video and image compression methods using neural networks have received much attention. In MPEG standardization, Video Coding for Machine (VCM) is a newly arising topic which attempts to compress features/images for the purpose of machine vision tasks. Especially, compressing features has advantages in terms of privacy protection and computation off-loading. In this paper, we propose an effective feature compression method equipped with a super-resolution (SR) module for features. Our main motivation comes from the observation that features are somewhat robust to spatial distortions (e.g., AWGN, blur, quantization distortions, coding artifacts), which leads us to integrating an SR module into the compression framework. We also further explore the best training strategy of the proposed method, i.e., finding the best combination of various losses and proper input feature shapes. Our comprehensive experiments show that the proposed method outperforms the baseline in the original VCM anchor scenario on various QP values with Versatile Video Coding (VVC). Specifically, the proposed framework achieved up to 50% BD-rate reduction compared to the conventional P-layer feature map compression method for the object detection task on the OpenImage dataset.

2. NeurIPS

   Towards efficient image compression without autoregressive models

   Ali, Muhammad Salman, Kim, Yeongwoong, Qamar, Maryam, Lim, Sung-Chang, Kim, Donghyun, Zhang, Chaoning, Bae, Sung-Ho, and Kim, Hui Yong

   Advances in Neural Information Processing Systems 2023

   Abs PDF

   Recently, learned image compression (LIC) has garnered increasing interest with its rapidly improving performance surpassing conventional codecs. A key ingredient of LIC is a hyperprior-based entropy model, where the underlying joint probability of the latent image features is modeled as a product of Gaussian distributions from each latent element. Since latents from the actual images are not spatially independent, autoregressive (AR) context based entropy models were proposed to handle the discrepancy between the assumed distribution and the actual distribution. Though the AR-based models have proven effective, the computational complexity is significantly increased due to the inherent sequential nature of the algorithm. In this paper, we present a novel alternative to the AR-based approach that can provide a significantly better trade-off between performance and complexity. To minimize the discrepancy, we introduce a correlation loss that forces the latents to be spatially decorrelated and better fitted to the independent probability model. Our correlation loss is proved to act as a general plug-in for the hyperprior (HP) based learned image compression methods. The performance gain from our correlation loss is ‘free’ in terms of computation complexity for both inference time and decoding time. To our knowledge, our method gives the best trade-off between the complexity and performance: combined with the Checkerboard-CM, it attains 90% and when combined with ChARM-CM, it attains 98% of the AR-based BD-Rate gains yet is around 50 times and 30 times faster than AR-based methods respectively

3. NeurIPS-WS

   Can Segment Anything Model Improve Semantic Segmentation?

   Qamar, Maryam, Kim, Donghoon, Ali, Muhammad Salman, Zhang, Chaoning, and Bae, Sung-Ho

   In NeurIPS Workshop on Failure Modes in the Age of Foundation Models 2023

   Abs PDF

   Recently, Segment Anything Model (SAM) has gained considerable attention in the field of computer vision establishing itself as a pioneering foundation model for segmentation. Notably, SAM excels in generating high-quality segmentation masks, yet it lacks in semantic labels. In contrast, conventional semantic segmentation models generate rather accurate semantic labels but often produce suboptimal segmentation masks. The notion of leveraging SAM’s superior mask quality to enhance the performance of conventional semantic segmentation models appears intuitive. However, our preliminary experiments reveal that the integration of SAM with these models does not result in any discernible improvement. Specifically, when assessing the performance of SAM’s integration into two baseline semantic segmentation models, DeepLab and OneFormer, we find no significant enhancements in the mean Intersection over Union (mIoU) on the Pascal VOC and ade20k datasets. Consequently, we conclude that, as it stands, the highly acclaimed foundational model is not the preferred solution for the semantic segmentation task. Instead, a more cautious and thoughtful approach is imperative to unlock any potential benefits in this context.

4. VCIP

   Exploring the Optimal Bit Pair for a Quantized Generator and Discriminator

   Yang, Subin, Ali, Muhammad Salman, Uddin, AFM Shahab, and Bae, Sung-Ho

   In 2023 IEEE International Conference on Visual Communications and Image Processing (VCIP) 2023

   Abs PDF

   Generative Adversarial Networks (GANs) are hindered from real-world applications due to their high computational cost and memory requirements. Model compression techniques, such as quantization, pruning, and knowledge distillation, can compress neural networks, lower memory requirements, and model size. However, quantizing generators often leads to a suboptimal solution. In this paper, we propose a novel method to stabilize GAN quantization by quantizing the generator and discriminator with different bit precision. Our method maximizes the quantization efficiency by jointly quantizing the generator and discriminator, which we found to be dependent on each other’s quantization. Specifically, quantizing the discriminator enhances the performance of the quantized generator, while the discriminator’s optimal quantization bit depends on the generator’s quantization bit and architectural type. We conducted extensive experiments on various GAN models, including BigGAN, SAGAN, and SNGAN, using different quantization methods, such as LSQ, PACT, and DoReFa, on benchmark dataset (CIFAR10). The experimental results demonstrate that our joint quantization method achieves higher compression rates while offering better performance in Frechet Inception Distance (FID) and Inception Score (IS).

2022

1. KIISE

   Analyzing Latent Space Behavior with Learnable Quantization and Sparsity in Image Compression

   Ali, Muhammad Salman, Awais, Muhammad, Kim, Hui Yong, and Bae, Sung-Ho

   Proceedings of the Korean Information Science Society Conference 2022

   Abs PDF

   Deep learning has revolutionized the field of computer vision, with the recent surge of deep learningbased approaches for classification, segmentation, and super-resolution there is a huge interest in the field of deep learning-based image compression methods. The image is compressed through an encoder architecture and then entropy encoded after quantization. However, loss of information during quantization occurs due to which accurate image reconstruction is not possible. Current methods employ uniform quantization methods for quantizing the latent space. We used learnable quantization methods in this paper to further study the behavior of compression. Learnable quantization methods due to their high representational capacity show higher entropy rates. We in this paper investigate the reason and delve deep into the usability of learnable quantization method for image compression.

2021

1. ICCV

   Distilling Global and Local Logits with Densely Connected Relations

   Kim, Youmin, Park, Jinbae, Jang, YounHo, Ali, Muhammad Salman, Oh, Tae-Hyun, and Bae, Sung-Ho

   In Proceedings of the IEEE/CVF International Conference on Computer Vision 2021

   Abs PDF

   In prevalent knowledge distillation, logits in most image recognition models are computed by global average pooling, then used to learn to encode the high-level and task-relevant knowledge. In this work, we solve the limitation of this global logit transfer in this distillation context. We point out that it prevents the transfer of informative spatial information, which provides localized knowledge as well as rich relational information across contexts of an input scene. To exploit the rich spatial information, we propose a simple yet effective logit distillation approach. We add a local spatial pooling layer branch to the penultimate layer, thereby our method extends the standard logit distillation and enables learning of both finely-localized knowledge and holistic representation. Our proposed method shows favorable accuracy improvement against the state-of-the-art methods on several image classification datasets. We show that our distilled students trained on the image classification task can be successfully leveraged for object detection and semantic segmentation tasks; this result demonstrates our method’s high transferability.

2. IEEEAccess

   Binary-Classifiers-Enabled Filters for Semi-Supervised Learning

   Kumar, Teerath, Park, Jinbae, Ali, Muhammad Salman, Uddin, AFM Shahab, Ko, Jong Hwan, and Bae, Sung-Ho

   IEEE Access 2021

   Abs PDF

   A typical semi-supervised learning-based scheme is based on training a single model for labeled data. For unlabeled data, it uses the pseudo-labeling method to obtain labels. However, the samples during pseudo-labeling are often filtered using a probability threshold, which suffers from the challenge of effective threshold selection. In the case of a high probability threshold, correct samples may not be labeled, and in the case of a low threshold, samples can be wrongly labeled. This threshold issue degrades the overall performance of the model. This paper addresses this vital issue by proposing a novel approach of SSL named Binary-Classifiers-Enabled Filters for Semi-Supervised Learning (BSSL) for labeling the unlabeled data by using binary classifiers as data filters. That is, we train binary classifiers dedicated to each class. After training, we propose three methods for labeling the unlabeled data; cascading, non-cascading, and rank-based binary classifiers. Our extensive experiment shows rank-based binary classifiers are the best choice for labeling the data. Our approach eliminates threshold selection to improve the performance of the model. Comprehensive experiments are performed to demonstrate the effectiveness of our approach on a variety of domains, including image classification, text classification and audio classification, datasets including MNIST, fashion-MNIST, EuroSat, ESC10, Free Spoken Digit dataset, Audio Emotion recognition, reuter and mice protein dataset. Rank based binary classifiers (BSSL) approach achieves absolute performance of atleast 10% and 5% over supervised learning(SL) and SSL, respectively on audio datasets in different number of sample cases except RAVDESS dataset. Moreover, BSSL shows tremendous performance on image datasets specifically when number of samples is very small. Overall, BSSL outperformed the purely supervised learning approach and SSL pseudo-labeling approaches in different number of samples cases.

3. JBE

   Pruning for Robustness by Suppressing High Magnitude and Increasing Sparsity of Weights

   Cho, Incheon, Ali, Muhammad Salman, and Bae, Sung-Ho

   Journal of Broadcast Engineering 2021

   Abs PDF

   Abstract Although Deep Neural Networks (DNNs) have shown remarkable performance in various artificial intelligence fields, it is well known that DNNs are vulnerable to adversarial attacks. Since adversarial attacks are implemented by adding perturbations onto benign examples, increasing the sparsity of DNNs minimizes the propagation of errors to high-level layers. In this paper, unlike the traditional pruning scheme removing low magnitude weights, we eliminate high magnitude weights that are usually considered high absolute values, named ‘reverse pruning’to ensure robustness. By conducting both theoretical and experimental analyses, we observe that reverse pruning ensures the robustness of DNNs. Experimental results show that our reverse pruning outperforms previous work with 29.01% in Top-1 accuracy on perturbed CIFAR-10. However, reverse pruning does not guarantee benign samples. To relax this problem, we further conducted experiments by adding a regularization term for the high magnitude weights. With adding the regularization term, we also applied conventional pruning to ensure the robustness of DNNs.

4. IEEEAccess

   Dynamic Structured Pruning with Novel Filter Importance and Leaky Masking Based on Convolution and Batch Normalization Parameters

   Cho, Incheon, Shin, Eunseop, Ali, Muhammad Salman, and Bae, Sung-Ho

   IEEE Access 2021

   Abs PDF

   Various pruning methods have been proposed to solve the overparameterized problem in deep neural networks. Most of the structured pruning methods have used magnitude-based filter importance to remove unnecessary filters. Usually, Convolutional Neural Networks (CNN) consist of blocks that proceed with batch normalization (BN) operations after convolutional (Conv) operations. Each element is calculated through cooperation between Conv and BN, so both Conv and BN parameters must be considered together in pruning. However, previous pruning methods independently used the norm of parameters, either Conv weights or BN scales, as filter importance, ignoring these CNN structures. With this intuition, we propose a new magnitude-based filter importance method that considers both Conv weights and BN scales, and provide evidence of importance through experimental analysis on the rank of the feature map as well as mathematical analysis on the feature distortion. Furthermore, in recent works, dynamically applying a mask to recover weights has enabled more accurate pruning. However, when the weights are multiplied by zero mask values, the gradients become zero, which prevents updating the weights. We name this problem the zero gradient transferring. To solve this problem, we propose a leaky masking method that replaces the zero value of the mask with a positive constant. As a result, we solve the zero gradient transferring through the leaky masking method, enabling more accurate dynamic structured pruning. We denote our proposed method as CoBaL that incorporates Conv and BN based filter importance and Leaky mask into dynamic structured pruning. Experimental results show that our CoBaL compresses 50.09% parameters and 32.51% FLOPs on ResNet 56 with the CIFAR-10 with slightly improved accuracy. Also, we possess comparable results on ImageNet with 50.74% and 29.38% reduction in the number of parameters and FLOPs at ResNet18, respectively.

5. JBE

   Class Specific Autoencoders Enhance Sample Diversity

   Kumar, Teerath, Park, Jinbae, Ali, Muhammad Salman, Uddin, AFM Shahab, and Bae, Sung-Ho

   Journal of Broadcast Engineering 2021

   Abs PDF

   Semi-supervised learning (SSL) and few-shot learning (FSL) have shown impressive performance even then the volume of labeled data is very limited. However, SSL and FSL can encounter a significant performance degradation if the diversity gap between the labeled and unlabeled data is high. To reduce this diversity gap, we propose a novel scheme that relies on an autoencoder for generating pseudo examples. Specifically, the autoencoder is trained on a specific class using the available labeled data and the decoder of the trained autoencoder is then used to generate N samples of that specific class based on N random noise, sampled from a standard normal distribution. The above process is repeated for all the classes. Consequently, the generated data reduces the diversity gap and enhances the model performance. Extensive experiments on MNIST and FashionMNIST datasets for SSL and FSL verify the effectiveness of the proposed approach in terms of classification accuracy and robustness against adversarial attacks.

6. IEEEAccess

   A Feature Fusion Based Indicator for Training-Free Neural Architecture Search

   Tran, Linh-Tam, Ali, Muhammad Salman, and Bae, Sung-Ho

   IEEE Access 2021

   Abs PDF

   Neural Architecture Search Without Training (NASWOT) has been proposed recently to replace the conventional Neural Architecture Search (NAS). Pioneer works only deploy one or two indicator(s) to search. Nevertheless, the quantitative assessment for indicators is not fully studied and evaluated. In this paper, we first review several indicators, which are used to evaluate the network in a training-free manner, including the correlation of Jacobian, the output sensitivity, the number of linear regions, and the condition number of the neural tangent kernel. Our observation is that each indicator is responsible for characterizing a network in a specific aspect and there is no single indicator that achieves good performance in all cases, e.g. highly correlated with the test accuracy. This motivated us to develop a novel indicator where all properties of a network are taken into account. To obtain better indicator that can consider various characteristics of networks in a harmonized form, we propose a Fusion Indicator (FI). Specifically, the proposed FI is formed by combining multiple indicators in a weighted sum manner. We minimize the mean squared error loss between the predicted and actual accuracy of networks to acquire the weights. Moreover, as the conventional training-free NAS researches used limited metrics to evaluate the quality of indicators, we introduce more desirable metrics that can evaluate the quality of training-free NAS indicator in terms of fidelity, correlation and rank-order similarity between the predicted quality value and actual accuracy of networks. That is, we introduce the Pearson Linear Coefficient Correlation (PLCC), the Root Mean Square Error (RMSE), the Spearman Rank-Order Correlation Coefficient (SROCC), and Kendall Rank-Order Correlation Coefficient (KROCC). Extensive experiments on NAS-Bench-101 and NAS-Bench-201 demonstrate the effectiveness of our FI, outperforming existing methods by a large margin.

7. JBE

   An Analysis on the Properties of Features Against Various Distortions in Deep Neural Networks

   Kang, Jung Heum, Jeong, Hye Won, Choi, Chang Kyun, Ali, Muhammad Salman, Bae, Sung-Ho, and Kim, Hui Yong

   Journal of Broadcast Engineering 2021

   Abs PDF

   Deploying deep neural network model training performs remarkable performance in the fields of Object detection and Instance segmentation. To train these models, features are first extracted from the input image using a backbone network. The extracted features can be reused by various tasks. Research has been actively conducted to serve various tasks by using these learned features. In this process, standardization discussions about encoding, decoding, and transmission methods are proceeding actively. In this scenario, it is necessary to analyze the response characteristics of features against various distortions that may occur in the data transmission or data compression process. In this paper, experiment was conducted to inject various distortions into the feature in the object recognition task. And analyze the mAP (mean Average Precision) metric between the predicted value output from the neural network and the target value as the intensity of various distortions was increased. Experiments have shown that features are more robust to distortion than images. And this points out that using the feature as transmission means can prevent the loss of information against the various distortions during data transmission and compression process.

2020

1. KBME

   Soft Error Adaptable Deep Neural Networks

   Ali, Muhammad Salman, and Bae, Sung-Ho

   Proceedings of the Korean Broadcast and Media Engineers Conference 2020

   Abs PDF

   The high computational complexity of deep learning algorithms has led to the development of specialized hardware architectures. However, soft errors (bit flip) may occur in these hardware systems due to voltage variation and high energy particles. Many error correction methods have been proposed to counter this problem. In this work, we analyze an error correction mechanism based on repetition codes and an activation function. We test this method by injecting errors into weight filters and define an ideal error rate range in which the proposed method complements the accuracy of the model in the presence of error.

2. IEEEAccess

   ERDNN: Error-Resilient Deep Neural Networks with a New Error Correction Layer and Piece-wise Rectified Linear Unit

   Ali, Muhammad Salman, Iqbal, Tauhid Bin, Lee, Kang-Ho, Muqeet, Abdul, Lee, Seunghyun, Kim, Lokwon, and Bae, Sung-Ho

   IEEE Access 2020

   Abs PDF

   Deep Learning techniques have been successfully used to solve a wide range of computer vision problems. Due to their high computation complexity, specialized hardware accelerators are being proposed to achieve high performance and efficiency for deep learning-based algorithms. However, soft errors, i.e., bit flipping errors in the layer output, are often caused due to process variation and high energy particles in these hardware systems. These can significantly reduce model accuracy. To remedy this problem, we propose new algorithms that effectively reduce the impact of errors, thus keeping high accuracy. We firstly propose to incorporate an Error Correction Layer (ECL) into neural networks where convolution is performed multiple times in each layer and majority reporting is conducted for the outputs at bit level. We found that ECL can eliminate most errors while bypassing the bit-error when the bits at the same position are corrupted multiple times under the simulated condition. In order to solve this problem, we analyze the impact of errors depending on the position of bits, thus observing that errors in most significant bit (MSB) positions tend to severely corrupt the output of the network compared to the errors in the least significant bit (LSB) positions. According to this observation, we propose a new specialized activation function, called Piece-wise Rectified Linear Unit (PwReLU), which selectively suppresses errors depending on the bit positions, resulting in an increased model resistance against the errors. Compared to existing activation functions, the proposed PwReLU outperforms with large accuracy margins of up-to 20% even with very high bit error rates (BERs). Our extensive experiments show that the proposed ECL and PwReLU work in a complementary manner, achieving comparable accuracy to the error-free networks even at a severe BER of 0.1% on CIFAR10, CIFAR100, and ImageNet.

2019

1. KIISE

   A Novel Error-Resilient Memristor-Based Neuromorphic Architecture Using an Error Correction Layer

   Ali, Muhammad Salman, and Bae, Sung-Ho

   Proceedings of the Korean Information Science Society Conference 2019

   Abs PDF

   Memristors due to their energy efficiency and high computing capability are a good choice for implementation of neuro-morphic systems. However, it has been observed that memristors are not entirely accurate in their computation tasks. In such an error prone environment neural network models fails and their accuracy drops significantly. We introduce an error correction layer for neural network implementation in memristors that reduces the error in memristor considerably. With our approach we are able to substantially increase the accuracy of the model.