A mutual learning framework for pruned and quantized networks

Model compression is an important topic in deep learning research. It can be mainly divided into two directions: model pruning and model quantization. However, both methods will more or less affect the original accuracy of the model. In this paper, we propose a mutual learning framework for pruned a...

Descripción completa

Guardado en:
Detalles Bibliográficos
Autores principales: Li, Xiaohai, Chen, Yigiang, Wang, Jindong
Formato: Articulo
Lenguaje:Inglés
Publicado: 2023
Materias:
Ciencias Informáticas
Model compression
Network pruning
Quantization
Mutual learning
Compresión de modelo
Poda de red
Cuantificación
Aprendizaje mutuo
Acceso en línea:http://sedici.unlp.edu.ar/handle/10915/152119
Aporte de:
SEDICI (UNLP) de Universidad Nacional de La Plata
id I19-R120-10915-152119
record_format dspace
spelling I19-R120-10915-1521192023-04-25T20:03:58Z http://sedici.unlp.edu.ar/handle/10915/152119 issn:1666-6038 A mutual learning framework for pruned and quantized networks Un marco de aprendizaje mutuo para redes podadas y cuantificadas Li, Xiaohai Chen, Yigiang Wang, Jindong 2023-04 2023-04-25T17:55:46Z en Ciencias Informáticas Model compression Network pruning Quantization Mutual learning Compresión de modelo Poda de red Cuantificación Aprendizaje mutuo Model compression is an important topic in deep learning research. It can be mainly divided into two directions: model pruning and model quantization. However, both methods will more or less affect the original accuracy of the model. In this paper, we propose a mutual learning framework for pruned and quantized networks. We regard the pruned network and the quantized network as two sets of features that are not parallel. The purpose of our mutual learning framework is to better integrate the two sets of features and achieve complementary advantages, which we call feature augmentation. To verify the effectiveness of our framework, we select a pairwise combination of 3 state-of-the-art pruning algorithms and 3 state-of-theart quantization algorithms. Extensive experiments on CIFAR-10, CIFAR-100 and Tiny-imagenet show the benefits of our framework: through the mutual learning of the two networks, we obtain a pruned network and a quantization network with higher accuracy than traditional approaches. La compresión de modelos es un tema importante en la investigación del aprendizaje profundo. Se puede dividir principalmente en dos direcciones: poda de modelos y cuantización de modelos. Sin embargo, ambos métodos afectarán más o menos la precisión original del modelo. En este artículo, proponemos un marco de aprendizaje mutuo para redes podadas y cuantificadas. Consideramos la red podada y la red quantized como dos conjuntos de características que no son paralelas. El propósito de nuestro marco de aprendizaje mutuo es integrar mejor los dos conjuntos de funciones y lograr ventajas complementarias, lo que llamamos aumento de funciones. Para verificar la efectividad de nuestro marco, seleccionamos una combinación por pares de 3 algoritmos de poda de última generación y 3 algoritmos de cuantificación de última generación. Extensos experimentos en CIFAR- 10, CIFAR-100 y Tiny-imagenet muestran los beneficios de nuestro marco: a través del aprendizaje mutuo de las dos redes, obtenemos una red pruned y una red de cuantificación con mayor precisión que los enfoques tradicionales. Facultad de Informática Articulo Articulo http://creativecommons.org/licenses/by-nc/4.0/ Creative Commons Attribution-NonCommercial 4.0 International (CC BY-NC 4.0) application/pdf
institution Universidad Nacional de La Plata
institution_str I-19
repository_str R-120
collection SEDICI (UNLP)
language Inglés
topic Ciencias Informáticas
Model compression
Network pruning
Quantization
Mutual learning
Compresión de modelo
Poda de red
Cuantificación
Aprendizaje mutuo
spellingShingle Ciencias Informáticas
Model compression
Network pruning
Quantization
Mutual learning
Compresión de modelo
Poda de red
Cuantificación
Aprendizaje mutuo
Li, Xiaohai
Chen, Yigiang
Wang, Jindong
A mutual learning framework for pruned and quantized networks
topic_facet Ciencias Informáticas
Model compression
Network pruning
Quantization
Mutual learning
Compresión de modelo
Poda de red
Cuantificación
Aprendizaje mutuo
description Model compression is an important topic in deep learning research. It can be mainly divided into two directions: model pruning and model quantization. However, both methods will more or less affect the original accuracy of the model. In this paper, we propose a mutual learning framework for pruned and quantized networks. We regard the pruned network and the quantized network as two sets of features that are not parallel. The purpose of our mutual learning framework is to better integrate the two sets of features and achieve complementary advantages, which we call feature augmentation. To verify the effectiveness of our framework, we select a pairwise combination of 3 state-of-the-art pruning algorithms and 3 state-of-theart quantization algorithms. Extensive experiments on CIFAR-10, CIFAR-100 and Tiny-imagenet show the benefits of our framework: through the mutual learning of the two networks, we obtain a pruned network and a quantization network with higher accuracy than traditional approaches.
format Articulo
Articulo
author Li, Xiaohai
Chen, Yigiang
Wang, Jindong
author_facet Li, Xiaohai
Chen, Yigiang
Wang, Jindong
author_sort Li, Xiaohai
title A mutual learning framework for pruned and quantized networks
title_short A mutual learning framework for pruned and quantized networks
title_full A mutual learning framework for pruned and quantized networks
title_fullStr A mutual learning framework for pruned and quantized networks
title_full_unstemmed A mutual learning framework for pruned and quantized networks
title_sort mutual learning framework for pruned and quantized networks
publishDate 2023
url http://sedici.unlp.edu.ar/handle/10915/152119
work_keys_str_mv AT lixiaohai amutuallearningframeworkforprunedandquantizednetworks
AT chenyigiang amutuallearningframeworkforprunedandquantizednetworks
AT wangjindong amutuallearningframeworkforprunedandquantizednetworks
AT lixiaohai unmarcodeaprendizajemutuopararedespodadasycuantificadas
AT chenyigiang unmarcodeaprendizajemutuopararedespodadasycuantificadas
AT wangjindong unmarcodeaprendizajemutuopararedespodadasycuantificadas
AT lixiaohai mutuallearningframeworkforprunedandquantizednetworks
AT chenyigiang mutuallearningframeworkforprunedandquantizednetworks
AT wangjindong mutuallearningframeworkforprunedandquantizednetworks
_version_ 1765660074739499008

Ejemplares similares