Sharing Weights in Shallow Layers via Rotation Group Equivariant Convolutions
文献类型:期刊论文
作者 | Zhiqiang Chen; Ting-Bing Xu; Jinpeng Li; Huiguang He |
刊名 | Machine Intelligence Research |
出版日期 | 2022-04 |
卷号 | 19期号:2页码:115-126 |
DOI | 10.1007/s11633-022-1324-5 |
英文摘要 | Convolution operation possesses the characteristic of translation group equivariance. To achieve more group equivariances,Rotation Group Equivariant Convolutions(RGEC) is proposed to acquire both translation and rotation group equivariances. However,previous works paid more attention to the amount of parameters and usually ignored other resource costs. In this paper,we construct our networks without introducing extra resource costs. Specifically,a convolution kernel is rotated to different orientations for feature extractions of multiple channels,meanwhile,much fewer kernels than previous works are used to ensure that the output channel does not increase. To further enhance the orthogonality of kernels in different orientations,we construct the non-maximum-suppression loss on the rotation dimension to suppress the other directions except the most activated one. Considering that the low-level features benefit more from the rotational symmetry,we only Share Weights in the Shallow Layers (SWSL) via RGEC. Extensive experiments on multiple datasets (i.e.,ImageNet,CIFAR,and MNIST) demonstrate that SWSL can effectively benefit from the higher-degree weight sharing and improve the performances of various networks including plain and ResNet architectures. Meanwhile,the convolutional kernels and parameters are much fewer (e.g.,75%,87.5% fewer) in the shallow layers and no extra computation costs are introduced. |
语种 | 英语 |
源URL | [http://ir.ia.ac.cn/handle/173211/50895] |
专题 | 类脑智能研究中心_神经计算及脑机交互 |
通讯作者 | Huiguang He |
推荐引用方式 GB/T 7714 | Zhiqiang Chen,Ting-Bing Xu,Jinpeng Li,et al. Sharing Weights in Shallow Layers via Rotation Group Equivariant Convolutions[J]. Machine Intelligence Research,2022,19(2):115-126. |
APA | Zhiqiang Chen,Ting-Bing Xu,Jinpeng Li,&Huiguang He.(2022).Sharing Weights in Shallow Layers via Rotation Group Equivariant Convolutions.Machine Intelligence Research,19(2),115-126. |
MLA | Zhiqiang Chen,et al."Sharing Weights in Shallow Layers via Rotation Group Equivariant Convolutions".Machine Intelligence Research 19.2(2022):115-126. |
入库方式: OAI收割
来源:自动化研究所
其他版本
除非特别说明,本系统中所有内容都受版权保护,并保留所有权利。