In this paper, we propose BNAS-v2 to further
improve the efficiency of BNAS, which employs Broad Convolutional
Neural Network (BCNN) as the search space. In BNAS, the
single-path sampling-updating strategy of an over-parameterized
BCNN leads to terrible unfair training issue, which restricts the
efficiency improvement. To mitigate the unfair training issue,
we employ a continuous relaxation strategy to optimize all
paths of the over-parameterized BCNN simultaneously. However,
continuous relaxation leads to a performance collapse issue
that leads to unsatisfactory performance of the learned BCNN.
For that, we propose the Confident Learning Rate (CLR), and
introduce the combination of partial channel connections and
edge normalization. Experimental results show that 1) BNAS-v2
delivers state-of-the-art search efficiency on both CIFAR-10 (0.05
GPU days, which is 4 faster than BNAS) and ImageNet (0.19
GPU days) with better or competitive performance; 2) the above
two solutions are effective alleviating the performance collapse
issue; and 3) BNAS-v2 achieves powerful generalization ability
on multiple transfer tasks, e.g., MNIST, FashionMNIST, NORB
and SVHN.