Different from other deep scalable architecturebased
NAS approaches, Broad Neural Architecture Search
(BNAS) proposes a broad scalable architecture which consists
of convolution and enhancement blocks, dubbed Broad Convolutional
Neural Network (BCNN), as the search space for
amazing efficiency improvement. BCNN reuses the topologies
of cells in the convolution block so that BNAS can employ
few cells for efficient search. Moreover, multi-scale feature
fusion and knowledge embedding are proposed to improve the
performance of BCNN with shallow topology. However, BNAS
suffers some drawbacks: 1) insufficient representation diversity
for feature fusion and enhancement and 2) time consumption of
knowledge embedding design by human experts. In this paper,
we propose Stacked BNAS, whose search space is a developed
broad scalable architecture named Stacked BCNN, with better
performance than BNAS. On the one hand, Stacked BCNN
treats mini BCNN as a basic block to preserve comprehensive
representation and deliver powerful feature extraction ability.
For multi-scale feature enhancement, each mini BCNN feeds
the outputs of deep and broad cells to the enhancement cell.
For multi-scale feature fusion, each mini BCNN feeds the
outputs of deep, broad and enhancement cells to the output
node. On the other hand, we propose Knowledge Embedding
Search (KES) to learn appropriate knowledge embeddings in a
differentiable way. Moreover, the basic unit of KES is an overparameterized
knowledge embedding module that consists of all
possible candidate knowledge embeddings. Experimental results
show that 1) Stacked BNAS obtains better performance than
BNAS-v2 on both CIFAR-10 and ImageNet, 2) the proposed KES
algorithm contributes to reducing the parameters of the learned
architecture with satisfactory performance, and 3) Stacked BNAS
delivers a state-of-the-art efficiency of 0.02 GPU days.