Data Representation has become the fundamental task in computer vision and machine learning community. Feature learning, especially via deep leaning, has abandoned the traditional pipeline (feature extractor and classifier) of solving computer vison problems by building a hierarchical architecture in which representation and following induction, prediction and recognition could be trained together. Although Feature learning, especially under the structures of DBN and CNN, has made a huge breakthrough in both academic and industrial community, there are still some problems need to be solved. First, few researches have taken advantage of memory which are very important for human vision system. Second, there does not exsit united framework to deal with all transformations. Last, huge amount of data are needed for training and the reason why deep models work remains unsolved. On the contrary, human vision system could easily handle these problems with a high speed. With the help of memory, we could recognize the target which may contain all kinds of transformations withonly a few samples. Recently, researchers on human vision system ultilize new technology
such as Microelectrode recording technique to analyze the firing rate of one
specific cell, which tremendously improved the research on the functional properties
of retina and visual cortex area. These achievements provided reseachers in computer
vison community with new direction to solve the above problems.
Illumination adaptation and shape recognition are two essential tasks of human
vision system, and illumination change as well as shape variation are two main obstacles
in computer vision society. Hence, we took the advantage of some achievements
on retina and cortex and made several improvements on illumination and shape invariance.
Our main contributions are illustrated as follows:
We proposed a locally nonlinear tone mapping algorithm. Inspired by the
principle of our vision system (more sensitive to changes in dark region), we proposed
a new locally nonlinear tone mapping algorithm to compress high dynamic range image
into low dynamic range image so that the illumination problem caused by the nonlinear
mapping of digital response function and quantization loss from analogue signals to
digital codes could be solved. Our model is based on physiological experiments (
Weber-Fechner states that subjective sensation is proportional to the logarithm of the
stimulus intensity) and coincides with the mechanism of retina. When solving our
model, we first estimate two guided images according to the physical explaination of
the parameters in our model. Then we adopt these two images as two constraints for
the final close-from solutions. Our model has solved the distortion problem of the
local linear model and achieved halo-free results which are common draws in filterbased
approaches. In addition, our model has linear complexity. Both subjective and
objective evaluations have demonstrated the effectiveness of our model.
We proposed a retina based real-time tone mapping framework for HDR
image and video. Inspired by the work mechanism of photoreceptor and horizontal
cell，we gradually simplify tone mapping as an estimation of a mapping matrix and
then provide some rules to estimate this mapping matrix according to its physical interpretation.
Our framework achieves similiar scores in the subjective and objective
evaluations with the state-of-the-art methods while speeds up about one hundred to
one thousand times. Meanwhile, with the help of the adaptation mechanism of our vision
system，we proposed a real-time tone mapping framework for HDR video. Compared
with the mainstream approaches, our framework could achieve high contrast and
ghost/flickering-free results with real-time speed.
We built a hierarchical network for general transformation such as affine
transformation, out-plain rotation and background variation Prof. Tomaso Poggio
and his team explored how our memeory affect on information processing in human
Ventral Stream and then proposed a united framework, M-theory, to get invariance of
general transformation, which provide a new direction for solving the above problems.
However, the shallow network the M-theory will result in a problem that the memory
dataset will increase exponentially when the categories of invariance grow up. Accordingly, we build a hierarchical network to solve this drawback. Experiments indicated that M-theory could be built on the top of other handcrafted descriptors or learning
features to get better performance.
We expanded the application of M-theory to non-geometric transformations.
The current applications of M-theory are limited to geometry transformations such as
affine transformation and out-plain rotation, we expand it to non-geometry transformations.
We theoretically proved that we could get robust invariant features via M-theory
when the transformation can be expressed as a convolution with a linear symmetric1
filter, such as Gaussian-blurring transformation. Then we demonstrated the correctness
of our theory in face identification under blur and degraded circumstances. High
performance could be achieved even if we adopt only random images, such as noise dot
image, as human memory. Compared with state-of-the-art methods, we could achieve
much higher accuracy, especially under severe circumstances.