Associate Professor Li Jianfeng proposed a brand new framework of concept representation, in collaboration with Professor Chen Jeff Z.Y. from the University of Waterloo, making a breakthrough in the interdisciplinary area of deep learning and physics. Li is a member of Zhang Hongdong’s research team at the Department of Macromolecular Science, Fudan University. Their paper “Structural Prediction and Inverse Design by a Strongly Correlated Neural Network” was published on Physical Review Letters (123, 108002).
Concepts, usually words and physical entities, need to be represented fully and appropriately before deep learning and neural networks can be applied to the study of natural language and other areas. To do that, researchers have resorted either to “one-hot encoding” or to “vector representation”, while both methods could describe the features of the concept, they failed to reflect its function or the influences of the environment on its properties.
“I’ve been always wondering whether it is true that the brain expresses a concept or a physical entity simply with a static vector? Not necessarily, I believe. For instance, the concept of ‘bicycle’ should have the ability to ‘construct’ its image or other features of the concept. It also has something to do with the environment, as its image varies under different lights. Therefore, we want to make improvement by representing a concept with operators instead of vectors,” Li Jianfeng said.
The teams thus decided to express concepts in the form of operators, allowing each sub-neural-network to represent a physical entity. The input can be seen as the influence of the environment in which the physical entity is located, and the output of the network represents the physical attributes exhibited by the physical entity in the environment.
The environment is then determined by a combination of physical properties of other entities existing in the environment. Through such self-consistent loop, strong correlations are built between the entity’s physical properties and the environment, which largely improves the accuracy of neural network prediction. The proposed framework is applied to and works well in the structural prediction and inverse design of the HP model. In addition, the team also finds that the new method particularly suitable for solving problems involving degeneracy.
