Graph Neural Networks

Graph neural networks

Graph neural networks (GNNs) with an irregular structure that describe an object with a set of features and additional information about connections are a logical continuation of the development of artificial intelligence software technology, showing effectiveness in recommendation systems, recognition of language, text, and visual content.

Graph theory appeared in 1736. When mathematician Leonhard Euler became interested in the problem of 7 bridges in Konigsberg. The condition is to cross all the bridges without crossing any of them twice. For clarity and simplification, route drawings were used. Based on which the answer was received - it is impossible to set a route with the main condition. While working on the problem, the scientist deduced several principles that have become fundamental.

Later it turned out that any object can be represented by a system of points and vectors. Such a theory could not go unnoticed when developing machine learning methods and creating AI algorithms.

Large corporations have already become interested in the development. For example, LinkedIn has created an open source Performance Adaptive Sampling Strategy that improves the accuracy of recommendations in social networks.

What are graph neural networks, how they are used, and how they are trained, let’s figure it out.

What is it

Graph neural networks are built on graphs. In mathematics, it is an abstract representation consisting of vertices (nodes) and faces (vectors). They look like a molecular diagram of a drug, a simple roadmap or a diagram connecting the owner’s account with subscribers, friends on social networks.

Graphs can be:

• simple – without parallel vectors, loops;
• complex – with a comprehensive description of each element;
• static – with non-changing representations of elements;
• dynamic – with amortized query processing.

To adapt such objects into a classical stable architecture, algorithms for vectorization or transformation of the information contained in the faces into numerical values are needed. Programmers use ready-made Python Spektral library tools.

They work by analogy with convolutional neurons, and if simplified, they are a generalization of this type of AI. That is, layered processing of input data, with parallel collection from neighbors, transformation and updating of values at the vertex.

But since convolution is more of a concept, without clear rules of construction, different frameworks are used in the implementation, depending on the accompanying conditions. For example:

• local – image recognition, search personalization tailored to the interests of users;
• global – approximation, tasks at the intersection of computer science and natural sciences, for predicting the properties of substances, their effects and compatibility, and tolerability in the creation of new medicines.

Or understanding the process of turning sand into a glassy state. Neurons emulate the movement of water and sand particles, visually recreating the dynamics of their interaction. As for training, the gradient descent method is usually used, based on minimizing the value of the error function. It looks like a graph in which you need to find a point with a zero (approximate to zero) derivative.

Despite the promising approach, it also has its drawbacks, for example, over-smoothing. When, during iterations (running through a mathematical algorithm), the representations of nodes become, if not identical, then similar, which strongly affects the correctness of predictions.

You can fix it by changing the ratio of the number of layers – reducing aggregation, increasing features. The architecture of graph neural networks of the GRU type is used, which is capable of “remembering” previous data when converting current data.

In addition, deep machine learning requires significant computing resources, which may limit mass commercial use due to profitability indicators.

Differences

While the principle of GNN is similar to convolutional, it has its differences:

• the principle of storing data in vertices and faces simultaneously, separately;
• heterogeneity of information – discrete, real values;
• undemanding to the same size of images;
• dynamism;
• high precision;
• accelerated processing;
• combinatorial optimization in banking and finance;
• consideration of the context during interpretation.

The last point gives Graph Neural Network a significant advantage over other models. Because GNNs better “understand” the relationship between objects.

To make it clearer, here is an example. On any booking service, participants exchange photos, reviews, and contacts. Other AI systems take into account only apartment owners and tenants, ignoring additional parameters. This simplifies implementation, but reduces productivity, since contextual information is taken into account - locations, groups of participants associated by a specific attribute (landlord, guests).

When implementing the Graph Neural Network, you can get better navigation for site visitors and improve the user experience. It will be possible to track the relationship - location-owner-tenants-reviews independently, without contacting users.

Magic Leap has offered a SuperGlue product that can match video files and create a three-dimensional image, recognize specific places, and build maps. Such a solution can be integrated into SLAM approaches (navigation systems, including in unmanned vehicles).

As for accuracy, convolutions in modeling during the covid pandemic showed results close to reality, better than dynamic ones that treat objects as homogeneous, without taking into account additional attributes.

Kinds

Graph neurons can be recurrent, spatial, and spectral convolutional. The difference is in the type of architecture and the distribution of weights. The simplest options consist of 2 layers.

Ready-made solutions for the transformation of structures:

• SCN is a topology–based convolution developed by Yann Lecun for pattern recognition;
• LightGCN is easier to implement, using a weighted investment amount;
• NGCF – works on the principle of collaborative filtering;
• GAT – with additional layers, a self-attention mechanism, with indirect resolution of the independent element-by-element distribution of weights.

The latter “learns” longer, but shows more accurate results. The explanation for this is that in convolutions, neighboring nodes have the same weights, and in GAT, each connection can be assigned (or allowed by AI to do so) a weighting factor.

This is noticeable even visually. If you assign your own shade to the vertex and try to schematically sketch the architecture, say, with only colored dots, the Graph Attention Network will look structured, with homogeneous areas of the same color and will not resemble a chaotic cluster of multicolored dots.

How they are used

Graph Neural Network is good at text recognition, computer vision technologies, and recommendation algorithms. In social networks and on marketplaces, when the model analyzes previous queries and displays cards of already viewed products.

However, they have limitations in terms of volume and geographical features. But China’s Alibaba has launched an AI service that clusters millions of customers and products at the same time. Developers with manual writing of code, add-ons, selection of structure, parameters would not be able to cope with such a volume. But the programmers of the platform have created the Aligraph pipeline, which builds a structure of 400 links in 4-5 minutes.

The introduction to the article on the Cornell University resource says that Fabula AI has managed to create a program that will have important practical applications in the near future. AI analyzes diverse parameters by identifying fake news in social networks - profile, frequency of visits, content, distribution. The comparison of factors and the use of available areas allows you to track a fake within 2 hours after publication and reposts. Considering that scientists’ concerns about AI are related to the flow of fakes and the difficulty in identifying outright lies, this tool will help filter out content and prevent it from spreading.

The advanced MDSCAN is a powerful tool in adaptation, visualization of underwater photography, with good performance and optimal image processing speed, which help in scientific research.

GNN is even used to create automation services for monitoring and analyzing the processes of the Large Hadron Collider. Graph visualization (built by Russian scientists) tracks network routes connecting computing centers and registers network anomalies.

The plans include a technology for clustering log files. For understanding, the software performs 2 million operations per day, at least 10% of which are completed with errors. If you put it in numbers, it’s hundreds of thousands of text messages. It probably doesn’t need to be said how impossible this is when checking manually. There may also be problems with automatic verification, since text is not a very convenient format. The essence of clustering is the separation of single patterns and anomalies from messages that have not been encountered before.

The same anomaly detection method is implemented in online banking software to analyze user behavior, evaluate transactions and identify atypical transactions, and protect against fraudsters. When requesting a transfer of money to unknown accounts, the transaction is automatically blocked until the client confirms it.

To get acquainted with AI, you can use ready-made solutions. For example, to test neurobots with integrated AI on the site, which are suitable for creative work.:

• text generation;
• working with images;
• creating art.

The models are constantly updated and are available in VK, messengers, or browser extensions. At the same time, you can choose a paid and free tariff with a limit on requests.

Results

This is a promising area, even taking into account the difficulties. The emergence of frameworks simplifies development and increases the attractiveness of the technology for optimization-combinatorial tasks. Google’s research has shown that platforms combining multiple approaches and machine learning methods improve performance, accuracy, and system performance, while simultaneously increasing profitability in commercial implementation.