Yifan Xu – CSCC46 2020 Course Blog

Explain GAN and Triple-GAN from the Perspective of Game Theory

Generative adversarial network (GAN) is an exciting recent innovation in machine learning. “Given a training set, this technique learns to generate new data with the same statistics as the training set. For example, a GAN trained on photographs can generate new photographs that look at least superficially authentic to human observers, having many realistic characteristics.” (Wiki)

The algorithm of GAN can be understood as a “Minimax Zero-Sum Non-Cooperative Game” that two neural networks, generative network, and discriminative network, are contesting against each other in the game. The generative model is trained to produce authentic images to fool the discriminator, and the discriminator is trained to distinguish between fake images produced by the generative model and the real images.

Researchers found that it is difficult to train GAN as the two networks cannot reach the optimal at the same time. This phenomenon can be explained by why it is difficult to find the Nash Equilibrium using gradient descent.

Consider a Minimax game with two players A and B, which control the value of x and y, respectively. Player A wants to maximize the value xy while B wants to minimize it. Analytically, we know that the equilibrium reaches when x=0 or y=0.

However, if we update the parameter x and y based on the gradient of the value function V, from figure 5, we find x and y are oscillating around 0 and do not converge. Hence, gradient descent has flaws to find the Nash equilibrium.

Researchers have also found it is often the case that the discriminator can reach near-optimal, and the generator is unable to model the distribution of the true data. Triple-GAN was proposed to improve the performance of the generator by introducing a third player, classifier. The utilities of the generator and discriminator differ from the ones in GAN slightly. The generator and the classifier characterize the conditional distributions between images and labels, and the discriminator solely focuses on identifying fake image-label pairs. The authors of the paper proved that when the class conditional distribution between the classifier and the generator becomes close, the generator and classifier can nearly model the true data distribution. Hence, Triple-GAN introduces a term R_L that penalizes the loss function if the class conditional distribution between the classifier and the generator diverges too much.

Now, take a look at how Triple-GAN has reshaped the dynamics of the game. The generator and classifier are trained to fool the discriminator, and the discriminator is trained to distinguish fake image-label pairs. But this time, the cooperative characteristic is introduced to the game. As mentioned in the paragraph above, the loss function is penalized if the class conditional distribution between classifier and generator diverges too much. In other words, the classifier and generator lose points if they have the same class distribution. And thanks to the cooperation with the classifier , the generator is able to choose a better strategy for itself and can model the true distribution data more closely.

References:

1: https://arxiv.org/pdf/1703.02291.pdf

2:https://arxiv.org/pdf/1406.2661.pdf

3:https://jonathan-hui.medium.com/gan-why-it-is-so-hard-to-train-generative-advisory-networks-819a86b3750b

4:https://en.wikipedia.org/wiki/Generative_adversarial_network

Strong Ties Versus Weak Ties in Word-of-Mouth Marketing

Before talking about the paper, let me ask you a question: Have you ever formed a hobby because of your close friends? For example, I started doing sports, playing League of Legends, using Gmail, and so on because the classmates around me were doing so.

Weak ties are more potent in many real-life applications because weak ties can disseminate brand new information more effectively while the information circulating within strong-tie relationships is often redundant.

In the paper, Strong ties versus weak ties in word-of-mouth marketing, the authors suggest that weak ties are overestimated because of its structural advantages over strong ties. There is growing evidence to show that the closed triadic relationship is a more predominant factor that promotes consumers to purchase products. This paper compares the importance of three types of marketing activities: advertising, word-of-mouth of strong ties, and word-of-mouth of weak ties. And it is mainly focusing on comparing the two interpersonal ones.

The researchers conducted a simulation called agent-based modeling (ABM) to prove their hypothesis that strong ties have more aggregate impact than weak ties on the spread of word-of-mouth, which promotes sales. And the figure above is the bar graph of this experiment, that divides the process into 5-stages: 0- 2.5%, 2.5-16%, 16-50%, 50-84%, and 84-100% of the market being reached, where a, b, c, r represents the impact of advertising, strong-tie word-of-mouth, weak-tie word-of-mouth, and r is the ratio between b and c. This graph shows that advertising plays a vital role in driving product growth at the initial stage (0-2.5%). Starting from the second phase, word-of-mouth of strong ties becomes the primary driver of the product growth while advertising experiences a dramatic decrease in promoting sales. Although the impact of weak-tie word-of-mouth is increasing, it is still much less influential than strong-tie word-of-mouth.

This experiment challenges the widely-accepted idea of “weak-tie versus strong-tie,” but it is limited to interpersonal ties. The authors argue that the importance of weak-tie word-of-mouth is sabotaged heavily by firms that advertise their products because advertising and weak interpersonal ties share the same ability to disseminate information globally, but advertising is more powerful than weak-tie word-of-mouth at spreading messages. The writers also bring in the findings from other papers, which suggests that word-of-mouth is seven times more effective than newspapers and magazines, four times more effective than personal selling, and twice as effective as radio advertising in influencing consumers to switch brands. Combining with the truism that people are more likely to buy a product because of the introduction of close friends than the introduction of acquaintances, it is obvious to see that strong-ties can play an essential role in marketing.

References:

https://www.sciencedirect.com/science/article/pii/S2340943618304535#:~:text=Strong%20ties%20have%20a%20greater%20aggregate%20impact%20than,on%20the%20spread%20of%20WOM.&text=As%20the%20level%20of%20advertising,weak%20ties%20becomes%20more%20pronounced.