Epistemic Game Theory : Putting Algorithms to Work

The aim of this study is to construct an epistemic model in which each rational choice under common belief in rationality is supplemented by a type which expresses such a belief. In practice, the finding of type depends on manual solution approach with some mathematical operations in scope of the theory. This approach becomes less convenient with the growth of the size of the game. To solve this difficulty, a linear programming model is constructed for two-player, static and non-cooperative games to find the type that is supporting that player’s rational choice is optimal under common belief in rationality and maximizing the utility of the game. Since the optimal choice would only be made from rational choices, it is first necessary to eliminate all strictly dominated choices. In real life, the games are usually large sized. Therefore, the elimination process should be performed in a computer environment. Since software related to game theory was mostly prepared with a result-oriented approach for some types of games, it was necessary to develop software to execute the iterated elimination method. With this regard, a program has been developed that determines the choices that are strictly dominated by pure and randomized choices in two-player games. Two functions named “esdc” and “type” are created by using R statistical programming language for the operations performed in both parts, and these functions are added to the content of an R package after its creation with the name EpistemicGameTheory.


Introduction
In order to evaluate the possible results of the decision, it is very important to constitute a belief about opponents' feasible preferences which can affect their choices.In addition to this, the precondition of making a good choice requires having a reasonable belief about opponents' choices.However, in general each belief of players may not be reasonable according to their opponents.The player should determine the opponent's possible idea about his opponent with putting himself into his opponent's shoes, to decide which choice is reasonable for his opponent or which choice would not be preferred by him.In other words, before surmising an idea about opponents' choices, it is compulsory to reason their system of thought.Indeed, Oskar Morgenstern who is one of the earliest founders of game theory, has highlighted this subject in his article "Perfect Foresight and Economic Equilibrium" which was published in 1935.In his article, Morgenstern has explained the significance of having idea about beliefs of the opponents, analyzing the opponents' systems of thought properly and establishing a reasonable relation to make a good decision (Morgenstern, 1935).However, the importance of this concept frequently has been underestimated in the studies on game theory, which have been published in last sixty years.Morgenstern's bold idea of using the tools of formal logic to talk about how members of a social system think, about how they think about what other members think, and so on, was far ahead of its time.However now, in the form of epistemic game theory, it has found a home (Brandenburger, 2010).
The discipline that studies these patterns of reasoning, and how they influence the eventual choices of the players, is called epistemic game theory (Perea, 2012). 1pproximately twenty-five years ago, conceptual changes have emerged with the introduction of epistemic game theory.This new branch of science has brought game theory back to its fundamental concepts, its background.In other words, it has brought game theory back to reasonable modeling of players' beliefs about their opponents.At the core of epistemic game theory, there is the fact that people have different tendencies to reason under same circumstances in a game.Therefore, it is not true reasoning in a unique way and claiming that it is the best option.Under such conditions, it can be said that there are only different reasoning ways, and it should be avoided that claiming on which one is better.In epistemic approach, the aim is to define the methods of reasoning, which can be used in the game, and to examine how the method affects the result of the game.

Belief about opponents' choices
The belief of a player about choices of his opponents is a probability distribution which is defined over the set where C i is the set of player i's choices.The probability value which is assigned by player i for his opponents' each choice combination, is obtained by b i (c 1 , . . ., c i−1 , c i+1 , . . ., c n ).
Let us symbolize the utility function of player i with u i and the belief about the choices of his opponents with b i .Accordingly, expected utility of player i from choosing the choice c i is calculated by the equation below.

Belief hierarchies and type
The concept of belief hierarchies is the basic element of epistemic approach.In an n-player game, the belief hierarchies for player i is as follows: 1.The belief that player i has about his opponents' choice-combinations, 2. The belief that player i has about the beliefs that his opponents have about their opponents' choice-combinations, 3. The belief that player i has about the beliefs that his opponents have about the beliefs that their opponents have about the other players' choice-combinations, and so on, ad infinitum.The first belief for player i is his first-order belief, the second belief is his second-order belief, and so on.
The belief hierarchies have some disadvantages both theoretically and practically.In theory, it is difficult to make a mathematical description of the hierarchy.In practice, it is often impossible to write and express each stage of the infinite hierarchy (first-order belief, second-order belief, etc.).For this reason, an approach that can express the hierarchy in a shorter and more formal way is needed.
The concept of infinite belief hierarchy has brought to game theory by John Harsanyi.He has worked on incomplete information games, also called Bayesian games, where players have incomplete information about the parameters of the game.It is aimed to model the beliefs of each player about the unknown parameters of the game, each player's beliefs about the other players' beliefs about these parameters, and so on ad infinitum.This may be called the explicit approach and is in fact feasible.However, the explicit approach is mathematically rather cumbersome and hardly manageable.Indeed, this was a major obstacle to the development of the theory of games with incomplete information at its early stages.The breakthrough was provided by John Harsanyi in a seminal work (Harsanyi, 1982) that awarded him the Nobel Prize in 1994 after thirty years.While Harsanyi actually formulated the problem verbally, in an explicit way, he suggested a solution that 'avoided' the difficulty of having to deal with infinite hierarchies of beliefs, by providing a much more workable implicit, encapsulated model (Zamir, 2013).
The concept of a type is the basis of the Harsanyi model.As a concept, type can be considered as a precise definition of players' belief hierarchies about unknown parameters of the game.It is a special representation of the player's beliefs about the actual parameters involved and the answers to the types of other players.This characteristic of the types gives the player the ability to self-reference inevitably that is, the ability to decide the types of other players through their own types in an interactive decision-making environment.
The construction that Harsanyi proposed in the context of a game with incomplete information on the preferences of the players was very simple: For every player, define a set of types, and for every type define a utility function, together with a probabilistic belief about the opponents' types (Harsanyi, 1982).
The content of the type in epistemic game theory differs from that of Harsanyi.While Harsanyi forms belief hierarchies for parameters of the game, epistemic game theory deals with the choices of the players.In the studies (Armbruster and Boge, 1979) and (Boge and Eisele, 1979), belief hierarchies have been used to describe the beliefs of players about their opponents' choices.
Let us denote the belief hierarchy with t c i which supports player i's any choice c.In the literature, belief hierarchy is also used as epistemic type (or briefly type).Therefore, t i represents an epistemic type of the player i.

Epistemic model
Let us consider an n-player game.The epistemic model, firstly, indicates possible types for each player.Let us symbolize the set of all possible types for every player i with T i .Each type t i stores information about the beliefs on the choices of player i's opponents and their types.The problem that arises here is how this belief can be expressed mathematically.
As it is known, every player i's belief about choices of his opponents is a probability distribution b i which has been defined on the set should have information about not only the belief about choices of his opponents but also information about his opponents' types.Thus, t i represents the choice-type combinations of player i's opponents.
The set that contains all possible choice-type combinations of any opponent j of player i is C j × T j .In parallel with this definition, the set of all choice-type combinations of player i's opponents' becomes An epistemic model specifies probability distribution b i (t i ) defined on the set of all possible choice-type combinations for every player i and each type t i ∈ T i .The probability distribution of b i (t i ) represents the belief that type t i has about the opponents' choices and types.
Consider a type t i for player i within an epistemic model.The choice c i is rational for type t i if it maximizes the expected utility for the belief that t i holds about the opponents' choice-type combinations.
The entire belief hierarchy can be expressed with an epistemic model.Constructing an epistemic model is easier than establishing a belief hierarchy.This is an important advantage of epistemic model.Another advantage of the epistemic model is that it can be defined by a mathematical expression conveniently.

Deciding Under Common Belief In Rationality
Type t i is said to believe in the opponents' rationality if for every opponent j, and every choice-type pair (c j , t j ) ∈ C j × T j to which t i assigns positive probability, the choice c j is rational for type t j .
In order to define common belief in rationality, firstly, the definition of k-fold belief in rationality is needed and explained as follows.

k-fold belief in rationality
Consider an epistemic model.
1. Type t i expresses 1-fold belief in rationality if t i believes in the opponents' rationality.
2. Type t i expresses 2-fold belief in rationality if t i only assigns positive probability to the opponents' types that express 1-fold belief in rationality.
3. Type t i expresses 3-fold belief in rationality if t i only assigns positive probability to the opponents' types that express 2-fold belief in rationality.
And so on.Thus, k-fold belief in rationality can be recursively defined for every number k.

Rational choice under belief in the opponents' rationality
The choice c * 1 is the optimal choice for player i, if the expected utility of the choice i is optimal with reference to player i's belief about the opponents' behavior patterns, then c * i is called as a rational choice.Player i believes that his opponents are rational if he assigns positive probability for only his opponent's rational choices.
If the choice c i of player i is optimal for some belief b i about the choices of his opponents' while believing in the opponents' rationality, then the choice c i is a rational choice for player i under belief in the opponents' rationality.
The concept of rational choice in game theory sometimes causes confusion.In the context of this study, the word "rational" is used as follows: If a player has built a belief about his opponent's choices, and has made the optimal choice for himself under this belief, he has made a rational choice.However, The R Journal Vol.10/1, July 2018 ISSN 2073-4859 this player may have had an unreasonable belief about the opponent, and making rational choice does not guarantee making the reasonable choice.Reasonability carries a subjective meaning and depends on the mindset of the person.Something that is reasonable for one may not be reasonable for another.Therefore, it is impossible to make a single definition of reasonable choice.
The reasonable choice should not only be rational under the belief in the opponents' rationality, but at the same time, it should be optimal according to a reasonable belief about the opponents' choices.What is open to debate is when does a belief about the opponent be reasonable?Epistemic game theory examines the answer to this question.

Rational choice under common belief in rationality
In an epistemic model, if the type t i expresses k-fold belief in rationality for every k, it can be said that t i expresses common belief in rationality.If t i expresses common belief in rationality and the choice c i is optimal for the type t i , then c i is a rational choice under common belief in rationality.
Common belief in rationality does not only mean that you believe that your opponents choose rationally, but you also believe that your opponents believe that their opponents will choose rationally, and that your opponents believe that their opponents believe that the other players will choose rationally, and so on.
Consider an epistemic model, which contains type sets T 1 , . . ., T n .If the player i's choice c i is not supported to express common belief in rationality by any type in T i , this does not mean that the choice c i cannot be chosen under common belief in rationality.There may be another epistemic model that has a type supporting the choice c i to express common belief in rationality.The purpose of this work is to seek an answer to the question that how this epistemic model can be detected.
Algorithm 1 (Choices that can be rationally chosen under common belief in rationality): An algorithm has been required to use in order to find the choices that can be rationally chosen under common belief in rationality.Algorithm 1 is based on the following theorem.
Theorem 1: A choice c i is irrational if and only if it is strictly dominated by another pure or randomized choice.In other words, a choice c i is rational if and only if it cannot be strictly dominated by another pure or randomized choice2 (Pearce, 1984).
By Theorem 1, the algorithm is explained as follows.
1) Eliminate all strictly dominated choices in the original game.
2) Eliminate all strictly dominated choices in the reduced game obtained after the step (1).
3) Eliminate all strictly dominated choices in the reduced game obtained after the step (2).
. . .Repeat this process until no strategy can be eliminated.
This algorithm ends with a finite number of steps and gives a set of choices that are not empty for each player if the game is finite.The order of elimination and speed do not affect the result.
Theorem 2: (Brandenburger and Dekel, 1987) and (Tan and Werlang, 1988) proved that, if the choices can be rationally made under k-fold belief in rationality for each k ≥ 1, then these choices are also survived (k + 1)-fold elimination.This can be generalized as; the rational choices under common belief in rationality are the choices that survive the iterated elimination of strictly dominated choices.

Algorithms For Finding Types That Express Common Belief In Rationality For Optimal Choices
In practice, the finding of type depends on non-computer based approach with some mathematical operations in scope of the theory.This approach becomes less convenient with the growth of the size of the game.For this reason, we construct a linear programming model for two-player, static and non-cooperative games to find the type that is supporting that player i's rational choice c i is optimal under common belief in rationality and maximizing the utility of the game.Since the optimal choice would only be made from rational choices, it is first necessary to eliminate all strictly dominated choices.By the reason of software related to game theory was mostly prepared with a result-oriented approach for some solution methods and some types of games, it was necessary to develop software to execute the iterated elimination method.With this regard, we developed a computer program that determines the choices that are strictly dominated by pure and randomized choices in two-player games.Başer transformed the operations performed in both parts to software by using R Statistical Programming Language and created a package with the name EpistemicGameTheory (Baser, 2017a).
The EpistemicGameTheory R package containing functions named esdc and type for both purposes explained above.The package roxygen2 was used to prepare the documentation when the R package was created (Wickham and et al, 2015).

esdc function
As it is known, since the optimal choice is made only from rational choices, it is first necessary to make iterated elimination of strictly dominated choices.For this purpose, the steps given in Algorithm 1 for two-player games need to be coded into software.We developed Algorithm 2 to make the steps of this algorithm suitable for programming architecture.
Let n be the number of choices of the first player; and m be the number of choices of the second player; the utility matrix of the first player would be; the utility matrix of the second player would be; Each entry of matrix A represents a utility level of the first player while each entry of matrix B represents a utility level of the second player.The two subscripts of these entries indicate the strategies chosen by the two players where the first subscript refers to the strategy chosen by the first player and the second subscript refers to the strategy chosen by the second player.
Algorithm 2 (Creating the Algorithm to be Used in Comparing the Choices of the First Player): 1) A combination matrix (C A ) is generated that contains all combinations of (n, (n − t)) with t = 1 at the initial point.Each row of C A shows how to compare the choices.
It is aimed to obtain a randomized choice for each row by using its elements and then compare with the utility level of the choice which does not exist in that row.For instance, let c * 1 be the randomized choice that is obtained by randomization of the choices (c 11 , c 12 , . . ., c 1(n−t) ) .Then the utility level of c * 1 is compared with the utility level of the choice that does not exist in the first row.The randomization process is made with the procedures in the following steps.
2) An index matrix (N) with rows equal to the number of rows of the matrix C A is generated.Each row of the index matrix is equal and consists of a number sequence of 1 to n.
3) A comparison is made between the rows of C A and N matrices.The elements existed in N and not in C A are assigned to the corresponding row of the difference matrix.The purpose of this step is identifying the choice for each row which is compared with the randomized choice that is obtained in step 1.
The R Journal Vol.10/1, July 2018 ISSN 2073-4859 4) For every row of C A , a probability vector P is generated from a uniform distribution on the (n − t − 1)− dimensional simplex.
According to this probability vector, the first player chooses the first choice with the probability p 1 , the second choice with the probability p 2 , ..., (n − t)th choice with the probability p (n−t) .
5) Using the obtained probability values, the expected utility for the first player as a result of choosing the randomized choice is calculated and compared with the utility provided by the choice not existed in the related row of the combination matrix.
If the first player believes that second player will choose his first strategy, the expected utility for the first player would be; If the first player believes that second player will choose his second strategy, the expected utility for the first player is calculated as; If the first player believes that second player will choose his last strategy, the expected utility for the first player is as below; If all expected utilities (E 1 , E 2 , . . ., E m ) obtained are greater than the expected utility of the choice pointed by the element in the corresponding row of matrix D, then the strategy is strictly dominated by that randomized choice.This process is made for every row of C A .
Two situations can arise here: Elimination occurs: In this case, the reduced utility matrix is obtained.The new C(n * , (n * − t)) combination matrix is created (where n * is the number of rows of the reduced utility matrix) and the above steps are repeated.
Elimination does not occur: In this case, a new probability vector is generated, and the above steps are repeated.Here it is very important to decide the number of iterations that determine how many times the probability vector will be generated and it depends on the dimension of the game.Therefore, the number of iterations must be sufficiently large to be able to determine the strictly dominated choices (iteration −→ ∞).If there is no strictly dominated choice according to the utility matrix, the value of t is increased by "1" to form a new combination matrix C(n, (n − t)) and the above steps are repeated.
6) When (n * − t) < 1 the algorithm ends and the last reduced utility matrix is obtained.The last reduced utility matrix determined by the above algorithm consists of the rational choices of the first player.The same steps are performed for the second player.The specified steps are written in the R statistical programming language, and a function named "esdc" is created.This function gives the reduced game after the iterated elimination of all strictly dominated choices.The properties of the function are shown in Figure 1

Value
This function works for the games with two players.

Figure 1: esdc Function
The esdc function uses seven arguments.These are; the choice number of the first player (n), the choice number of the second player (m), the utility matrix of the first player (A), the vector consisting of the choice names of the first player (choices.A), the utility matrix of the second player (B), the vector consisting of the choice names of the second player (choices.B), and the number of repetitions of the algorithm (iteration).As a result of the execution of this function, the reduced utility matrices of the players' that are obtained after eliminating strictly dominated choices as output.

type function
It is aimed to show that for every player i and every rational choice c i (i = 1, 2, . . ., n), there is a type t i , which supports that c i is optimal under common belief in rationality.
At the beginning of the game, a type is defined for each element in the player's choice set.However, the "type" function generates types only for rational choices because the player do not choose irrational strategies in practice.

Construction of Epistemic Model:
Let A * be the first player's reduced utility matrix.
The first player believes that the second player will likely choose the first choice with the probability q 1 , the second choice with the probability q 2 , • • • , and the final choice with the probability q m .In this case, according to the preferences of the second player, the utilities of the first player can be expressed by a linear equation system.
Under this belief, first player will get the utility U 1 , if he chooses his first strategy; he will get the utility U 2 , if he chooses his second strategy; . . . he will get the utility U i , if he chooses his ith strategy,

Application of the Traveler's Dilemma
We applied the two functions to Basu's well-known The Traveler's Dilemma (Basu, 1994) game to underscore the practical usefulness of this work.Each player has 99 strategies between 2 and 100.The utility matrices for the players are created according to the utility function of the Traveler's Dilemma game.
Applying esdc Function The arguments for esdc function are assigned as given below.For this numerical example, the "iteration" argument is taken as 500.The name of choices of both players ("choices.A" and "choices.B") are denominated by the numbers between 2 to 100.n = 99 m = 99 iteration = 500 esdc(n,m,A,choices.A,B,choices.B,iteration) The Reduced Utility Matrices With the execution of "esdc" function the algorithm eliminates all strictly dominated choices from the utility matrices A and B. For these matrices 196 times elimination occurred.The reduced utility matrices are displayed below.In the last reduced game, the players have only one choice left, which is choosing a price of "2".Therefore, they can only rationally choose a price of "2" under common belief in rationality.
[1] "ELIMINATION IS OVER." [1] "The Last Reduced Matrix For Player 1:" Applying type Function The type function is executed for finding types for rational choices under common belief in rationality given in the reduced matrices for both players.The definitions of arguments are shown below.
The R Journal Vol.10/1, July 2018 ISSN 2073-4859 A<-matrix(c(2),1,1) B<-matrix(c(2),1,1) choices.A = c("2") choices.B = c("2") type(A,B,choices.A,choices.B) The Output of type Function The output of type function includes the coefficients of the linear equation system, the types that supports relevant choice under common belief in rationality and the maximum utility of the player.For this example, the output of type function is displayed below.type(A,B,choices.A,choices.B) [1] "The utility matrix of Player 1:" 2 2 2 Player 1 s type for the strategy 2 : 1 Success: the objective function is 2 [1] "The utility matrix of Player 2:" 2 2 2 Player 2 s type for the strategy 2 : 1 Success: the objective function is 2 The epistemic model of the first player was created as follows using belief probabilities from the output of type function that make every rational choice of the first player optimal under common belief in rationality.
Type: T 1 ={t 2 1 } Belief for the first player: 2 ) The epistemic model is constructed by using the values for the second player from the output of type function as follows: Type: T 2 ={t 2 2 } Belief for the second player: b 2 (t 2 2 ) = (2, t 2 1 ) Consequently, the first player will choose his choice "2", if he believes that second player will choose "2" with the probability 1.In a similar way, the second player will choose "2", if he believes that first player will choose "2" with the probability 1 and they will end up getting two units of money each.

Conclusion
Game theory has been investigated with epistemic approaches in recent years.Theorists and practitioners do research for analyzing the logic underlying game theory in a broader and more realistic perspective.Although epistemic game theory has a strong theoretical background, there is a lack of tools to work on large-scale problems in practice.This study based on common belief in rationality which can be considered as the heart of epistemic game theory.In two-player games, we developed a systematic way to find out types that optimize their rational choices under common belief in rationality for every player together with maximizing their utility.Thus, this study brings flexibility to producing solutions of large-scale problems within the scope of epistemic game theory.The algorithms which have been used are transformed into software.R is preferred because of its open-source software feature, and an R package has been created to bring the program into use.It is thought that this study will serve as an example for future work in computational epistemic game theory field and it is planned to adapt this approach to n-person games as well.

Bibliography
strictly dominated choices.Usage esdc(n,m,A,choices.A,B,choices.B,iteration) Arguments n an integer representing the number of choices of player 1 m an integer representing the number of choices of player 2 A an nxm matrix representing the payoff matrix of player 1 choices.A a vector of length n representing the names of player 1's choices B an nxm matrix representing the payoff matrix of player 2 choices.B a vector of length m representing the names of player 2's choices iteration an integer representing the iteration number of algorithm Details This function works for the games with two players.
W. Armbruster and W. Boge.Bayesian game theory.Game theory and related topics, 17:28, 1979.[p371] B. Baser.EpistemicGameTheory: Constructing an Epistemic Model for the Games with Two Players., The R Journal Vol.10/1, July 2018 ISSN 2073-4859 Journal Vol.10/1, July 2018 ISSN 2073-4859 type Finding types that express common belief in rationality for optimal choicesDescriptionThis function takes the reduced payoff matrices and finds out the probabilities for the types that expresses common belief in rationality for optimal choices.