I'm hoping some game theorists will chime in here; it doesn't matter if you don't get macro. I need your help, and want your thoughts on my intuitions:
Not all Nash equilibria are created equal.
Game A. There are n identical players who move simultaneously. Player i chooses Si to minimise a loss function Li = (Si-Sbar)2 + (Si-Sbar)(Sbar-S*), where Sbar is defined as the mean Si over all players, and S* is a parameter that is common knowledge to all players.
This game has a unique Nash equilibrium Si = Sbar = S*.
Game B is exactly the same as game A, except the loss function is now Li = (Si-Sbar)2 + (Si-Sbar)(S*-Sbar). (I flipped the sign of the last bracketed term.)
This game also has a unique Nash equilibrium Si = Sbar = S*.
I think the Nash equilibrium in game A is plausible, but the Nash equilibrium in game B is implausible.
To help you understand why I think that, let's make an apparently trivial change to both games. Let Si be bounded from above and below, so Sl <= Si <= Su, where Sl < S* < Su.
Game A still has the unique Nash equilibrium Si = Sbar = S*.
Game B now has three Nash equilibria: the original Si = Sbar = S*; the lower bound Si = Sbar = Sl; and the upper bound Si = Sbar = Su. And I find the second and third equilibria equally plausible, and the first (interior) equilibrium very implausible.
If n is large, so that dSbar/dSi = 1/n approaches zero (individuals ignore the effect of their own choice on the average choice), the reaction functions for the two games are:
Game A: Si = S* + 0.5(Sbar-S*)
Game B: Si = S* + 1.5(Sbar-S*)
[Did I get the math right?]
The reaction functions for the two games look like this:
For game A, the green reaction function crosses the black 45 degree line (for a symmetric Nash equilibrium) only once, at S*.
For game B, the red reaction function crosses the black 45 degree line three times: at S*; at the lower bound Sl; and at the upper bound Su. That's why we get three Nash Equilibria.
I think the interior equilibrium in game B is much less plausible than the two degenerate equilibria at the upper and lower bounds.
[Update: I would not board a ferry if I knew that, if the ferry leaned starboard/port, each passenger on the ferry would want to be further to starboard/port than the average passenger.]
If this were a repeated game, I could talk about learning. It would be hard for players to learn the interior equilibrium in game B, because any mistakes they make in predicting what other players do will tend to be self-reinforcing. For example, if all players expect Sbar to be S* + epsilon, the actual Sbar will be S* + 1.5epsilon.
But even in a one-shot game I do not think the interior equilibrium in game B is plausible.
I was trying to figure out what would happen if the n players moved sequentially (observing previous players' moves before making their own move), and each player had a small probability of making a totally random move anywhere between Sl and Su (trembling hand). The math was too hard for me, but I think it would make the interior equilibrium in game B very improbable, if n was large. Am I right?
(The two games here are highly stylised versions of a New Keynesian macroeconomic model. The players are price setting firms, Si is the amount by which firm i chooses to raise its price, so Sbar is the economy-wide inflation rate. Assume for simplicity the natural real rate of interest is 0%.
In game B, S* is the nominal interest rate set by the central bank.
In game A, S* is the central bank's inflation target, and there is a prior stage in the game where the central bank announces that it will set the nominal interest rate in accordance with the Howitt/Taylor principle, after observing Sbar.
The upper and lower bounds on inflation represent the idea that if inflation gets too high or too low the central bank will change the game, by adopting QE, for example.
Strictly speaking, I should add a third term like (S*-Sbar – m)2 to the loss function, where m is the degree of monopoly power, to represent the losses from monopoly power in a New Keynesian model. But I ignored it for simplicity, since it doesn't affect my point here.)
Worthwhile Canadian Initiative 
Nick, I’ve played around with this and the closest I can get to it is something like this:
There are N players from 1,…,N. Each player makes an optimal choice based on previous choices and anticipating the choices of those who come after them. e(i) is the error made by player i, s(i) is their choice. The optimal strategy of player N-t is then
s(N-t) = a(N-t) Sum_i{1,N-t-1} (s(i)+e(i)) + b(N-t) s*
where a(N-t) and b(N-t) are ratios of polynomials in N. a(N-t) is a t+1 degree polynomial divided by a t degree polynomial – but there’s also a sum of N-t-2 e’s and s’s there so effectively you have something that’s t+1 degree over t+1 degree. b(N-t) is t degree over t degree. So if you take limits there with respect to N, they’d go to a constant.
So it’s like an ARMA process except the coefficients depend on N and t. You solve that backwards you get something like
s(N-t) = Product_i{1,N-t-1} (1+c(i)) e(i) + b(N-t) s*
where c(i) is a polynomial ratio of the same degrees as the a’s. Note the 1 in the parentheses. This means that all the errors get amplified by subsequent choices. It’s an explosive process.
The one weird thing, which I mentioned above, is that keep getting that lim b(N-t) as N goes to inf is 1/2 not 1. But maybe I’m dropping something somewhere.
Nick, we may be talking past each other a bit here so let me back up and run down what we have here as I see it.
1. In the OP you have a simultaneous move game which is fairly simple in which the NE are the three that you mentioned in the post as well as your last comment.
2. What you describe in the comments is a different game in which firms move sequentially. This is a significant difference from the original game described in the post. In this game you need to find a sub-game perfect NE in which players predict the actions of future players. Note also, that you can’t really use a traditional game tree since the strategy space is continuous. However, this game is probably manageable and they equilibria you describe probably are equilibria when there are no errors.
3. Whether they are THP or not requires you to solve a perturbed game in which they make mistakes with some probability. This is the game that will be difficult to solve. It might not be prohibitively difficult but off the top of my head, it’s not clear that it isn’t. Then when you shrink the probability of error to zero, if there is an equilibrium in the perturbed game that approaches the equilibrium in the base game, that equilibrium is THP. (Note that it is THP if there is ANY way to collapse the distribution of errors to zero which causes the equilibrium to approach the proposed equilibrium.)
Not also that the equilibrium to the sequential move game with errors will specify a contingent plan of action for each player dependent on sbar when it is time for them to act. This plan may be different for each firm depending on their position. This means that the equilibrium path of sbar may not be deterministic (it may be a function of the errors). You need to distinguish between a tendency of sbar to drift toward s* or away from s* as the game progresses from the tendency for the equilibrium strategies to converge to the proposed equilibrium strategies in the base game as the probabilities for error approach zero. I suspect (though I may be wrong) that the former is more along the lines of what you have in mind. The latter is what determines THP. I suspect (and this is very speculative) that if you can solve the perturbed game, all three equilibria in the base game will end up being THP. It will probably be somewhat difficult to solve though.
P.S. This may seem nitpicky but it’s usually worth the trouble of being specific about things like this in these game theory conversations. The equilibria in the sequential move game (with no errors) wouldn’t be s*, sl, and sm. They would be something like: play s* if sbar’=s*, play sl if sbar'<s*, play sh if sbar’>s* for all firms after the first where sbar’ is the average s up to that point and then play either s*, sl, or sm for the first firm. (I’m doing this very casually, so I may not be getting the equilibria right but the idea is that you have to specify a complete plan of action for each firm contingent on the information they have at the time.) This likely means that the first firm will determine the actual path (probably a constant level) of s and so if one level is preferable to the first firm, they will be able to select that level. This will actually probably rule out s* as a solution to this game all together.
Mike: “1. In the OP you have a simultaneous move game which is fairly simple in which the NE are the three that you mentioned in the post as well as your last comment.
2. What you describe in the comments is a different game in which firms move sequentially. This is a significant difference from the original game described in the post. In this game you need to find a sub-game perfect NE in which players predict the actions of future players. Note also, that you can’t really use a traditional game tree since the strategy space is continuous. However, this game is probably manageable and they equilibria you describe probably are equilibria when there are no errors.”
Agreed. Except, in the OP, I did also consider the sequential variant on the simultaneous game (see the bolded paragraph in the OP). And on thinking it over, in the comments, I came to the conclusion that the sequential variant is probably the more promising game from which to explore the question of THP. (Plus, the sequential variant is closer to the NK macro model, so it’s more interesting for that reason too.)
“You need to distinguish between a tendency of sbar to drift toward s* or away from s* as the game progresses from the tendency for the equilibrium strategies to converge to the proposed equilibrium strategies in the base game as the probabilities for error approach zero. I suspect (though I may be wrong) that the former is more along the lines of what you have in mind. The latter is what determines THP.”
Good point. Agreed. I think I was trying to do the latter, though I may not have been clear on this.
Mike: “This likely means that the first firm will determine the actual path (probably a constant level) of s and so if one level is preferable to the first firm, they will be able to select that level.”
Hmmm. Good point.
notsneaky: The way you are setting up the problem sounds right to me. I’m not following the math, but that’s my fault.
Could you take a look at my September 15 09.26 am comment please, where I make my own attempt at the math. Mine looks simpler, (if it’s right).
Hold on, some stuff coming up…
Sept 15 09.26 looks right (for big N), although I think it’s important to separate out the choice made by a person, s(i), from the outcome, say r(i), which is the sum of the choice and the error r(i)=s(i)+e(i). So that should be almost right, except that person i+1 will take into account the past errors that have been made (all the e(j) for j < i) but not the future errors that will be made (assuming those are mean zero). So their choice will be s* + weighted average of past s and future s + weighted past errors. Hence a person further down in the “queue” will have more past realized errors to deal with.
Ugh, inequality cut off comment again.
…all the e(j) for j less than i. So actual optimal choice is something like s=(a1)s*+(a2)(past s + past e). a1 and a2 are coefficients. The reason why “future s” do not appear in there is because they can be solved out (i.e. they’re there we just iterate until they’re gone).
Nick, I get what you are saying, I’m just trying to be clear about the various versions we are kicking around here, I’m not trying to play gotcha regarding who said what when. My main point is just that you need to be pretty careful about extending the equilibrium from the simultaneous move version to the sequential move version. Things can change a lot and when you try to add errors and do THP, it gets complicated pretty quickly. Ultimately I don’t think THP is the concept you need to make your point.
Then actually since we know that if nobody ever makes any errors, the optimal choice is in fact s(i)=s* for all i, we got to have a2=(1/(i-1))(1-a1). Those a1’s are functions (rational functions) of N and i. We can also figure out that the very first person to move will choose s(1)=s. This is because no errors have been made yet and all future errors are zero in expectation, hence s* is the best they can do in expectation. But after that the realized errors start accumulating…
Ok. Let me give it another stab. Correct some things and be less sloppy.
First, as Mike says this has nothing to do with a Trembling Hand Equilibrium, nor does it have anything to do with some kind of special new equilibrium. It’s just a regular sequential game with uncertainty in it. It’s just a boring Subgame Perfect Nash equilibrium (in expectation). If you draw the game tree out then you have Player 1 make a move, then a player called “Nature” makes a move, then Player 2 makes a move, then Nature makes a move, then Player 3 makes a move and so on. But because we want to consider the case where the number of players, N, is large, computationally it’s a big pain. Actually since it involves ratios of polynomials and since each player anticipates future moves by others, the order of these polynomials increases quite quickly so even in low-N case, like 4, it’s messy.
Second, the question of whether s(i)=s* is an equilibrium is ill posed. A strategy here is a contingent plan. “If nature chooses e(1), e(2), …, e(i-1), and previous players choose s(1), s(2),…,s(i-1), then I will choose s(i)”. If nature chooses different e’s and players choose different s’s I will choose a different s(i). Etc.” So is s(i)=s* an equilibrium? Of course not, unless all errors that have already happened are zero.
So we have to rephrase the question in a way that makes sense. For example, “before the game begins is the expected choice of person i, Es(i)=s“? (Actually yes if errors are mean zero). Or even “after person 1 has made a move and nature has chosen the first error, is the expected choice of person i, for i>1, Es(i)=s? If not, given e(1), how much does it deviate from s? Does this deviation increase with i and N? Does the variance of s(i) increase? Will the errors cancel out in s(i) for big enough i and N? Given that people make errors can we expect that the choices of s(i) will stay within some neighborhood of s? Or will they – even if the errors themselves are bounded – diverge further and further out? Etc.
Ok, math next. But I’ll post that later so as not to flood the comment thread.
notsneaky, you’re getting closer to the right approach here but this part is a little off:
“before the game begins is the expected choice of person i, Es(i)=s“? (Actually yes if errors are mean zero). Or even “after person 1 has made a move and nature has chosen the first error, is the expected choice of person i, for i>1, Es(i)=s? If not, given e(1), how much does it deviate from s? Does this deviation increase with i and N? Does the variance of s(i) increase? Will the errors cancel out in s(i) for big enough i and N? Given that people make errors can we expect that the choices of s(i) will stay within some neighborhood of s? Or will they – even if the errors themselves are bounded – diverge further and further out? Etc.”
You need to start at the end and ask “what is the last firm’s optimal behavior, contingent on sbar at that point? Then you have to go one person forward. This will be kind of annoying because they need to anticipate the action of the last person being what you found as a result of the first question. However, this will be probabilistic if there are errors. So you have to figure out the expected behavior by the last person given any choice by the second to last player and then find the optimal strategy by the second to last player for every value of sbar up to that point. Now go forward one more person and do this again. But, of course, every time you go one person forward, figuring out the probabilities of everything that could happen in the future will become more complicated. It won’t take long for this approach to become unwieldy. If you really want to solve it, I would do it with two people. Then try doing it with three people. At point you may notice a pattern that you can expand to a n-person game. If you don’t figure out such a pattern, a brute-force approach with a large n will probably be pretty difficult.
I skipped this part because I forgot that it’s not necessarily obvious but when you get through all of the last n-1 actors, you figure out what value of s maximizes the first guy’s expected payoff (or minimizes their loss function or whatever) and then everything follows from that. This is why you can’t start with “before the game begins is the expected choice of player 1……whatever” The behavior of player 1 is not random, it’s determinate, (the error is random but that’s something else) you just have to figure out what everyone else will do in every situation in order to actually determine it.
First some notation since it’s hard to write math in a blog comment. N players. s_{N-t} is the choice made by player who moves t periods before the last person. So s_{N} is the last person to move, s_{N-1} is the next to last person … up to s_{1}. Likewise e_{N-t} is the error made by the person who moves t periods before the last one. Then define A_{N-t} as the sum of all s_{i} for i up to N-t, and likewise, V_{N-t} the sum of all e_{i} for i up to N-t. I’m going to use capital S instead of s*, so that I can use * as a multiplication sign. a_{N-t}, b_{N-t} and c_{N-t} (lower case) are going to be coefficients which depend on N and t.
We solve it by backward induction. First we minimize the loss function of the Nth person who gets to observe all the previous choices and errors. The optimal choice is
s_{N}=((3N-4)(A_{N-t-1}+V_{N-t-1})-N(N-1)S)/(2(N-1)(N-2))
Person N does anticipate that they will make an error, e_{N}, but if the error is mean zero and uncorrelated with previous errors, then this will not affect their choice.
So we have s_{N} as a function of s_{N-1}, s_{N-2}, …, s_{1}, and also the errors and S. We take this s_{N} and plug it into player N-1’s loss function and maximize that with respect to s_{N-1}, making sure not to forget the effect of s_{N-1} on s_{N}, which is (3N-4)/(2(N-1)(N-2)). This is where it’s start getting messy but basically we get s_{N-1} as a function of s_{N-2}, s_{N-3}, … , s{1} as well as all the errors up to N-2 and S. To fully solve it we would keep doing this but the algebra gets very convoluted. However we can figure out what kind of functions the coefficients on all the lags, for s’s, e’s and S, are. For s_{N-1} we have
s_{N-1}=(a_{N-1}(A_{N-2}+V_{N-2})-(b_{N-1})S
We can check that if all the past errors are zero then s_{N}=s_{N-1}=S, so we’re on the right track.
a_{N-1} and b_{N-1} are ratios of polynomials in N. Let’s leave them alone for now.
More generally, for s_{N-t} we have
s_{N-t}=(a_{N-t})(A_{N-t-1}+V_{N-t-1})-(b_{N-t})S
So it’s like an ARMA(N-t,N-t) process with time varying coefficients. If we take limits of a_{N-t} and b_{N-t} as N goes to infinity they converge to some constants (possibly 0), which means that for large N we could treat this as just a regular ARMA process. Anyway, iterating backwards we get
s_{N-t}=[(a_{N-t}Product_i{1,N-t-1} (1+a_{N-t-i})]s_{1}-{big coefficient}S+MA_{N-t}
where we don’t have to worry about {big coefficient} too much for reasons explained right below and MA is the moving average term that has all the errors from 1 to N-t-1 in it.
So we have s_{N-t} as a function of N, t, all the errors up to N-t-1 and the first choice made, s_(1). But we know that if all the errors are zero then the optimal choice is always S, so we have to have
s_{N-t}=[big mess]s_{1}-[big coefficient]S=S
Since this has to hold for t=N-t-1, and there are no errors before 1, we know that s{1}=S. So [big coefficient]=[(a_{N-t}Product_i{1,N-t-1} (1+a_{N-t-i})]-1.
Okay, let f(N,t)=[(a_{N-t}Product_i{1,N-t-1} (1+a_{N-t-i})]. So we have so far
s_{N-t}=f(N,t)s(1)+(1-f(N,t))S+MA=S+MA
So we’re reduced the choice of player to just the value S and all the errors that have been made before they make a move.
What is this MA term? Again, iterate to get
MA_{N-t}=(a_{N-t})(e_{N-t-1})+(Sum_i{2,N-t-1} of c_{N-t-i}e_{N-t-1})
That still leaves the coefficients in the sum, c_{N-t-i}. These are given by
c_{N-t-i}=(a_{N-t})*Product_i{j,i-1} (1+a_{N-t-j})
Like I said, it’s a big mess. But that’s pretty much the solution right there.
Now we can take expectations. In particular note that if we take expectation “before the game starts” then Es_{i}=S. The errors are unpredictable and mean zero so that’s not surprising. If somehow every player had to commit to a choice before observing errors and other’s choices, they’d choose S.
The more interesting question is what happens as N goes to infinity to the coefficients and their product, and what happens to the variance of s_{N-t} as t approaches N. Still working on that…
Let me try to clarify/re-state/modify my own view, in the light of Mike’s and notsneaky’s comments:
Assumptions: sequential moves game; the number of players N is very large, so each player ignores the effect of his S on Sbar; each player has a reaction function S = S* + bE(Sbar-S); Nature moves only once, at some time T in the sequence of moves, adding a mean-zero shock e to Sbar (“Nature trembles player T’s hand”).
For 0 < b < 1 the equilibrium is straightforward. All the players who move before Nature moves play S=S, and all the players who move after Nature moves play S=S+b(Sbar-S) where Sbar=S* + [(N-T)/N][1/(1-b)]e. Notice that Sbar approaches S* in the limit as e approaches 0. (So I want to say that S* is a THP equilibrium for 0 < b < 1.)
But, as b approaches 1, [1/(1-b)] approaches infinity, so we can no longer say that Sbar approaches S* in the limit as e approaches 0.
And, if b > 1, it is not true that the equilibrium Sbar is well-defined and approaches S* in the limit as the variance of e approaches zero. (So I want to say that S* is not a THP equilibrium for b > 1.)
Remember to put a space either side of < , to stop Typepad having conniptions.
Mike, yes, that what I did in the last comment. In the comment you responded to I was essentially skipping ahead to the solution. After we iterate backwards (or ‘forward’) as you say, each person’s optimal strategy involves reacting to the choice of the first person (the person who moves first, not the person whose optimization problem we solve first) and all the errors that have been made up to that point. But for person 1, no errors have been made yet so their best choice, in expectation, is to just choose s*.
Nick, the difficulty with saying “N is large” before solving the game – as opposed to solving the game, then letting N be large – is that even though one’s person’s choice will not affect the average in that period, a forward looking player will realize that their choice will affect the choice of everyone who moves after them. Like in a grocery store queue, if I put an extra item in my basket that increases not just my time in the queue but also the time in queue for everyone standing behind me.
So even though my choice only changes the average today by something like constant/N, it changes it N-i times (everyone who’s still going to move) overall, so the total effect is something like constant*(N-i)/N, which is no longer infinitesimal even as N goes to infinity.
Also – “All the players who move before Nature moves play S=S*” – just call all those players who move before Nature moves “Player 1” and let Nature move 2nd.
This is also a special case of the game I wrote out above where we just set all but one of the errors equal to 0
One more comment for now (sorry, I’m having fun with this).
Mike, the case N=2 won’t work because then any s is an optimal strategy. I think someone pointed this out above. You need at least 3 players, maybe 4 (since first player’s choice is going to be just s*)
Nick, let me see if I understand how you’re posing the question. Let’s say player 1 moves, then Nature chooses an error, then everyone else moves with no more errors. The question you’re asking is “if we go forward to player t’s choice will that error made t periods before be amplified or dampened?”
It seems like there’s two questions here. One is, what happens to the choice of s as we move “far into the queue”. That is, given a fixed N, how much will the choice of person N-t diverge from s given that there was an error made earlier on. This is basically asking what happens as t — > 0.
The other question is what happens as we let N go to infinity. It’s sort of confusing to try and do both things at once.
Actually come to think of it, having only one error at time T (say, 2) is not going to work here for you Nick. With N large, just like any one player’s choice will have a negligible impact on Sbar, so will any single error.
Player 1 moves first and chooses s*. Nature moves and chooses e. Now it’s player 2’s turn. Suppose player 2 thinks everyone after them will choose s*. The actual average is s+e/N which is approximately s. So player 2 ignores e/N and plays s*. Then players 3,… reason the same way. You still have everyone playing s* if there’s only one error. EVEN IF that error is NOT arbitrarily small (say, bounded away from zero)
What you need is that everyone can make an error so that their (weighted) average can have a chance of being non-negligible.
Nick,
I’m now pretty sure that I was on target when I said this:
“You need to distinguish between a tendency of sbar to drift toward s* or away from s* as the game progresses from the tendency for the equilibrium strategies to converge to the proposed equilibrium strategies in the base game as the probabilities for error approach zero. I suspect (though I may be wrong) that the former is more along the lines of what you have in mind. The latter is what determines THP.”
Your latest version imposes reaction functions on the players (rather than finding optimal reaction functions) so once you do that, you’re not really talking about a Nash Equilibrium any more, you’re just saying “if everyone acted this way, this is what would happen.” You’re not really saying whether or not it makes sense for them to act that way. This means you can’t really do a THP analysis. Such an analysis depends on whether the strategies converge to the proposed equilibrium strategies as the probability of error decreases. You have now made the strategies exogenous so this question becomes nonsensical.
This being said, I think I get the point you are trying to make (at least in a limited way, I’m not sure I see the connection to macro) but I think you just need to say “see in this case sbar approaches s* over time and in this other case, it is likely to wander off to one side or the other.” My advice is to forget about THP, I don’t think it is the droid you are looking for. But if you just say something like that, I suspect it will make more sense to more people (including economists) anyway.
Mike: “You need to distinguish between a tendency of sbar to drift toward s* or away from s* as the game progresses from the tendency for the equilibrium strategies to converge to the proposed equilibrium strategies in the base game as the probabilities for error approach zero. I suspect (though I may be wrong) that the former is more along the lines of what you have in mind. The latter is what determines THP.”
I’ve got that point. I thought I was doing the latter. At least in the case where b < 1, S jumps up or down when Nature makes a move, but then stays constant for all remaining players.
But I think you may be on target in your next paragraph (beginning “Your latest version…”
Point taken.
notsneaky: “Nick, the difficulty with saying “N is large” before solving the game…”
I was trying to simplify, by assuming each player is small compared to the macroeconomy, but I also recognise the problem, that as the equilibrium of the remaining players gets very sensitive to one tremble, even a very small player could affect the average, by affecting everyone else’s play.
I think one – better? – way to formulate it would be something like this. Fix N. N-t is the person who moves t periods from last. Every player can make an error.
Examine the game in expectations. In other words, suppose you’re asked to forecast person N-t’s strategy before the game actually starts. Your best forecast IS in fact s*, for any t. All the errors are unpredictable, mean zero, and uncorrelated so that is your best guess. BUT the variance of the strategies will increase as t — > 0, at least in the Game B version. So effectively the “standard error” on your forecast s_{N-t}=s* gets larger.
Another way. Before the game starts, ask “what is the probability that [s_{N-t}-s] will be within +/- delta of zero?”. I.e. what is P(-delta < [s_{N-t}-s] < delta). Then see what happens to that probability as t gets close to N.
[s_{N-t}-s] is a random variable with mean zero (since the errors are mean zero and uncorrelated) and some variance which depends on N and t, say sigma(N,t). Assume the distribution of e is symmetric. The probability then is 2F(delta/sigma(N,t))-1. Now let N be “large” and t approach N. The variance sigma(N,t) will increase so F(.) will decrease so the probability that you are still “delta-close” to s* goes to zero. Basically the interval over which you’re integrating that pdf gets smaller and smaller the further out you go.
This is not true in the version A of the game. I think…
Since we have come this far, I will try briefly to make my point about distinguishing between a convergence of s to s* and a convergence of strategies in the perturbed game to strategies in the base game. The key point to notice is that the s played by a player is not their strategy, it is an action. The strategy is a plan of action stating which s they will play in every possible contingency.
So take your base, sequential game (from the bold paragraph and subsequent comments) with no errors. An equilibrium MAY consist of strategies like: play s* for sbar’=s*, play s1 for sbar’ > s* and play s2 for sbar’ < s* where sbar’ is the average s up to that point for every player after the first and then the first player chooses s*. (As I said, I don’t think that will be an equilibrium given either of the objective functions proposed in the OP but let’s just pretend it is). Now when this game is played, everyone will play s* and sbar at all times will be s*
If you add an error into this game and everyone plays the same strategies sbar will shoot off to one side. But the strategies don’t change, only the actions change. If these strategies are an equilibrium to the perturbed game then they are a THP equilibrium in the base game even though the path of sbar is likely to be different in the perturbed game, that’s not the point. (I believe that IS your point, but it is not the point of THP.)
notsneaky: I like that. It looks promising.
“Now let N be “large” and t approach N. The variance sigma(N,t) will increase so F(.) will decrease so the probability that you are still “delta-close” to s* goes to zero.”
I think that’s right. Do we know it’s right?
“Basically the interval over which you’re integrating that pdf gets smaller and smaller the further out you go.”
Is that a re-statement of the previous sentence? Or a conjecture? Or a reason to believe the previous sentence?
“This is not true in the version A of the game. I think…”
I think so too. In game A, the effects of the mean zero independent errors should tend to cancel out as N gets large.
Mike: OK. I think I follow you now.
If we can show that, in the Neo-Fisherite game (Game B), if the players make arbitrarily small independent errors, the outcome will almost certainly be an arbitrarily large distance away from the Neo-Fisherite equilibrium, (for large N), that would be worth showing. It means we can ignore the Neo-Fisherite equilibrium.
Plus, it would maybe be of more general interest in game theory? It would be applicable to any game with an “unstable” (in the old-fashioned sense) equilibrium.
If my algebra is correct then yes we know that. At least for game B. Look at the “MA” term in my post above:
MA_{N-t}=(a_{N-t})(e_{N-t-1})+(Sum_i{2,N-t-1} of c_{N-t-i}e_{N-t-1})
This is how previous errors affect person N-t’s choice. Square that thing and take expectation to get variance of s_{N-t}-s*. Take derivative with respect to t. I’m pretty sure it works for any N>3 but obviously if there are only few players then the errors won’t have many opportunities to get amplified.
The part I’m not as sure about is what happens in Game A but my intuition is same as yours.
Nick, yes, I think we are basically simpatico at this point. I do think it’s possible that there is some interesting broader game theory application here. I’ve been trying to think of a way to eliminate the original suspect equilibrium in the simultaneous move game (the sequential move game is too complicated to work with for this purpose I think). So far I haven’t made a breakthrough. In the back of my mind is a strong sense that we are not the first people to consider this issue and so the concept we are looking for is probably pretty elusive. Still, it’s hard not to ponder.
Mike, mixed strategies? Then make it into an evolutionary game with types surviving or not?
Mike and notsneaky: we seem to be converging 😉
The concept we are exploring might(?) be called robustness/fragility. For large N: S* is a “robust” equilibrium in Game A; and a “fragile” equilibrium in Game B. Suitable names?
I think this is analogous to the mixed strategy equilibrium in coordination games. For example, suppose two risk-neutral players each choose left or right, and get a payoff of 1 if they match and 0 if they do not match. As a simultaneous move game there are three Nash equilibria: (left,left), (right,right), and (left with p=1/2, left with p=1/2). In the first two equilibria are ‘stable’ in the sense that neither player would want to change their move if they knew the other was going to deviate by a small amount. The interior mixed strategy equilibrium does not have this property: Each player is only willing to play left with probability 1/2 if they know that the other player will also play left with probability exactly 1/2. A tiny bit higher, and the best response is left; a tiny bit lower and the best response is right.
Nick, these types of equilibria have always struck me as implausible as well. It is hard to see how real people would find their way to this knife edge balance and stay there. I have always assumed there must be a well known equilibrium refinement which eliminates them, but I haven’t seen it yet. I’m no game theorist though – I hope someone on this thread has definitive answer.
Brad: I agree. And I think this is important.
Brad,
OK, I kind of agree, and I feel like I am basically the game theory guy on this thread so I consider this a pretty informed answer but it will be far from definitive. As far as I know, there is no such refinement (that’s not definitive but it’s fairly well informed). But I’m not sure the equilibrium you want to rule out is any more implausible than the other two. If you did an experiment where you put two people in separate rooms and made them choose left or right and told them the payouts you described, you would probably get about half the people choosing left and half choosing right (and if you got more one way or the other, it would probably either be due to a random sampling error or to some kind of mental or social bias or something but you wouldn’t get everyone chooses right or everyone chooses left. This is because there is no way to coordinate on one or the other and that is a feature of the game as it’s defined. If there is some way to coordinate, then it is a different game. For instance if you have people declare what they are going to do in one stage and then observe what the other one declares and then choose, that’s a different game. Or if two people choose and then two more people observe what the first two chose and then choose and that goes on forever, that’s a different game. There are many different variations of this game which are kind of similar but significantly different in their structure that have significantly different equilibria. If you actually had to play that game, neither one of the pure strategies would be any “better” in any sense than the equilibrium mixed strategy.
So if you are using a simplified game to represent a more complex situation, then that’s not the equilibrium concept that is flawed, it’s the setup of the game. With all of that said, there may be some concept which could be useful in all of this, but we need to be careful about how to define it. The most obvious way to rule out that equilibrium is to just rule out every mixed strategy NE. Every (maybe not every but nearly every) mixed strategy NE has the characteristics that annoy you about that one (that if the other player changed slightly, one player would not want to mix any more). But hopefully I don’t need to argue that this is too extreme an approach. (What would you do with matching pennies? And for that matter, you it wouldn’t even give you a better grasp of the game you propose for reasons I mention above.) It’s a pretty delicate web. It’s difficult to go chopping things out without causing problems elsewhere.
Ultimatley I think one of the most difficult concepts to get across to people about game theory is that it doesn’t always tell you what will happen. We all know this in theory but you have to constantly remind yourself when you’re doing it.
Mike: true. But if we converted Brad’s game into a sequential moves game, I think the 50:50 mixed strategy equilibrium is a lot less plausible than the other two.
Mike,
Thanks for your response. You have a good point about putting people in a room and having them play the one-shot game as written. The problem is that Nash Equilibria in general are not good predictors in this situation – it’s only after people have played the game for a dozen or so rounds that they generally converge on something that looks like an equilibrium. My concern is that they are unlikely to converge on an equilibrium like the one above (I have only my intuition to go on here – I’m going to have to do a little digging in the experimental literature). And as Nick points out, that equilibrium still exists in the repeated version of a coordination game.
Still, you have convinced me that part of the problem is using the simultaneous game as a model for more complicated dynamic situations. Unfortunately, people do that all the time (economists and non-economists alike). Perhaps it’s because repeated games are so complicated, and often subject to “anything goes” folk theorems. People are trying to get the right intuition by thinking about a simpler simultaneous move game. Unfortunately, I think these types of equilibria might give exactly the wrong intuition (or formal predictions). That’s an untested hypothesis, but that is my concern.
It’s also not just about mixed strategy vs pure strategy. The mixed strategy in a game of Rock-Paper-Scissors (or whatever you call your local cultural variant) is unique and very compelling. Here the simultaneous game gives a good intuition for what you will find in long-term repeated play.
-Brad
One way to reformulate the result above, “theorem-like” would be:
Given a delta > 0 (a neighborhood), and a p, 1 le p > 0, (a required probability), in Game B, there always exists an N and a t < N, such that the probability of player N-t playing (s* +/- delta) is less than p.
In other words, if you require that every player plays “close enough” to s* with at least some given probability, I can always find you a player in a “long enough” game who violates that requirement.
This could be taken as a definition of some notion of “non-robustness” in sequential game with uncertainty, although it depends on N.
Ideally, it’d be better to have a definition of “robustness” rather than “non-robustness” but the converse of “I can always find” is not “I can never find” but “I can sometimes not find”. For example, in Game A it may be the case that such a player can be found for some delta but not another (this also mirrors the assumption made in New Keynesian models which assume that the adjustment paths always stay “in a neighborhood” of the steady state no matter what).
Also in Game A, while intuition suggests, as Nick notes, that the errors will cancel out, so the variance of strategies will not explode, this isn’t obvious. That’s because errors made by different players get weighted differently, with earlier errors being amplified – and weighted – more.
Nick, if you make it sequential move game, (where they take turns moving) I’m sure that equilibrium will go away. If you make it a repeated game like Brad suggests (where they move at the same time but do it over and over again) it is probably still there but it’s also still not obvious to me that it is an implausible equilibrium. It would be interesting to do an experiment like I suggested where people play the game repeatedly. They have to start out by guessing. My suspicion is that most of the time, once they got a match, they would keep playing that pure strategy from then on. If they both played a strategy like “randomize until there is a match and then choose that action until the other guy deviates, then randomize again” it would also be a NE. It’s not obvious that people would always do this though. And there isn’t a clear (to me) set of criteria for ruling out any of these equilibria. All of the complaints basically come down to some version of “well if the other person changed their strategy, then it wouldn’t make sense to do that any more” but that’s usually the case with NE. If you want to find a concept, it has to say something very specific about the WAY the optimal strategies react to changes by other players. It’s sort of a tricky business.
And yeah, like I said, one thing we all agree on is that NE is not always predictive. We just have to keep reminding ourselves.
BTW, note that the repeated game has all kinds of unconvincing equilibria. For instance: Both players alternate L,R,L,R no matter what happens. No reason to believe that people should behave that way. But if they did, neither one of them could be better off by unilaterally changing.
That equilibrium doesn’t go away in a sequential game.
the mixed strategy equilibrium does go away. If the second player observes the firs player’s move, they will choose the same thing with probability 1.
Sorry, I thought we were still talking about the s* game.
Noah: “But some economists would reject that use of the word `unstable’. I am trying to restate that objection in another way.”
Stick to “unstable”. Equilibrium is either stable or unstable (occasionally neutral). As for those economists who reject this use of the word “stable”, they are fools. The only question is whether they can be redeemed (educable) or they are irredeemable, in which case the project at hand is to discredit their competence and get them out of economics.
John: (You mean Nick, not Noah.)
“Equilibrium” means “what the model says will happen”.
“Stable/unstable” traditionally means “will/will not return to equilibrium if hit with a temporary shock”.
But a prediction of what will happen if hit with a temporary shock is itself the equilibrium of another model.
So you get into an infinite regress.
For example, whether an equilibrium is “stable/unstable” in the old-fashioned sense may depend on how players update their expectations of the other player’s future actions. But then you are implicitly talking about the dynamic equilibrium path of a new model which incorporates assumptions about learning. And whether those assumptions about learning do or do not make sense depends on the information available to the players, etc.
“Robustness/fragility” asks instead “will the equilibrium change by a very large amount if the assumptions change by a very small amount?”
The critique of the traditional sense of “stable/unstable” is a valid critique. But the correct way to respond to that critique is to try to reform the traditional sense, not to ignore the valid core of the traditional sense, nor to ostracise the critics.
@Nick, not sure I follow your last comment. Perhaps I am not familiar with what economists mean these concepts?
Equilibrium is defined in the context of a single dynamic model. Stable/unstable means whether the dynamic model predicts a return or a divergence from the original state when there is a small shock. This is not in the context of a new model, and what will happen is not typically referred to as an equilibrium state — eg in most systems, you will get a path that describes oscillations around the original state if it was a stable equilibrium state. It is generally considered that unstable equilibria are implausible states in which to find a dynamic system, because the system can exist in that state only if it is very carefully prepared. This need not have anything to do with whether the actual equilibrium state is sensitive to assumptions or not.
For example, consider a simple pendulum made of a rigid rod. It has an unstable equilibrium when the rod is pointing straight up. This is an implausible state in which to find this system, and this is not because it is sensitive to any assumptions. No matter what the mass of the rod or its length, this unstable equilibrium looks pretty similar.
Robustness/fragility the way you are describing them, sound more like asking whether the dynamic system is chaotic or not. This does not have anything to do with plausibility. It is a property of the system. The existence of states which are not only stable equilibria but have the additional property that the system converges to them eventually, no matter where it starts from, can be a very useful feature for predictability. But if the system lacks such states, that does not make it any less plausible as a model, it just makes the behavior of the system more complex (and arguably more interesting to study).
Nick,
I it’s in quotes but I missed who you are quoting and I’m not sure if you are agreeing with it or not but just for the record I think “what the model says will happen” is not a good definition of equilibrium. I would say something like “a state in which, if attained, will persist.” Of course, the application to a one-shot game probably requires a bit more nuance but that’s the general idea. Obviously, when there are multiple equilibria, the model doesn’t tell us “what will happen” only the various states in which, if you were there, no force within the model would tend to push you away from that spot. What you seem to want to do is add a force which pushes you away a little and distinguish between an equilibrium for which there is a force in the model which will tend to PUSH YOU BACK and those for which there is a force which will tend to push you farther away. A worthwhile goal but tricky because most NE hang on a knife edge in one way or another. For instance, if you take the matching pennies game, which has only one NE it is still the case that if one players strategy deviates by a tiny amount, the other player’s optimal strategy swings all the way to one extreme. Yet, I don’t think we want to eliminate this equilibrium.
John,
That is terrifying, I hope it’s a comedic bit….
Mike: I made up that “quote”; but I think it’s an accurate reflection of what many economists think, and I lean towards thinking it myself.
” I would say something like “a state in which, if attained, will persist.”” But, in a model with a unique equilibrium, doesn’t the model say it will pertain and persist? Which means it’s the same as “what the model says will happen”. And if there are multiple equilibria, what prevents jumps from one to another?
nive: there is a danger in pushing the physics/engineering analogy. Because people have expectations. What happens depends on what they expect to happen.
The inverted pendulum is an unstable equilibrium, and also “fragile”, in the sense that a small gust of wind (like a trembling hand) will cause a large deviation.
@Nick, how people form expectations and what they do as a result is what the model is supposed to be modeling, no?
“What the model says will happen” is how the system will evolve from a given initial state. In a model with a unique equilibrium (assuming it’s stable, which if it’s unique is “likely”), as well as some analog of “friction”, as well as the absence of exogenous factors, the system will eventually settle down in the equilibrium state. So you need a few more assumptions before equilibrium is “what the model says will happen”.
All unstable equilibria are fragile in that sense — that’s the definition of unstable. But there are other equilibria that are stable, but are still subject to large deviations from small effects, and we do not view them as implausible as a result. For example a spinning top, when it comes to rest will be fallen over on the floor and pointing in some direction. For a real top, the final direction will be very sensitive to exactly how it was set spinning, any imperfections in the floor on which it’s spinning etc. But we do not get surprised when we see a top lying on the floor pointing in some particular direction. We would be extremely surprised, however, to see a top in its unstable equilibrium, at rest but standing upright.
OK, here is a simple two person non-zero sum game.
Each player chooses a number from {0, 1, 2}. The payoff for each player is 0, except in four cases. If the player chooses 1 and the other player chooses 0 or 2 the payoff for the player is -1. If one player chooses 0 and the other player chooses 2, the payoff is -2.
Here is my question. What does each player expect the other player to do?