I'm hoping some game theorists will chime in here; it doesn't matter if you don't get macro. I need your help, and want your thoughts on my intuitions:
Not all Nash equilibria are created equal.
Game A. There are n identical players who move simultaneously. Player i chooses Si to minimise a loss function Li = (Si-Sbar)2 + (Si-Sbar)(Sbar-S*), where Sbar is defined as the mean Si over all players, and S* is a parameter that is common knowledge to all players.
This game has a unique Nash equilibrium Si = Sbar = S*.
Game B is exactly the same as game A, except the loss function is now Li = (Si-Sbar)2 + (Si-Sbar)(S*-Sbar). (I flipped the sign of the last bracketed term.)
This game also has a unique Nash equilibrium Si = Sbar = S*.
I think the Nash equilibrium in game A is plausible, but the Nash equilibrium in game B is implausible.
To help you understand why I think that, let's make an apparently trivial change to both games. Let Si be bounded from above and below, so Sl <= Si <= Su, where Sl < S* < Su.
Game A still has the unique Nash equilibrium Si = Sbar = S*.
Game B now has three Nash equilibria: the original Si = Sbar = S*; the lower bound Si = Sbar = Sl; and the upper bound Si = Sbar = Su. And I find the second and third equilibria equally plausible, and the first (interior) equilibrium very implausible.
If n is large, so that dSbar/dSi = 1/n approaches zero (individuals ignore the effect of their own choice on the average choice), the reaction functions for the two games are:
Game A: Si = S* + 0.5(Sbar-S*)
Game B: Si = S* + 1.5(Sbar-S*)
[Did I get the math right?]
The reaction functions for the two games look like this:
For game A, the green reaction function crosses the black 45 degree line (for a symmetric Nash equilibrium) only once, at S*.
For game B, the red reaction function crosses the black 45 degree line three times: at S*; at the lower bound Sl; and at the upper bound Su. That's why we get three Nash Equilibria.
I think the interior equilibrium in game B is much less plausible than the two degenerate equilibria at the upper and lower bounds.
[Update: I would not board a ferry if I knew that, if the ferry leaned starboard/port, each passenger on the ferry would want to be further to starboard/port than the average passenger.]
If this were a repeated game, I could talk about learning. It would be hard for players to learn the interior equilibrium in game B, because any mistakes they make in predicting what other players do will tend to be self-reinforcing. For example, if all players expect Sbar to be S* + epsilon, the actual Sbar will be S* + 1.5epsilon.
But even in a one-shot game I do not think the interior equilibrium in game B is plausible.
I was trying to figure out what would happen if the n players moved sequentially (observing previous players' moves before making their own move), and each player had a small probability of making a totally random move anywhere between Sl and Su (trembling hand). The math was too hard for me, but I think it would make the interior equilibrium in game B very improbable, if n was large. Am I right?
(The two games here are highly stylised versions of a New Keynesian macroeconomic model. The players are price setting firms, Si is the amount by which firm i chooses to raise its price, so Sbar is the economy-wide inflation rate. Assume for simplicity the natural real rate of interest is 0%.
In game B, S* is the nominal interest rate set by the central bank.
In game A, S* is the central bank's inflation target, and there is a prior stage in the game where the central bank announces that it will set the nominal interest rate in accordance with the Howitt/Taylor principle, after observing Sbar.
The upper and lower bounds on inflation represent the idea that if inflation gets too high or too low the central bank will change the game, by adopting QE, for example.
Strictly speaking, I should add a third term like (S*-Sbar – m)2 to the loss function, where m is the degree of monopoly power, to represent the losses from monopoly power in a New Keynesian model. But I ignored it for simplicity, since it doesn't affect my point here.)
Worthwhile Canadian Initiative 
Think of the equilibrium-plus-one extension of the game, where one single player gets to “peek” at everyone else’s move before deciding theirs. In this case, Sbar and S* are both observed quantities, so the loss function becomes:
(Si-Sbar)^2 + (Si-Sbar)k
where k is Sbar-S.
The loss function is most negative (representing a gain) when Si-Sbar = -k/2, or Si = -k/2+Sbar = 0.5(Sbar+S)
That is, the last-mover wants to pick Si as the average of the target and everyone else’s choice. This is even equivalent to the “guess 2/3 of the average” game, where observed results approach two to three “iterations” of rationality.
Now, if k was S-Sbar, the optimum point for our last-mover is instead Si = 1.5Sbar – 0.5Si = 0.5(Sbar+Si) + (Sbar – Si) — the last-mover wants to be further from the target than the average, and it wants to be (Sbar-Si) away from the good-loss-function case.
Taking these results, we can extend them to a fully iterative game. The (Sbar-S) case results in a stable convergence to the Nash equilibrium, whereas (S-Sbar) results in a divergence away from it if Si is not bounded.
Majro: thanks for your comment. I don’t fully understand it yet (maybe just me), but it seems to be along the right lines.
(I need to set interest rates in 30 minutes, and prepare for a canoe trip, so I may not respond to comments immediately.)
I think the concept you’re looking for is “Trembling hand perfect equilibrium”, and related concepts. Basically, this is a Nash Equilibrium that is robust to vanishingly small random deviations.
An analogy I like is a boulder perched at the top of a hill vs. in a valley. Both are equilibria, but the former is not robust to small perturbations, because then the boulder rolls down the hill.
jonathan: I think you are right. I have a very rough understanding of trembling hand perfection, but can’t figure out if the interior equilibrium in game B is trembling hand perfect (with sequential moves?). I think it isn’t, but can’t prove it.
Nick: Actually, I looked at the payoff functions you wrote down, and I think you meant to write something else.
For example, for Game A, Li = (Si-Sbar)^2 + (Si-Sbar)(Sbar-S), and so *any choice of S is a nash equilibrium as long as everyone coordinates on it, because both terms will be 0 as long as Si = Sbar, which will always be true when Si = Sj for all i,j.
Wait, never mind. I was thinking that the loss function took a minimum value of 0, but in fact it can go negative (I was thrown off by its being a quadratic-looking loss function).
jonathan: phew! You are right to check my math; I so often get it wrong.
@Nick Rowe:
Majro: ” Version #1 [A] would have the firm set its price such that Sbar moves (epsilon) towards S*, Version #2 [B] would have the firm set its price such that Sbar moves away from S*.”
But Sbar is the average not just of past prices, but of future prices too, so it’s not quite that simple, unless we assume purely backward-looking expectations in the sequential-moves version of the game.
In the A game the equilibrium is stable: Suppose sbar > s*. Best response si is below the 45 degree line. Everybody plays it. So sbar moves closer to s*. New best response is still below 45 degree line. So sbar moves closer to s*. Etc., and same for sbar < s*
In the B game the middle equilibrium is unstable. Suppose sbar > s*. Best response si is above the 45 degree line. Everybody plays it. So sbar moves further away from s* towards that high value si. Etc. If sbar < s*, si moves closer to the low value si. Unstable.
What matters is not that there’s a lower and upper bound but the fact that in game A the reaction function is flatter than 45 degree line and in game B it’s steeper.
Unstable equilibria are implausible.
(I only looked at the graph, didn’t do the math)
@Nick Rowe:
“Game A. There are n identical players who move simultaneously. Player i chooses Si to minimise a loss function Li = (Si-Sbar)2 + (Si-Sbar)(Sbar-S), where Sbar is defined as the mean Si over all players, and S is a parameter that is common knowledge to all players.”
OK. Li = (Si-Sbar)2 + (Si-Sbar)(Sbar-S)
= (Si-Sbar)(Si-S) ≧ 0
OC, if Si = S* then Li = 0.
If Si > S* then Si ≥ Sbar, and if Si < S* then Si ≤ Sbar. That can only be true for each i if Si = Sbar or Sbar = S*. In the first case, Li = 0, and in the second case, Li = (Si-S)2. Neither case can be assumed, I think.
“Game B is exactly the same as game A, except the loss function is now Li = (Si-Sbar)2 + (Si-Sbar)(S-Sbar).”
OK. Li = (Si-Sbar)2 + (Si-Sbar)(S-Sbar) ≥ 0
∂Li/∂Si = 2Si – 3Sbar + S = 0, approximately, with n large.
Then Si = (3Sbar-S)/2 and
Si-Sbar = (Sbar-S)/2 and
S-Sbar = -2(Si-Sbar) and so
Li = (Si-Sbar)2 – 2(Si-Sbar)(Si-Sbar) = -(Si-Sbar)2 ≥ 0
which is true only if Si = Sbar. That is, the only way Player i can set Si = (3Sbar-S)/2 is when Si = Sbar and Si = S*. Again, I doubt if we can assume those conditions.
OC, none of this holds if we are talking about payoffs instead of loss functions.
if the n->large and the game is symmetric then
A: the cost function is (at the limit) a parabola with a min at S*,
B: the cost function is (at the limit) a hyperbola with a MAX at S*
if [Sl,Su] is not symmetric around S* then the cost function has a min at that end of the interval further away from S*
if the interval is symmetric then the cost function has a min (the same) at both ends
It seems that at the non-symmetric interval case what matters is the degree of asymmetry. If the short end is on the Sh side then I think there are values that Sh can take that will make the other end either less desirable or more desirable. On the symmetric interval case on the other hand I can’t seem to find a reason for either end to be more desirable.
making the game sequential shouldn’t change something since everybody is insignificant and does the same thing
now if n is not large enough and/or everybody’s different I don’t know, it seems too messy
“B: the cost function is (at the limit) a hyperbola with a MAX at S”
should read
“B: the cost function is (at the limit) a hyperbola with a SADDLE POINT at S”
sorry
notsneaky: I edited your comment to put a space either side of the < . Otherwise, Typepad freaks out, because it thinks there’s a link coming.
“Unstable equilibria are implausible.”
I tend to agree. But some economists would reject that use of the word “unstable”. I am trying to restate that objection in another way.
Majro: “It still works with forward-looking expectations.”
Aha. Interesting, if we can prove it. (My brain is still out canoeing, so I can’t wrap my head around the rest of your comment.)
Min: you lost me, sorry.
john: If the n players colluded (minimised their joint losses), they would choose S* in game A, and either Sl or Su in game B. But we are assuming there is no collusion. Each player chooses Si to minimise his own losses taking other players’ Sj’s as given. (I’m not sure whether or not you are assuming that.)
Nick – yeah, but that’s on them. Do you have specific example in mind?
I think the point is also that the existence of the lower and upper bound doesn’t really matter; it’s not what makes the B interior equilibrium implausible. It’s the slope of the reaction function that makes it implausible (if you had the bounds in A economy nothing would change)
One thing I’m a little hung up on though is that you want these guys to move sequentially and you want n large, which means something like “n goes to infinity”. But wouldn’t that mean that effectively every players just gets to move once (have to wait until infinity to make another move)?
I guess you could assume a continuum of agents and the moves would be sort of like a seconds-hand on a clock “sweeping” the circle. If that was in “model time” (within each period) then it’d be the same as the static game. If that was actual time… that’d be hard to analyze (I think).
Nick,
Should you not be using an absolute value function?
Game A:
Li = ABS ((Si-Sbar)^2 + (Si-Sbar)(Sbar-S)) = ABS ((Si – Sbar)(Si – S))
Game B:
Li = ABS ((Si-Sbar)^2 + (Si-Sbar)(S* – Sbar)) = ABS ((Si – Sbar)(Si + S* – 2 Sbar))
Under game A, the minimum loss occurs when player i sets Si = Sbar or S*.
Under game B, the minimum loss occurs when player i sets Si = Sbar or 2 Sbar – S*.
Under game A, a player knowing S* and without knowing Sbar, would chose to set his Si = S*. Under game B, a player knowing S* and without knowing Sbar, might set his Si = some multiple of S*:
Si = k x S*
Rewriting the loss function for game B:
Li = ABS ((k x S* – Sbar)(k x S* + S* – 2 Sbar))
Setting Li = 0
0 = ABS ((k x S* – Sbar)(k x S* + S* – 2 Sbar))
Since a player does not know what Sbar is, he would pick a k that minimizes both ABS(k x S* – Sbar) and ABS((k + 1) x S* – 2 Sbar).
0 = ABS (k x S* – Sbar) = ABS(k x S* + S* – 2 Sbar)
Sbar = k x S* = ( k x S* + S* ) / 2
k = 1
And so, even under game B
Si = 1 * S* = S*
If we are not using the absolute value function, the we need to use negative infinity as our minimum value for Li.
notsneaky:
Well, I do have specific neo-fisherian examples in mind, but, this point goes beyond neo-fisherians. Many economists will say: “Well, if the model says S* is the equilibrium, then the model says S* will happen. Period. And “unstable” simply means that S(t) moves over time, and does not return to its original position following a shock.”
“I think the point is also that the existence of the lower and upper bound doesn’t really matter; it’s not what makes the B interior equilibrium implausible. It’s the slope of the reaction function that makes it implausible (if you had the bounds in A economy nothing would change)”
I tend to agree. Introducing the bounds was partly a rhetorical device. But partly to answer the question: “OK, so you don’t think S is what will happen in game B. So, what will happen??”
“I guess you could assume a continuum of agents and the moves would be sort of like a seconds-hand on a clock “sweeping” the circle. If that was in “model time” (within each period) then it’d be the same as the static game.”
Not necessarily the same as the static game. Sometimes, who moves first matters, because you can observe the others’ moves before making your own, so you can make your move contingent on theirs.
I want to write a blog post with the title: “Nash equilibria that are ‘unstable’ (in the old-fashioned sense) are not trembling hand perfect”. If I could prove it. That would resurrect and validate that old-fashioned sense of “unstable”.
You could do something like this with this game:
Suppose there’s a (countably) infinite number of players where each players is indexed by the time period in which they make their move. This is sort of “small n” for initial players but “large n” for later players set up. My understanding of what you’re trying to do is that a player’s choice of s(t) (since we’re doing time I’m switching subscripts from i to t) affects sbar contemporaneously. So sbar=(s(t)+Q(t-1))/(t+1), where Q(t-1) is the sum of all previous choices up to time t. There’s no “taking average as given” fudging here, so players understand that by choosing s(t) they’re also affecting sbar. We start at t=0 and assume that s(0)=s(1)=s* (we have two give it two time periods because the n=2 case is weird).
Let r(t) denote the actual outcome of choice s(t), where r(t)=s(t)+e(t) for t>1, e(t) is white noise. Since each player moves only once they do the best they can in that period so we don’t have to worry about future forecasts and all that.
algebra
In Game A the optimal choice of s(t) is s(t)=((t+1)Q(t-1)+t(t+1)sbar)/(2t^2+2t)
In Game B the optimal choice of s(t) is s(t)=((3t-1)Q(t-1)-t(t+1)sbar)/(2t^2-2t)
*more algebra
In Game B we have
s(t)-s=((3t-1)/(2t-2))(Q(t)-s)
That coefficient is greater than 1. So s(t)-s diverges from Q(t)-s* and s(t) explodes unless Q(t)=s* for all t. (remember Q(t) is the average so far)
Alternatively we could compute r(t)-s* explicitly. It’s basically r(t)-s=e(t)+a(t-1)e(t-1)+a(t-2)e(t-2)+… . The a(t)’s are time dependent coefficients, which are really tedious to compute (ratios of time polynomials). It’s a moving average. We can use that to compute variance and autocovariances and check if its stationary. Strangely enough the variance is actually bounded (I think). But the process is non-stationary so any mistake gets build into subsequent choices and never dies out. s(t) goes to +/- infinity.
(You can also just simulate this in a spreadsheet with little trouble)
Here’s the somewhat weird part and it actually occurs in Game A version. That process is stationary and the effect of mistakes die out but … s(t) does not converge to s but rather to 1/2 s*. Maybe my algebra was wrong but you can see it above:
s(t)=((t+1)Q(t-1)+t(t+1)sbar)/(2t^2+2t)
In “long run” Q(t)=sbar, so
s(t)=((t+1)^2)/(2t^2+2t)s –> (1/2)s*
The mistakes don’t die out fast enough, stay in the process long enough so that you never get to the optimum.
This’d be clearer if one could write math in a blog comment.
I’m also pretty sure that at the last ASSA I saw somebody doing comparative statics on a model with an unstable equilibrium and obtaining “counter intuitive results”
Notsneaky,
I think the reason you are seeing the Si = (1/2) x S* term is that choosing that value can generate a negative Li value (negative loss = positive gain?). If Si = 1/2 x S* then:
Li = -1/4 x S^2 + 1/2 x S x Sbar
If S* >> Sbar, then the -1/4 x S^2 becomes the dominant term. Suppose S is 1000 and Sbar is currently 100. i starts at 1.
L1 = -1/4 x 1 million + 1/2 x 1000 x 100 = -200,000
That is the maximum negative loss (positive gain) that can be obtained. If instead you use the absolute value:
Li = ABS ((Si – Sbar)(Si – S))
Then it becomes clear that a player should set Si = S to minimize Li.
I don’t think that’s it though I’d have to think through it. Rather it’s that si=s* is only optimal if sbar=s*. This is always true in the static version. But in the dynamic version because people make mistakes (the errors) these mistakes, which die out but not fast enough, accumulate and keep sbar away from s*.
Notsneaky,
I don’t think people need to make mistakes to keep Sbar away from S*. If negative losses (positive gains) are allowed in the loss function, then people are sometimes encouraged to keep Sbar away from S*. Notice that if the opposite is true (S* << Sbar) then people should choose an Si = 2 x S*.
Li = ((2S* – Sbar)(2S* – S)) = 2 x S^2 – S* x Sbar
This time suppose S* is 100 and Sbar is currently 1000. i starts at 1.
L1 = 2 x 10,000 – 100 x 1000 = -80,000
Frank, here there’s nothing special about negative losses. All that matters is that for some si, loss is less than for some other si. You can add an arbitrarily large constant to that loss function if you want. It’s the ordering, not the sign that matters.
If there are no errors and initial choices are s* then everyone will keep playing s*. When you say above “Suppose s* is 1000 and sbar is currently 100” you are implicitly assuming that some mistakes have been made.
Ah, I think I see what you’re saying. I’m tripping up over the same thing that got jonathan momentarily confused above. s* is not actually the optimum since the function can go negative. Got it.
Nick Rowe: “Min: you lost me, sorry.”
Let me analyze Game A more carefully. 🙂 We have this loss function for player, i:
Li = (Si-Sbar)2 + (Si-Sbar)(Sbar-S) = (Si−Sbar)(Si−S) ≥ 0
The equation is an algebraic simplification and the inequality is because a loss function is non-negative. As it turns out, it is easy for each player to minimize the loss function by choosing Si = S*, since the minimum is 0. However, let us look at the loss function itself. In fact, let us look at all of the loss functions.
If Si = S* then Li = 0, and the inequality is satisfied. If Si > S*, then Si > Sbar, and if Si < S*, then Si < Sbar. A priori, we do not know how close any S* will be to Sbar, but if S* − Sbar = ε > 0, then Si − Sbar ≥ ε for every Si > Sbar, and similarly for Si < Sbar. I said that to be sure that each Li was, in fact, ≥ 0, regardless of player i’s choice of Si, Si had to equal S* or Sbar, or S* had to equal Sbar. Now, it is possible, as indicated above, for each Li to be non-negative when none of those conditions hold, but it is not possible to guarantee it if the choice of Si is unconstrained. And you have to guarantee it for each Li to be a loss function.
In real life, I don’t think that you can guarantee it, and so the Li’s are not really loss functions.
As others have also pointed out, in game B it is easy for Li to be negative, so it is not a loss function, either.
I think this may be clearer about Game A. If S* != Sbar, then it is possible that for some i, Si falls in between S* and Sbar, and then Li < 0, and Li is not a loss function. 🙂
notsneaky: unfortunately, it’s harder than that. let n=100. Consider the 7th player in the sequential game. Sbar is the weighted average of three things:
1. The S of the 6 previous players;
2. his own S(7);
3. his expectation of the S of the 93 remaining players.
If n is large, we can ignore 2, and we can maybe (though I’m not sure of this) ignore the effect of 2 on 3, but we can’t ignore 3.
Min: put a minus sign in front of the loss function, and call it a utility function, if you like.
I see. I was trying to set it up so as to not have to deal with 3.
So player 7’s payoff is the loss they experience in period 7 (when they make a choice) plus the loss they experience in periods 8+. In those 8+ periods they’re done making choices but still are affected by the choice of others. I’m assuming there’s some discounting going on in there to make that sum finite. Is that how it’s suppose to work?
notsneaky: yep, if my games are to represent the NK model, we have to include 3. Since what matters is the individual firm’s P(i) relative to the general price level P.
We don’t need discounting to make the sum finite, since we divide the sum by n.
But it’s the sum of Li’s, not the sum of si’s.
So it’d be something like this: V_t(s(t)) is the value function for player moving at time t. The Bellman is deceptively simple
V_t=L(s(t))+bEV_(t+1)
(maybe the constraint that r(t)=s(t)+e(t) needs to be made explicit)
The first order condition is then
Sum{j=0,inf) (b^j)(dL(t+j)/ds(t))=0
s(t) appears in L(t+j), j>0 because of its effect on the average.
To be able to even begin figuring this out I think you’ll probably need to assume that the errors, e(t)’s, are bounded (so they can’t be white noise – this is an assumption made in all the NK model which I always thought weird and I’ve never seen anyone really dig into that)
Second, I think at that point you pretty much have to assume that the optimal sequence s(t) is bounded otherwise this isn’t defined, even with discounting. That pretty much rules out Game B, unless you put those lower and upper bounds in there.
(there’s also another issue and that’s whether or not decision makers anticipate that they’ll make an error. I.e. “if I choose s(t), the actual variable will depend on s(t)+e(t), so I should account for that. If I was risk neutral this wouldn’t matter but I’ve got a quadratic loss function so…”)
Nick Rowe: “Min: put a minus sign in front of the loss function, and call it a utility function, if you like.”
OK, but the (Si−Sbar)2 is a dead giveaway for a loss function, since squaring the difference makes the term non-negative. In addition, the (Si−S) factor makes it easy for the players to set Si = S to obtain the minimum loss of 0, which is the desired theoretical result, right? So I suspect that the Li’s for game A were originally intended in New Keynesian theory to be loss functions, as advertised.
Let’s go back to the game-as-specified, without dividing it into rounds. We will, however, add a degree of Bayesian expectations.
We’ll start the game with a prior distribution for Sbar, such that we believe Sbar to be a random variable with mean S0 plus some small random ε. That S0 is a prior fixation point — perhaps a previous Sbar, or perhaps a number whispered to everyone upon entering the room.
Now, expand the standard loss function in terms of ε to get Li = (Si-S0)^2 + (Si-S0)(S0-S) – 2(Si-S0)ε – (S0-S)ε + (Si-S0)ε + ε^2 – ε^2 = (Si-S0)^2 + (Si-S0)(S0-S) – (Si-S)ε.
This loss function has a couple of mathematically nice features: there is no ε^2 term so our expected loss is not influenced by the uncertainty in our belief about Sbar, and the loss-from-mean-error term depends only on Si and S* and not S0.
Now, our expected loss is given by taking the ensemble mean of the above, which means the ε term drops out with the assumption that it is zero-mean. That gives <Li> = (Si-S0)^2 + (Si-S0)(S0-S), which in turn is minimized for Si = 1/2(S0+S).
Now, this is our “irrational expectations” solution, or what we get if we assume everyone else is an idiot. We can extend this result to partially rational expectations by modifying our prior. Rather than assume Sbar = S0 + ε, we’ll assume that Sbar = (1-γ)S0 + γSi + ε, where 0 ≤ γ ≤ 1 and Si is the value we get by running through the above. We can do so easily by replacing S0 with S’ = (1-γ)S0 + γSi, which in turn gives Si = 1/2((1-γS0 + γSi + S), or after a bit of algebra Si = ((1-γ)S0 + S)/(2 – γ). As γ → 1, Si → S* and our expectation about Sbar also → S*.
Better yet, this is a “comfortable” equilibrium. Rationally, if we think that our guess is very far away from S0, our expectation about ε would also increase — but provided it is still zero-mean it does not affect our expected loss.
This changes for the abnormal loss function. Expanding in terms of ε again gives Li = (Si-S0)^2 + (Si-S0)(S-S0) – 2(Si-S0)ε – (S-S0)ε – (Si-S0)ε + 2ε^2 = (Si-S0)^2 + (Si-S0)(S-S0) – (3Si + S – 4S0)ε + ε^2.
Now, under the ensemble mean we get <Li> = (Si-S0)^2 + (Si-S0)(S-S0) + <ε^2>, which is an uncomfortable loss function: we expect to lose in proportion to the uncertainty of our expectation. Still, the optimum choice of Si is only given by the deterministic part, giving us 2(Si-S0) = (S0-S) or Si = 1/2(3S0-S) for irrational expectations.
Blending our prior with our choice of Si again gives Sbar = (1-γ)S0 + γSi + ε and S’ = (1-γ)S0 + γSi, but this time the algebra gives us Si = (3(1-γ)S0 – S)/(2 – 3γ).
This is terrible news. We still recover Si = S for γ = 1, but getting there is a problem. If we don’t assume that the game is more than 2/3 rational, our optimum play has the opposite sign of the near-fully-rational optimum. Even worse, near γ=2/3 we don’t even have a convergent result, so any Si (and by extension Sbar) is a possibility, strongly suggesting that we’d also update our ε to one with higher variance.
In turn, that is extremely uncomfortable, because with this loss function uncertainty about our expectation of Sbar translates directly into a greater expected loss.
notsneaky: “But it’s the sum of Li’s, not the sum of si’s.”
Ah! That’s what you meant. I had in mind a simpler model. Firms take turns to announce their prices in advance, then when all firms have announced their prices, there is one period in which customers buy goods, and the Li are revealed, then the game ends. So only one Li for each firm, and no discounting is needed.
“To be able to even begin figuring this out I think you’ll probably need to assume that the errors, e(t)’s, are bounded (so they can’t be white noise – this is an assumption made in all the NK model which I always thought weird and I’ve never seen anyone really dig into that)”
Simplest(?) assumption: each firm has a probability p of choosing the loss-minimising Si, and a probability 1-p of a trembling hand, where Si has a uniform distribution between the upper and lower bounds. We solve for the equilibrium, then take the limit as p approaches one, and see if it approaches S*.
Nick, wouldn’t that be easier to solve since you could just do it by backward induction (for finite #, then take limit)?
There’s N players. Nth player observes all the previous values. Chooses sN based on current average plus expectation of error. N-1 th player anticipates Nth player’s choice (in expectation), chooses sN-1. And so on. Going all the way back you’re going to get
s1=s+f(N)sigma where f(N) is some function of N (I’m guessing a ratio of polynomials of Nth degree in N) and sigma is the variance of the error term (assuming iid. It’d be a real mess if errors were autocorrelated, but more interesting also). Then you take the limit f(N) as N goes to infinity. Or compute s(t,N)-s and see if this gets bigger or smaller as t goes to N.
With a finite number of players we don’t have to worry about the errors being bounded either since it’s a finite number of equations in the same number of unknowns.
Tedious, very tedious, but doable.
notsneaky: “Nick, wouldn’t that be easier to solve since you could just do it by backward induction (for finite #, then take limit)?”
Yes. But as you say, tedious.
Or, maybe some proof by contradiction?
Or, eyeballing that reaction function graph for game B, start with p=1 (where p is probability of trembling hand and uniform distribution of S between Sl and Su), then slowly reduce p, and watch the distribution start to mass up at Sl and Su, because the reaction function is non-linear, so if E(Sbar) is anywhere near Sl (or Su) you play Sl (or Su) if your hand does not tremble. So in the limit, as p approaches zero, the distribution is bimodal at Sl and Su. (I’m not sure if that was clear.)
It’s gonna be some ratio of polynomials. So you could do the case N=3 (I think N=2 might not work), N=4, maybe N=5 and see what the sequence of the coefficients is.
Nick, I’m not sure what you meant in the OP by an equilibrium being more or less plausible. Plausibility is not a feature of a mathematical result. And I haven’t understood all the math that followed.
But if what you meant was that the games are supposed to represent stylized models of the real economy in equilibrium, and that game A was a more plausible model of the real economy than game B, then I think the argument is pretty clear. It is not plausible to represent the economy as being in a state which it would not be in if it experienced small shocks, because the real economy is always experiencing small shocks.
Heath: solving for the Nash equilibrium is simply math. But in some cases, game theorists find the Nash Equilibrium to be…implausible. For example, where someone makes a conditional threat “if you do A I will do B” that it would not be rational to follow through with. It all has to do with counterfactual conditionals — what one player would do if another player made a non-equilibrium move. The idea behind trembling hand perfect equilibria is to deal with those counterfactual conditionals, by supposing there is a vanishingly small probability that every possible move (whether rational or not) does get made.
Nick,
I believe that both ARE in fact trembling hand perfect. The issue with trembling hand perfection is whether or not you can shrink the chance of error arbitrarily and have an equilibrium approach the equilibrium you have in mind. In both cases I believe you CAN do this and that means that both interior equilibria are THP. Essentially, this is because if there is some small chance of each other player choosing a value of s not equal to s*, then
1. If the probability distribution of their errors is symmetric around s*, I think the best response will still be s* (didn’t do the math but this seems right).
2. Even if it is NOT symmetric and you know there is a greater chance of a higher or lower sbar, your best response will still be “in the neighborhood” of s* and it will get closer to s* as you shrink the probability of error.
The way to go about the analysis (I believe) is to do something like this: Let each player select a target: sti. Then let sbar=average[sti+ei] where ei is distributed somehow. Then solve for an interior NE. Then collapse the distribution of ei to zero and see if you have an equilibrium which approaches s*. I suspect you will.
In order to eliminate an equilibrium as not THP, you basically need a tiny possibility of error to make everyone’s best response jump to one of the extremes. You have it moving away slightly. In one game the best response will be between s* and sbar and in the other it will be outside of that range but in either case, it will approach s* as the expected error approaches zero (again, I believe). I think what you really want is to invent another equilibrium concept. This could be a big deal if you could. I will think about it some more. If I come up with anything, I’ll let you coauthor 🙂
For what it’s worth, here is a variation where I think you can use THP to eliminate the middle equilibrium.
Assume the number of players is so large that any one player cannot affect sbar (or else assume that their payoffs are dependent on sbar-i which is the average of all other players s). Then let players try to maximize (it’s easier to for me to think in terms of maximizing for some reason but obviously you can take the inverse and minimize it) of U=(sbar-s)^2(si-sbar)^2. Then if everyone plays s, they will all be indifferent between all s and so it will be a NA. However, if there is any small chance that others will make a mistake, then everyone will want to choose sl or sm. It won’t converge to s* for any convergence of the error (that I can see).
I doubt this is of any use to your point about macro (off the top of my head, I can’t think of an analogous slight variation on the payoff function that would be THP) but it might help to illustrate the concept of THP. An interesting, though probably off topic aspect of this is that you could also have a NE where half the people chose sl and half chose sh (or one in which everyone mixed with probability 1/2) (this is assuming that s* is half way between sl and sh but you get the idea). This equilibrium, I suspect (and the degree of speculation here is increasing dramatically) WOULD be THP. And yet, you would still have a case where if one person defected to the other side, everyone else would want to follow.
For the record, I am doing this very cavalierly so mistakes are likely.
Mike: thanks for this.
I will come back to it later today.
Mike: thanks again.
Some thoughts:
1. I understand your variation, in your second comment, but it doesn’t really work to make my point about macro. Because in my game B, each individual has an interior optimum for any given Sbar, as long as Sbar does not get too close to Sl or Su.
2. “Even if it is NOT symmetric and you know there is a greater chance of a higher or lower sbar, your best response will still be “in the neighborhood” of s* and it will get closer to s* as you shrink the probability of error.”
I think I get what you are saying there, but we must be careful. It is important that the trembling hand error could be large, even if the probability of that error is small. Anything is possible, even if improbable. That’s why I want to think in terms of a uniform distribution of errors, when an individual’s hand trembles.
3. Let me sketch my first thoughts towards a proof that S* is not THP:
Let the slope of the reaction function be “b”. (So b=1.5 in my Game B)
Assume the players move sequentially (each observes previous moves) in a one-shot game.
Assume only one player has a hand that trembles, but it could be any player. Call that player t, and let that player be t in line to move. So 1 < = t < = n. And player t makes an error e.
Suppose that S* is a THP equilibrium (I’m trying to derive a contradiction).
If t=n (the trembling hand player is the last player) then all other players choose S*, and so Sbar=S* + e/n.
If t = n-1, then the last player will also deviate from S*, so Sbar = S* + e/n + (be/n)
If t = n-2, then the last two players will also deviate from S*, and the second from last player will know the last player will deviate, so Sbar = S* + e/n + 2(be/n) + 2(bbee/nn) + 2(bbbeee/nnn) + etc. [I think I got that right.]
Now for general 1 < t < n, (with large n) we know that if 0 < b < 1 that infinite sequence converges to Sbar = S* + [(n-t)/n][1/(1-b)]e , (just like the Old Keynesian multiplier, where a fraction [(n-t)/n] of the population respond to the shock).
But if b > 1 (like in my Game B), that infinite sequence may not converge to a finite number. Which means that Sbar does not approach S* in the limit as e approaches zero, provided the player whose hand trembles comes early enough in the sequence. In fact, if t=1 (the first player’s hand trembles), then b > 1 is sufficient for non-convergence.
Which contradicts my initial assumption that S* is THP.
I think I’ve got that (roughly) right.
I think what you are saying here makes sense, it’s just not the definition of THP, it’s something else all together. Here is from Wikipedia.
“First we define a perturbed game. A perturbed game is a copy of a base game, with the restriction that only totally mixed strategies are allowed to be played. A totally mixed strategy is a mixed strategy where every pure strategy is played with non-zero probability. This is the “trembling hands” of the players; they sometimes play a different strategy than the one they intended to play. Then we define a strategy set S (in a base game) as being trembling hand perfect if there is a sequence of perturbed games that converge to the base game in which there is a series of Nash equilibria that converge to S.”
You are making a variation which is a dynamic game and you are talking about what it converges to (or doesn’t converge to) over time. That is a different convergence than what the equilibria in the perturbed game (the one with errors) as it converges to the base game. This means that you can’t assume a uniform distribution of errors, because you have to make it converge to the base game. (Although, I’m not sure that matters for my point anyway. If it’s a simultaneous move game then a small probability of a large error will still cause what I would call a “local” shift of the equilibrium, unlike my game where a small chance of error causes everyone to jump to the extreme values. That’s what you need to eliminate something with THP. So notice that the reaction functions in my model is not smooth, it is vertical at s* and horizontal everywhere else.)
I realize that the convergence of s over time is what youa are actually interested in, so you are probably approaching it along the right lines, but it’s just not a case of THP. Once you do it as a dynamic game with sbar (potentially) changing over time, I think you may actually be able to use simpler terms. I’m not really sure what those terms are but it seems like that becomes something a lot like what we did in first year macro (also in diff eq/linear algebra) where you draw those graphs with time on the horizontal and s (in this case) on the vertical and you fill in arrows everywhere showing all the paths over time depending on where you start and you get some horizontal lines that are “equilibria” but sometimes the arrows funnel into them and sometimes they funnel away from then. Then you call them “stable” or “unstable” or something like that. It seems like what you need to do is get the setup of the game right so that you can describe it in those terms and not need THP.
As an aside, you don’t really run across THP that often in the literature. I suspect it’s because it’s kind of a thorny concept due to the “if there is A sequence of games that converges” part of the definition. It can be kind of annoying to prove that no such sequence of games exists when a game gets even moderately complicated. Plus it isn’t really that strong in the end (it basically only rules out a pretty specific type of equilibrium). A lot of the stuff that you kind of feel like should be ruled out it doesn’t eliminate (as in your case).
Thanks again Mike.
“You are making a variation which is a dynamic game and you are talking about what it converges to (or doesn’t converge to) over time.”
I am not sure if we understand each other on that point. The NK macro model is indeed a dynamic game, that continues forever, with each player making multiple moves. But the game I am considering here is a very simplified stylised version of the NK game. My game has only one “period”; each player moves only once, but they take turns to move. So I am not really considering convergence over time. Firm 1 sets its price, firm 2 sets its price,…,firm n sets its price, then the customers observe all the prices, and decide how much to buy from each firm, and then time ends.
OK, I see what you mean but in that case, it’s not even clear off the top of my head what is an equilibrium in the base game. (Note that this is a significant change from the game in the OP Once you know that, to determine whether it is THP or not you would still have to define a perturbed version of that base game and solve the perturbed game for a NE and see if it converges as the rate of error gets small. This might be difficult.
It seems like you might actually want to do something like what I was thinking. For instance: Each period one (or some number of) firm(s) set their price and receives a payoff that period which is a function of their price relative to the average price of all other firms who acted previously. Then you will have the case where, if their optimal action is to price in between s* and sbar, it will converge over time to s*. If their optimal action is to price outside of that range, it will diverge. You could make their payoff depend on actions taken after they move also (as you mentioned in a previous comment) but then it starts getting complicated unless you can come up with some kind of recursive relationship that will reduce it to something convenient. It seems like this might be kind of close to what already occurs in a NK model but I’m not fully clear on that or on exactly how your fundamental point fits into that analysis.
When I look at this, I kind of feel like there may be a policy application as well. Every month central banks set policy sequentially after observing the policy choices of each previous central bank, etc. If the FED sets policy “too tight” or “too loose” what effect does that have on Canada’s optimal policy? Does that effect cause some kind of spiral toward the zero bound on one side or hyperinflation on the other? (For the record I think not, but that’s because I already think I know what causes a downward spiral toward the zero bound and it’s something different.)
Mike, in the game I have in mind, Sbar is the (expected) average S of all the firms, both those that have already set S and those that have not yet set S.
Suppose n=101. The 51st firm will set its S = S* + b[0.5 Average S of the previous 50 firms + 0.5 E{average S of the remaining 50 firms}].
In other words, Sbar is partly backward-looking and partly forward-looking.
Yeah, I think I get what you have in mind now. But do you know what the equilibrium(a) is (are)?
Mike: the Nash Equilibria are S*, Sl, and Su. But I think the only THP equilibria are Sl and Su (the lower and upper bounds on S).
Translated into macro-speak, the lower and upper bounds are when the central bank changes the game (adopts QE, for example) because inflation has got too low or too high.
Nick Rowe: “in the game I have in mind, Sbar is the (expected) average S of all the firms, both those that have already set S and those that have not yet set S.”
If you mean “expected” in the mathematical sense, the expected average may not exist.