?

Log in

No account? Create an account

Mon, Jan. 20th, 2014, 09:05 pm
One Mystery Hunt thought for now

Hunt was awesome -- the best in my memory. Delightful theme, but more importantly the puzzles had been really well edited and testsolved and factchecked, and it showed.

At wrapup, the editorial director explained their overall philosophy, which included "Make the Hunt more enjoyable by smaller teams." She then added "I'm not sure how well that worked out."

It worked out amazingly well.

My team never got swamped by so many puzzles that we had to abandon ones that were giving us trouble. And in combination with the high quality of the puzzles and the policy of keeping HQ open until 6pm Sunday, this meant that we saw the endgame for the first time since I've started hunting, and we had a "complete" rate of over 90%. This makes us feel really good about the Hunt experience and about ourselves.

At the post-wrapup gathering, someone from a large team commented to me that he didn't like the fairly linear release mechanism, because it meant that on a large team like his, there were often periods when there weren't enough puzzles to go around.

This is a very important statement that he made. Let me repeat it:

Some people on a large team were sometimes bored when there weren't enough puzzles to go around.

For years, many people have debated how to handle the growing team size gap. Partly this becomes an issue of fairness: A team like mine, with 30-40 solvers, simply cannot be truly competitive with teams that are three times our size.

There's also an "arms race" between writing teams and solving teams. If a megateam can solve the entire hunt in 24 hours, does the writing team have to ramp up the scale or difficulty to compensate? And what does that do to the smaller teams' ability to see a sizable fraction of the Hunt, let alone the endgame? I think that was one cause (among others) of the problems with the scale and difficulty of the 2013 Hunt.

Those who feel that this is a problem have struggled with what measures could overcome it: various forms of sliding scales to make it more "costly" (in terms of points or time) for a large team, or imposing a team size limit, have been suggested. But those are bad ideas: they are unfair, they are hard to enforce (or, in some cases, to define), and the size gap is so large that it would be impossible to design a penalty that can overcome it.

As one of my favorite fictional characters said when all lines of attack on the enemy military force seemed futile: "It's not hopeless. My friends, we are going to hit them in the payroll!"

Now, solvers don't join teams for a paycheck; our reward is enjoyment.

If the hunt structure and release mechanism impose inherent limitations on how many puzzles a team can have open at once, that means parallel solving of scores of puzzles just can't happen. And that will simply change the natural dynamic so that being on a large team will be somewhat less fun than being on a smaller one.

It's not a penalty. It's not a tax or an imposed limit. If a solver feels that they will still have more fun on a large team, nothing interferes with that freedom.

But just as in recent years the Hunt community has been making the philosophical shift from "When the race is won, the race is done" to "HQ will stay open until 6pm so that all teams can continue to enjoy the game", I believe it would be good for the future of the Hunt for running teams to consider what [Alice Shrugged] accomplished this year. [Alice] took a damaged franchise and made it exciting, invigorating, and thoroughly enjoyable by adopting the design philosophy that they did.

I think targeting Hunt at smaller teams was a wonderful thing to try. I'd applaud it even if it hadn't worked. But from where I sit, looking at solving stats and being awed by the low team-dropout rate as Hunt progressed, and reflecting on the high satisfaction among my teammates, this approach worked great.

Best. Hunt. Ever.

Please, may I have another?

Tue, Jan. 21st, 2014 02:48 am (UTC)
dougo

Hear hear! The Hunt went from "worst to first"...

There are a number of milestones that a team can accomplish in the course of the weekend:
1. Solve a puzzle.
2. Solve a metapuzzle.
3. Solve a supermeta.
4. Complete the hunt.
5. Win the hunt.

Our team (Central Services, about the same size as your team, or slightly bigger) will typically get to #3 by Friday night or Saturday morning, but we have never gotten to #4. I'm glad that your team got to #4, and I hope that we can get there soon also. But I think there's a very large gap between #3 and #4, and I think there are a large number of teams who fall into this gap. It would be nice to add a new milestone in between.

This hunt came close. At the wrap-up, they made a distinction between "Wonderhole" rounds and "Wondercore" rounds, and in fact our team solved all three "Wonderhole" metas (though the last two happened in the last hour of the hunt and involved a small amount of handholding from HQ). But I would have liked this to have been a more clear goal, and to have had a more satisfying reward (akin to the deck of cards from the MIT supermeta, or the coin for winning the whole hunt-- there were also some consolation prizes for finishing the hunt).

I hope Random is conscious of this. It's great to serve the needs of both small teams and large teams, but I think it would be even greater to serve medium-sized teams better too.

Edited at 2014-01-21 03:13 am (UTC)
(Deleted comment)

Tue, Jan. 21st, 2014 03:38 am (UTC)
dougo

Yes, that big bubble was particularly pronounced for us this year:

http://web.mit.edu/puzzle/www/2014/solved_puzzles_by_team.svg

We were decidedly 13th place (86 solved, of 120), with nobody near us in either direction (99 and 78 were the closest). And we were basically that way from Saturday 8pm on.

Tue, Jan. 21st, 2014 03:06 am (UTC)
mabfan

Wow. I am delighted that you had such a great time, and find your analysis thought-provoking. (For an event that I don't attend...)

Tue, Jan. 21st, 2014 03:52 am (UTC)
brokenwndw

Interesting post!

I'm on Codex, which did not lose that many people after 2012 and is still over 150 people. For me, this hunt only felt release-constrained towards the beginning. I don't recall feeling like we didn't have enough open puzzles at any time after we entered Wonderland. So I might be a data point on the other side.

I also need to note, as I have before, that although our philosophy was not to starve large teams, we still never intended our release to be as rapid as it was. We wanted teams to always have the same number of puzzles to work on where possible, unless we needed to time-release puzzles so they could see them before we shut down. But we assumed that teams would accumulate abandoned puzzles as the hunt went on, and so the constants were tuned so that the total number of open puzzles increased over time. For various reasons our hunt had far fewer abandoned puzzles than we anticipated, and so the effect was to flood people. I wish in retrospect that we had more adaptible math, or a knob to turn, to avoid this effect, and it's still one of my top regrets about the hunt.

If this hunt seems restricted by comparison to its predecessors, I think it's because we had a glitch in the math, and 2013 ran so long that time release caught up to everybody. I actually think this year's release rate was pretty good, even for large teams, and rather close to what we wanted but failed to accomplish.

Tue, Jan. 21st, 2014 04:07 am (UTC)
brokenwndw

Addendum:

I do have to note that while Codex is nominally 150 people, a lot of us are new or relatively casual solvers. (I don't mean that disparagingly-- in fact quite the opposite. One of the things I love about an open-invitation team is how easily we bring in people who are just trying it out, or even just there to hang out and gawk.) We definitely did not bring anywhere near 150 people to bear at all times, especially after the first day.

So the total expert, highly-driven brainpower on any given puzzle was probably comparable to what you would find on a much smaller team of veterans. I can't speak to the experience of teams who really do have 100+ veteran puzzlers... if such even exist.

Tue, Jan. 21st, 2014 05:14 am (UTC)
rlangmit

To be fair, the "mini hunt" philosophy was pioneered by Evil Midnight in the Hell Hunt of 2007, though there it was extra easy. Inner Zyzzlvaria in 2009 was, in my mind, very similar to this year's MIT round. I remember that it was certainly a great thrill to finish that Sunday afternoon (which is probably why I enjoyed that hunt so much despite more advanced teams having problems). And I'm pretty sure other hunts since then have included similar ideas, though I'm too tired to think of them now.

But it's fantastic that Alice Shrugged kept to that philosophy and, perhaps, perfected it. The accomplishment of solving "stage 1" of the hunt, the plot twist that comes with it...excellent!

On the other end of the scale, we at Up Late didn't quite reach the endgame this year, but we're getting *so close*. I hope future hunts are just like this one--well-paced and run long enough so that teams like ours continue to have a good shot at finishing. (Though I will admit last year's was fun in the "omg we will never forget how epic this was" way...variety is the spice of life...).

TL, DR: I agree wholeheartedly. :)

Tue, Jan. 21st, 2014 01:38 pm (UTC)
thespelvin

The MIT round was not my idea, but I would be lying if I said I didn't have Inner Zyzzlvaria in mind when I contributed to it. IZ was conceived as a meaningful subhunt goal that every team could accomplish, so much so that we originally gated the rest of the Hunt behind it (a plan we had to change Saturday morning). But we [Evil Midnight] overestimated the average team, expecting them to get through five easier but still meaty metas and a very involved board game ubermeta, so the idea of getting everybody to Captain Blastoid was not fully realized.

With the MIT round, Alice Shrugged was much more consistent in terms of differentiating the concept of an "MIT puzzle" and a "Wonderland puzzle" throughout the year, and we were much better at making the MIT metas than the average meta. Plus, no offense to Captain Blastoid, but our little skit at the end was not as much of a reward as unlocking a vorpal sword, venturing into a dark room and hunting a ten-foot-tall Jabberwock.

I think the Zyzzlvaria Hunt had a lot of great innovations (multi-round structure mashup, advancement by point system, theoretically accessible subhunt) that were appreciated in their first incarnation but not totally baked. It makes me smile every time a team goes back to them and gets them right.

Tue, Jan. 21st, 2014 03:19 pm (UTC)
noahspuzzlelj

I completely agree with Dan's last paragraph. Indeed, in 2011 when we reused all three of those ideas, one thing I was hoping was that it would set the precedent that these ideas were big enough and flexible enough that they could and should be copied over and over again. Prior to that, my feeling was that hunt structures had one-off ideas that people wouldn't reuse (say all metas output two answers, or every round has a puzzle hidden in the map), but "each round has a different structure" and "accessible subhunt" are not one-off ideas like that but instead a new model for how a hunt can work which is an alternative to the older "homogenous hunt with one major structural innovation" model.

Tue, Jan. 21st, 2014 05:46 pm (UTC)
vettecat

Very glad it went so well and you had fun! (40 people is small? Yow.)