The Game Outcomes Project, Part 4: Crunch Makes Games Worse
游戏专案为何成功系列之四：加班反而会把事情搞砸
网志版：http://wp.me/pBAPd-qJ
原文网址：
http://gamasutra.com/blogs/PaulTozour/20150120/234443/The_Game_Outcomes_Project_Part_4_Crunch_Makes_Games_Worse.php
缩网址：http://tinyurl.com/m7kmuzf
撰文：Paul Tozour
繁体中文翻译：NDark
20150120
译按：本文是一篇统计学专业文章，若有翻译不正确的文句，请以原文为主。
This article is the fourth in a 5-part series.
Part 1: The Best and the Rest is also available here: (Gamasutra)
(BlogSpot) (in Chinese)
Part 2: Building Effective Teams is available here: (Gamasutra)
(BlogSpot) (in Chinese)
Part 3: Game Development Factors is available here: (Gamasutra)
(BlogSpot) (in Chinese)
This article is Part 4, and a Chinese translation will soon be available.
Part 5 will be published in late January 2015.
For extended notes on our survey methodology, see our Methodology blog
page.
Our raw survey data (minus confidential info) is now available here if
you'd like to verify our results or perform your own analysis.
The Game Outcomes Project team includes Paul Tozour, David Wegbreit, Lucien
Parsons, Zhenghua “Z” Yang, NDark Teng, Eric Byron, Julianna Pillemer, Ben
Weber, and Karen Buro.
本文是五篇系列中的第四篇。
第五篇将会在一月底释出。
想要知道问卷的方法论，请参阅部落格页面
:http://intelligenceengine.blogspot.com/2014/11/game-outcomes-project-methodology-in.html
我们问卷的原始资料在此，有兴趣的朋友可迳自取用分析。
"游戏专案为何成功"团队成员包含Paul Tozour，David Wegbreit，Lucien Parsons，
Zhenghua “Z” Yang，NDark Teng，Eric Byron，Julianna Pillemer，Ben Weber，及
Karen Buro。
The Game Outcomes Project, Part 4: Crunch Makes Games Worse
游戏专案为何成功系列之四：加班反而会把事情搞砸
Extended overtime (“crunch”) is a deeply controversial topic in our
industry. Countless studios have undertaken crunch, sometimes extending to
mandatory 80-100 hour work weeks for years at a time. If you ask anyone in
the industry about crunch, you’re likely to hear opinions stated very
strongly and matter-of-factly based on that person’s individual experience.
And yet such opinions are almost invariably put forth with zero reference to
any actual data.
延长工时（加班）在我们的产业中充满争议。无数的工作室都曾采取加班的手段，甚至一
周会工作八十到一百小时。假如我们询问业界加班的情形，我们会听到各种基于个人经验
的不同看法。
但这些意见纯粹都是主观意见，缺乏实际数据佐证。
If we truly want to analyze the impact of extended overtime in any scientific
and objective way, we should start by recognizing that any individual game
project must be considered meaningless by itself – it is a single data
point, or anecdotal evidence. We can learn absolutely nothing from whether a
single successful or unsuccessful game involved crunch or not, because we
cannot know how the project might have turned out if the opposite path had
been chosen – that is, if a project that crunched had not done so, or if a
project that did not employ crunch had decided to use it.
As the saying goes, you can’t prove (or disprove) a counterfactual – you’d
need a time machine to actually know how things would have turned out if you’
d chosen differently.
假如我们真的想要用科学化的方式分析加班带来的冲击，我们应该先认知道一点也就是：
分别从各专案的特例来看都是没有意义的。我们无法清楚的知道加班对于专案是否有影响
，因为我们从事后来看只能看到成功与失败，而不能用另一个方式再执行同一个实验，因
为我们还没发明时间机器。
Furthermore, there have undeniably been many successful and unsuccessful
games created both with and without crunch. So we can’t give crunch the
exclusive credit or blame for a particular outcome on a single project when
much of the credit or blame is clearly owed to other aspects of the game’s
development. To truly measure the effect of crunch, we would need to look at
a large sample, ideally involving hundreds of game projects.
更进一步，无法否认的有很多成功或失败的专案都可能由加班或不加班的情形下完成。所
以我们不能完全用加班来指责单一专案的成败，因为造成他们的成功与失败可能来自不同
的要素。真正要测量加班的影响，我们应该用巨观，数百个专案的数据来做。
Thankfully, the Game Outcomes Project survey has given us exactly that. In
previous articles, we discussed the origin of the Game Outcomes Project and
our preliminary findings, and our findings related to team effectiveness and
many additional factors we looked at specific to game development. We also
wrote up a separate blog post describing the technical details of our
methodology.
In this article, we present our findings on extended overtime based directly
on our survey data.
刚好"游戏专案为何成功"的问卷给了我们这个机会。在先前的系列文章中，我们已经谈论
了游戏专案为何成功这个计划的来由与初步结果，找到与团队效率之间的关系，以及游戏
制作领域的额外要素。我们也在部落格撰写了我们的方法论。
本篇文章中，我们从问卷的数据中持续寻找加班相关的线索。
Attitudes Toward Crunch
Developers have surprisingly divergent attitudes toward the practice of
crunch. An interview on gamesindustry.biz quoted well-known industry figures
Warren Spector and Jason Rubin:
对于加班的不同态度
令人惊讶地，开发者对于加班的态度也很分歧。在Gamesindustry.biz的访问中我们可以
引述Warren Spector 与 Jason Rubin 的说法：
“Crunch sucks, but if it is seen by the team members as a fair cost of
participating in an otherwise fantastic employment experience, if they value
ownership of the resulting creative success more than the hardship, if the
team feels like long hours of collaboration with close friends is ultimately
rewarding, and if they feel fairly compensated, then who are we to tell them
otherwise?" asked Rubin.
Rubin说："加班确实糟透了，但假如从团队成员的角度看来，那可能也是一个美妙的团队
经验，假如他们认为创作的结果高过痛苦，假如他们认为与亲密战友长时间的合作是一个
终极的满足，假如他们能够获得回馈，那么谁又有资格跳出来阻止他们？"
[…] "Look, I'm sure there have been games made without crunch. I've never
worked on one or led one, but I'm sure examples exist. That tells me
something about myself and a lot about the business I'm in," said Spector.
Spector继续说："...听着，我确信一定有不需要加班就产出的游戏，但我待过的开发案
从未这样，虽然我相信一定有例外。这就是我所作的工作与产业。"
[…] "What I'm saying is that games - I'm talking about non-sequels,
non-imitative games - are inherently unknowable, unpredictable, unmanageable
things. A game development process with no crunch? I'm not sure that's
possible unless you're working on a rip-off of another game or a low-ambition
sequel.
"...我不是说那些续作专案，抄袭游戏，而是说完全原创，完全未知的产物。这种游戏开
发案子怎么可能不加班？除非你正在抄袭或是只是作没有野心的续作。"
“[…] Crunch is the result of working with a host of unknown factors in
creative mediums. Since game development is always full of unknowns, crunch
will always exist in studios that strive for quality […] After 30 years of
making games I'm still waiting to find the wizard who can avoid crunch
entirely without compromising at a level I'm unwilling to accept.”
"...加班是因为我们是与未知参数的艺术媒体战斗过程的产物。只要游戏开发充满了不确
定性，为了追求品质，加班就是必然。但说起来简单，三十年的游戏制作机验后，我仍等
待某个魔术师来告诉我不需要加班就可以做出我可以接受的关卡。"
On the other side of the fence is Derek Paxton of Stardock, who said in an
interview with Gameranx:
在Gameranx的访问中，Stardock 的 Derek Paxton 表达了另一个角度的看法：
“Crunch makes zero sense because it makes games worse. Companies crunch to
push through on a specific game, but the long-term effect is that talented
developers, artists, producers and designers burn out and leave the industry.
"加班一点也没有意义，因为它只会把游戏搞砸。公司会用加班来压缩特定游戏专案，但
长期来看会把有才能的开发者，美术人员，制作人，设计人榨干，逼得他们不得不离开这
个产业。"
“Companies and individuals should stop wearing their time spent crunching as
a badge of honor. Crunch is a symptom of broken management and process.
Crunch is the sacrifice of your employees. I would ask them why crunch isn’t
an issue with other industries. Why isn’t crunch an issue at all game
studios?
"公司与开发者应该停止把[时间花在加班]当作荣誉的象征。加班是崩坏管理与流程的病
征。加班是员工的牺牲品。我问其他产业为何他们不需要加班？为什么不是每个游戏工作
室都需要加班？"
“Employees should see it as a failure. Gamers should be concerned about it,
because in the long term the hobby they love is losing talent because of it.
Companies should do everything in their power to improve their processes to
avoid these consequences.”
"员工应该把这件事视为失败。游戏开发者应该认真关心此事，因为长期来看，他们对游
戏的爱会因此而遗失。公司应该要尽其可能改善流程来避免这些后果。"
So who is right – Spector and Rubin, or Paxton?
所以谁才是对的？Spector 及 Rubin，还是 Paxton？
[Full disclosure: team member Paul Tozour leads Mothership Entertainment,
whose flagship game is being published by Stardock.]
[抢先报：Paul Tozour 率领母舰娱乐这间公司，他们的主打游戏会被 Stardock 发布。]
In the Game Outcomes Project survey, we provided 3 text boxes at the end that
respondents could use to tell us about their industry experiences. Where
they mention crunch, they invariably mention it as a net negative. One
respondent wrote:
在游戏专案为何成功的问卷中，我们设计了三个开放字段给回答者，让他们告诉我们产业
的经验。关于提到加班的部分，不约而同地都提出负面的说法。其中一个回应这样写着：
“The biggest issue we had was that the lead said ‘Overtime is part of game
development’ and never TRIED to improve. As sleep was lost, motivation
dropped and the staff lost hope ... everything fell apart. Hundred-hour
weeks for nine months, and I'm not exaggerating. Humans can't function under
these conditions ... If you want to mention my answer feel free. I'm sure
it'd be familiar to many devs.”
"我们最大的问题就是管理者说：[加班是游戏开发的一个部分]，而从未试着改善，当睡
眠不足，热情与希望也会随之降低与剥离。我说真的，人类不能在九个月每周上百个小时
的加班这样的环境下工作... 假如你们希望我老实讲，我敢保证其他团队状况都相同。"
Another developer put it more bluntly:
另一个开发者说得更难听：
“Schedule 40 hours a week and you get 38. Schedule 50 and you get 39 and
everyone hates work, life, and you. Schedule 60 and you get 32 and wives
start demanding you send out resumes. Schedule 80 and you’re [redacted] and
get sued, jackass.”
"一周四十个小时的工作，那么工作效率差不多是三十八小时。如果排五十个小时，那么
就会得到三十九小时外加痛恨工作，痛恨人生，及痛恨管理阶级的员工。如果排了六十个
小时，那么会得到三十二个小时的效率外加离职潮。排八十个小时的工作，那么只会收到
存证信函。"
In this article, we will be getting a final word on the subject from the one
source that has yet to be interviewed: the data.
这篇文章中，我们会访问我们的案例，也就是那些我们手中的数据资料，对这个题目给一
个总结。
The “Extraordinary Effort” Argument
"超凡努力（加班，代表着热情）"理论
We’ll begin by formulating the “pro-crunch” side of the discourse into
testable hypotheses. Although no one directly claims that crunch is good per
se, and no one denies that it can have harmful effects, Spector and Rubin
clearly make the case in the article above that crunch is often (if not
usually, or even always) a necessary evil.
虽然没人直接声称加班本身就是好事，也没有人否认它有害，Spector 与 Rubin 清楚地
在前面的说法也证实通常（并非总是）加班是必要之恶。但我们先试着以"加班是好事"这
个论点来做个整理。
According to this line of thinking, ordinary development with ordinary
schedules cannot produce extraordinary results. We believe an accurate
characterization of this viewpoint from the gamesindustry.biz article quoted
above would be: “Extraordinary results require extraordinary effort, and
extraordinary effort demands long hours.”
也就是假设这样的思路下去思考，正常工期的开发方式没办法制作杰出作品。也就是相信
Gamesindustry.biz的访问中所提到论点："超凡的成果来自于超越极限的努力（超凡的努
力），而超越极限的努力需要长时间付出，也就是加班。"
This position (we’ll call it the “extraordinary effort argument”) leads
directly to two falsifiable hypotheses:
1. If the “extraordinary effort argument” is correct, there should be a
positive correlation between crunch and game outcomes, and higher levels of
crunch should show a measurable improvement in the outcomes of game projects.
2. If the “extraordinary effort argument” is correct, there should be
relatively few, if any, highly successful projects without crunch.
这个论点（我们姑且称为超凡努力的论点）直接就会发现两个反证：
# 假如超凡努力的论点是对的，那么在我们的问卷中加班与游戏产出分数上会有正相关
，越加班，就应该会产出优秀的作品。
# 假如超凡努力的论点是对的，那么应该不可能发生没加班却高度成功的专案。
Luckily for us, we have data from hundreds of developers who took our survey
with no preconceptions as to what the study was designed to test, and which
we can use to verify both of these statements. We’ll agree to declare
victory for the pro-crunch side if EITHER of these hypotheses remains
standing after we put it in the ring with our data set.
很幸运地，我们从问卷中得到数百分开发者的回应，可以透过这些数据来证实这两件事，
尤其是在我们设计之初并没有故意去设计这样的实验。假如数据中告诉我们超凡努力论点
是对的，那么我们也会宣告加班是胜利之道，并将其放在结论的皇冠上。
Crunching the Numbers
We’ll approach our analysis in several phases, carefully determining what
the data does and does not tell us.
加班数字
我们接着从数个步骤来分析，小心地看那些数据透露，或没有透露的事。
Our 2014 survey asked the following five questions related to crunch, which
were randomly scattered throughout the survey:
#“I worked a lot of overtime or ‘crunched’ on this project.”
#“I often worked overtime because I was required or felt pressured to.”
#“Our team sometimes seemed to be stuck in a cycle of never-ending crunch
/ overtime work.”
#“If we worked overtime, I believe it was because studio leaders or
producers failed to scope the project properly (e.g. insufficient manpower,
deadlines that were too tight, over-promised features).”
#“If I worked overtime, it was only when I volunteered to do so.”
我们在2014年的问卷中问了以下关于加班的问题，在问卷中我们还把它们都随机排列：
# 我在专案中超时工作。
# 因为感受到压力，我常常超时工作。
# 我们的团队常常感觉到受阻碍，并陷入无止尽的加班。
# 需要加班的原因是领导层与制作人在时程上搞砸了。（人力不足，估计期限过短，过
度承诺）
# 我加班是因为我自愿加班。
Here’s how the answers to those questions correlate with our aggregate
project outcome score (described on our Methodology page). On the horizontal
axis, a score of -1.0 is “disagree completely” and a score of +1.0 is “
agree completely."
这里是这些答案与总和专案产出分数的相关分数（方法论在我们的部落格已描述），水平
轴是从-1.0的完全不同意，到1.0的完全同意。
Figure 1. Correlation of each crunch-related question with that project’s
actual outcome (aggregate score). Each of the 5 questions is shown, as an
animated GIF with a 4-second delay. Only the horizontal axis changes.加班相关
的问题与总和产出分数的关联性，每个问题以一个四秒的周期显示出来
The correlations are as follows: -0.24, -0.30, -0.47, -0.36, +0.36 (in the
same order listed in the bullet-pointed list above). All five of these
correlations have statistical p-values well below 0.001, indicating that they
are statistically significant. Note how all the correlations are strongly
negative except for the final question, which asked whether crunch was solely
voluntary.
关联性依序是-0.24，-0.30，-0.47，-0.36，0.36（顺序如问题序）。五个关联性都有少
于0.001的统计p值。也就是具有统计表征。注意除了最后一个问题自愿加班之外，这里关
联性都是强烈的负向。
“But wait,” a proponent of crunch might say. “Surely that’s only because
you’re using a combined score. That score combines the values of questions
like ‘this project met its internal goals,’ which are going to give you
lower values, because they're subjective fluff. Of course people who are
unhappy about crunch are going to give that factor low scores – and that’s
going to lower the combined score a lot. It’s a fudge factor, and it’s
skewing your results. Throw it out! You should throw away the critical
success, delays, and internal goals outcomes and JUST look at return on
investment and I bet you’ll see a totally different picture.”
但加班的支持者可能会说："等等，这一定是因为这里是一个总合分数，包含了内部满意
度，这当然会有负分，因为那是主观意见，加班就是会让人不开心，才会导致总合分数这
样发展，应该要排除在外！我们应该要只看利润的产出分数，一定可以看到不同的结果。
"
OK, let’s do that:
那么我们也从善如流：
Figure 2. Correlation of each of the 5 crunch-related questions with that
project’s return on investment (ROI). As with Figure 1, each of the 5
questions is shown, as an animated GIF with a 4-second delay. Only the
horizontal axis changes. Note that many of the points shown represent
multiple coincident points. See our Methodology page for an explanation of
the vertical axis scale.五个加班相关问题对上专案利润的关联度，如图一相同，每个
问题以一个四秒的周期显示。只有在水平轴不同。每个点都可能代表重合在一起的点。在
垂直轴的缩放方式请参照我们的部落格网页。
Notice how the lines have essentially the same slopes as in the previous
figure. The correlations with ROI are as follows (in the same order): -0.18,
-0.26, -0.34, -0.23, and +0.28. All of these correlations have p-values
below 0.012.
注意到回归线仍与前一张图相同吗？利润的关联性是：-0.18，-0.25，-0.34，-0.23，及
0.28。全部关联性都有小于0.012的统计p值。
Still not convinced? Here are the same graphs again, correlated against
aggregate reviews / MetaCritic scores.
不相信吗？这张图也一样，对上网页分数的关联性：
Figure 3. Correlation of each of the 5 crunch-related questions with the
project’s aggregate reviews / MetaCritic score (note that the vertical axis
does not represent actual MetaCritic scores but is a normalized
representation of the answers to this question; see our Methodology page for
more info). As with Figures 1 and 2, each of the 5 questions is shown, as an
animated GIF with a 4-second delay. Note that many of the points shown
represent multiple coincident points. Only the horizontal axis changes.五个加
班问题对上网页分数的关联性（注意垂直轴并非表示MetaCritic真正分数，而只是一个对
问题经过正规化的数值。更多资讯，请看我们的部落格）如同图一与图二，五个答案都以
一个四秒周期的方式显示。每个点都可能代表重合在一起的点。只有在水平轴的参数是不
同的。
The results are essentially identical, and all have p-values under 0.05.
结果一样，全部都具有小于0.05的统计p值。
So if our combined score has a negative correlation with ALL our crunch
questions except the one about crunch being purely voluntary (which itself
does not imply any particular level of crunch), that means that we’ve
disproven the first part of the “extraordinary effort argument” – the
correlation is clearly negative, not positive.
总合的分数对上除了自愿加班之外的所有加班问题都是负向的关联。意思是我们能够推翻
超凡努力论点，很清楚，就是没有正相关。
Now let’s look at the second testable hypothesis of the “extraordinary
effort argument.”
In Figure 4 (below), we’re looking at the two most relevant questions
related to overall crunch for a project. The vertical axis is the aggregate
outcome score, while the horizontal axis represents the scale from “disagree
completely” (-1) to “agree completely.” The black lines are trend lines.
As you can see, in both cases, higher agreement with each statement
corresponds to inferior project outcomes.
接着来看看我们对于超凡努力理论的第二个辩证。
在下面的图四中，我们取出两个对加班问题中最相关的问题。垂直轴是总合产出分数，同
时水平轴是从完全不同意的-1，到完全同意。黑色的线是趋势线。如你可见，在两个问题
中，越高的同意带来越低的总和分数。
Figure 4. The two most relevant questions related to crunch compared to the
aggregate project outcome score.两个相关问题对上总和的产出分数。
We’ve added horizontal blue and orange lines to both images. The blue line
represents a score of 80, which will be our subjective threshold for “very
successful” projects. The orange line represents a score of 40, which will
be our threshold for “very unsuccessful” projects.
我们接着加上了的蓝色与橘色水平线。蓝色线是80，也就是我们主观认定非常成功专案。
橘色线则代表40。也就是我们主观认定非常不成功的专案。
The dots above the blue line tell a clear story: in each case, there were
more successful games made without crunch than with crunch.
在蓝线之上的点的分布清楚了代表一件事：多数成功的游戏没加班的数量比加班的多。
However, these charts don’t tell the full story by themselves; many of the
data points are clustered at the exact same spot, meaning that each dot can
actually represent several data points. So a statistical deep-dive is
necessary. We’re particularly interested the four corners of the chart –
the data points above the blue line on the extreme left and right sides of
each chart (below -0.6 and above +0.6 on the horizontal axis) and below the
orange line on the left and right sides.
然而，只是图并不能将细节全盘托出，很多数据点都重合在一起，分不清楚有几个。所以
我们需要再深一步的统计分析。我们对这张图的四个角落特别有兴趣。也就是蓝线以上的
左右端，以及橘线以下的左右端。（水平轴以-0.6及0.6为界线）
Looking solely at the chart on the top of Figure 4 (“I worked a lot of
overtime or ‘crunched’ on this project”), we observed the following
pattern. Note that the percentages are given in terms of the total data
points in each vertical grouping (under -0.6 or above 0.6 on the horizontal
axis).
独立看图四上半（我在专案中超时工作。）我们观察到后述的模式。注意那些比例是以水
平轴已经切开（为左右两群组）的群组来计算。
We can see clearly that a higher percentage of no-crunch projects succeed
than fail (17% vs 10%) and a much larger percentage of high-crunch projects
fail rather than succeeding (32% vs 13%). Additionally, a higher percentage
of the successful projects are no-crunch than high-crunch (17% vs 13%), while
a higher percentage of the unsuccessful projects are high-crunch vs no-crunch
(32% vs 10%).
我们可以很清楚地看到非加班的区块成功数量是高于失败数量（17%多过于10%），高度加
班的区块中，失败却高于成功（32%多过于13%）。成功专案中，不加班多于加班（17%多
过于13%）。失败专案中高度加班的情形多过于不加班（32%多过于10%）。
Here’s the same chart, but this time looking at the bottom question, “Our
team sometimes seemed to be stuck in a cycle of never-ending crunch /
overtime work.”
同样的图表中，我们看下半个问题：我们的团队常常感觉到受阻碍，并陷入无止尽的加班
。
These results are even more remarkable. The respondents that answered “
disagree strongly” or “disagree completely” were 2.5 times more likely to
be working on very successful projects (23% vs 9%), while the respondents who
answered “agree strongly” or “agree completely” were, incredibly, more
than 10 times more likely to be on unsuccessful projects than successful ones
(41% vs 4%).
结果更强烈。在不加班（回应强烈不同意与完全不同意）的案子中成功专案是超过不成功
专案的两倍半（23%对上9%）。在高度加班中（强烈同意与完全同意）的案子中不成功的
案子则是成功案子的四倍（41%对上4%）
Some might object to this way of measuring the responses, as it is an
aggregate outcome score which takes internal achievement of the project goals
into account – and this is a somewhat subjective measure. What if we looked
at return on investment (ROI) alone? Surely that would paint a different
picture.
有些人可能会质疑总和的分数包含了专案的内部满意度，当然就是主观意见。那假设我们
只看专案利润？会有不同的结果吗？
Here is ROI:
专案利润的图在此：
Figure 5. The two most relevant questions related to crunch compared to
return on investment (ROI).最相关的两个问题对上专案利润的关联性
The first question (top chart) gives us the following results:
第一个问题的结果如此：
The second question (bottom chart) gives us:
第二个问题的结果如此：
These results are essentially equivalent to what we got with Figure 4