The Modernity Cover Concept survey was open for 10 days and gathered 114 responses from posts on in Fate-related Facebook groups, subreddits, Twitter, and one the email newsletter for San Antonio Meetup group. There’s not a lot of interesting trends for responses by day of response, but the second day garnered the most responses which might lead one to conclude that reddit folks like to express their opinions and generally act immediately, if they’re going to act at all, but that’s only speculation based on the fact that the survey was posted to reddit the day after Facebook and Twitter: Survey Monkey doesn’t have a built-in mechanism for tracking referral source without making some changes to the survey structure. (Lesson #88 learned there.)
In quantitative terms, the two primary avenues of exploration were 5-point feeling thermometers for each of four concept images created to test several stylistic elements against the following questions:
- Based on this cover style, would you buy this game?
- Based on this cover style, what ESRB rating would you give this game?
Anecdotal, open-ended input was also requested for each concept.
Here’s how the data of responses graph out:
Overall, I’m happy with the quantity of anecdotes. It always falls off a little bit throughout a survey as people try not to repeat themselves or just feel like there’s nothing to comment on, but in this case the anecdote curve is nearly flat, which is great! I’m very glad (as a data scientist) that you’re willing to share. While I’m reading them, though, I have to send other my wanna-be creative and somewhat autistic selves on vacation. . . The Fate Fan Club isn’t shy about being brutally honest!
“Personally, I think the art style lacks polish and gives it an amateurish feel. I expect the game inside to be similarly less than finished.”
“Nope. Drag this out back and shoot it. This looks like it’s done by a 14 year old with a freshly pirated copy of Illustrator who brags that they’re gonna make 6 figures as a freelance design guru no really check out my deviantART.”
The next two demonstrate the trend of getting polar opposite feedback from members of the same audience about the same image. . .
“I like that the cover feature women in a way that’s not sexualised, but I kinda hate the monster and that 3D vibe. It breaks the noir mood.”
“Looks both clunky and overly sexualized”
Across concepts there were respondents that completely disagreed, which is to be expected. But I’m not sure what to do with an equal number of people who say opposite things about the same image:
- “Not Enough Detail!” and “Too Much Detail!”
- “It looks amateurish and lacks polish.” and “I love the look and the promise of what’s inside.”
- “I love the daemon” and “I hate the stupid demon”
- “It’s a bunch or mismatched stuff thrown together.” and “I like the different characters shown off, makes me associate the game with variety.”
- “Cthulhu is a nice touch” and “Meh, another RPG that uses Cthulhu-like artwork”
I shall ponder upon the duality of man and find a way to make peace with the fact that I can’t please them all. . .
In particular, I am curious about why #37 seems less rated M for Mature than the other three. Anecdotes for #17 and #23 seemed to indicate that a few of folks assumed the female spellcaster was naked (I’m curious about why some respondents came to that conclusion–does the color white indicated nakedness in some parts of the world?) and a minority of respondents were hyper-focused on the breasts of the same character (which somehow makes me the misogynist or the over-sexed teenager?), but there weren’t any breast-focused or porn-related commentary for #42 (as if all well-endowed women star in porn?) which would naturally lead to a similar above-Teen rating. Maybe Cthulhu Himself by definition automatically rates one ESRB level bump?
Awkward transition. Not sure what to do with this one:
“How old are you guys? Having women issues much? Wow…”
Short answer: No, not really.
Fortunately, the majority of respondents gave helpful feedback about specifically what worked for them or what didn’t.
“Preferred the demon to Cthulhu, although I can see having him might draw in certain folks. Prefer this card layout to the torn paper. Best design yet.”
“This one makes me think we’ll be dealing with Cthulhu, or some Cthulhu type themes, which makes me think of a more mature theme. Also, makes me want to get it more.”
I’m curious to know what specifically prompted this one (saying “everything about it” isn’t helpful, either):
Lots of stuff to think about. And discuss with my wife of 25 years and long-time business partner.
It goes without saying that I’m bummed that very few folks took these as prototypes for comparison purposes, instead judging them as finished work. But, I’ll take the feedback however I can get it. I already knew why most creatives refuse to share anything that’s not (in their own opinion) perfectly polished, which I believe prevents a lot of good feedback from flowing, but I also get that early feedback can often be unpleasant.
For the record, I didn’t get anywhere near a large enough sample to generalize broadly, and in retrospect there are a couple of important qualifying questions that I failed to ask (bad survey designer!) which would have shed more light on the WHY of the responses, which leaves me to focus on the WHAT of the responses we do have for this analysis. For example, I think that some of the hypernegative feedback were simply people who are averse to UF or ON as genres in the first place, but there’s no way to discern that from this survey as designed. Next time!
With #42 the only concept to receive any #5 rankings (“Hurry up and take my moneys!”) and the fewest #1 rankings (“Nope!”), it might seem the clear winner. But, #17 also clustered very heavily around a #3 ranking, it just had a couple more #1s to drag it down and no #5s to pull it up. Although there were anecdotes from people who preferred either #23 or #37, they were far fewer than those that prefer the alternatives and they didn’t love either enough to give either a single #5 ranking.
Although most respondents agreed that Modernity (regardless of cover concept) was Teen-rated, for some reason there’s something about #37 (the cover with ripped out tabs) that pulled huge for E10+, which drug the overall rating estimate down. Not sure if it’s the simpler coloring of the characters, using a head-n-shoulders view instead of full figure, the lack of Cthulhu, or some combination of the above. Curiouser.
Still, the data and the anecdotes will be enormously helpful in charting the course forward for Modernity, its art style, as I churn forward finalizing the next iteration of the product for 2018.
Thank you all very much. (Yes, you, too.)
The percentile-oriented analog random number generators had their say, and they decided that the winner of our $50 DriveThruRPG gift certificate is. . .
Congratulations to Lauren Woods!
The digital transmission has already happened, and Lauren has been encouraged not to spend it all in one place (although, that’s going to happen regardless the silly advice I gave). Perhaps at some future date Lauren will be willing to disclose what was purchased with the windfall? Inquiring minds and all that, you know.
I’m always open to whatever feedback whenever you’re willing to share through whatever medium you’re willing to share it. Some people need something to hate, and I’m OK with being the focus of evil in the modern world for those folks if that’s what they need to feel better about themselves. I learned a long time ago that unhelpful negativity from others isn’t really about me anyway.
For those of you who’ve offered specific actionable feedback or to help or collaborate in some way, you’re wonderful and I’m grateful for you. Be looking for follow up email from me soon. Until then, may we all be rewarded for our intentions, not our actions.