Analyse your reader survey in ten minutes

It's Wednesday night. Your board meets on Thursday, and you've got 212 survey responses in a spreadsheet. Column F is free text. Someone's written four paragraphs about the 2019 redesign, someone's written "good", and someone has asked for more recipes, which is odd, since you're an institute of structural engineers.

You ran the survey to find out what readers want. What you've got is a very long evening. So most surveys end the same way: you read the first thirty comments, take away whichever ones annoyed you most, and report that "members were broadly positive".

Claude can read all 212 before the kettle's boiled. The ten minutes in the title gets you a checked first read, which is usually what Thursday needs. A proper research report takes longer, and you don't need one.

Minute one: take the names out

Export the survey as a spreadsheet and make a copy. In the copy, delete the columns with names, email addresses and member numbers. Have a quick look through the comments as well, since people do sign them, and "as the only female surveyor in Wagga" identifies someone just as well as a name would.

Check your association's data policy and your Claude plan's terms before you upload anything. It's far less trouble now than it would be after.

Minutes two to three: get the counts

Attach the file and ask for the plain numbers first:

This is a reader survey for our member magazine. For each multiple-choice or rating question, give me the number of responses and the results as counts and percentages. Tell me how you worked them out. If a question has blanks, say how many. Don't interpret anything yet.

Keep the "don't interpret". You want to look at the numbers before you've been told what they mean.

Minutes four to six: find the themes

Now the part that would have taken your evening:

Read every comment in column F. Group them into the main themes, and give me the number of comments in each. For each theme, give three quotes, word for word, with the row number. Add a separate list of any comment that's unusual, specific or that asks for something nobody else asked for.

The row numbers keep everyone honest, and that last sentence is there for a reason. The common themes will mostly confirm what you already suspected. It's the one comment from someone who prints the magazine for the staff room in a mine site, because there's no reception, that changes how you think about your readers.

Minutes seven to eight: check it

Claude reads quickly, and it can still be wrong. Pick five quotes and look them up by row number. Check that they exist, that they're word for word, and that they mean what the summary says they mean. Then count one theme yourself by filtering the column for a keyword. If a quote's been tidied or a count looks off, ask Claude to go back and explain. If something's still wrong after that, don't use that part.

Then read twenty comments yourself, chosen at random. It takes two minutes, and it stops you presenting a summary of something you haven't read.

Minute nine: ask what you shouldn't conclude

This is the question most people skip:

Based on who answered and how many, what would it be wrong to conclude from these results? What can't this survey tell us?

Here's the catch with every reader survey: the people who answer are the people who read the magazine and care enough to fill in a form. "87% of respondents enjoy the magazine" may be telling you about the 87% of a small group of fans. The 3,000 members who've never opened it weren't asked, or didn't reply. So always give the response count, and say what share of your readers that is.

Minute ten: compare it with what they did

People are honest in surveys, but they're also a bit aspirational. They'll tell you they want long features on policy. Then they'll click the checklist.

So put your survey next to your numbers. Zeenly shows you reads, unique readers, link clicks and video plays for each issue, so you can see if the things people asked for are what they used. If you've connected Claude, you can ask for those figures in the same chat with "how many people read the September issue, and what got clicked?", then:

Here's what readers said they want, and here's what they read and clicked. Where do they match, and where do they disagree? Give me the three most useful differences.

Where they disagree is where it gets interesting. The five numbers worth tracking will give you more to compare over time.

Get it onto one page

Your board doesn't want the spreadsheet. Ask for a draft and edit it:

Write a one-page summary for our board: how many responded, the three clearest findings, two quotes, what we can't conclude, and three changes we recommend. Plain English, no adjectives. Use only the figures above.

The recommendations are yours to change, because you know what's affordable and who'll object. Claude doesn't know the treasurer. Then do what most surveys never do, and tell readers what you heard. "You asked for shorter features, so we've cut them to 800 words" in the next editor's letter gets more people to answer next time.

You don't need to wait for your next survey to try this. Dig out the last one, the one nobody got round to analysing, and give it ten minutes. As for the recipes, that's your call.