How to tally survey responses by hand: the tally sheet, multi-select answers and the check total

The method in seven steps

Hand-tallying a paper survey takes seven steps: number the forms, make one tally sheet per question, write down your rules, tally one question at a time, check every batch against the forms, code the write-ins, and divide by a base you name. One worked example runs through this guide: a feedback form handed out at a free community event that 75 people attended and 60 people returned. The event and its answers are made up for illustration; every figure worked out from them is computed, and each one can be checked from the tables.

1. Number every form, 1 to 60, in a corner, before you read any answer. A form you cannot find again is a form you cannot recount. University of Wisconsin-Extension's method for open answers starts the same way: "assign each form a unique identification number" (Quick Tips #20).
2. Make one tally sheet per question, with a row for every printed answer and the rows that catch everything else (below).
3. Write your rules down before the first stroke: what counts as a tick, what a crossed-out tick means, what to do with an answer in the margin. "Keep track of the decisions you make by writing them down," says UW-Extension's tip sheet on data accuracy (Quick Tips #22), so you can apply them the same way on form 60 as on form 1 and explain them afterwards.
4. Tally one question at a time, through the whole stack, in batches of ten forms.
5. Check each batch before you start the next: on a single-choice question, ten forms make ten strokes.
6. Code the write-ins from any "Other, please specify" box.
7. Work out percentages, and state what you divided by.

One tally sheet per question

Give each question its own sheet, with a row for every answer printed on the form, then a row for "left blank" and, on a single-choice question, a row for "two or more ticked". Those extra rows are what make the sheet checkable: every form lands in exactly one row, so the rows must add up to the number of forms.

Going through the stack once per question keeps your eye on one place on the page and one set of rows. Reading each form from top to bottom and marking several sheets handles each form only once, but every form means switching sheets, and a stroke in the wrong question's row is the error that follows. Both work; the check totals below catch either kind of slip.

The basics of the sheet itself, the question written at the top and every category listed before the first stroke, are in the guide to setting up a tally chart. What a survey adds is the forms that do not give one clean answer, and the questions that allow several. If you would rather tap than draw, the tally counter keeps one named count per row with an undo button, which a paper sheet does not have.

Blank answers, two ticks and "does not apply"

A blank, a double tick and an answer to a question the form told someone to skip are three different things, and each needs a rule written down before tallying, so that none is lost and none is counted as an answer.

Two ticks on a single-choice question have no single answer, and choosing one for the respondent puts your judgement into the data. Election counts treat the same case strictly. The Texas Secretary of State's hand counting procedures say that when a voter marks more candidates than allowed, "none of the voter's selections in that specific race can be counted", and that overvotes and undervotes should still go on the tally lists "for reconciliation purposes to verify the number of ballots that have been counted". A survey is not bound by election law, but the reasoning carries over: the double-tick row keeps the count of forms whole.

An answer that should not be there counts as missing. Where a form says "if no, go to question 5" and someone answers question 4 anyway, UW-Extension's advice is that "usually you will want to consider their inappropriate answers as 'missing' data" (Quick Tips #22). For questions that do not apply to some people, its guide to analysing data says that "only the number of persons who actually answer the particular question is used in calculating the percentage" (Analyzing Quantitative Data, G3658-6).

In the example, question 1 asked "Overall, how would you rate the event?" Four forms left it blank and two ticked two boxes, so 54 of the 60 forms gave one valid answer.

Check-all-that-apply questions

On a tick-all-that-apply question, tally every box as its own count, one stroke per tick, and keep a second tally of how many boxes each form ticked; the option rows will add up to more than the number of forms, and that is correct. A form with three ticks puts one stroke in three rows.

Question 2 in the example asked "How did you hear about today's event? Tick all that apply", with five boxes: a poster in the library, social media, the email newsletter, a friend or family member, and "Other (please write in)".

That second tally gives two checks no single row can. Its forms column must equal the 60 forms returned, and its ticks column must equal the 84 strokes in the option rows. If the option rows come to 83 or 85, a stroke was missed or doubled, and the batch totals (below) say which ten forms to look at again. The multi counter suits this question well: one card per option and one per "boxes ticked" count, all on screen at once in grid view.

The percentages will not add up to 100, and should not

The shares on a tick-all question add up to more than 100% because one person can be in several rows. University of Wisconsin-Extension's guide says it plainly: "Percentages are added only when categories are mutually exclusive (do not overlap). This is not the case in multiple choice questions where the respondent may select several answers" (G3658-6). Forcing the column to 100 by dividing by total ticks changes the question being answered, as the percentages section shows.

An empty box is not a "no"

A box left unticked on a tick-all list means "not ticked", not "no", and reporting it as no understates how many people the option applies to. Pew Research Center asked the same items both ways in a survey of 4,581 US adults from 30 July to 12 August 2018: as one select-all list, and as a separate yes-or-no question for each item. The share saying they had been denied coverage by a health insurance company was 19% when asked yes or no, and 13% on the checklist, and Pew has since adopted "a policy of using a forced-choice format instead of a select-all-that-apply list" in its online surveys "whenever possible" (Pew Research Center, 9 May 2019).

Two practical consequences for a paper form. Report a tick-all result as "ticked", never as "said yes". And when a real yes or no matters, ask it as a yes-or-no question for each item, which tallies as two rows per item on a yes / no counter. Pew's lists also offered a "none of these" box, which separates "none applies" from "skipped the question": a blank form cannot tell you which.

"Other, please specify" write-ins

Read every write-in, sort it with a written list of categories, and move it into a printed option only when it clearly means that option and the form has not already ticked it. UW-Extension's ten-step method for written answers (Quick Tips #20) builds a "coding scheme" of categories that "do not overlap", keeps "a written record of the coding decisions that you make", has a second reader code the same answers with "a minimum of 80 percent" agreement as the target, and turns answers that fit nothing into "emergent" categories.

The last row is the one that gets miscounted. That form already has a stroke for the poster; a second one would count one person twice for one option and push the poster's share of respondents up. Instead the form loses its Other tick and now has one tick, not two, so the second tally moves too: 37 forms with one tick and 14 with two.

The checks still hold: 3 blank forms and 57 forms with at least one tick make 60, and 37 × 1 + 14 × 2 + 6 × 3 = 83 ticks, the total of the option rows.

The check total, batch by batch

Every single-choice sheet must add up to the number of forms returned, and checking it every ten forms rather than once at the end tells you which ten forms to recount when it fails. Draw a line under each batch on the sheet, or keep a column per batch.

Here is what the check is for. On the first pass, forms 31 to 40 added up to 11, with five strokes for Good. Recounting those ten forms alone found form 34 tallied twice; Good went back to 4 and the batch to 10. Without batch totals the sheet would simply have come to 61, and every one of the 60 forms would have needed reading again.

Two people, one count

For a count that has to stand up, have one person read each answer aloud and two others tally it on separate sheets, then compare the sheets at the end of every batch. That is how Texas election workers hand-count ballots: a counting team of two or more, in which one "caller" announces each vote and the others are "talliers" who record it on their own tally list. Three tally lists are kept per team and compared during the count, and a difference that cannot be resolved means the ballots "must be recounted" (Texas Secretary of State, hand counting procedures). UW-Extension suggests the same pairing for typing survey data into a computer: "have one person read the values aloud and another person check the screen" (Quick Tips #22).

Percentages: one count, three answers

A percentage is a count divided by a base, and a survey offers at least three: the people you asked, the forms returned, and the forms that answered the question; a tick-all question adds a fourth, all the ticks. UW-Extension's guide puts it as a rule: "It is important to use the right base and to indicate which base you've used." Its own example is 100 forms with 70 yes, 20 no and 10 blank: 70% yes of everyone, 78% yes of the 90 who answered (G3658-6).

The headline moves with the base. Excellent or Good together is 41 forms: 54.7% of the people who came, 68.3% of the forms and 75.9% of the valid answers, so "three in four rated it excellent or good" and "just over half did" are both honest readings of the same strokes. The usual choice for a rating question is the valid answers, with the blanks and double ticks reported beside it, which is one of the two options UW-Extension gives; the 75 people who came are the base for the response rate, 60 of 75 or 80%. The rounded columns add up to 99.9 and 100.1, not 100: the rounding section of the tally to percentage calculator explains why and how to make them total exactly.

The share of ticks answers a different question from the share of people. "30% heard about it from the poster" is wrong: 30.1% is the poster's share of all 83 ticks, while 43.9% of the people who answered ticked it. Statistics software reports both by default; IBM's SPSS documentation says a multiple-response frequency table contains "the count for each value, the percentage of responses, and the percentage of cases" (MULT RESPONSE, FREQUENCIES subcommand), where cases are people.

The respondent column carries its own check. It adds up to the average number of ticks per person, times 100: 83 ticks ÷ 57 people = 1.456, which is the 145.6% at the foot of the column. If your column adds up to something else, a percentage was divided by the wrong base. To get these shares without the arithmetic, enter 57 as the total in the tally to percentage calculator, which then reports each option on its own instead of forcing the column to 100.

Percentages of part of the sample

When you report a percentage for part of the sample, such as members only or under-30s, give the number of people that part contains. The American Association for Public Opinion Research's code lists "the unweighted sample size(s) on which one or more reported subgroup estimates are based" among the facts a survey must disclose on request (AAPOR Disclosure Standards, Code of April 2021, section III.B.6). A share of 12 people moves by more than 8 points with each person, and the margin of error grows as the group shrinks.

What this guide does not cover

This guide covers counting paper or printed survey answers by hand: the tally sheet, the rows for answers that do not fit, tick-all questions, write-ins, the check totals and the base for a percentage. It does not cover weighting a sample, testing whether two groups differ, or margins of error, which the guide to how many people to survey explains; nor the average of a rating scale, which UW-Extension's G3658-6 works through; nor the full analysis of long written answers. The election procedures quoted above are a model of careful counting, not rules that apply to a survey, and the event, its 75 attendees and its 60 forms are an example made up for this guide.

Simple Online Counter is a free, browser-based set of counters and calculators. Its tally counter and multi counter keep named counts in the browser with an undo, and its tally to percentage calculator divides a set of counts by whichever total you choose.