The Waiter Problem: Why Authors Need True Product Testing

Treat your novel like a product, not just art. Traditional feedback loops (like paid developmental edits on full manuscripts) are often expensive, biased, and inefficient because they rely on "unnatural" reading behaviors. Instead, authors should adopt an iterative product-testing mindset. By using the "retelling technique"—observing a reader interact with a single chapter and having them recount the story—you can objectively verify if the reader understands the plot and feels the intended emotions, allowing you to fix "bugs" before they destroy the reader's experience.

Unlike other customer service industries, publishing has an unavoidable parasocial factor. It best resembles lifestyle cosmetics or athletic gear. Buying Selena Gomez's Rare Beauty Soft Pinch Liquid Blush or Air Jordans, for example, begins a largely imagined and one-sided affiliation. Then, when products perform well, stronger bonds and more sales follow. Meanwhile, a failure to meet expectations often triggers animosity and bad word of mouth. Because satisfying customers comes first (and always), lifestyle companies invest substantial time and effort testing products. Authors face the same challenge: satisfying reading earns loyalty, even as bad experiences destroy once promising relationships. Yet, authors' and publishers' product testing falls far short of comparable industries. We want to trust "professional" intuition, a big mistake that will cost you money in upfront costs and a less compelling product.

Part of the problem comes from novels' complexity. As individual creative acts, they're unique in ways lipstick and shoes aren't. Still, they're almost identical in how readers invest and experience books in isolation. Fine dining illustrates what novelists and similar producers can never do. Imagine a waiter, perhaps from The Bear," constantly monitoring whether the meal is going well. They hover, always ready to refill a water glass while watching for the right moment to bus the table and bring the next course. With lipstick, shoes, and novels, the creator is far away, sometimes dead. Too often, we release our children into the world and hope for the best. Sometimes, we hire a professional or trust a writer friend to review our work. Neither approach works. You need direct product testing to take the place of that waiter. The idea is for you to anticipate and solve problems before they happen and ensure that your reader gets the story, feels the intended emotions and has the desired overall experience.

How do you product test a novel? The watchwords are natural and spontaneous. Critically, this approach looks nothing like what you normally do, namely, ask for comments. By asking real genre readers to retell the story in their own words and surreptitiously observing other reactions, you'll be able to directly assess a reader's understanding as well as their subjective experience. This practice leads to reader satisfaction by eliminating confusion and boredom in favor of the desired emotion. It also lets you get to know potential fans. Finally, retelling makes revision more efficient and largely eliminates the need for expensive development editors. You should try it at least once in your career to see.

The technique, in a nutshell, replicates actual reading. You set your volunteer reader up in a comfortable place and have them read a chapter while surreptitiously watching them. Let them stop anytime they have questions. Afterwards, you or an assistant instruct them to retell the chapter in their own words, then -- and only when they're done -- will you probe for more specific information. Your notes from this session, with Bardsy's other tools, will guide your revisions. Our next blog covers this process in detail.

You can use this test as necessary, especially on key chapters. Over time, you'll become more familiar with your audience, learn to anticipate their reactions, and become a more effective writer. You also won't need paid editors or as much help from colleagues.

The standard way to beta read involves finding a fellow author or professional, handing them an entire novel and getting pages of comments. This post's observations are based on an examination of paid developmental editing reports. While some editors share more useful observations than others, the overall tendency is clear. Authors value the emotional support on offer, but it comes at a substantial cost—for developmental edits, often in the neighborhood of a few thousand dollars. In terms of producing a satisfying manuscript, this practice is inefficient at best and often completely ineffective. Four reasons explain this conclusion.

A large number of authors come to Bardsy with a stack of comments, up to 30 pages, and ask the same question: what do I do next? Of course, asking for help is better than the alternative, which is to address the comments one by one, then self-publishing or querying the manuscript. We always answer the same way: let's try to prioritize the remarks and address the first one. In prioritizing, the author should select the comment whose revision will lead to the biggest overall improvement in the reader's experiences. We recommend using the Minimum Complete Story (MCS) to decide which of these comments is the most relevant, the one to address first. Some developmental editors try to be helpful by adding an action plan that suggests what to do first. Either way, the author revises. Because big problems come before little ones, the highest-priority revision often involves multiple, if not the majority, of the chapters. So far so good; what's next?

Before saying move on to problem two, think about the state of the novel. After just one significant change, it's a different product. A major revision leads to a new draft, especially if the author cares enough to ensure coherence, updating every part of the writing to align with the change. Now, problem two may no longer be problem two. It may have been solved, perhaps, or it may not have been affected by the revision. More likely, however, is that the new draft's biggest issue is not problem two; instead there's a different problem or a related but distinct version of problem two. Following tradition, a conscientious author would submit the whole novel, ask for comments and pay whatever fee.

Iteration supplies the logical solution. You want to find the biggest problem, fix and repeat until your product is ready to go. Of course, that takes time, which is unavoidable, and raises two other issues. We'll address the first -- readiness -- next time. The second issue involves the length of analysis.

Put simply, you can't expect anyone to provide actionable comments on an entire novel. Why? It's impossible for a human to hold a creative work this large in their head and provide specific feedback. At best you can point to big flaws in the story, which motivates using the MCS. By the same token, writers can't revise an entire novel without tools. Commenters depend on an unnatural and unspontaneous procedure, described below, for this reason.

Orientation supplies one, particularly glaring problem, with whole novel feedback. Properly breaking novels into chapters is critically important for reader satisfaction. Remember, each chapter's beginning has to reestablish the continuity, including the reader's empathy for the main character. Put another way, its start must have enough reminders to put reader and character literally on the same page. Absent that orientation, it takes effort for readers to figure out what's going on and they may get lost. This problem arises because ordinary readers don't pay much attention, even as they take irregular, sometimes quite long, breaks.

In fine dining, the waiter tracks the meal. After clearing the main course, for instance, it's time for dessert. The reader needs the same kind of support throughout the novel. We've never seen a paid editor comment on chapter orientation despite the fact that most authors don't provide nearly enough. Most likely, paid commenters pay unnatural attention or read one chapter right after another. The best commenters will stop and analyze after every chapter; nevertheless, points regarding orientation and other reader support rarely come up. Ignoring this necessity is a symptom of a larger problem.

The only things to reasonably expect from whole novel reads are: Amazon-type reviews, some emotional boost and a very loose predictor of sales (providing other conditions are met).

When people are asked to critique a novel, they stop being readers. They start looking for things to fix, which destroys the natural reading experience. The primary issue concerns whether the comments reflect how an actual reader would react. Further, any comments are neither natural nor spontaneous. Study after study shows people can't read and observe themselves at the same time. Moreover, stopping for evaluation changes an audience's experience. Psychologists and market researchers alike use other ways to measure message effectiveness. With the scope of the task: reading an entire novel, commentators are forced to adopt an unnatural process in place of superhuman brainpower.

Paid editors generally read and make notes as they go, periodically stopping to summarize. In practice, many forget to take notes and have to catch up later. Ironically, they are most likely to miss when the story is particularly gripping or boring. They also have to reconstruct their emotions when making these notes, which is very difficult for anyone to do accurately. In fact, someone else can judge your emotions by looking at your face better than you can state how you're feeling.

Two more things suggest that paid editors won't turn in good evaluations. The first is they have to finish. Rarely if ever does the recipient of a few thousand dollars say this is great; it's ready to publish. Instead, they have to make comments, where the volume indicates the quality of effort. What if they don't have much to say? Too many will essentially make things up, i.e. invent problems where none exist. Then, there's the other end of the spectrum.

Followers of Bardsy's content will remember that the biggest problem authors face is DNF or Did Not Finish. Paid editors can never assess this danger because they have committed to finishing the novel. They might say this gets to a point here where it's so bad a reader is likely to give up. With an iterative process, the author would stop there and try to figure things out. Commenters could stop to refund you if paid or otherwise ghost you, but it's more likely that they'll continue. An especially mean editor, or one uninterested in repeat business, could mark all the places a reader might stop. Everything after the first failure is wasted.

The last point raises Bardsy's hackles. Sometimes editors insist on changes to align with their vision, not the authors. For example, we saw an editor suggest dropping a romance although that was the story the author wanted to tell. Fellow writers often try to impose their own style or vision onto your story rather than helping you achieve yours. This may or may not increase book sales, but it definitely causes unwarranted stress. Such attempts to undermine vision may be cloaked in the guise of crossing our "boring" parts. Knowing your story is the only defense against this violation. Only a contract, after someone has bought your story, obliges you to listen to this type of comments.

Please email a comment or question if you find this analysis suspect. We think it's a good prima facie case for investigating other approaches to feedback cum product testing. We'll detail our preferred method -- retelling -- next time.