{"id":48073,"date":"2024-08-12T16:29:44","date_gmt":"2024-08-12T16:29:44","guid":{"rendered":"https:\/\/www.demandgenreport.com\/?p=48073"},"modified":"2024-08-12T16:29:44","modified_gmt":"2024-08-12T16:29:44","slug":"7-misconceptions-about-email-a-b-testing","status":"publish","type":"post","link":"https:\/\/www.demandgenreport.com\/demanding-views\/7-misconceptions-about-email-a-b-testing\/48073\/","title":{"rendered":"7 Misconceptions About Email A\/B Testing"},"content":{"rendered":"<p>Few <a href=\"https:\/\/www.demandgenreport.com\/resources\/what-s-working-in-email-marketing-optimizing-content-across-devices-increasing-personalization\/8038\/\" target=\"_blank\" rel=\"noopener\">email marketing<\/a> initiatives are as important \u2014 or as misunderstood \u2014 as A\/B testing. Over the years, we\u2019ve guided brands with A\/B testing initiatives that help drive growth and insights, educating and empowering our clients along the way. In that spirit, let\u2019s identify and correct the most persistent misconceptions about A\/B testing in email campaigns.<!--more--><\/p>\n<h3>Misconception 1: It\u2019s All About The Subject Line<\/h3>\n<p>For lots of marketers, \u201cA\/B email testing\u201d starts and stops at the subject line. The truth is that there\u2019s several variables you can and should test, such as:<\/p>\n<ul>\n<li>Open rates at different times of the day and week, along with varying subject lines; and<\/li>\n<li>Post-open engagement and conversion factors, such as body copy, headline copy, offer\/promo, CTA, background colors or the order of content blocks.<\/li>\n<\/ul>\n<p>You should make a point of updating your list periodically with other elements to test. The more you test, the more you learn.<\/p>\n<p>One last note on this subject: Changes in open rate are relatively murky post-iOS 15, which is even more reason to expand the scope of variables to test.<\/p>\n<h3>Misconception 2: You Can Test Everything, Everywhere, All At Once<\/h3>\n<p>Clean A\/B tests involve one variable, not many: Don\u2019t switch up a subject line and time of send and day of week, or you won\u2019t know which one(s) made a difference. Beyond that, they require focus on specific elements within that variable.<\/p>\n<p>Since people love to talk about subject lines, let\u2019s use that as an example. Wildly different subject lines will perform differently, but you won\u2019t understand why. On the other hand, if you\u2019re testing specific differences like emojis versus no emojis, question versus statement, etc., you\u2019ll have a clear takeaway from the data.<\/p>\n<h3>Misconception 3: Instant Wins Abound<\/h3>\n<p>One of the biggest misconceptions is that each A\/B test might just produce a silver bullet. In fact, A\/B testing results are best seen in aggregate, over the long term. If you\u2019re structuring your tests well and getting information to optimize future sends, the wins will mount steadily over time.<\/p>\n<p>One reason you shouldn\u2019t expect instant results is that the audience for the test will be split in half: Even if one version knocks the other out of the park, 50% of your audience will still be underperforming in comparison.<\/p>\n<p>Last thing here: Depending on the test, you might not find any kind of winner. Inconclusive findings are fairly common, and it doesn\u2019t mean the test was flawed. Instead, it simply means the variable you introduced \u2014 e.g., changing the copy format to bullet points \u2014 didn&#8217;t move the needle one way or the other.<\/p>\n<h3>Misconception 4: Once A Winner, Always A Winner<\/h3>\n<p>A winning test from six months ago might not be a winner if the same test were run today. That\u2019s not to say you should relaunch tests in rigid six-month intervals, but you should keep an eye on any winning campaigns to make sure their performance remains strong. If you see engagement trending steadily down over time (rather than witnessing one bad send and over-reacting), it might be your signal to take another look.<\/p>\n<h3>Misconception 5: The Truth Is Universal<\/h3>\n<p>We love a good best practice at DMi, but we also know there are exceptions to every rule that you won\u2019t know until you test. For instance, for one client, we personalized a CTA to reflect the person\u2019s location (e.g. \u201cFind businesses in {city}!\u201d), and performance was far lower than the email with the non-personalized CTA.<\/p>\n<p>In other words, work with guidelines \u2014 not assumptions \u2014and be prepared for some surprises along the way.<\/p>\n<h3>Misconception 6: A\/B tests Are Always A Good Idea<\/h3>\n<p>Speaking of universal truths, while we love A\/B tests, they aren\u2019t always a good strategic move! If you need to push out an email for the sake of engagement and performance, put your best foot forward (informed by all the learnings from past A\/B tests, of course) rather than segmenting your audience and risking a less-than-optimal performance from 50% of the list.<\/p>\n<h3>Misconception 7: There\u2019s No Value In Failure<\/h3>\n<p>If you test a great idea that falls flat, it can be hard on the ego \u2014 but it\u2019s also important knowledge that you need to file away for reference. Remember that the aggregate value of A\/B testing comes in the insights you can apply over time, and you\u2019ll reframe \u201cfailure\u201d as a valuable piece of information.<\/p>\n<p>How important is it to get A\/B email testing right? Well, as media costs soar and first-party data usage becomes even more important, email marketing has a very important role in holistic marketing strategies. Great A\/B testing that helps you raise the levels of your email engagement over the long term is one of the best investments you can make for your marketing growth.<\/p>\n<hr \/>\n<p><em>Lauren McGrath is an Associate Email Strategy &amp; Execution Manager at digital marketing agency <a href=\"https:\/\/www.dmipartners.com\/\" target=\"_blank\" rel=\"noopener\">DMi Partners<\/a>, which she joined in 2021. Lauren, a New Jersey native who earned a bachelor&#8217;s degree in fashion merchandising and an MBA from Thomas Jefferson University, builds and optimizes email campaign execution processes for a portfolio of brands across B2C, CPG and B2B verticals. She began her email marketing career in 2018 and is passionate about raising the bar for email performance standards across the marketing industry.<\/em><\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Few email marketing initiatives are as important \u2014 or as misunderstood \u2014 as A\/B testing. Over the years, we\u2019ve guided brands with A\/B testing initiatives that help drive growth and insights, educating and empowering our clients along the way. In that spirit, let\u2019s identify and correct the most persistent misconceptions about A\/B testing in email [&hellip;]<\/p>\n","protected":false},"author":64,"featured_media":48074,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"_et_pb_use_builder":"","_et_pb_old_content":"","_et_gb_content_width":"","footnotes":""},"categories":[19181],"tags":[19364],"topic":[19132],"class_list":["post-48073","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-demanding-views","tag-dmi-partners","topic-demand-generation"],"acf":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v22.2 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>7 Misconceptions About Email A\/B Testing - Demand Gen Report<\/title>\n<meta name=\"description\" content=\"Few email marketing initiatives are as important \u2014 or as misunderstood \u2014 as A\/B testing. Over the years, we\u2019ve guided brands with A\/B testing initiatives\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.demandgenreport.com\/demanding-views\/7-misconceptions-about-email-a-b-testing\/48073\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"7 Misconceptions About Email A\/B Testing - Demand Gen Report\" \/>\n<meta property=\"og:description\" content=\"Few email marketing initiatives are as important \u2014 or as misunderstood \u2014 as A\/B testing. Over the years, we\u2019ve guided brands with A\/B testing initiatives\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.demandgenreport.com\/demanding-views\/7-misconceptions-about-email-a-b-testing\/48073\/\" \/>\n<meta property=\"og:site_name\" content=\"Demand Gen Report\" \/>\n<meta property=\"article:published_time\" content=\"2024-08-12T16:29:44+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.demandgenreport.com\/wp-content\/uploads\/2024\/08\/Demanding-Views-Template-22.png\" \/>\n\t<meta property=\"og:image:width\" content=\"560\" \/>\n\t<meta property=\"og:image:height\" content=\"315\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"author\" content=\"Kelly Lindenau\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Lauren McGrath, DMi Partners\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"5 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.demandgenreport.com\/demanding-views\/7-misconceptions-about-email-a-b-testing\/48073\/\",\"url\":\"https:\/\/www.demandgenreport.com\/demanding-views\/7-misconceptions-about-email-a-b-testing\/48073\/\",\"name\":\"7 Misconceptions About Email A\/B Testing - Demand Gen Report\",\"isPartOf\":{\"@id\":\"https:\/\/www.demandgenreport.com\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/www.demandgenreport.com\/demanding-views\/7-misconceptions-about-email-a-b-testing\/48073\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/www.demandgenreport.com\/demanding-views\/7-misconceptions-about-email-a-b-testing\/48073\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/www.demandgenreport.com\/wp-content\/uploads\/2024\/08\/Demanding-Views-Template-22.png\",\"datePublished\":\"2024-08-12T16:29:44+00:00\",\"dateModified\":\"2024-08-12T16:29:44+00:00\",\"author\":{\"@id\":\"https:\/\/www.demandgenreport.com\/#\/schema\/person\/0dfdd766d4003d62ac31d47f7eb0d232\"},\"description\":\"Few email marketing initiatives are as important \u2014 or as misunderstood \u2014 as A\/B testing. Over the years, we\u2019ve guided brands with A\/B testing initiatives\",\"breadcrumb\":{\"@id\":\"https:\/\/www.demandgenreport.com\/demanding-views\/7-misconceptions-about-email-a-b-testing\/48073\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.demandgenreport.com\/demanding-views\/7-misconceptions-about-email-a-b-testing\/48073\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.demandgenreport.com\/demanding-views\/7-misconceptions-about-email-a-b-testing\/48073\/#primaryimage\",\"url\":\"https:\/\/www.demandgenreport.com\/wp-content\/uploads\/2024\/08\/Demanding-Views-Template-22.png\",\"contentUrl\":\"https:\/\/www.demandgenreport.com\/wp-content\/uploads\/2024\/08\/Demanding-Views-Template-22.png\",\"width\":560,\"height\":315,\"caption\":\"Email A\/B testing concept with author headshot\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.demandgenreport.com\/demanding-views\/7-misconceptions-about-email-a-b-testing\/48073\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/www.demandgenreport.com\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"7 Misconceptions About Email A\/B Testing\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.demandgenreport.com\/#website\",\"url\":\"https:\/\/www.demandgenreport.com\/\",\"name\":\"Demand Gen Report\",\"description\":\"\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.demandgenreport.com\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/www.demandgenreport.com\/#\/schema\/person\/0dfdd766d4003d62ac31d47f7eb0d232\",\"name\":\"Kelly Lindenau\",\"url\":\"\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"7 Misconceptions About Email A\/B Testing - Demand Gen Report","description":"Few email marketing initiatives are as important \u2014 or as misunderstood \u2014 as A\/B testing. Over the years, we\u2019ve guided brands with A\/B testing initiatives","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.demandgenreport.com\/demanding-views\/7-misconceptions-about-email-a-b-testing\/48073\/","og_locale":"en_US","og_type":"article","og_title":"7 Misconceptions About Email A\/B Testing - Demand Gen Report","og_description":"Few email marketing initiatives are as important \u2014 or as misunderstood \u2014 as A\/B testing. Over the years, we\u2019ve guided brands with A\/B testing initiatives","og_url":"https:\/\/www.demandgenreport.com\/demanding-views\/7-misconceptions-about-email-a-b-testing\/48073\/","og_site_name":"Demand Gen Report","article_published_time":"2024-08-12T16:29:44+00:00","og_image":[{"width":560,"height":315,"url":"https:\/\/www.demandgenreport.com\/wp-content\/uploads\/2024\/08\/Demanding-Views-Template-22.png","type":"image\/png"}],"author":"Kelly Lindenau","twitter_card":"summary_large_image","twitter_misc":{"Written by":"Lauren McGrath, DMi Partners","Est. reading time":"5 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/www.demandgenreport.com\/demanding-views\/7-misconceptions-about-email-a-b-testing\/48073\/","url":"https:\/\/www.demandgenreport.com\/demanding-views\/7-misconceptions-about-email-a-b-testing\/48073\/","name":"7 Misconceptions About Email A\/B Testing - Demand Gen Report","isPartOf":{"@id":"https:\/\/www.demandgenreport.com\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.demandgenreport.com\/demanding-views\/7-misconceptions-about-email-a-b-testing\/48073\/#primaryimage"},"image":{"@id":"https:\/\/www.demandgenreport.com\/demanding-views\/7-misconceptions-about-email-a-b-testing\/48073\/#primaryimage"},"thumbnailUrl":"https:\/\/www.demandgenreport.com\/wp-content\/uploads\/2024\/08\/Demanding-Views-Template-22.png","datePublished":"2024-08-12T16:29:44+00:00","dateModified":"2024-08-12T16:29:44+00:00","author":{"@id":"https:\/\/www.demandgenreport.com\/#\/schema\/person\/0dfdd766d4003d62ac31d47f7eb0d232"},"description":"Few email marketing initiatives are as important \u2014 or as misunderstood \u2014 as A\/B testing. Over the years, we\u2019ve guided brands with A\/B testing initiatives","breadcrumb":{"@id":"https:\/\/www.demandgenreport.com\/demanding-views\/7-misconceptions-about-email-a-b-testing\/48073\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.demandgenreport.com\/demanding-views\/7-misconceptions-about-email-a-b-testing\/48073\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.demandgenreport.com\/demanding-views\/7-misconceptions-about-email-a-b-testing\/48073\/#primaryimage","url":"https:\/\/www.demandgenreport.com\/wp-content\/uploads\/2024\/08\/Demanding-Views-Template-22.png","contentUrl":"https:\/\/www.demandgenreport.com\/wp-content\/uploads\/2024\/08\/Demanding-Views-Template-22.png","width":560,"height":315,"caption":"Email A\/B testing concept with author headshot"},{"@type":"BreadcrumbList","@id":"https:\/\/www.demandgenreport.com\/demanding-views\/7-misconceptions-about-email-a-b-testing\/48073\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.demandgenreport.com\/"},{"@type":"ListItem","position":2,"name":"7 Misconceptions About Email A\/B Testing"}]},{"@type":"WebSite","@id":"https:\/\/www.demandgenreport.com\/#website","url":"https:\/\/www.demandgenreport.com\/","name":"Demand Gen Report","description":"","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.demandgenreport.com\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/www.demandgenreport.com\/#\/schema\/person\/0dfdd766d4003d62ac31d47f7eb0d232","name":"Kelly Lindenau","url":""}]}},"parsely":{"version":"1.1.0","meta":{"@context":"https:\/\/schema.org","@type":"NewsArticle","headline":"7 Misconceptions About Email A\/B Testing","url":"http:\/\/www.demandgenreport.com\/demanding-views\/7-misconceptions-about-email-a-b-testing\/48073\/","mainEntityOfPage":{"@type":"WebPage","@id":"http:\/\/www.demandgenreport.com\/demanding-views\/7-misconceptions-about-email-a-b-testing\/48073\/"},"thumbnailUrl":"https:\/\/www.demandgenreport.com\/wp-content\/uploads\/2024\/08\/Demanding-Views-Template-22-150x150.png","image":{"@type":"ImageObject","url":"https:\/\/www.demandgenreport.com\/wp-content\/uploads\/2024\/08\/Demanding-Views-Template-22.png"},"articleSection":"Demanding Views","author":[{"@type":"Person","name":"Kelly Lindenau"}],"creator":["Kelly Lindenau"],"publisher":{"@type":"Organization","name":"Demand Gen Report","logo":"https:\/\/www.demandgenreport.com\/wp-content\/uploads\/2024\/01\/dgr_v3_funnel-1.png"},"keywords":["dmi partners"],"dateCreated":"2024-08-12T16:29:44Z","datePublished":"2024-08-12T16:29:44Z","dateModified":"2024-08-12T16:29:44Z"},"rendered":"<script type=\"application\/ld+json\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@type\":\"NewsArticle\",\"headline\":\"7 Misconceptions About Email A\\\/B Testing\",\"url\":\"http:\\\/\\\/www.demandgenreport.com\\\/demanding-views\\\/7-misconceptions-about-email-a-b-testing\\\/48073\\\/\",\"mainEntityOfPage\":{\"@type\":\"WebPage\",\"@id\":\"http:\\\/\\\/www.demandgenreport.com\\\/demanding-views\\\/7-misconceptions-about-email-a-b-testing\\\/48073\\\/\"},\"thumbnailUrl\":\"https:\\\/\\\/www.demandgenreport.com\\\/wp-content\\\/uploads\\\/2024\\\/08\\\/Demanding-Views-Template-22-150x150.png\",\"image\":{\"@type\":\"ImageObject\",\"url\":\"https:\\\/\\\/www.demandgenreport.com\\\/wp-content\\\/uploads\\\/2024\\\/08\\\/Demanding-Views-Template-22.png\"},\"articleSection\":\"Demanding Views\",\"author\":[{\"@type\":\"Person\",\"name\":\"Kelly Lindenau\"}],\"creator\":[\"Kelly Lindenau\"],\"publisher\":{\"@type\":\"Organization\",\"name\":\"Demand Gen Report\",\"logo\":\"https:\\\/\\\/www.demandgenreport.com\\\/wp-content\\\/uploads\\\/2024\\\/01\\\/dgr_v3_funnel-1.png\"},\"keywords\":[\"dmi partners\"],\"dateCreated\":\"2024-08-12T16:29:44Z\",\"datePublished\":\"2024-08-12T16:29:44Z\",\"dateModified\":\"2024-08-12T16:29:44Z\"}<\/script>","tracker_url":"https:\/\/cdn.parsely.com\/keys\/demandgenreport.com\/p.js"},"_links":{"self":[{"href":"https:\/\/www.demandgenreport.com\/wp-json\/wp\/v2\/posts\/48073"}],"collection":[{"href":"https:\/\/www.demandgenreport.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.demandgenreport.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.demandgenreport.com\/wp-json\/wp\/v2\/users\/64"}],"replies":[{"embeddable":true,"href":"https:\/\/www.demandgenreport.com\/wp-json\/wp\/v2\/comments?post=48073"}],"version-history":[{"count":2,"href":"https:\/\/www.demandgenreport.com\/wp-json\/wp\/v2\/posts\/48073\/revisions"}],"predecessor-version":[{"id":48076,"href":"https:\/\/www.demandgenreport.com\/wp-json\/wp\/v2\/posts\/48073\/revisions\/48076"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.demandgenreport.com\/wp-json\/wp\/v2\/media\/48074"}],"wp:attachment":[{"href":"https:\/\/www.demandgenreport.com\/wp-json\/wp\/v2\/media?parent=48073"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.demandgenreport.com\/wp-json\/wp\/v2\/categories?post=48073"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.demandgenreport.com\/wp-json\/wp\/v2\/tags?post=48073"},{"taxonomy":"topic","embeddable":true,"href":"https:\/\/www.demandgenreport.com\/wp-json\/wp\/v2\/topic?post=48073"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}