AI in Email Marketing: What’s Worth Testing (And What’s Not)

Artificial Intelligence (AI) in email marketing is changing how teams operate. But as Marina Carroll (CRM strategist at New Standard Co.) puts it, "AI is in its massive beta phase right now." 

While marketers are experimenting with AI tools for copy, testing, send time, and segmentation, the results are mixed. 

Some use cases save hours. Others risk damaging the customer experience.

So, what exactly are experts like Marina and Erik Huberman (Founder of Hawke Media) using AI for today? What works? What doesn’t? And where should you test it next?

This blog unpacks how experts use AI in their workflows and which use cases drive impact and are worth your time.

Where AI is working: Early wins marketers are seeing

While AI is unreliable in many creative and strategic areas, there are a few aspects where teams are already seeing real operational gains. Marketers are able to leverage AI to speed up execution.

Let’s break down how you can use AI to execute your advanced email marketing strategies:

1. Copywriting 

For brands sending multiple campaigns a week, AI is now a reliable tool to create copy for the first draft. A lot of the marketing teams are using AI to generate starter copy for low-risk promotional sends and then making edits to polish it up and maintain brand tone and hierarchy.

The workflow is simple: 

2. Reporting & hypothesis validation

Teams can move faster on campaign reporting and test result validation with a little help from AI.

Instead of manually writing performance summaries or digging through dashboards for insights, marketers are using tools like ChatGPT to synthesize key takeaways. The output isn’t a replacement for proper analysis, but it speeds up draft reporting and internal decision-making.

Here’s how to use AI within campaign reporting and result validation: 

• Write quick post-send summaries.
• Spot early performance trends.
• Sanity-check hypotheses before deeper analysis.

‍

3. Send-time & segment optimization

AI-driven features in ESPs like Klaviyo and Attentive are proving to be beneficial in two areas:

• Predict optimal send times based on user engagement patterns
• Auto-suppress disengaged profiles to protect inboxing and improve click rates

For brands with large lists, these automations reduce manual segmentation work and help lift Revenue Per Email (RPE) at scale.

‍

What’s not working (yet): Where AI creates more risk than value

AI has proved to be very beneficial in areas like send-time optimization and deliverability-driven suppression. But when it comes to creative control, attribution accuracy, and customer-facing content, it’s still more risk than reward.

Here are the four areas where AI is more likely to create more work or revenue risk than meaningful impact.

Building a brand voice

Even after training AI models with brand docs and feeding past campaign examples, most teams find that AI output still misses tone or context.

It’s not just about getting the copy right. The bigger risk is pushing out messaging that sounds off-brand, or worse, something that feels robotic, generic, or tone-deaf for your audience.

‍

Here’s where things go wrong:

• Inconsistent voice across campaign streams.
• Use of banned phrases, emojis, or off-brand terminology.
• Missed nuance on audience personas (e.g., talking to first-time buyers like long-term VIPs).

Attribution and performance claims

AI tools often claim higher revenue or engagement, but most attribution models behind these claims are still flawed.

Here’s where the risk shows up:

• AI features auto-claim credit for any downstream purchase from exposed users.
• Lack of clean BAU vs AI holdout groups.
• Short-term engagement spikes are mistaken for sustainable revenue lift.

Brands running AI-based experiments without clean holdouts or proper control groups risk making false assumptions about lift. 

Hallucination risks in customer messaging

AI-generated copy sounds convincing, even when it’s factually wrong. That’s the core hallucination risk. For marketing teams, more than tone, it’s sending false information to your customers that can damage brand reputation.

Here’s where this risk shows up most:

• Dynamic product blocks where AI pulls the wrong SKU.
• SMS campaigns where AI suggests offers or promo codes that don’t exist.
• Last-minute promotional emails built on AI-generated templates without QA.

Weak creative direction and message positioning

AI can help you write faster, but it can’t decide what story you should be telling. That’s where many teams go wrong—letting AI generate copy without sharing a clear creative brief, messaging priority, or customer insights.

This shows up clearly in your execution:

• Campaigns where the offer is buried three sections down.
• Emails that miss the emotional or seasonal hook.
• Flows where AI fills in generic content that doesn’t align with product launch timing or customer lifecycle stage.

‍

What to test first: Practical AI experiments for email marketers

‍

If you want to see real revenue impact, you need to run focused, high-leverage experiments without wasting time on aspects that won’t scale or won’t move core KPIs.

Here are 3 testing areas where you can leverage AI to optimize your email strategy.

Segment clean-up and suppressions

Deliverability is the fastest way to lift revenue at scale. AI can help you reduce sends to risky contacts, helping you drive better inboxing, higher click rates, and a direct RPE bump.

‍

How to execute the test: 

Let AI models suggest suppressions based on engagement (e.g., no clicks in the last 10 days, or users with high bounce/spam history). Then compare:

• Business-as-usual engaged segment vs
• AI-suggested suppressed list (filtered by engagement risk)

How to validate results:

• Keep creative and timing identical.
• Run the BAU list against the AI-trimmed list for at least 3-4 campaigns.
• Track inbox placement, spam complaint rate, RPE lift, and click rates.

Send-time testing

AI-based send-time tools like Klaviyo Smart Send Time or Attentive AI adjust each send to land in inboxes when subscribers are most likely to engage. 

For large campaigns (100,000+ sends), optimizing for engagement windows can drive noticeable gains in both CTR and RPE. But small lists likely won’t see statistically reliable results.

‍

How to execute the test: 

Compare performance between AI-optimized send times and a fixed, manually chosen send time.

• Run a 50/50 test for at least 2 weeks (to cover day-of-week patterns).
• Monitor engagement window shift. Check if clicks and orders are coming in at different hours.
• Track RPE delta, click-to-open rate, and unsubscribe rate.

AI-assisted product recommendations in post-purchase flows

If your brand is using AI for product recommendations, post-purchase flows are the safest touchpoint to test it first. 

Since moving first-time buyers to a second purchase is a critical revenue driver, AI can help surface next-best product categories based on past customer behaviour. 

Note: Getting too granular (SKU by SKU) can backfire.

‍

How to execute the test: 

Set up an A/B test within your post-purchase flow with two variants:

• Test group: AI-driven, category-based next-product recommendations vs
• Control group: Static bestsellers or one-size-fits-all upsells

Track over 30 days. Monitor metrics like second purchase conversion rate, RPE, and unsubscribe.

Pro-tip: Match your testing window to your sales cycle

If your brand has a longer purchase window (for example, high-AOV products or multi-touch conversion journeys), adjust your test timeline to match. Short-term spikes (or dips) in opens, clicks, or even short-window RPE can be misleading if your typical customer takes weeks (or months) to convert.

This is especially important for flows like welcome or post-purchase, where conversion often happens days or weeks after the first email.

‍

Are you using AI to speed up execution or replace creativity?

AI is undeniably becoming a key part of email workflows, but as both Carroll and Huberman reinforce, it's not a replacement for strategy. It shines when used to eliminate friction, scale the execution of repetitive tasks, and test faster, as long as you stay critical of its output.

‍

For most brands, the real challenge in email marketing isn’t slow execution. It’s stagnant list growth and poor deliverability. Tie helps you fix both.

Tie is an identity resolution and email deliverability platform with features built for better ecommerce conversions:

• Uncover anonymous visitors: Tie matches anonymous visitors to real customer profiles based on consented third-party sources, helping you grow your email list and capture high-intent shoppers.
• Land in more inboxes: Tie's domain warming and prospect filtering tools improve your email deliverability, increasing visibility and opens.  
• Send hyper-relevant emails: With comprehensive and accurate customer profiles, you can use demographic and behavioural attributes to create personalized emails that resonate with shoppers. 

Book a demo to see how Tie can improve your email performance.