Name: Mailchimps New Bot Filtering
Uploaded: 2026-01-15T20:16:06.484Z
Duration: 55 min 25 s
Description: Mailchimps New Bot Filtering

Transcript for "Mailchimps New Bot Filtering": everyone. My name is Taylor Mattson. I'm a staff product manager here at Mailchimp, specifically working on our reporting and analytics products, and I have recently launched a bot filter. And super excited to talk to you guys about that today and, of course, answer any questions that people have as well. Perfect. And then I'm Jordan. I'm one of our customer success program managers. So I manage and run all of the webinar events. So thank you all so much for being here today to learn about this new feature and just sharing all of your questions and insight. We take everything. We read it, and then we, adopt all of your feedback. So thank you again for being here. And we can kick to the next slide where I just wanted to cover a couple housekeeping items before we dive in. The first is at the end of today's session, I am gonna launch a survey, and we're just looking for feedback. What do you wanna hear? Any sessions for future webinars? Did you like today's session? Just all of the things. And as we're both speaking today, please drop any question that comes to mind in the q and a tab. So where y'all are writing in right now in the chat, there's a q and a tab. And we have our team backstage monitoring all of your questions. So we'll answer some of them via the chat, but we are gonna leave some for our live q and a, so definitely drop anything that comes to mind. And we have a demo, of course, showing you this new feature. So feel free to follow along in app as Taylor navigates the bot filtering feature, or feel free. Sit back, relax. Just take it all in. The session will be recorded. So, yes, it's going to be recorded, and it'll be sent to your email tomorrow. So twenty four hours after the session right now, you will get the recording. So, again, feel free, sit back, relax, just really absorb everything, and don't worry. And then we will kick to our next slide where before we dive in and I kick it over to Taylor, I do wanna launch a poll. So I'm just curious. How confident are you in your understanding of bot filtering right now? Are you very confident? Are you somewhat confident? Are you slightly confident? Or you're like, I have no idea. That is exactly why I'm here, Jordan. So we wanna hear from you. And I'll leave the poll open so that you guys can submit your answers. But with that, Taylor, I'm gonna kick it to you to take us through the rest. You would think I know when to unmute myself, but still learning. Alright. So let's kick it off. Thanks for that intro. Let's talk a little bit about why our bot filter exists and why accurate data matters. So as you all probably know, you need accurate data to be able to confidently measure your campaign success. Open rates and click rates are huge indicators on whether or not your campaign was successful, your subject lines or your content, and being able to measure different campaigns against each other with these metrics is is integral to your job. It's also helpful to have accurate data to optimize send times based on real behavior, understand true audience engagement, and improve your ROI with trustworthy insights. So all this to say that with better data, you make better marketing decisions. So we have launched a bot filtering, capability because today, or for a while actually, there has been a bot or nonhuman, metrics inside of data. And a lot of these things are this, the reason why this exists is because there's a lot of anti, spam, different security platforms that are actually opening and clicking links on behalf of end users or your end customers to make sure that those emails are safe and that the content in the email is safe for their users. So a great example of a company that does this is Apple. For any Apple devices or for any customers who are using an Apple device and specifically using the Apple Mail app, Apple actually has a feature called, MPP, and what they do is they, take the email that is in the user's inbox using the Apple map, or Apple Mail app and, remove our pixel and replace it with a copy of that email to protect the customer's, behavior post email. And that what that looks like is a real open in our database before the Apple MPP filtering. The same thing can be said for clicks. So, for example, there's a ton of AI companies now, and how they learn everything and incorporate all the knowledge of the world into their models is actually scraping the Internet. So clicking links, reading things, and learning so that their models can become better. Those were registered as human clicks in our database as well. So some key benefits for the bot filter is that our service automatically identifies and filters, invalid automated activity when that filter is on. This feature works across email and SMS. So I gave two examples, one for email, one for for, mobile. So something like Apple MPP will affect mobile. But Google and Android have a similar technology, for any text messages that come through as Android as well. Our service is designed to protect integrity for your performance data, so, making sure that we can accurately, with a 100% confidence, determine whether or not activity is a bot, is some something that was one of our forefront principles while, developing this this feature. And then the service also helps ensure metrics reflect real human engagement. So what does all this mean? All this means that we give you an accurate picture of your data by removing all nonhuman, clicks and opens from your engagement metrics so that you can fully feel confident in the strategies that you come up with because you have trustworthy data. All right, so honest attribution you can trust. So because attribution is based off of opens and clicks, the more accurate the opens and clicks are, the more accurate your attribution is. So bot activity is excluded by default. Revenue and ROI reflect real human behavior, so you don't have to worry about attributing sales to bots. And then it also prevents false performance spikes caused by bots. So for example, there could be a huge spike of clicks because, I don't know, Anthropic is testing a new model, and they're doing a lot of scraping. Right? So we by filtering the bot activity out, you're able to ignore that noise and just focus on what's real. Alright. So here's a before and after just so we can see the difference that clean data makes. So before the bot filter feature, we had inflated open and click rates, unclear engagement patterns because of those non human clicks and opens in the data, and then as a result, our metrics were hard to trust. But now after the filter, you have more accurate performance metrics, clearer audience behavior, and confidence in decision making. Now you can feel good about the decisions you're making. But as we this filter is is brand new and because the nature of some of the bots, like bots that we detect by IP address, fun fact that I learned while building this product, is that, IP addresses are actually recycled every month to three months. So an IP address that shows up as a bot today could very well not be a bot three months from now. And we have a third party that we work with to to make sure that we have the most accurate IPs for bots. And a lot of these companies like Google and Apple, they actually post their IPs publicly, so we're able to to stay up on the know on their recent IPs as well. But you might wanna compare data year over year, and a year ago, we did not have this filtering. And maybe some of your your leadership, for example, I've heard, may be used to seeing open and click metrics with that nonhuman activity included. So, being able to see data with and without bots is is important to a lot of different customers just depending on their use case. And so because of that, our filter is actually a setting that you can turn on and off. And so you'll see a picture of kinda what it looks like on the right here. But every page in the app with reporting and analytics metrics that has this filter, you'll see this indicator pop up that says bot filtering is on or off. Within the filter or within the setting, you can turn specific filtering on. So all filtering that we have available today is removing bots from email opens, also formerly known as Apple MPP. So that if you guys are familiar with the Apple MPP in context setting that was on some of the pages like marketing dashboard and email report, we took that away and replaced it with a more broader setting. You could turn filtering on for email clicks. And then also if you are an SMS enabled account, you can also turn on filtering for bot behavior for SMS clicks. And all this is to maintain visibility into real engagement while excluding that invalid activity. So you may see a notification in your app when you log in. Sorry. And this will pop up to let you know that we have launched this feature. And if you did wanna learn more, we have a link to some articles here. And then also you can manage the setting directly from this notification that you either have or will see in your account soon. And then, of course, whenever you're done with this notification, just dismiss it, you can move on with your life. Alright. So we tried our very best to to get bot filtering everywhere in the app all at one time, but we have started out with our most important areas, and we'll talk a little bit about areas that we still need to add the filter to so you have a good handle on where the bot filter is today and where we just need a little bit more time, but we wanted to give you guys as much value ahead of holiday season and right after holiday season so you can see exactly what your true metrics are. So today, the bot filtering is activated on the Mailchimp homepage, marketing dashboard, much of the email report, SMS report, custom reports, and recipient activity. All right, so here's just a few things that you need to know. We will rely on precise industry leading detection. We use deterministic signals, not guesswork. So how we determine behavior is human or nonhuman is actually not just based off of one signal. We look at a large number of signals, IP address, user agent. We also work with a couple of third parties who are able to give us threat intelligence groups. And so all of that is taken into play. We are very conservative. And so that's what I meant by not just looking at one signal. We're looking at all these signals before we determine whether or not it is a bot, and that helps us reduce the false positives. So a false positive is, you know, marking something as a bot when it's not. Quick heads up for everybody. Bot filtering is automatically enabled in your account. So you may log in and immediately see lower open and click rates, and this is to help ensure that data reflects real human engagement. So if you want to see open rates and click rates closer to what you're used to, meaning they do have that nonhuman activity in it, you would have to turn the setting off, and your opening click rates will then look very similar to what they look like before the filter released. Alright. The moment you've all been waiting for. I will give you a live demo. So let me just switch screens. And alright. So here I am in an experimental test count account. So any anything crazy in here, just know that this is a test count. This is not a real user, and sometimes look things look a little silly in here. But what does not look silly is our bot filter. So as mentioned, earlier in the deck, you'll see this bot indicator or filtering indicator. Here, my account is showing on. Anywhere in the app, if you see this, you can easily click it, and you can see which metrics the filtering is on for. So in this account, I have all three filters on, email opens, email bot clicks, and then also SMS bot clicks. But let's say for whatever reason I don't want these filters on, I wanna compare year over year, for example, I can just click that manage settings right from that pop up and I'm taking to the analytics settings page. Now this page has the star of the show today in the bot activity setting, but also worth noting that we also have an attribution window setting here too. So if you wanted to change attribution windows for different audiences, that is also on the same page. And as we release new settings for analytics, those will also be on this page. So maybe a new page for for some folks on the call. Once you take a look at your bot setting, and maybe this might be your first time, you can learn more about the bot filters by clicking by clicking our, what we call a point of need here. And then now on the right hand side will be a lot more information about a lot of the things I've already said in this call today, but more about bot filtering, and you can go ahead and directly reach support from here if you have more questions. But just a nice little reminder, if there's other folks at your your place of employment that you need to pass information on for, this is a great place. But turning filters on and off is as simply as just clicking the button here. You'll see down here, this is called a Toast here, but it's a little notification pops up to let you know that the bot setting is saved and updated, and then you can easily return to the previous page you were on. So, for us, we were on all campaigns where a lot of you probably go, and, it's a quick way to just, you know, pick up where you left off, try not to invade your space too much. So yeah. So you can turn on, like I mentioned before, one or the other. And then once you go back to a page, let's say maybe an email report, which I already had, you can pop that open. The bot filtering is here, and you can see which filters are on and off as well. So that's it's pretty there's nothing nothing too much more to show you, on marketing dashboard and in all the other products that we have too. It shows up similarly. So I really tried to make it, make it super easy for you guys. And, you know, no matter where you are, it looks the same. So with that, I will go back to the deck. Perfect. And then love everybody's questions. We are answering them. We're tackling them one by one, and we have a ton of time for our live q and a segment. So don't worry. If you don't haven't seen a response yet, we're gonna try and get to it. But, while we're on the topic of updating our product, I did wanna let you all know that our team is hosting a transactional office hour for developers. And if you have no idea what transactional is, this is a great time to join us and learn about it. But Mailchimp has revamped our transactional. So the one to one, some of you may know it as like, if you have a password reset or if you need to send someone a confirmation email, it's not a marketing message. It's more transactional based. That entire interface for Mailchimp has been revamped and updated, and it's amazing. And so just wanted to call that out that we do have an office hours on the twenty seventh with our developer team should you want to attend. I'll drop those links in the chat. Don't wanna spend a ton of time on this because I know the focus is for bot filtering. So with that, Taylor, we can can dive into the live q and a. Let me I'm gonna be our emcee. Yay. Okay. Great. Y'all don't get to lose me. So I'm going to push the questions. These are the ones that y'all have been submitting. Hannah's question was, if the bot filtering is on in our account, is that filtering also automatically applied to any data coming through the reporting API? That is a great question. And so you might remember during my presentation when I said we tried really hard to get the bot filter on all surfaces. One of the surfaces that did not make it in the first launch is the reporting API. We are actively working on getting bot filtering and other features that are missing today from the reporting API. But today, if you use that, it does include bots. And once we are able to to launch the version that does not include bots, we'll we'll definitely let you know. Perfect. And I see in our future too a part two of this webinar series of bot filtering of when we do expand its capabilities, we can have another webinar session to just highlight where it's been expanded to and then have another round of questions because I know we have a ton right now. So just wanna make sure everybody feels confident in. these changes as they come. And we we do have a knowledge based article. If ever anyone is up to date or knows about those articles, we'll we'll update the article with the surface areas as well as, like, as new ones come up. Amazing. Okay. That'll be helpful. Alright. Emma, will the bot filtering ever reflect in the recipient activity so a clean list can be exported for clicks and opens? We are still getting the bots in our list when exporting the recipients. Yes. The recipient activity is also on that list. Part of the reason why we were unable to launch it on all surface areas is we're actually modernizing or updating our our data source. And so we couldn't update everything at once, and recipient activity is actually, again, being worked on as well. And as soon as we get that reading from our our new database, the bot filtering is automatically applied. And, again, similar answer to, the last question and what Jordan and I had said about, updates But as soon as we have it available, we'll let you know. But in. the meantime Yeah. oh, never mind. That doesn't really help. I was gonna say custom reports, but we don't have recipients in there either yet. Never. mind. Also, can we see this is brand new? We're all learning. And and, again, this feedback that you all are asking about submitting, we take that into account as we work to improve the product. So definitely keep it coming. Taylor, this next question is kind of off of what you just said. Haven't been able to figure out how to see the recipients that are being filtered out as bots. Is there a way for us to export and tag those addresses to monitor them? Sounds like it's a this visibility is coming soon. Yes and no. So I wanna dig a little bit deeper on this question. So seeing the recipient activity filtered for bots is coming soon, similar to the last question. But I did wanna kinda dig deeper on the second part of the question with, is there a way for us to export and tag those addresses to monitor them? Like, we haven't included that yet. And the main reason for that is a lot of times when I talk to customers or partners or anybody really about bots, this assumption that there's bad actors is mostly false in a lot of cases. Like for sure, there's bad actors in this world who are clicking and opening like Python user agents, for example, sometimes are malicious. But a lot of people who use Apple devices have Apple MPP turned on automatically. Like myself, I use all Apple devices. And so for a lot of companies, I register as an Apple MPP open, but I also open my emails too. And so, like, tagging me as as a bot in in your audience would be kind of misinformed. Because while my opens are bots, I still open emails and I still click, and you can still track my human clicks and purchases as well. So it is good to kinda, like, learn. Like, for example, universities or company email addresses tend to have more bot opens or clicks because they have higher security measures on their anti spam software. But like that that's a learning, but you wouldn't wanna really kind of, like, tag them as, like, bad actors or or definitely don't clean them out of your list. So just wanted to to take a second to talk about that because, yeah, not all bots are bad, but definitely don't clean clean them out of your list unless, you know, you know they're bad. Thank you for touching on that. Yeah. Okay. Let's keep it going. I lost okay. Let's do this one. Emma, will bot filtering work for multivariate test? It does not seem to apply to these tests. Yes. That is planned for our next quarter of work. So our new quarter starts in February, and this is is something that we'll start working on for that quarter. Perfect. Okay. Elizabeth, there seems to be a large drop in opens in some accounts since the bot filter launch even though Apple MPP was already on. Is there a reason for this? This one, we could dig into the account and take a look. There shouldn't have been a big change because, like you mentioned in the question, Apple MPP was already here. Our bot detection service is mostly the new work that we did focused on bot clicks, and we just incorporated the work we had already done for Apple MPP. So if you have a CSM, definitely reach out to them. Otherwise, I think we can we can find you, Elizabeth, and we can take a look at your account and see what's going on. Perfect. And, Elizabeth, I know I believe you're one of our partners, so we could definitely sync with you, assuming you're working on multiple accounts. But part two, Elizabeth had another question. And you kind of touched on this, Taylor, in. one of your answers, but what happens if there is a bot open or click and then a real one happen? Yes. So bot so all clicks and all opens are still being stored in our database. We've basically added a column that says, is this real or is this not real? So when you have the filter on it, literally just filters out anything that is not real. So one step further is, when I went through Apple MPP, Apple MPP actually removes the Intuit Mailchimp and any other company. We're not dealing with this issue. All all people in the industry are facing this issue. So Apple actually removes the pixel from the email, and places a copy in the inbox. So that means we cannot track any opens after that Apple MPP open from an Apple device using the mail app. However, me, Apple user, if I have an Apple MPP open on my phone, the person who sent me the marketing email can still track an open if I'm on my computer and I open up the Gmail app, for example. That still registers as a very real open. And so we are able to track Apple open specifically if they happen off of an Apple device, off of the mail app. For clicks, though, that does not apply at all. So we we we track all clicks. We determine whether or not it's a bot based off of those things we mentioned earlier, IP address, user agent, and those are way easier to to track. And Apple or any other company does not remove our pixels, so all bot and human clicks are coming into our database. Got it. Okay. Good reassurance. Yeah. I will keep us going. So, Shane, does bot filtering apply to automation flows? We have found a large amount of bot activity on our site that triggers automations like abandoned cart automations, which does drive bounce right bounce rates exceptionally high. Yeah. Absolutely. This one is is currently being worked on as well. Similar answer to the other ones. We had to move automation's data over, which is a lot of data, a lot of complexity. So the same answer. We're we're working on it as soon as we launch it, but it is planned. Our our 100% goal is to get every single metric in our app, filterable to remove bots. And so it's just kind of like a process. We we launched with something that felt good and to deliver immediate value, and now we're constantly going to be delivering on this list that we were unable to include in the first one. Amazing. Mhmm. Okay. I'm just gonna keep them going. We have a ton, which is awesome. So Kevin said with Apple privacy in place, does that only affect Apple, like, email addresses, these examples, or all emails using Apple Mail apps like Gmail, Yahoo, etcetera? The the latter. So it's all email addresses using the Apple Mail app. So that's for for non Apple users. If you've never seen it, that's that's this guy down here, the blue one. And so we only the it only affects if a customer has their email set up on an Apple device using the mail app. So if they were on an iPhone and only use uses the Gmail app, then it there would be no Apple MPP open. You physically have to set up the email address on the mail app. And I saw a few question. I'm gonna go off book for one second, Jordan. Do it. Let's a. few questions come through that are, related to this that I wanted to address too. So for Apple MPP, if you think about it, Apple has a huge market share, and especially in The US. So if you're a US customer, Apple is a market dominator. For Apple MPP, Apple actually turned this feature on for all, customers, and it happened in, like, a software update, like, four years ago. But, basically, you know, when you update your iPhone or your computer, you basically just click through everything to get to using your device after it's updated. And all those clicks, one of them was Apple MPP. It was turned on automatically. People probably didn't know what it was. They kept clicking. So for a lot of users who have iPhones and who use the mail app, Apple MPP is on, and they they probably don't know about it. And so when you see open rates dropping 50%, 17%, like, that's because of the fact that Apple has turned this feature on automatically, and Apple has a large market share. So maybe somebody in Asia, some country in Asia where Apple is not the market dominator won't see as big a drop in Apple MPP opens as maybe somebody in The United States where Apple is the dominant phone company of choice. So just kinda want to address the big drop. But oh, go ahead, Jordan. No. Was like, I love that. I'm learning, we're learning extra. So and when we first ran the analysis, we saw anywhere between 4555% of the opens that were coming into our system as Apple Open. So if you're a customer who's seen an open rate drop around 50%, like, while alarming, I'm no doubt. It definitely raises eyebrows. It's actually right in line with, you know, if you're a US customer and just judging off the Apple market share, kind of why that happens. Wow. That does make sense. And I didn't think of it like that. And thanks for grouping some of the other questions too. That really helps answer all of it. David said, is it possible to have a click time down to the second so that we can validate? Oh, hey, David. I talked to David earlier. Yeah. So it's definitely possible. We would have to update our our app, obviously, which is why you don't see it today. So thinking about where we could do this on the email report, we do have, like, recipient activity, which you can filter your recipients based off of who clicked. And I believe we we track last click time there. There's a lot of organic places we can put it. I know we have the data, so I'll make sure that it gets added to our backlog of feature requests. And I think it's something especially for you because I know a little bit about you and how you like to track the data. It's definitely something we can do, and and I imagine would be helpful for quite a few of you. Perfect. Okay. Laura asked, does the Apple MPP filter all clicks and opens from Apple even legitimate ones? You kind of touched on that, but it's good to just. So. Apple MPP only affects opens. So your clicks are unaffected when Apple MPP is enabled. And Apple MPP only opens once, and so that one is counted. But if customer who is an Apple MPP open opens anywhere else outside of their Apple Mail app, that open is still counted, which is a legitimate one. So yes. But Apple MPP and clicks have nothing to do with each other. All clicks are okay with Apple MPP. Thank you. I. feel like there's so many terms that we're saying, so it's good to break it down to reiterate. And the good. thing everyone remember, one, you'll get this recording. And, also, we have a knowledge based article, and we also have that our point of need, our little pop out on the settings page too. So don't feel like you have to memorize this at all because it's a lot. It is a lot. And I'll as you answer this next question, I will drop that knowledge base article on the chat so that everybody has it. Oh, Simone beat you. did it. Thank you. Our team backstage is awesome. Yes. So Bruce asked, you've mentioned many reporting areas that the bot filtering does not yet touch on. Can you tell us what reporting areas it does? I was hoping to see that in the demo. I know we have, like, a list in the deck. I don't know if you wanna pull that back up or Yes. I can't share with you sharing the question, oh, though. of course, you can't. Here we go. So let me see. Let me see if I'm still logged in to that account. Yes. Okay. Okay. So I. can show you the areas. The homepage has it, but I'm not gonna lie. The homepage was acting weird before I logged in. Again, this is a test account, but you'll see in the the on the homepage, we have the bot filtering on. On all campaigns, because we have metrics in these rows, bot filtering is also applied on all campaigns. In the email report and SMS report, so if we filtered this for email, we click a report, the filtering is on the email report. On some pages like click performance and recipient activity, we briefly touched on some of them. Like click performance, you don't see the indicator here because it's not not yet available on click performance. So, wait. I think it opened into a new tab. oh, no. Sorry. We can only see the all campaigns in the list form. Oh, no. Okay. Sorry, guys. I know. me let me learn how to use technology. See, I was trying to be too fancy and and just share. Let me just do it. a tab. So No. All good. It's help. it's helpful to watch you click through everything, so thank you for doing this. Yeah. Perfect. Okay. Can you see the email report now? Yeah. Okay. Cool. So on the overview, you'll see the filter. On recipient activity, you'll see the filter. But on click performance, we do not see the filter, and that's because it's not, yeah, on click performance. Where else is it? It is on the SMS report. So let me see if this test account has an SMS report. Does not. If it had an SMS report, it would be there. And then we have marketing dashboard. You'll see the bot filtering on here and then also custom reports. Something weird is going on here. Here's the the filter as well. So, again, sorry. This is not the best test account I had, but it is the one with this feature. But all those surface areas here have the bot filtering. And you'll know right away if you're on a surface area, just look for bot filtering on or off. If this is showing up on that area, that means you have this page is enabled. If you do not see this on a page, that means that's on our list that we still need to to get to. And then when you click on that, I just wanted to see that drop down again when you click on bot filtering on. Yes. And this is clickable that's just single page. This will take you right to the setting. Okay. So if we click on that, now we're back to the page with the setting. Cool. Perfect. Cool. Yes. It's all connected. Okay. So would you say tracking the clicks makes more sense than open rate with the current bot world we're in? Yes. I would say click rate is more reliable than open rate, specifically because I know how Apple MPP works. Right? Like, I'm unique because I'm at work sometimes, and I get a little bored. And I wanna open up Gmail to check an email on the web. But for most people, like, they use their phone for everything. And so if a customer one of your customers or people you're trying to reach is exclusively using the Apple Mail app, because of the way Apple MPP is implemented and because it removes the pixel, you'll never see a real open from them, unfortunately, despite they could be very much opening your email. And so because clicks do not have that same problem, for me I would say click rate is more reliable than open rate. Click rate is more reliable than open rate. Retweet. Okay. So I'm gonna burn that in my brain forever. Let me find I just saw a question. So is it safe to assume, Taylor, that open rates are now deflated if recipients only open in Apple Mail? Yes. It is safe to assume. I like I like to use the words a more accurate open rate, but truly, yes. Like, your open rate is deflated because we lose the ability to track any open after an Apple MPP open on the Apple Mail app. So for people who are exclusively using that, your open rate is deflated, which is why literally timely. The question before this, I say click rate is probably more reliable as a benchmark than open rate. Okay. So this next question, Kevin has you had mentioned previously. So he asked when Apple strips the pixel and makes a copy and someone still opens it, that does count as a human opening. Right? Which is the net tracking we have in our metrics now? So it only counts as a human open if a human opens it off the Apple app at a different time. Because when they strip the pixel, that's how we know whether or not the email was opened again. So the u the the end customer, the person you're trying to reach, doesn't know that Apple did this. There's no indicator that says Apple did this. It It happens automatically. So no is the answer to the question unless they open on a different device or a different app, and it is because that pixel is stripped. So the the information about the open is in the pixel. Okay. Okay. Cool. Yeah. So the second part, which is the net tracking we have in our metrics now. So if you have the filter off, your opens include Apple MPP and humans. If you have the filter off, then or sorry. If you have the filter on, then your opens are all opens minus the apple opens, which would give you all known human opens. Okay. Got it. I, like, had to, like, paint that out. Kevin wrote in the chat, got it. Thank you. So thank you, Taylor. Yeah. No problem. Sarah has a great question. Will bot filtering apply retroactively to past emails or only new ones moving forward? They will apply to emails that were sent after the November. And a lot of the reason is because what something I mentioned earlier, which is the fact that IP addresses are recycled. So we can't, in good faith, say that an IP that was bought today was a bot six months ago or one year ago. And if we did, then we would inaccurately count something as a bot. These bots are these anti spam softwares, they're sneaky. So our our service is is constantly having to be fed new data and and new indicators that, things are bots. And so, yeah, retroactively applying it to to data would be a a really bad idea and, to maintain integrity and and make sure you guys can actually trust the metrics, we we did not do that. That's so interesting. And I see a lot of questions too. So our most and I know you might not have this answer. Are most bots just IT spam filtering? No. Well, there a lot of them are, but there's also some from, like, the AI models I mentioned. Mhmm. There are some from some bad actors. So there's a plethora of reasons, but a lot of them are anti spam, especially if you see a lot and you're you're sending to corporate email addresses or e d u email addresses a lot, because our spam like, stuff in my work inbox from people at my job go to spam all the time because of how high our anti spam software is, and and that's where a lot of the behavior comes from. But not all of it comes from there. Okay. Yeah. That is true. Camilla asked, with these cleaner metrics, which KPIs do you recommend prioritizing now? Yeah. I think open open rate is still a good metric to to track. I would like I mentioned before, probably prioritize click rate over it. But, like, with open rate, you can still create benchmarks for yourself because Apple and PP Apple users are very loyal. And so, like, if I have an iPhone today, I'm I'm I've had an iPhone since, I don't even know, 2009 or. something. Right? So, like, I'm I'm not getting an Android unless something crazy happens probably. And so, like, my open rate for my like, me would be very stable. And so once you kinda establish a baseline for open rate, I think that metric is okay. But, again, I would trust click rate more. Okay. Yeah. It's so funny because we. were trained, I feel like, open rate, open rate. And so now we're just having to retrain and establish click rate is a more reliable given the nature that we're in with technology and bots. Yeah. So the sad thing, but job security, the bots. are ever evolving too. So that's just something else to kinda think about. Like, we've launched these features. We have this knowledge now. The Apple could be watching this and know that we were on to them. We figured it out, and now they need to come up with something new. Right? And so we'll we'll always be evolving. And like I mentioned before too, like, this isn't a Mailchimp thing. Right? Like, everyone in our industry is dealing with the same thing. Even, you know, companies who are focused on web traffic, like, those there's bots and web traffic as well too. So it's definitely a technology that we're all kind of running to understand and as it evolves, we'll also need to evolve. Absolutely. Okay. Had a really great question. Is there a reason to not turn the bot filtering on? Is there any data that we would be losing or not. tracking? You don't lose any data. You're not losing tracking or anything. Like I mentioned earlier, we are taking all opens and clicks into our database. We're just identifying them as human or nonhuman, and our filter takes care of the math for us. Thank goodness. Yeah, any reason to not turn filtering on, I would I think I mentioned a few of them, but, like, if you're comparing year over year, well, you might wanna turn the filter off just to get apples to apples since last year we did not have the bot filter. If you have somebody you're reporting numbers to who hasn't quite adopted this this bot mindset or understand bots and they just kinda want, again, an apples to apples comparison, you might turn the filter off. So there's different kind of, like, small moments where you might wanna turn the filter off to to just be able to have your normal context of what these metrics are, and that's fine. Nothing not like I mentioned, nothing happens in terms of your data. It's literally just math that we're doing on the back end to either filter the nonhumans in or out. So feel free to turn it on and off as many times as you want to, and everything should be fine. Perfect. That's a good reassurance too for folks who were, like, debating about using this feature. Okay. So Anne asked, are you able to export both filtered and non filtered numbers through the analytics tab or see them side by side? Yeah. We are toying with the idea of showing, both bot and non bot together. And some of the research that we did ahead of launching to the, this product, a lot of our customers and partners that we talked to just wanted clean data and they didn't really care. They didn't really care how many bots they had or where the places the bots were coming from. And after launching the feature and talking to more customers, I think people are maybe changing their mind a bit like this question too. So it is something that's in our backlog to to consider and retest with our users to see if it's, you know, something that we should do, because now there seems to be a little bit more interest than when I initially spoke to a lot of you guys. So, it's on our road map. It's not yet prioritized, but I can definitely see this being launched in the future. Okay. Sorry. My Wi Fi? Oh, no. For Apple MPP, Okay. the filtering will go back to June 2024. And then for the bot clicks, that info will go back to, the November 2025. And we'll double check the KB articles to make sure that's on there. If not, we'll get it added so you don't have to remember that. Okay. I was just going to check that actually myself and see if that date was on there. Yeah. Please. And let us know if it's not, because, yeah, that came from my brain. Impressive. Thank you. Okay. We have a little bit of time left. I'm gonna push a couple more questions. So if you have anything left, definitely let us know. Jen asked, I may have missed this, but will Apple MPP be integrated with bot filtering, or will they always be separate? They're integrated today. So it it's all one setting, but there's three settings within the one setting. So you can there's Apple MPP, bot clicks for email, and bot clicks for SMS. They're all part of that one setting on the settings page, but you can toggle which one of the three you want on or off. So if you have was what you Oh, sorry. Go ahead. wait. No. I was just saying that was what you showed us in app, right, when. you clicked on it and then the drop down came. up? Okay. Yep. And so if. you this was an experiment for a while. So if you haven't seen this filter, that's probably why. But as of last night, I did roll this out, to almost all users, so you should see it popping up. And so when you see it, you will not see that Apple MPP checkbox that was on marketing dashboard or or the email report anymore because we got rid of that checkbox, and now it's part of that setting. I love that. Okay. That's a good call out. Perfect. And then I just saw in the chat, Sally said, so awesome to be able to see the real data now with this filter. Thank you so much. Sally, don't make me cry on camera. Think, you know, our our team I I obviously, there's a ton of people to thank for this. Even our backstage help, Simone, has helped with all of our knowledge based articles. My engineering team love them. Everyone worked super, super hard to try to get this out to you guys. And we know we didn't get every surface area, but we really felt it was very important for you guys to get data in as many surface areas as possible as soon as possible. And so I owe a lot to that to my team. I stressed them out a little bit, but, I'm really happy that you guys are happy with it. I love that. Yes. It's so nice. And the exciting thing is that it's only gonna get better, and we're only gonna continue to add more. So I will look at we can do one more question. Let's see. Okay. I have to call this out. I just saw in the chat, Emma had mentioned we were considering leaving Intuit Mailchimp before this came out. This was really important for our company, and it's really great to have accurate data. Screenshot. So thank you. I know times a 100. I love that. I love that. Thank you, Emma. You made my day. Now I can be happy for the rest of the day. But I'm I'm glad we saved you, and, you know, just let us know how else we can continue to save you in the future. Yes. No kidding. And these webinars are part of it. Like, hearing your feedback, what do you guys want? Where do you feel like the product is missing? We truly take that. We loop in people like Taylor, who's a product manager, and we give your feedback to her, and she can help action on it. So thank you for coming and sharing your feedback. So, yes, let me grab, okay. One more question. We can do Shane's. Is email name one of the deterministic signals? I found that a lot of the bots we have dealt with have a bit of a template email address, something like this, for example. Do have any insight? No. This is very interesting. That's not something that we considered ourselves. I mentioned we did we do work with a couple of third parties that help us determine whether or not something is a bot. I have to double check and see if they're using something like that. But for us, our team didn't personally implement anything that looks at kinda risky, sketchy looking email addresses, but that's super interesting. Yeah. That is a lot of spam. Now that I think about it, I'm like, yeah. There is kind of, like, a templatized Yeah. format, but something to consider. While we wind down, I do wanna launch one last poll question. As we've answered a ton of questions, hopefully, we got to everybody's. After today's session, how likely are you to use our bot filtering to inform your marketing decisions? I have my hopes of what the answer is, but we just wanna gauge, kind of how everybody is feeling because we can absolutely have a part two to this webinar. Expand more. Of course, I'm gonna have a part two once we launch bot filtering more, an app on the various places that y'all have mentioned. And as we just wind down, today's session will be recorded. It's recorded. It's going to be emailed to you tomorrow. And I'm gonna also go ahead and launch a survey. Within the survey, there is a question in particular where we ask, do you want someone from our customer success team to reach out to? We covered a ton, and I know there were some questions that I saw come through that are very specific to people's account. And I wanna make sure that we can answer those questions. And so if you do want someone from our success team to reach out to you, please just select yes, and I'll be sure to get our team in contact with you. And why that's so crucial is because we can set dedicated time to look in your account and help, one, see the data that you have questions about and be able to provide relevant recommendations rather than kind of like a blanketed answer. So please, in that survey, submit yes. So you want someone to reach out, and we will get to you. And this is not gonna be the last of this bot filtering series. I see a couple more in our future, so definitely keep an eye out in your email for those invites. And, yes, just thank you, Taylor, for being you, for building this and bringing this product to life, and thanks everybody for joining us. Thank you all. Yeah. Yay, team. Happy Thursday, everybody. Thank you so much. We'll see you soon. Bye.