10/17/2017 | Episode 8
Maritza Dominguez is Trust & Safety Lead at Patreon.
Kevin Lee is a Trust & Safety Architect at Sift Science.
A former Marketplace Manager at Wanelo, Courtney Bode has since founded OpsTales.
Evan: Welcome to “Trust & Safety in Numbers,” presented by Sift Science. I’m your host, Evan Ramzipoor. It’s October so while you’re dunking your pumpkin spiced Oreos into your pumpkin spiced lattes, TM, there is a little trademark there, you just couldn’t see it, retailers across the country are stocking their shelves with Santa hats. Love them or hate them, the winter holidays are coming and with them, the fraudsters. This episode is a veritable cornucopia of holiday fraud tips. We’ll hear from expert fraud fighters on how they get ready for the holiday rush and what you can do to ensure you’re not overwhelmed on Black Friday. But first, let’s warm up with a quick fraud fact. Did you know that last year e-commerce fraud spiked by over 30% during the holidays? About 1 in every 97 transactions was fraudulent. For more on holiday fraud prep, check out, “Get Ready for the Holiday Rush,” on the Sift Science blog. Now, put on some Bing Crosby and let’s start the show. So, just how bad is the holiday fraud rush?
Courtney: When we go into the holiday season, we actually see our manual review queue sometimes up to tripling. We also can see our dispute rate doubling if we’re not really careful about it.
Evan: That was Courtney Bode, Workplace Operations Manager at Wanelo.
Courtney: We consider peak season to start around Black Friday. That’s when we see a lot of foreign IP Addresses and kinda crazy purchasing patterns begin. And that’s when it becomes really hard for us to start isolating what’s true fraud and what is user behavior just slightly shifting as they move addresses and ship to family members.
Evan: Kevin Lee, our Trust and Safety expert here at Sift Science and a seasoned fraud fighter, has this to say about the holiday fraud rush.
Kevin: This is essentially the Super Bowl for a lot of e-commerce companies. Whether you’re in digital goods or physical goods, you’re probably gonna see a spike in orders flowing through your system, so that can be pretty exciting. At the same time, it can be pretty stressful. It’s going to be a rush. And so depending on how your systems are set up and how much preparation you’ve done, it can be somewhat taxing. So, know that there is some light at the end of the tunnel. Christmas is eventually gonna come. And so it will end, but it will be quite stressful for about a three week period.
Evan: What is it about the Holidays that drives fraudsters to make such bold moves? Maritza Dominguez, head of Trust and Safety at Patreon has some ideas.
Maritza: Fraudsters tend to think that they can squeak in between your other purchases. And so it’s always a huge thing that we always had to prepare for.
Evan: Stopping these sneaky fraudsters can seem like a daunting task. How do you even get started? Courtney from Wanelo shares some tips.
Courtney: First, we do a quick industry scan you know, what’s been happening in the past. What are other people seeing? And how do we integrate that into our process? And then we look at our internal data. What did we see last year and in our other past holiday seasons? The biggest question for us is, what false negatives and positives did we see? And how can we take that learning and adjust our new holiday season based off of our past mistakes?
Evan: Kevin Lee has similar advice.
Kevin: In the past, it started off with certainly communications within the company. And so understanding what the company first did in the past, so what did the holiday rush look like last year, really studying the transaction flow coming into the system, talking to the marketing team or whoever is running promotions, just so you’re ready to expect those spikes. There’s gonna be a spike regardless post-Thanksgiving, but in terms of marketing and other promotions that are running, expect to see some bumps there.
Maritza: I’ve done a few different things that have helped me throughout the year. One, and something that I preach to anyone who will listen is become friends with fraud fighters at other companies.
Evan: That’s Maritza from Patreon. Although Patreon doesn’t experience seasonal fraud, Maritza says she’s worked in verticals where it can be a lot to deal with.
Maritza: It’s super helpful to share tips and things that you’re doing and finding out trends that are happening in other places. They may not be the exact same as yours, but it can spark an idea of something you can try that might be helpful for you. I know I’ve picked up a lot of tips going to fraud forums, or becoming friends with fraud fighters at other companies. Also going to conferences and listening to experts speak about things they’ve done. It’s super helpful to learn from other people’s mistakes and from their successes.
Evan: That’s a good starting point. So, step one, take the lay of the land and recall what you’ve learned from the past. Courtney from Wanelo takes us to step two.
Courtney: In addition, looking at company priorities. A big question for us is, how risk-averse do we wanna be, is a bigger priority for us to accept new orders and new customers? Or do we wanna prevent fraud? How much can we allow for our dispute rate to increase and what decline rate do we want to maintain?
Evan: Maritza’s next suggestion is something we’re all intimately familiar with in San Francisco, layer up.
Maritza: It may not be something you do year round but in preparation for a big fraud pump, there is no shame in adding extra products or services that will help you out.
Evan: As many of us know, a robust manual review process is key to a good fraud fighting strategy. This is true year round but it’s especially true during the busier months.
Courtney: After we look at the industry and our internal data, the next thing we look at is our manual review process. Based on the past false positives, we try to see what types of shopping trends we need to be aware of and how our manual review team was able to handle this in the past. A big question for us is whether our team has the ability to quickly analyze new trends, or whether that’s something that I need to particularly train them on this year. And again, that all comes out of our past historical data and we can see where we failed and how we can improve.
Kevin: If you’re using rules, then I’d strongly consider revamping or adjusting some of those. If you have machine learning in place, that will probably be a bit easier for you, especially if it’s been through a holiday rush before just because it will expect to see some anomalies in there and kind of account for them ahead of time.
Evan: So you’ve gathered data, you’ve got your industry-wide intelligence, you’ve layered up, what’s next?
Courtney: And the last thing we do with all of this information, we start collecting this information. At the end of September, beginning of October is to start making adjustments to our team and hire the new people. For reference, our team grew last year from about three people on manual review, up to 12 people at it’s highest. So, there can be some big changes that need to happen.
Evan: More than anything, Courtney says it’s vitally important to know your user base.
Courtney: A lot of the typical fraud trends like changing shipping address and changing credit card information or IP address, a lot of that becomes really common during the holidays. So, you really need to get in touch with your growth team and your product team and see what user behavior is normal today and what they predict to be normal in a month, and you need to compare that with your fraud process and teach your manual review team what might have looked bad three months ago is actually okay during Christmas time.
Evan: Dig into your data, look at the trends, and make adjustments as you see fit.
Maritza: Maybe five day holds on certain types of criteria if you have you know, three of five of these things. That has always helped out for us. I’ve been able to catch a lot of things before they’ve made it out the door because of the extra steps that I put in place.
Evan: But Maritza cautions that there is no magic formula for catching fraudsters.
Maritza: Unfortunately, there’s not one silver bullet that will take care of everything, but if you’re anticipating something bad coming along, adding extra catches or adding extra rate limiting or an extra service that would catch certain kinds of fraud.
Evan: And though that’s true, Kevin says preparation and planning are key to a successful holiday season. Make your list and check it twice.
Kevin: There’s an old quote from John Wooden that comes to mind in terms of, “Failure to prepare is preparing to fail.” And so really, as much as you can before Black Friday or whenever you’re going to see this big push, understand that your team is going to need a lot of preparation, whether that’s your existing team or maybe you’re going to have to cross-train folks on the customer support team, or maybe you’re going to have to hire temp staff or do something with another team, whether they’re in the office or offshore. So, creating as much policy and procedure and kind of doing a few test runs and stress tests of the system, whether it’s on people or policies or rules or what have you, to understand what it’s gonna look like in real life and kind of practicing for that kind of main event.
Evan: All right. So, we’ve taken in a lot of information here, but is there anything we’re missing?
Maritza: I also expect long nights.
Evan: Oh, yeah. That. Well, hopefully, they won’t be too long now that you’ve listened to this podcast. Thanks for joining us on “Trust & Safety in Numbers.” Until next time, stay vigilant fraud fighters.
Learn more about what sets Sift Science’s machine learning apart.
With billions of compromised credentials already in criminals’ hands, how do you protect your users’ accounts, your brand, and your bottom line?