TTEC, [NASDAQ: TTEC], a company used by some of the world’s largest brands to help manage customer support and sales online and over the phone, is dealing with disruptions from a network security incident resulting from a ransomware attack, KrebsOnSecurity has learned.

While many companies have been laying off or furloughing workers in response to the Coronavirus pandemic, TTEC has been massively hiring. Formerly TeleTech Holdings Inc., Englewood, Co.-based TTEC now has nearly 60,000 employees, most of whom work from home and answer customer support calls on behalf of a large number of name-brand companies, like Bank of America, Best Buy, Credit Karma, Dish Network, Kaiser Permanente, USAA and Verizon.

On Sept. 14, KrebsOnSecurity heard from a reader who passed on an internal message apparently sent by TTEC to certain employees regarding the status of a widespread system outage that began on Sunday, Sept. 12.

“We’re continuing to address the system outage impacting access to the network, applications and customer support,” reads an internal message sent by TTEC to certain employees.

TTEC has not responded to requests for comment. A phone call placed to the media contact number listed on an August 2021 TTEC earnings release produced a message saying it was a non-working number.

[Update, 6:20 p.m. ET: TTEC confirmed a ransomware attack. See the update at the end of this piece for their statement]

TTEC’s own message to employees suggests the company’s network may have been hit by the ransomware group “Ragnar Locker,” (or else by a rival ransomware gang pretending to be Ragnar). The message urged employees to avoid clicking on a file that suddenly may have appeared in their Windows start menu called “!RA!G!N!A!R!”

“DO NOT click on this file,” the notice read. “It’s a nuisance message file and we’re working on removing it from our systems.”

Ragnar Locker is an aggressive ransomware group that typically demands millions of dollars worth of cryptocurrency in ransom payments. In an announcement published on the group’s darknet leak site this week, the group threatened to publish the full data of victims who seek help from law enforcement and investigative agencies following a ransomware attack.

One of the messages texted to TTEC employees included a link to a Zoom videoconference line at ttec.zoom.us. Clicking that link opened a Zoom session in which multiple TTEC employees who were sharing their screens took turns using the company’s Global Service Desk, an internal TTEC system for tracking customer support tickets.

The TTEC employees appear to be using the Zoom conference line to report the status of various customer support teams, most of which are reporting “unable to work” at the moment.

For example, TTEC’s Service Desk reports that hundreds of TTEC employees assigned to work with Bank of America’s prepaid services are unable to work because they can’t remotely connect to TTEC’s customer service tools. More than 1,000 TTEC employees are currently unable to do their normal customer support work for Verizon, according to the Service Desk data. Hundreds of employees assigned to handle calls for Kaiser Permanente also are unable to work.

“They’ve been radio silent all week except to notify employees to take another day off,” said the source who passed on the TTEC messages, who spoke to KrebsOnSecurity on condition of anonymity. “As far as I know, all low-level employees have another day off today.”

The extent and severity of the incident at TTEC remains unknown. It is common for companies to disconnect critical systems in the event of a network intrusion, as part of a larger effort to stop the badness from spreading elsewhere. Sometimes disconnecting everything actually does help, or at least helps to keep the attack from spreading to partner networks. But it is those same connections to partner companies that raises concern in the case of TTEC’s ongoing outage.

In the meantime, if you’re unlucky enough to need to make a customer service call today, there’s a better-than-even chance you will experience….wait for it…longer-than-usual hold times.

This is a developing story. Further details or updates will be noted here with a date and time stamp.

Update, 5:37 p.m. ET: TTEC responded with the following statement:

TTEC is committed to cyber security, and to protecting the integrity of our clients’ systems and data. We recently became aware of a cybersecurity incident that has affected certain TTEC systems.  Although as a result of the  incident, some of our data was encrypted and business activities at several facilities have been temporarily disrupted, the company continuous to serve its global clients. TTEC immediately activated its information security incident response business continuity protocols, isolated the systems involved, and took other appropriate measures to contain the incident. We are now in the process of  carefully and deliberately restoring the systems that have been involved.

We also launched an investigation, typical under the circumstances, to determine the potential impacts.  In serving our clients TTEC, generally, does not maintain our clients’ data, and the investigation to date has not identified compromise to clients’ data. That investigation is on-going and we will take additional action, as appropriate, based on the investigation’s results. This is all the information we have to share until our investigation is complete.

On Thursday evening, KrebsOnSecurity was the subject of a rather massive (and mercifully brief) distributed denial-of-service (DDoS) attack. The assault came from “Meris,” the same new “Internet of Things” (IoT) botnet behind record-shattering attacks against Russian search giant Yandex this week and internet infrastructure firm Cloudflare earlier this summer.

Cloudflare recently wrote about its attack, which clocked in at 17.2 million bogus requests-per-second. To put that in perspective, Cloudflare serves over 25 million HTTP requests per second on average.

In its Aug. 19 writeup, Cloudflare neglected to assign a name to the botnet behind the attack. But on Thursday DDoS protection firm Qrator Labs identified the culprit — “Meris” — a new IoT monster that first emerged at the end of June 2021.

Qrator says Meris has launched even bigger attacks since: A titanic and ongoing DDoS that hit Russian Internet search giant Yandex last week is estimated to have been launched by roughly 250,000 malware-infected devices globally, sending 21.8 million bogus requests-per-second.

While last night’s Meris attack on this site was far smaller than the recent Cloudflare DDoS, it was far larger than the Mirai DDoS attack in 2016 that held KrebsOnSecurity offline for nearly four days. The traffic deluge from Thursday’s attack on this site was was more than four times what Mirai threw at this site five years ago. This latest attack involved more than two million requests-per-second. By comparison, the 2016 Mirai DDoS generated approximately 450,000 requests-per-second.

According to Qrator, which is working with Yandex on combating the attack, Meris appears to be made up of Internet routers produced by MikroTik. Qrator says the United States is home to the most number of MikroTik routers that are potentially vulnerable to compromise by Meris — with more than 42 percent of the world’s MikroTik systems connected to the Internet (followed by China — 18.9 percent– and a long tail of one- and two-percent countries).

It’s not immediately clear which security vulnerabilities led to these estimated 250,000 MikroTik routers getting hacked by Meris.

“The spectrum of RouterOS versions we see across this botnet varies from years old to recent,” the company wrote. “The largest share belongs to the version of firmware previous to the current stable one.”

It’s fitting that Meris would rear its head on the five-year anniversary of the emergence of Mirai, an IoT botnet strain that was engineered to out-compete all other IoT botnet strains at the time. Mirai was extremely successful at crowding out this competition, and quickly grew to infect tens of thousands of IoT devices made by dozens of manufacturers.

And then its co-authors decided to leak the Mirai source code, which led to the proliferation of dozens of Mirai variants, many of which continue to operate today.

The biggest contributor to the IoT botnet problem — a plethora of companies white-labeling IoT devices that were never designed with security in mind and are often shipped to the customer in default-insecure states — hasn’t changed much, mainly because these devices tend to be far cheaper than more secure alternatives.

The good news is that over the past five years, large Internet infrastructure companies like Akamai, Cloudflare and Google (which protects this site with its Project Shield initiative) have heavily invested in ramping up their ability to withstand these outsized attacks [full disclosure: Akamai is an advertiser on this site].

More importantly, the Internet community at large has gotten better at putting their heads together to fight DDoS attacks, by disrupting the infrastructure abused by these enormous IoT botnets, said Richard Clayton, director of Cambridge University’s Cybercrime Centre.

“It would be fair to say we’re currently concerned about a couple of botnets which are larger than we have seen for some time,” Clayton said. “But equally, you never know they may peter out. There are a lot of people who spend their time trying to make sure these things are hard to keep stable. So there are people out there defending us all.”

In May 2015, KrebsOnSecurity briefly profiled “The Manipulaters,” the name chosen by a prolific cybercrime group based in Pakistan that was very publicly selling spam tools and a range of services for crafting, hosting and deploying malicious email. Six years later, a review of the social media postings from this group shows they are prospering, while rather poorly hiding their activities behind a software development firm in Lahore that has secretly enabled an entire generation of spammers and scammers.

The Manipulaters’ core brand in the underground is a shared cybercriminal identity named “Saim Raza,” who for the past decade across dozens of cybercrime sites and forums has peddled a popular spamming and phishing service variously called “Fudtools,” “Fudpage,” “Fudsender,” etc.

The common acronym in nearly all of Saim Raza’s domains over the years — “FUD” — stands for “Fully Un-Detectable,” and it refers to cybercrime resources that will evade detection by security tools like antivirus software or anti-spam appliances.

The current website for Saim Raza’s Fud Tools (above) offers phishing templates or “scam pages” for a variety of popular online sites like Office365 and Dropbox. They also sell “Doc Exploit” products that bundle malicious software with innocuous Microsoft Office documents; “scampage hosting” for phishing sites; a variety of spam blasting tools like HeartSender; and software designed to help spammers route their malicious email through compromised sites, accounts and services in the cloud.

For years leading up to 2015, “admin@manipulaters.com” was the name on the registration records for thousands of scam domains that spoofed some of the world’s top banks and brand names, but particularly Apple and Microsoft. When confronted about this, The Manipulaters founder Madih-ullah Riaz replied, “We do not deliberately host or allow any phishing or any other abusive website. Regarding phishing, whenever we receive complaint, we remove the services immediately. Also we are running business since 2006.”

Two years later, KrebsOnSecurity received an email from Riaz asking to have his name and that of his business partner removed from the 2015 story, saying it had hurt his company’s ability to maintain stable hosting for their stable of domains.

“We run web hosting business and due to your post we got very serious problems especially no data center was accepting us,” Riaz wrote in a May 2017 email. “I can see you post on hard time criminals we are not criminals, at least it was not in our knowledge.”

Riaz said the problem was his company’s billing system erroneously used The Manipulators’ name and contact information instead of its clients in WHOIS registration records. That oversight, he said, caused many researchers to erroneously attribute to them activity that was coming from just a few bad customers.

“We work hard to earn money and it is my request, 2 years of my name in your wonderful article is enough punishment and we learned from our mistakes,” he concluded.

The Manipulaters have indeed learned a few new tricks, but keeping their underground operations air-gapped from their real-life identities is mercifully not one of them.

ZERO OPERATIONAL SECURITY

Phishing domain names registered to The Manipulaters included an address in Karachi, with the phone number 923218912562. That same phone number is shared in the WHOIS records for 4,000+ domains registered through domainprovider[.]work, a domain controlled by The Manipulaters that appears to be a reseller of another domain name provider.

One of Saim Raza’s many ads in the cybercrime underground for his Fudtools service promotes the domain fudpage[.]com, and the WHOIS records for that domain share the same Karachi phone number. Fudpage’s WHOIS records list the contact as “admin@apexgrand.com,” which is another email address used by The Manipulaters to register domains.

As I noted in 2015, The Manipulaters Team used domain name service (DNS) settings from another blatantly fraudulent service called ‘FreshSpamTools[.]eu,’ which was offered by a fellow Pakistani who also conveniently sold phishing toolkits targeting a number of big banks.

The WHOIS records for FreshSpamTools briefly list the email address bilal.waddaich@gmail.com, which corresponds to the email address for a Facebook account of a Bilal “Sunny” Ahmad Warraich (a.k.a. Bilal Waddaich).

Warraich’s Facebook profile says he works as an IT support specialist at a software development company in Lahore called We Code Solutions.

A review of the hosting records for the company’s website wecodesolutions[.]pk show that over the past three years it has shared a server with just a handful of other domains, including:

-saimraza[.]tools
-fud[.]tools
-heartsender[.]net
-fudspampage[.]com
-fudteam[.]com
-autoshopscript[.]com
-wecodebilling[.]com
-antibotspanel[.]com
-sellonline[.]tools

FUD CO

The profile image atop Warraich’s Facebook page is a group photo of current and former We Code Solutions employees. Helpfully, many of the faces in that photo have been tagged and associated with their respective Facebook profiles.

For example, the Facebook profile of Burhan Ul Haq, a.k.a. “Burhan Shaxx” says he works in human relations and IT support for We Code Solutions. Scanning through Ul Haq’s endless selfies on Facebook, it’s impossible to ignore a series of photos featuring various birthday cakes and the words “Fud Co” written in icing on top.

Yes, from a review of the Facebook postings of We Code Solutions employees, it appears that for at least the last five years this group has celebrated an anniversary every May with a Fud Co cake, non-alcoholic sparkling wine, and a Fud Co party or group dinner. Let’s take a closer look at that delicious cake:

The head of We Code Solutions appears to be a guy named Rameez Shahzad, the older individual at the center of the group photo in Warraich’s Facebook profile. You can tell Shahzad is the boss because he is at the center of virtually every group photo he and other We Code Solutions employees posted to their respective Facebook pages.

Shahzad’s postings on Facebook are even more revelatory: On Aug. 3, 2018, he posted a screenshot of someone logged into a WordPress site under the username Saim Raza — the same identity that’s been pimping Fud Co spam tools for close to a decade now.

“After [a] long time, Mailwizz ready,” Shahzad wrote as a caption to the photo:

Whoever controlled the Saim Raza cybercriminal identity had a penchant for re-using the same password (“lovertears”) across dozens of Saim Raza email addresses. One of Saim Raza’s favorite email address variations was “game.changer@[pick ISP here]”. Another email address advertised by Saim Raza was “bluebtcus@gmail.com.”

So it was not surprising to see Rameez Shahzad post a screenshot to his Facebook account of his computer desktop, which shows he is logged into a Skype account that begins with the name “game.” and a Gmail account beginning with “bluebtc.”

KrebsOnSecurity attempted to reach We Code Solutions via the contact email address on its website — info@wecodesolutions[.]pk — but the message bounced back, saying there was no such address. Similarly, a call to the Lahore phone number listed on the website produced an automated message saying the number is not in service. None of the We Code Solutions employees contacted directly via email or phone responded to requests for comment.

FAIL BY NUMBERS

This open-source research on The Manipulaters and We Code Solutions is damning enough. But the real icing on the Fud Co cake is that sometime in 2019, The Manipulaters failed to renew their core domain name — manipulaters[.]com — the same one tied to so many of the company’s past and current business operations.

That domain was quickly scooped up by Scylla Intel, a cyber intelligence firm that specializes in connecting cybercriminals to their real-life identities. Whoops.

Scylla co-founder Sasha Angus said the messages that flooded their inbox once they set up an email server on that domain quickly filled in many of the details they didn’t already have about The Manipulaters.

“We know the principals, their actual identities, where they are, where they hang out,” Angus said. “I’d say we have several thousand exhibits that we could put into evidence potentially. We have them six ways to Sunday as being the guys behind this Saim Raza spammer identity on the forums.”

Angus said he and a fellow researcher briefed U.S. prosecutors in 2019 about their findings on The Manipulaters, and that investigators expressed interest but also seemed overwhelmed by the volume of evidence that would need to be collected and preserved about this group’s activities.

“I think one of the things the investigators found challenging about this case was not who did what, but just how much bad stuff they’ve done over the years,” Angus said. “With these guys, you keep going down this rabbit hole that never ends because there’s always more, and it’s fairly astonishing. They are prolific. If they had halfway decent operational security, they could have been really successful. But thankfully, they don’t.”

Some of the most successful and lucrative online scams employ a “low-and-slow” approach — avoiding detection or interference from researchers and law enforcement agencies by stealing small bits of cash from many people over an extended period. Here’s the story of a cybercrime group that compromises up to 100,000 email inboxes per day, and apparently does little else with this access except siphon gift card and customer loyalty program data that can be resold online.

The data in this story come from a trusted source in the security industry who has visibility into a network of hacked machines that fraudsters in just about every corner of the Internet are using to anonymize their malicious Web traffic. For the past three years, the source — we’ll call him “Bill” to preserve his requested anonymity — has been watching one group of threat actors that is mass-testing millions of usernames and passwords against the world’s major email providers each day.

Bill said he’s not sure where the passwords are coming from, but he assumes they are tied to various databases for compromised websites that get posted to password cracking and hacking forums on a regular basis. Bill said this criminal group averages between five and ten million email authentication attempts daily, and comes away with anywhere from 50,000 to 100,000 of working inbox credentials.

In about half the cases the credentials are being checked via “IMAP,” which is an email standard used by email software clients like Mozilla’s Thunderbird and Microsoft Outlook. With his visibility into the proxy network, Bill can see whether or not an authentication attempt succeeds based on the network response from the email provider (e.g. mail server responds “OK” = successful access).

You might think that whoever is behind such a sprawling crime machine would use their access to blast out spam, or conduct targeted phishing attacks against each victim’s contacts. But based on interactions that Bill has had with several large email providers so far, this crime gang merely uses custom, automated scripts that periodically log in and search each inbox for digital items of value that can easily be resold.

And they seem particularly focused on stealing gift card data.

“Sometimes they’ll log in as much as two to three times a week for months at a time,” Bill said. “These guys are looking for low-hanging fruit — basically cash in your inbox. Whether it’s related to hotel or airline rewards or just Amazon gift cards, after they successfully log in to the account their scripts start pilfering inboxes looking for things that could be of value.”

According to Bill, the fraudsters aren’t downloading all of their victims’ emails: That would quickly add up to a monstrous amount of data. Rather, they’re using automated systems to log in to each inbox and search for a variety of domains and other terms related to companies that maintain loyalty and points programs, and/or issue gift cards and handle their fulfillment.

Why go after hotel or airline rewards? Because these accounts can all be cleaned out and deposited onto a gift card number that can be resold quickly online for 80 percent of its value.

“These guys want that hard digital asset — the cash that is sitting there in your inbox,” Bill said. “You literally just pull cash out of peoples’ inboxes, and then you have all these secondary markets where you can sell this stuff.”

Bill’s data also shows that this gang is so aggressively going after gift card data that it will routinely seek new gift card benefits on behalf victims, when that option is available.  For example, many companies now offer employees a “wellness benefit” if they can demonstrate they’re keeping up with some kind of healthy new habit, such as daily gym visits, yoga, or quitting smoking.

Bill said these crooks have figured out a way to tap into those benefits as well.

“A number of health insurance companies have wellness programs to encourage employees to exercise more, where if you sign up and pledge to 30 push-ups a day for the next few months or something you’ll get five wellness points towards a $10 Starbucks gift card, which requires 1000 wellness points,” Bill explained. “They’re actually automating the process of replying saying you completed this activity so they can bump up your point balance and get your gift card.”

The Gift Card Gang’s Footprint

How do the compromised email credentials break down in terms of ISPs and email providers? There are victims on nearly all major email networks, but Bill said several large Internet service providers (ISPs) in Germany and France are heavily represented in the compromised email account data.

“With some of these international email providers we’re seeing something like 25,000 to 50,000 email accounts a day get hacked,” Bill said.  “I don’t know why they’re getting popped so heavily.”

That may sound like a lot of hacked inboxes, but Bill said some of the bigger ISPs represented in his data have tens or hundreds of millions of customers.

Measuring which ISPs and email providers have the biggest numbers of compromised customers is not so simple in many cases, nor is identifying companies with employees whose email accounts have been hacked.

This kind of mapping is often more difficult than it used to be because so many organizations have now outsourced their email to cloud services like Gmail and Microsoft Office365 — where users can access their email, files and chat records all in one place.

“It’s a little complicated with Office 365 because it’s one thing to say okay how many Hotmail connections are you seeing per day in all this credential-stuffing activity, and you can see the testing against Hotmail’s site,” Bill said. “But with the IMAP traffic we’re looking at, the usernames being logged into are any of the million or so domains hosted on Office365, many of which will tell you very little about the victim organization itself.”

On top of that, it’s also difficult to know how much activity you’re not seeing.

Looking at the small set of Internet address blocks he knows are associated with Microsoft 365 email infrastructure, Bill examined the IMAP traffic flowing from this group to those blocks. Bill said that in the first week of April 2021, he identified 15,000 compromised Office365 accounts being accessed by this group, spread over 6,500 different organizations that use Office365.

“So I’m seeing this traffic to just like 10 net blocks tied to Microsoft, which means I’m only looking at maybe 25 percent of Microsoft’s infrastructure,” Bill explained. “And with our puny visibility into probably less than one percent of overall password stuffing traffic aimed at Microsoft, we’re seeing 600 Office accounts being breached a day. So if I’m only seeing one percent, that means we’re likely talking about tens of thousands of Office365 accounts compromised daily worldwide.”

In a December 2020 blog post about how Microsoft is moving away from passwords to more robust authentication approaches, the software giant said an average of one in every 250 corporate accounts is compromised each month. As of last year, Microsoft had nearly 240 million active users, according to this analysis.

“To me, this is an important story because for years people have been like, yeah we know email isn’t very secure, but this generic statement doesn’t have any teeth to it,” Bill said. “I don’t feel like anyone has been able to call attention to the numbers that show why email is so insecure.”

Bill says that in general companies have a great many more tools available for securing and analyzing employee email traffic when that access is funneled through a Web page or VPN, versus when that access happens via IMAP.

“It’s just more difficult to get through the Web interface because on a website you have a plethora of advanced authentication controls at your fingertips, including things like device fingerprinting, scanning for http header anomalies, and so on,” Bill said. “But what are the detection signatures you have available for detecting malicious logins via IMAP?”

Microsoft declined to comment specifically on Bill’s research, but said customers can block the overwhelming majority of account takeover efforts by enabling multi-factor authentication.

“For context, our research indicates that multi-factor authentication prevents more than 99.9% of account compromises,” reads a statement from Microsoft. “Moreover, for enterprise customers, innovations like Security Defaults, which disables basic authentication and requires users to enroll a second factor, have already significantly decreased the proportion of compromised accounts. In addition, for consumer accounts, adding a second authentication factor is required on all accounts.”

A Mess That’s Likely to Stay That Way

Bill said he’s frustrated by having such visibility into this credential testing botnet while being unable to do much about it. He’s shared his data with some of the bigger ISPs in Europe, but says months later he’s still seeing those same inboxes being accessed by the gift card gang.

The problem, Bill says, is that many large ISPs lack any sort of baseline knowledge of or useful data about customers who access their email via IMAP. That is, they lack any sort of instrumentation to be able to tell the difference between legitimate and suspicious logins for their customers who read their messages using an email client.

“My guess is in a lot of cases the IMAP servers by default aren’t logging every search request, so [the ISP] can’t go back and see this happening,” Bill said.

Confounding the challenge, there isn’t much of an upside for ISPs interested in voluntarily monitoring their IMAP traffic for hacked accounts.

“Let’s say you’re an ISP that does have the instrumentation to find this activity and you’ve just identified 10,000 of your customers who are hacked. But you also know they are accessing their email exclusively through an email client. What do you do? You can’t flag their account for a password reset, because there’s no mechanism in the email client to affect a password change.”

Which means those 10,000 customers are then going to start receiving error messages whenever they try to access their email.

“Those customers are likely going to get super pissed off and call up the ISP mad as hell,” Bill said. “And that customer service person is then going to have to spend a bunch of time explaining how to use the webmail service. As a result, very few ISPs are going to do anything about this.”

Indictators of Compromise (IoCs)

It’s not often KrebsOnSecurity has occasion to publish so-called “indicators of compromise” (IoC)s, but hopefully some ISPs may find the information here useful. This group automates the searching of inboxes for specific domains and trademarks associated with gift card activity and other accounts with stored electronic value, such as rewards points and mileage programs.

This file includes the top inbox search terms used in a single 24 hour period by the gift card gang. The numbers on the left in the spreadsheet represent the number of times during that 24 hour period where the gift card gang ran a search for that term in a compromised inbox.

Some of the search terms are focused on specific brands — such as Amazon gift cards or Hilton Honors points; others are for major gift card networks like CashStar, which issues cards that are white-labeled by dozens of brands like Target and Nordstrom. Inboxes hacked by this gang will likely be searched on many of these terms over the span of just a few days.