VMware is bringing VMs and containers together, taking advantage of Heptio acquisition

At VMworld today in San Francisco, VMware introduced a new set of services for managing virtual machines and containers in a single view called Tanzu. The product takes advantage of the knowledge the company gained when it acquired Heptio last year.

As companies face an increasingly fragmented landscape of maintaining traditional virtual machines, alongside a more modern containerized Kubernetes environment, managing the two together has created its own set of management challenges for IT. This is further complicated by trying to manage resources across multiple clouds, as well as the in-house data centers. Finally, companies need to manage legacy applications, while looking to build newer containerized applications.

VMware’s Craig McLuckie and fellow Heptio co-founder, Joe Beda, were part of the original Kubernetes development team They came to VMware via last year’s acquisition. McLuckie believes that Tanzu can help with all of this by applying the power of Kubernetes across this complex management landscape.

“The intent is to construct a portfolio that has a set of assets that cover every one of these areas, a robust set of capabilities that bring the Kubernetes substrate everywhere — a control plane that enables organizations to start to think about [and view] these highly fragmented deployments with Kubernetes [as the] common lens, and then the technologies you need to be able to bring existing applications forward and to build new application and to support third party vendors bringing their applications into [this],” McLuckie explained.

It’s an ambitious vision that involves bringing together not only VMware’s traditional VM management tooling and Kubernetes, but also open source pieces and other recent acquisitions including Bitnami and Cloud Health along with Wavefront, which it acquired in 2017. Although the vision was defined long before the acquisition of Pivotal last week, it will also play a role in this. Originally that was as a partner, but now it will be as part of VMware.

The idea is to eventually cover the entire gamut of building, running and managing applications in the enterprise. Among the key pieces introduced today as technology previews are the Tanzu Mission Control, a tool for managing Kubernetes clusters wherever the live and Project Pacific, which embeds Kubernetes natively into VSphere, the company’s virtualization platform, bringing together virtual machines and containers.

Screenshot 2019 08 26 08.07.38 1

VMware Tanzu. Slide: VMware

McLuckie sees bringing virtual machine and Kubernetes together in this fashion provides a couple of key advantages. “One is being able to bring a robust, modern API-driven way of thinking about accessing resources. And it turns out that there is this really good technology for that. It’s called Kubernetes. So being able to bring a Kubernetes control plane to Vsphere is creating a new set of experiences for traditional VMware customers that is moving much closer to a kind of cloud-like agile infrastructure type of experience. At the same time, Vsphere is bringing a whole bunch of capabilities to Kubernetes that’s creating more efficient isolation capabilities,” he said.

When you think about the cloud native vision, it has always been about enabling companies to manage resources wherever they live through a single lens, and this is what this set of capabilities that VMware has brought together under Tanzu, is intended to do. “Kubernetes is a way of bringing a control metaphor to modern IT processes. You provide an expression of what you want to have happen, and then Kubernetes takes that and interprets it and drives the world into that desired state,” McLuckie explained.

If VMware can take all of the pieces in the Tanzu vision and make this happen, it will be as powerful as McLuckie believes it to be. It’s certainly an interesting attempt to bring all of a company’s application and infrastructure creation and management under one roof using Kubernetes as the glue, and with Heptio co-founders McLuckie and Beda involved, it certainly has the expertise in place to drive the vision.

Why now is the time to get ready for quantum computing

For the longest time, even while scientists were working to make it a reality, quantum computing seemed like science fiction. It’s hard enough to make any sense out of quantum physics to begin with, let alone the practical applications of this less than intuitive theory. But we’ve now arrived at a point where companies like D-Wave, Rigetti, IBM and others actually produce real quantum computers.

They are still in their infancy and nowhere near as powerful as necessary to compute anything but very basic programs, simply because they can’t run long enough before the quantum states decohere, but virtually all experts say that these are solvable problems and that now is the time to prepare for the advent of quantum computing. Indeed, Gartner just launched a Quantum Volume metric, based on IBM’s research, that looks to help CIOs prepare for the impact of quantum computing.

To discuss the state of the industry and why now is the time to get ready, I sat down with IBM’s Jay Gambetta, who will also join us for a panel on Quantum Computing at our TC Sessions: Enterprise event in San Francisco on September 5, together with Microsoft’s Krysta Svore and Intel’s Jim Clark.

The Good, the Bad and the Ugly in Cybersecurity – Week 34

The Good

It’s common knowledge that threat actors are going after the lucrative target of payment systems. The most popular attack today is ATM cash out, ending in money withdrawn. To combat this increasing threat, Visa have announced improved measures to combat fraud and to beef up the cybersecurity of payment systems. Visa clients can take advantage of the new payment security capabilities without incurring any additional charges or registration.

After Apple’s long-awaited news of an expanded bug bounty program comes more good news for bug hunters and software endusers alike as Microsoft this week announced a new Edge Insider Bounty Program. Although Edge is a chromium-based browser (and Google have their own bug bounty program for Chrome), Microsoft are only rewarding bounties for vulnerabilities that are unique to the Edge browser. The Redwood outfit are offering between $1000 and $30000 in awards for qualifying submissions.

image of edge bug bounty announcement

The Bad

This year’s epidemic of ransomware attacks on local governments continued with 22 new attacks confirmed in Texas this week. The attackers are demanding $2.5 million, but so far there’s no indication that any ransomware has been paid. The link between these systems is they are all managed by the same service provider. We’ve seen other cities and municipal authorities (to name a few: City of Atlanta, City of Baltimore) also falling victim to ransomware attacks. The toll this year so far has reached at least 40 municipalities that we know of. From major cities like Balitmore to small towns like Lake City, it seems local governments are deliberately being targeted as threat actors are attracted by the combination of taking down mission-critical services and local governments’ lack of budget to resist paying and “toughing out” the economic damage. Combine that with many cities’ failure to deploy robust anti-ransomware security solutions and it seems like the attackers will be enjoying plenty more paydays in 2019. 

image of tweet from Texas DIR

Things seem to be going from bad to worse for Apple at the moment. Bluetooth data leaks are one thing, but it now seems that Apple’s iOS system is suffering from a security meltdown. A recent update for iOS 12.4 has undone the good work of a previous patch, and the latest version of the mobile phone operating system is once again vulnerable to the possibility of running unsigned code. On top of that, a new Bluetooth vulnerability (KNOB) affects 12.3 and earlier versions of iOS, so it appears almost every iOS device has some security issue or other. Presumably, Apple are hard at work on an update!

The Ugly

Popular web-based sys admin tool Webmin has been carrying an RCE backdoor that was maliciously inserted into its source code by attackers. Bad, of course. But ugly, too: the code was manipulated as far back as July last year without anyone noticing

image of tweet about webmin vulnerability

What do Cylance, Kaspersky, Trend Micro and Bitdefender all have in common?  The surprising answer is security flaws. Bitdefender was this week the latest security product to be found containing a serious vulnerability. The flaw would allow an attacker to take complete control of a target device with an unsigned DLL that runs as NT AUTHORITY/SYSTEM – an account with the highest level of privileges on the local system. The privilege escalation vulnerability, CVE-2019-15295, was patched by the vendor on Wednesday.

Like this article? Follow us on LinkedIn, Twitter, YouTube or Facebook to see the content we post.

Read more about Cyber Security

How Pivotal got bailed out by fellow Dell family member, VMware

When Dell acquired EMC in 2016 for $67 billion, it created a complicated consortium of interconnected organizations. Some, like VMware and Pivotal, operate as completely separate companies. They have their own boards of directors, can acquire companies and are publicly traded on the stock market. Yet they work closely within Dell, partnering where it makes sense. When Pivotal’s stock price plunged recently, VMware saved the day when it bought the faltering company for $2.7 billion yesterday.

Pivotal went public last year, and sometimes struggled, but in June the wheels started to come off after a poor quarterly earnings report. The company had what MarketWatch aptly called “a train wreck of a quarter.”

How bad was it? So bad that its stock price was down 42% the day after it reported its earnings. While the quarter itself wasn’t so bad, with revenue up year over year, the guidance was another story. The company cut its 2020 revenue guidance by $40-$50 million and the guidance it gave for the upcoming 2Q 19 was also considerably lower than consensus Wall Street estimates.

The stock price plunged from a high of $21.44 on May 30th to a low of $8.30 on August 14th. The company’s market cap plunged in that same time period falling from $5.828 billion on May 30th to $2.257 billion on August 14th. That’s when VMware admitted it was thinking about buying the struggling company.

Insider Threats | From Malicious to Unintentional

In this post, SCB’s Principal Security Consultant Tal Eliyahu discusses the case for changing our perception of the threat of malicious insiders. Simple human error is far more likely to be the cause in many cases, he argues.

What is an Unintentional Insider Threat?

An unintentional insider threat (UIT) is a current or former employee, contractor, or business partner who has or has had an authorized access to an organization’s network, system, or data and who, through action or inaction without malicious intent, unwittingly causes harm or substantially increases the probability of future serious harm to the confidentiality, integrity, or availability of the organization’s resources or assets, including information, information systems, or financial systems.

Humans are inherently complex and multi-faceted creatures with our own agendas, influences, faults, beliefs, and priorities. No amount of secure network topologies and firewalls or security software can withstand a user innocently clicking on an email link, or being convinced to give up login details over the phone by someone pretending to be from the IT department. – Australian Computer Society Report

Based on various research papers, studies and reports, it seems that we should change our perception of a malicious insider threat, and get accustomed to the idea that in most cases, people do not hold malicious intent.

Around three in four businesses (75%) and charities (76%) identifying breaches or attacks think that their most disruptive case was intentional. – Cyber Security Breaches Survey 2019

While 61% of IT leaders believe employees have maliciously breached data at some point, 94% of U.S. employees and 87% of U.K. employees claim they have not intentionally broken company data sharing policies.- Insider Data Breach survey 2019

A 2018 study on the cost of insider threats reported that 64% of organizations found the “careless employee or contractor” as being the root cause of most insider threat incidents at their places of business. Furthermore, according to the Verizon Data Breach Report, misdelivery is the fourth most frequent action that results in data breaches across sectors and human error causing 35% of all data breaches in 2019. Misdelivery is a form of human error, occurring when those entrusted with the delivery of sensitive information either intentionally or unknowingly end up delivering sensitive information to unauthorized users. 

“The most significant threats to the exposure of sensitive or confidential data are employee mistakes.”  – 2019 Global Encryption Trends Study

On the topic of UIT’s, we also can observe an indication of continuous lack of awareness among employees. On the 2018 User Risk report, it is noted that 55% of working adults allow friends and family members to access their employer-issued devices at home. A friend or family member might access highly sensitive data such as the organizations’ bank accounts or customer data.

The Cost of an Unintentional Insider Threat report identified factors leading to UIT such as multitasking, burnout, and low situational awareness. This stands as an indicator that UIT is now changing its form as “Dave” is getting older since he was created by John Klossner.

We can blame only ourselves for not taking more responsibilities setting industry success factors and principals of awareness in organizations, as we realize that even the leading consulting firms fail to interpret their own advice.

cartoon of data security

The following are a few UIT examples covered in my earlier article on the subject of Insider Bank Threats:

Case Study: HSBC

In 2017, HSBC apologized after it e-mailed personal information on customers to other account holders. The e-mails contained names, e-mail addresses, countries of residence, the name of the customers’ relationship manager and HSBC customer identification numbers.

“An e-mail was sent to a small number of our retail banking customers which unfortunately included an attachment containing personal information of some of HSBC Bermuda’s customers.” – HSBC spokeswoman

In another case in April 2007, HSBS was fined with US$5.3M for the loss of an unencrypted floppy disk in the mail, containing the details of 1,917 pension scheme members, including addresses, dates of birth and national insurance numbers; while in February 2008 HSBC lost an unencrypted CD containing the details of 180,000 policyholders in the post.

“It is also worrying that increasing awareness around the importance of keeping personal information safe and the dangers of fraud did not prompt the firms to do more to protect their customers’ details.” – Margaret Cole from the Financial Services Authority (FSA)

Case Study: Wells Fargo

When a lawyer representing Gary Sinderbrand, a former Wells Fargo employee, subpoenaed the bank as part of a defamation lawsuit against a bank employee, Sinderbrand as well as himself expected to receive a selection of emails and documents related to the case. But what landed in Sinderbrand’s hands went far beyond what his lawyer had asked for: Wells Fargo had turned over — by accident, according to the bank’s lawyer — an unencrypted CD with confidential information of about 50,000 of the bank’s wealthiest clients.

The 1.4 gigabytes of files that Wells Fargo’s Angela Turiano lawyer sent included copious spreadsheets with customers’ names and Social Security numbers, paired with financial details like the size of their investment portfolios and the fees the bank charged them. Most are customers of Wells Fargo Advisors, the branch of the bank that caters to high-net-worth investors.

“This was the unfortunate result of an unintentional human error involving a spreadsheet,” – Shea Leordeanu, Spokeswoman for Wells Fargo Advisors

“I thought I was reviewing a complete set, when in fact, I only reviewed the first thousand documents.” – Angela Turiano

Like this article? Follow us on LinkedIn, Twitter, YouTube or Facebook to see the content we post.

Read more about Cyber Security

Oracle directors give blessing to shareholder lawsuit against Larry Ellison and Safra Catz

Three years after closing a $9.3 billion deal to acquire NetSuite, several Oracle board members have written an extraordinary letter to the Delaware Court, approving a shareholder lawsuit against company executives Larry Ellison and Safra Catz over the 2016 deal. Reuters broke this story.

According to Reuters’ Alison Frankel, three board members, including former U.S. Defense Secretary Leon Panetta, sent a letter on August 15th to Sam Glasscock III, vice chancellor for the Court of the Chancery in Georgetown, Delaware, approving the suit as members of a special board of directors entity known as the Special Litigation Committee.

The lawsuit is what is called in legal parlance a derivative suit. According to the site Justia, this type of suit is filed in cases like this. “Since shareholders are generally allowed to file a lawsuit in the event that a corporation has refused to file one on its own behalf, many derivative suits are brought against a particular officer or director of the corporation for breach of contract or breach of fiduciary duty,” the Justia site explained.

The letter went on to say there was an attempt to settle this suit, which was originally launched in 2017, through negotiation outside of court, but when that attempt failed, the directors wrote this letter to the court stating that the suit should be allowed to proceed.

As Frankel wrote in her article, the lawsuit, which was originally filed by the Firemen’s Retirement System of St. Louis, could be worth billions:

One of the lead lawyers for the Firemen’s fund, Joel Friedlander of Friedlander & Gorris, said at a hearing in June that shareholders believe the breach-of-duty claims against Oracle and NetSuite executives are worth billions of dollars. So in last week’s letter, Oracle’s board effectively unleashed plaintiffs’ lawyers to seek ten-figure damages against its own members.

It’s worth pointing out, as we reported at the time of the NetSuite acquisition, that Larry Ellison was involved in setting up NetSuite in the late 1990s and was a major shareholder at the time of the deal.

Oracle was struggling to find its cloud footing in 2016, and it was believed that by buying an established SaaS player like NetSuite, it could begin to build out its cloud business much faster than trying to develop something like it internally. A June Synergy Research SaaS marketshare report, while admitting the market was fragmented, still showed Oracle was far behind the pack in spite of that deal three years ago.

SaaS Q119 1

While there have been bigger deals in tech M&A history, including Salesforce’s acquisition of Tableau for $15.7 billion earlier this year, it’s still stands with some of the largest.

We reached out to Oracle regarding this story, but it declined to comment.


(function() { var scribd = document.createElement(“script”); scribd.type = “text/javascript”; scribd.async = true; scribd.src = “https://www.scribd.com/javascripts/embed_code/inject.js”; var s = document.getElementsByTagName(“script”)[0]; s.parentNode.insertBefore(scribd, s); })()


Ping Identity files for $100M IPO on Nasdaq to trade as ‘PING’

Some eight months after it was reported that Ping Identity’s owners Vista Equity had hired bankers to explore a public listing, today Ping Identity took the plunge: the Colorado-based online ID management company has filed an S-1 form indicating that it plans to raise up to $100 million in an IPO on the Nasdaq exchange under the ticker “Ping.”

While the initial S-1 filing doesn’t have an indication of price range, Ping is said to be looking at a valuation of between $2 billion and $3 billion in this listing.

The company has been around since 2001, founded by Andre Durand (who is still the CEO), and it was acquired by Vista in 2016 for about $600 million — at a time when a clutch of enterprise companies that looked like strong IPO candidates were going the private equity route and staying private instead.

But more recently, there has been a surge in demand for better IT security linked to identity and authentication management, so it seems that Vista Equity is selling up. The PE firm is taking advantage of the fact that the market’s currently very strong for tech IPOs, but there is so much M&A in enterprise right now (just yesterday VMware acquired not one but two companies, Carbon Black for $2.1 billion and Pivotal for $2.7 billion) that I can’t help but wonder if something might move here too.

The S-1 reveals a number of details on the company’s financials, indicating that it’s currently unprofitable but on a steady growth curve. Ping had revenues of $112.9 million in the first six months of 2019, versus $99.5 million in the same period a year before. Its loss has been shrinking in recent years, with a net loss of $3.1 million in the first six months of this year versus $5.8 million a year before (notably in 2017 overall it was profitable with a net income of $19 million. It seems that the change is due to acquisitions and investing for growth).

Its annual run rate, meanwhile, was $198 million for the first six months of the year, compared to $159.6 million in the same period a year ago.

The area of identity and access management has become a cornerstone of enterprise IT, with companies looking for efficient and secure ways to centralise how not just their employees, but their customers, their partners and various connected devices on their networks can be authenticated across their cloud and on-premise applications.

The demand for secure solutions covering all the different aspects of a company’s IT stack has grown rapidly over recent years, spurred not just by an increased move to centralised applications served through the cloud, but also by the drastic rise in breaches where malicious hackers have exploited vulnerabilities and loopholes in companies’ sign-on screens.

Ping has been one of the bigger companies building services in this area and tackling all of those use cases, competing with the likes of Okta, OneLogin, AuthO, Cisco and dozens more off-the-shelf and custom-built solutions.

The company offers its services on an SaaS basis, covering services like secure sign-on, multi-factor authentication, API access security, personalised and unified profile directories, data governance and AI-based security policies. It claims to be the pioneer of “Intelligent Identity,” using AI to help its system analyse user, device and network behavior to better identify potentially malicious activity.

More to come.

Breach at Hy-Vee Supermarket Chain Tied to Sale of 5M+ Stolen Credit, Debit Cards

On Tuesday of this week, one of the more popular underground stores peddling credit and debit card data stolen from hacked merchants announced a blockbuster new sale: More than 5.3 million new accounts belonging to cardholders from 35 U.S. states. Multiple sources now tell KrebsOnSecurity that the card data came from compromised gas pumps, coffee shops and restaurants operated by Hy-Vee, an Iowa-based company that operates a chain of more than 245 supermarkets throughout the Midwestern United States.

Hy-Vee, based in Des Moines, announced on Aug. 14 it was investigating a data breach involving payment processing systems that handle transactions at some Hy-Vee fuel pumps, drive-thru coffee shops and restaurants.

The restaurants affected include Hy-Vee Market Grilles, Market Grille Expresses and Wahlburgers locations that the company owns and operates. Hy-Vee said it was too early to tell when the breach initially began or for how long intruders were inside their payment systems.

But typically, such breaches occur when cybercriminals manage to remotely install malicious software on a retailer’s card-processing systems. This type of point-of-sale malware is capable of copying data stored on a credit or debit card’s magnetic stripe when those cards are swiped at compromised payment terminals. This data can then be used to create counterfeit copies of the cards.

Hy-Vee said it believes the breach does not affect payment card terminals used at its grocery store checkout lanes, pharmacies or convenience stores, as these systems rely on a security technology designed to defeat card-skimming malware.

“These locations have different point-of-sale systems than those located at our grocery stores, drugstores and inside our convenience stores, which utilize point-to-point encryption technology for processing payment card transactions,” Hy-Vee said. “This encryption technology protects card data by making it unreadable. Based on our preliminary investigation, we believe payment card transactions that were swiped or inserted on these systems, which are utilized at our front-end checkout lanes, pharmacies, customer service counters, wine & spirits locations, floral departments, clinics and all other food service areas, as well as transactions processed through Aisles Online, are not involved.”

According to two sources who asked not to be identified for this story — including one at a major U.S. financial institution — the card data stolen from Hy-Vee is now being sold under the code name “Solar Energy,” at the infamous Joker’s Stash carding bazaar.

An ad at the Joker’s Stash carding site for “Solar Energy,” a batch of more than 5 million credit and debit cards sources say was stolen from customers of supermarket chain Hy-Vee.

Hy-Vee said the company’s investigation is continuing.

“We are aware of reports from payment processors and the card networks of payment data being offered for sale and are working with the payment card networks so that they can identify the cards and work with issuing banks to initiate heightened monitoring on accounts,” Hy-Vee spokesperson Tina Pothoff said.

The card account records sold by Joker’s Stash, known as “dumps,” apparently stolen from Hy-Vee are being sold for prices ranging from $17 to $35 apiece. Buyers typically receive a text file that includes all of their dumps. Those individual dumps records — when encoded onto a new magnetic stripe on virtually anything the size of a credit card — can be used to purchase stolen merchandise in big box stores.

As noted in previous stories here, the organized cyberthieves involved in stealing card data from main street merchants have gradually moved down the food chain from big box retailers like Target and Home Depot to smaller but far more plentiful and probably less secure merchants (either by choice or because the larger stores became a harder target).

It’s really not worth spending time worrying about where your card number may have been breached, since it’s almost always impossible to say for sure and because it’s common for the same card to be breached at multiple establishments during the same time period.

Just remember that while consumers are not liable for fraudulent charges, it may still fall to you the consumer to spot and report any suspicious charges. So keep a close eye on your statements, and consider signing up for text message notifications of new charges if your card issuer offers this service. Most of these services also can be set to alert you if you’re about to miss an upcoming payment, so they can also be handy for avoiding late fees and other costly charges.

macOS Incident Response | Part 1: Collecting Device, File & System Data

In previous posts, we’ve looked at how malware persists on macOS, taken a practical tour of macOS malware hunting techniques and also discussed techniques for reversing macOS malware. One area that we haven’t discussed so far, and which we’ll offer an introduction to in this and subsequent posts, is macOS post-breach or post-infection incident response. We’ll get started today, in Part 1, by explaining how to quickly gather up vital data about file events, system configuration and the machine’s current environment. In later posts in this mini-series on macOS incident response, we’ll look at discovering user activity, retrieving things like browser history, email messages, notifications, and application usage, among other things. 

Throughout, we will learn about many of the hidden logs, text files and databases that are littered across both the user and system domains that can reveal suspicious or malicious activity, and we’ll see how to write our own scripts to collect and analyse that data along the way. Although this is a vast subject, we hope this short series will serve as an introduction and give you a taste to learn more, so let’s get started!

IR part 1 ftr image

Getting Started With Incident Response

When I’m dealing with a Mac that’s known to be compromised, the first step is to consider the client’s situation and the potential nature of the breach. For example, if the device may have been used in a crime or could become part of a criminal investigation, I would recommend the client to use a digital forensics lab that can image the device and recover artefacts from memory without polluting the evidence. This is quite a different process from what we will cover here, which is more akin to a SOC team investigation to determine what an intruder or a malware infection may have done that has not already been logged by detection software. Has there been lateral movement, has data been exfiltrated, has there been system manipulation? Are there other indicators of attack or compromise that we haven’t yet discovered? These are the questions that we want to set out to answer as quickly as possible in order to protect the business.

Let’s assume for the purposes of our scenario, then, that an employee has brought us a machine after discovering and removing a malware infection. The machine is still powered on, and we have the necessary credentials (and authority) to examine the machine fully. 

Say Hello to Sysdiagnose

With that out of the way, let’s set about collecting some initial information. Typically, investigators will want to list things like the system version, currently running processes, network configuration, Bluetooth set up, mounted volumes, install history, system log and much more besides. You could invest quite some time writing your own custom scripts to collect that and other information (we’ll do a bit of custom script writing later in this series), but if you have direct access to the machine you can save yourself a lot of work by leveraging the built-in sysdiagnose tool provided by Apple. 

The sysdiagnose tool was not designed for security or incident response purposes. Apple wrote it for macOS beta testers as a means of collecting just about everything they could ever want to know about a Mac when investigating OS bug reports. But for that reason, it’s ideal for our purposes, too. Here’s an image showing just some of the data that it collects.

image of sysdiagnose output (some of it)

What’s even nicer, from the point of convenience, is that if you have physical access to the machine you can kickoff the report simply by pressing this keychord:


If you get it right, you’ll see the display briefly flash indicating that the process has begun. 

If using the keychord is a problem for any reason, head over to the Terminal app. There are a bunch of command line options you can specify (see man sysdiagnose page), but for our purposes we will just run it in vanilla mode. Enter the following, type the admin password, and confirm that you want to proceed when prompted:

$ sudo sysdiagnose

In this case you will not see the display flash. 

Regardless of which way you invoke the tool, it’ll take a few minutes to complete, so you might want to take the opportunity to make a coffee, walk the dog, feed the cat, or while away the time as you see fit until sysdiagnose has done its thing. 

Exploring Files Collected by Sysdiagnose

When sysdiagnose has finished, it’ll pop a Finder window showing you the compressed result. Copy it off to your local machine, then double-click it to unpack it and have a quick scroll through what’s been collected. Yes, there’s a lot of juicy stuff in there: everything from a full ps to netstat, kextstat, system_profiler, top, Wifi scans and much, much more.  

image of sysdiagnose unzipped

When working with large amounts of text files I like to use BBEdit, which offers many useful functions for quickly searching and manipulating multiple files. The features I’ll use are all available in the free version, so if you don’t already have a copy of BBEdit just go ahead and download the free demo. Of course, if you have your own way of working with large sets of files, that’s fine, too. 

If you have BBEdit in the Dock, grab the sysdiagnose parent folder in the Finder and drag and drop it on top of the BBEdit Dock icon. When the project view opens up, scroll down to the logs folder in BBEdit’s Sidebar, click the disclosure triangle and scroll down again. You should see useful things like Install.log and InstallHistory.plist among many other goodies. 

Still in the logs subfolder, find the folder SystemExp, descend into that and open up the folder named “Dock” (followed by a date and timestamp). In here, you’ll find useful stuff such as CachedWindows.txt, which might tell you a little about the user’s recent activity (although much more to come on that in the following post!). 

image of CachedWindows

Also, take a look at dockextras.txt file, which may include info on things like the last time the user connected to Facetime, Messages and a bunch of other apps. 

image of Facetime Connection

Interlude – A Note About Timestamps

Before we move on, a note about the timestamps you see here, as you’ll encounter these elsewhere in macOS logs. Timestamps like this


may look like Unix epoch timestamps (that is, seconds since 1/1/1970), but if you try to convert them using Unix epoch time you’ll get nonsense dates. These are actually Cocoa timestamps, which are similar but the seconds are counted since 1/1/2001. To convert them, add the difference between Unix and Cocoa start dates in seconds (that’s a fixed integer of 978307200) and use the date command line utility with the -r switch. We remove the fraction of a second and just deal with the whole integer, like so:

$ date -r $((587381137 + 978307200))

image of cocoa timestamp

That returns the more human-friendly date of Tue 13 Aug 2019 16:25:37 +07 from the Cocoa timestamp.

Finding Traces of Malicious Activity

Just below the logs folder you should see a file lsappinfo.txt. Click on it to load it into the main editor window. This file contains a lot of useful data about currently running applications, but even more useful for incident response – when we’re likely faced with a situation where malware has been and gone – is to look in the two files below, the admin (501) and root (0) dumps of lsregister. These are dumps of the databases held by Launch Services and contain detailed information about every application that has been available to the user.

Let’s walk through a practical example of how we might use this information to learn more about an infection.

If you scroll through lsregister-0.txt, you’ll notice each record has a path field and many have a CFBundleIdentifier field. To make a cursory examination of this file, I’ll use BBEdit’s ‘Process Lines Containing’ function (from the Text menu) and copy all lines containing CFBundleIdentifier to a new document. 

In the resulting text window, I’ll use the same function only this time I’ll delete all lines containing “com.apple” to narrow down my search (as we’ve pointed out before, some malware likes to disguise itself by using the “com.apple” label, so bear that in mind).

image of process lines

On my suspect device, this gives the following results. The highlighted ones will stand out to anyone familiar with macOS malware. There’s a bunch of commodity adware/PUP programs, but the ones in red are particularly interesting.

image of Suspicious BIDs in LSRegister

Let’s see what more we can find out about them. We’ll start with the bottom one, since that kind of bundle identifier is a non-standard pattern rarely used by legitimate software. Using BBEdit’s Multi-File Search function (Shift-Command-F), we can rapidly search through all the files collected by sysdiagnose for this identifier and see what else is known about it. 

image of multi file search

Add the identifier to the “Find” field and choose “Frontmost project” from the “Search in:” panel below. Then click ‘Find All’.

image of Finding IoCs

Our search results have revealed the Path, full App Name and team ID (aka “Developer Signature”). But further investigation on the machine shows no evidence the application still exists. After trying searches on VirusTotal and other public search engines, the teamID led us to a Russian-language stackoverflow post.

image of Russian stackoverflow adware

It turns out that the developer signature was used to sign an “app” that was in fact a Bash script bundled in an Application wrapper. It looks very much like a variant of OSX.Shlayer. There’s a high probability that the item found on our machine was a variant of the same malware, given that they were both signed by the same developer.

Returning to our list of labels, note that the second item, com.lights.Oblivion, is a bundle identifier associated with OSX.CrescentCore.

And what about the other highlighted item, com.ableton.live? Ableton Live is a legitimate commercial program, but there’s also cracked versions on the internet that are used for cryptojacking

Again, using the Multi-File search, we can find more info in the sysdiagnose folder. This time a result in the install.log reveals that the app was delivered in an unsigned .pkg. Since there is no chance that a company like Ableton would be distributing their software without proper code signing, there’s a strong likelihood that this package is malware. 

image of ableton live cracked

It seems our user’s machine has seen quite a lot of action!

While the above example isn’t particularly methodical, it does hopefully give you an idea of what you can do with such a vast amount of data and a few multi-file searches. 

One Log To Rule Them All

Among the many other files worth exploring in the sysdiagnose folder, there is one other that deserves special mention. Scroll down (either in BBEdit or Finder) to a file called system_logs.logarchive.

image of logarchive

As the name suggests, this is a collection of macOS system logs, the sort that are typically viewed in the Console.app. The file is actually a directory, but its contents are unreadable in BBEdit; however, double-clicking it in BBEdit will open it in the Console.app. You can also read this format with the log command in the Terminal. The latter is a far more powerful and effective tool for investigative work, but it does take a little practice to master. As there are many good guides on the log command, such as here and here, as well as the man page itself, we won’t go into details here. However, there are a couple of oddities about the “unified logging” system that I haven’t seen covered elsewhere and which are worth being aware of. 

First, note that the system_logs.logarchive file collected by sysdiagnose only contains a subset of the logs available. You can see the range of information collected by using the stats command. For example, 

$ log stats --archive --overview

image of log stats sysdiagnose

In this case, we see logs collected from August 15th to 20th. Now let’s run the same command on the machine without specifying the name of the logarchive file in the sysdisagnose folder.

$ log stats --overview

With no logarchive file specified, the command returns the stats for the main system log datastore held on the device.

image of log stats live

That’s quite a lot more (and also quite a lot larger!) and covers around 30 days worth of logs, from July 22nd to August 21st. To collect all the log info, run a separate collection command.  Be sure to specify a destination that is safe to write to (such as a connected device or quarantined folder) as by default the collect verb will save to the current working directory.

$ sudo log collect --output

The other oddity of this tool is that if you run the stats command on your newly collected log file, you may find it contains logs reaching even further back in time than the previous output of --overview indicated. In this case, the collect command appears to have reached back an additional 4 days, to 18th July.

image of logs stats archive

The cause of these oddities is unknown (at least to me) – whether it’s a bug or intended behavior – but the vagaries of the log command are worth bearing in mind. 

Exploring fs_usage for File Activity

One other file we’ll mention in the sysdiagnose folder before moving on is fs_usage.txt. This gives you a capture of file activity when you ran the sysdiagnose utility. It is useful to see what was occurring at the time of collection. You can quickly parse fs_usage.txt to get a list of every process that was involved in file activity. Try to cd into the sysdiagnose parent directory, then use something like the following to uniquely list processes that were interacting with the file system:

$ awk '{print $NF}' fs_usage.txt | cut -d. -f1 | sort -u


We can do something similar to quickly get a list of all file paths that were accessed. Note we’re grepping out files accessed by sysdiagnose itself to ignore our own activity:

$ awk '{print $0}' fs_usage.txt | grep '/' | sort -u | grep -v -i sysdiagnose

However, as fs_usage only records file activity at the time we ran the utility, we need something better to provide historical records of file events. 

FSEvents – Old, Not Obsolete

Fortunately, such records of file system events are created in a hidden folder at the root of each volume or disk image. 


You can easily toggle visibility of this and other useful hidden folders in the Finder by using the keychord:


image of fsevents Finder

As we see in the image above, this folder is protected, so we will need to drop down to root on the command line to inspect it.

image of fsevents terminal

The .fseventsd folder contains data files compressed with gzip. Although we could manually unzip each file, hexdump it or extract the printable characters with strings, that all requires a lot of labor and the results are likely to lose context. A better solution is to use the free tool FSEventsParser. This has the ability to create both SQL database and spreadsheet output, giving us access to much more powerful queries and analysis. 

Running the tool in its most basic form requires specifying the source and destination folders (more recent versions also require the -t switch and either folder or image for a value). Depending on the number of records, this may take some time.

$ python FSEParser_V3.3.py -s -t folder /.fseventsd -o /Users/sentinel/Desktop/FSEvents_Out

image of FSEParser start

The output, however, is well worth it. With FSEvents, we can conduct queries such as which files were sent to the Trash, what devices were mounted, which files were accessed or what websites were visited on a particular date. 

image of file system events parser sql

Like the unified logs, .fseventsd will only reach back a limited timespan as the records are continually churned to save space. How far back depends on a number of factors, including how active the system is, but if your suspicious events occurred close enough to the collection time, you may well have some extremely rich data that you can mine for evidence of malicious activity. 

Be aware that activities like updating the OS will wipe out existing logs in the .fseventsd folder (you can use the install.log in the sysdiagnose folder to determine when the most recent update occurred), and it’s also not unheard of for some events to fail to be recorded at all, such as during especially heavy I/O activity.

Another issue to bear in mind is that users can deliberately prevent the system from recording FSEvent activity by creating a touch file inside the .fseventsd folder.

$ sudo touch /.fseventsd/no_log

What all that means is that you can’t assume something didn’t happen just because you didn’t find a record of it in .fseventsd. However, what you do find can often prove extrememly illuminating.


In this introduction to incident response on macOS, we’ve taken a look at three built-in tools – sysdiagnose, unified logging and FSEvents – that can help you quickly collect device, file and environmental data about a Mac. Due to the breadth of the subject, there’s a lot we didn’t cover here, but hopefully we’ve given you enough of a taste to explore further. In the next post in this series, we’ll take a look at some of the hidden databases that reveal user activity. See you there!

Like this article? Follow us on LinkedIn, Twitter, YouTube or Facebook to see the content we post.

Read more about Cyber Security

Splunk acquires cloud monitoring service SignalFx for $1.05B

Splunk, the publicly traded data processing and analytics company, today announced that it has acquired SignalFx for a total price of about $1.05 billion. Approximately 60% of this will be in cash and 40% in Splunk common stock. The companies expect the acquisition to close in the second half of 2020.

SignalFx, which emerged from stealth in 2015, provides real-time cloud monitoring solutions, predictive analytics and more. Upon close, Splunk argues, this acquisition will allow it to become a leader “in observability and APM for organizations at every stage of their cloud journey, from cloud-native apps to homegrown on-premises applications.”

Indeed, the acquisition will likely make Splunk a far stronger player in the cloud space as it expands its support for cloud-native applications and the modern infrastructures and architectures those rely on.

2019 08 21 1332

Ahead of the acquisition, SignalFx had raised a total of $178.5 million, according to Crunchbase, including a recent Series E round. Investors include General Catalyst, Tiger Global Management, Andreessen Horowitz and CRV. Its customers include the likes of AthenaHealth, Change.org, Kayak, NBCUniversal and Yelp.

“Data fuels the modern business, and the acquisition of SignalFx squarely puts Splunk in position as a leader in monitoring and observability at massive scale,” said Doug Merritt, president and CEO, Splunk, in today’s announcement. “SignalFx will support our continued commitment to giving customers one platform that can monitor the entire enterprise application lifecycle. We are also incredibly impressed by the SignalFx team and leadership, whose expertise and professionalism are a strong addition to the Splunk family.”