Thomas Hatch is the Co-founder and CTO of SaltStack, an event-driven automation platform that delivers threat-aware security compliance, fast and scalable control of any cloud, and configuration management for heterogeneous application environments and data center infrastructure. In this episode, Thomas describes how SaltStack has built tools and point solutions to monetize a very flexible open source platform for orchestration and configuration.

Transcript

Michael Schwartz: Welcome to Open Source Underdogs, the podcast where we talk to the people who make open source companies get up and go.

Today my guest is Thomas Hatch, CTO and Co-Founder of SaltStack.

The SaltStack automation platform delivers threat-aware security compliance, fast and scalable control of any cloud and configuration management for heterogeneous application environments and data center infrastructure.

This is one of their first remote interviews, and I forgot to get hellos and goodbyes, so we’re just going to dive right in with the first question. Here we go.

Origin

Tom, can you share with our listeners the journey that led you to start SaltStack, and why using the open source development methodology was important to you?

Thomas Hatch: To be honest, when I started writing Salt, my aspirations weren’t quite as high as they are now. I started writing Salt in my basement to pretty much solve a problem.

I’ve written a lot of other pieces of software that were proprietary owned, and owned by companies that I work for, but always as an internal tool, and I was kind of sick of having to rewrite it.

And so I thought, well, I’ll just write it this time and make it open source and clear it with this company that I’m going to make it open source. And after about ten months or so, people were using it. So I started looking at it saying, “I should try and make a business out of this.”

I mean, like you were saying about business models, it started off that journey, that journey of trying to figure out what are the optimal ways to be profitable, with an open source software platform.

Initial Struggles

Michael Schwartz: How do you survive those first two years?

Thomas Hatch: Well, back in 2011, I mean, that’s when I started writing the project to my basement, I still had a job, and at that point, it was me, spending my nights writing Salt, and my employer didn’t mind because they were using Salt internally, but it also made it a great test bed.

That company went out of business not because of their infrastructure, but went out of business at the end of 2011.Yeah, I mean, I still remember, I was in my basement, and I get the “hey, we are out of the business” phone call.

And I went upstairs, and I tell my wife, “hey, do you think I should make a business with this Salt thing, people are starting to use it?”

And she looked at me and said, “don’t you dare do anything but start a business! I don’t want to listen to you moan for the rest of our lives together about how you missed your big chance.”

So, then, we had a fair amount of money in savings and we pretty much burnt through that, through 2012, and we were honestly running pretty low on cash when we got our first seed money in the door, but yeah, I mean, that was basically it, that and a few consulting jobs.

Initial Community

Michael Schwartz: That’s amazing! Congratulations. What was the community like in, let’s say 2013, when you raised that seed funding, like how large was the community at that point?

Thomas Hatch: We had received contributions from maybe 400 people, community had grown pretty quickly through 2012. We had quite a few pretty large users. So some of the flagship users that we still have we’ve gained back then, like LinkedIn for instance.

Open Core

Michael Schwartz: I was watching the 2016 keynote that’s on YouTube, and in that keynote you mentioned you’re not a fan of open core. Have your thoughts changed at all on this topic?

Thomas Hatch: Well, I’d like to say that my thoughts have evolved a lot over the years. In the early days, my thinking was let’s just make a ton of open source software and figure it out.

But at the same time, I did want to avoid an open core issue because when you go with open core, it stifles the innovation that can happen inside of your community.

I really wanted to avoid stifling innovation, but at the same time, we ran into a problem which is pretty typical I think of a lot of open source software companies is that, we’re out there and we’re selling services and support, but once we solve the problem for our customer, they don’t really need us anymore.

I mean, we go back to the early, early years, and that was a lot of what we’re running into. So we decided to start working on an Enterprise product. I specifically architected the Enterprise product so that it was a separate piece of software.

The actual definition of open core is hotly-debated, I definitely argued that we are not open core, but every now and then someone comes out of the woodwork and accuses us of being such.

The way that I look at it is that open core means that you have one piece of software that is augmented with proprietary software directly in that codebase. What I wanted to do is create a separate piece of software that could sit above Salt and value.

One of the things that I find challenging with that was that, right off the bat, developing proprietary software is very different from developing open source software.

And I think that that’s one of the major problems that a lot of open source companies run into.

They say, “okay, we’ve been really successful, making a piece of open source software that people love, and we can make a proprietary one, but the way that software was developed is so dramatically different from open software.”

So we struggled a lot admittedly early on in making that piece of proprietary software. And then we hit another wall that, again, in retrospect, I think it’s a common wall for people to hit from business and monetization perspective, especially with add-on software.

It was that the proprietary software that we had just ended up, mostly at the beginning it just expressed what we already did with open source software.

So we found it was really hard to create differentiation. It was especially difficult to create that differentiation because Salt does a lot of things. I mean, it’s an incredibly powerful platform. So, then, we struggled with that.

A lot of where my head is at today, it’s actually along the lines of what I like to refer to as “peripheral engagement.”

So when we come back and we look at our core users of Salt, their needs are typically your DevOps professionals, SRE professionals, people who are very technical and low-level in being able to use the automation platform.

But then the power that we found from a sales perspective and a monetization perspective is to make sure that our Enterprise products are geared towards the needs, not of that core team as much, but more towards the groups that are slightly peripheral to that core team.

And it helps that core team, and then they are able then to expose the power of what they’ve built with Salt to these other teams.

So, right now, a lot of our success has come through our Enterprise software, exposing the functionality of Salt over to, say a NOC environment, or your IT maintenance teams, which makes just that day-to-day maintenance significantly easier.

And then we’re also in the process now, because we’ve seen some success there, we’re in the process now of creating a new security, a SecOps piece. And we’re really excited that’s going to come out, our Enterprise software first quarter of next year. And we’ve already got a lot of companies lining up to purchase this in their infrastructure.

And, again, it turned into the situation where we’ve got a powerful, open source foundation that is widely used.

That open source foundation needs to cover certain use cases that are broad in nature and are platform in nature.

And open source communities are really good at developing those, but we find over and over again that there are certain areas of software development, where open source communities, to be frank, don’t seem to be quite – I mean, I’m going to get burned for saying this, for saying anything disparaging about the effectiveness of open source communities and software development – but there are certain areas where open source communities seem to have difficulty building software that gets wide, mainstream traction despite the quality of the software that comes out.

And those are areas where open source through the years just hasn’t been able to penetrate.

One of my favorite examples there is the “year-of-the-open-source-desktop.” I mean, I’m running KTE, sure, of course, but Linux desktop hasn’t penetrated the masses whereas the Kernel has.

So you end up being able to kind of step back and say there are things that an open source community express as well, if we can build the targeted solutions, that solve business problems that aren’t going to be solved as easily in a pure open source environment. Then that is a way to market, and the way to augment your position.

Open V. Commercial

Michael Schwartz: How do you decide which software to make commercial?

Thomas Hatch: So early on, it was kind of, well, where does it belong? You’ve got an Enterprise platform, if it belongs in the Enterprise platform, it’s commercial. Otherwise, it goes in the Open.

At this point, it’s shifted a little bit. I mean, that’s still part of that philosophy, but it’s shifted in, proprietary is that which is tied to the solution that is being expressed in the proprietary software.

Actually, that core answer still really holds true. If that functionality belongs on the Salt agent, it’s got to be open source. If that functionality belongs on the master, it’s generally going to be open source. If hat functionality belongs in the Enterprise piece, it’s going to be proprietary.

Resource Allocation

Michael Schwartz: So how do you prioritize, with regard to like resource allocation, everyone’s resource-constrained, you know, you can’t develop everything you’d like to develop, but how do you decide how much to invest in a Commercial versus the open source?

Thomas Hatch: A lot of that has to do with really just hitting our goals. We have that blogmenting who’s working on what quite a bit.

We’ve got a few people who stay Open all the time, and we’ve got a number of people that stay Enterprise all the time, and we’ve got a few people that kind of bleed between teams.

It’s basically, Hey, we’ve got to get X, Y, Z then to meet our protocols of the Open, and we’re a little ahead on the Enterprise side. Then, we’ll shift him over and vice-versa.

It really depends on, frankly, sprint-by-sprint basis on where those needs are. It’s just this constant balancing.

Commercial & Open Source?

Michael Schwartz: Do Commercial features ever move into open source?

Thomas Hatch: Oh, definitely. It’s one of the interesting things too.

As I come back and look at it, occasionally there’s some component libraries that we make for the Enterprise piece that we move into the Open.

One of the things about having a separate codebase like we do, for the Enterprise platform, is that things that belong in that Enterprise platform, again, by their nature are more likely to stay proprietary, but in the places where there is some lead over.

So occasionally we’ll end up writing something that – actually, let me just kind of walk you through a life cycle of this.

So Salt’s pluggable, crazy pluggable, and so we’ll write plugins that are going to be running on the minion that need to be deployed by the Enterprise products. So they can, you know, do its work.

And, oftentimes, those plug-ins are very specific. So we don’t hide the code, but we don’t add them directly into the Open source product because their functionality is directly tied to the use case that’s being expressed in our Enterprise product.

But occasionally, those plug-ins are of such a value that they are generic, in which case, we end up putting them in the Open and the proprietary software at the same time, so that the proprietary software can then send those plug-ins out, really as a back port, to older versions, until our entire customer base migrated to a version where that’s just baked in.

It’s not always this philosophy of, “I’m going to make a piece of proprietary software, and it’s got a timeline until it becomes Open.”

That is something that I think that we’ll probably be adopting as some of our proprietary software matures a little bit, and its install base matures. And we’re able to continually make the right business cases and the right alignment.

But yeah, that’s one of those things we feel that we need to walk into, as opposed to trying to pursue the future too much, too early on.

Is New Model Working?

Michael Schwartz: You mentioned before what I call the open source dilemma, which is, customers only buy support when they need help.

You’ve moved towards a licensed strategy, where certain features are licensed, and certain features are open – how do you feel about commercial adoption so far since moving to that strategy? Has it met your expectations?

Thomas Hatch: It has certainly improved. It has certainly met some of my expectations.

I mean, entering into that strategy, there were still things that I was skeptical about.

A lot of my goal for a long time inside of the company, we need to express a certain amount of functionality from our proprietary software, to be able to just get to the point, where we can start expressing these targeted solutions that I was talking about.

As we built out the targeted solutions more, the sales improved significantly. It’s been really interesting to watch because those solutions that are built out in an Enterprise software, that augment the automation platform that we have in the Open, does start to drive sales, and does start to drive a business model that is much more sustainable as we grow.

As opposed to, I mean, we still run into a number of open source companies that struggle making that sustainability curve, they still need to have a significant amount of their business consumed by professional services.

Customer Segments

Michael Schwartz: Do you segment your customers at all?

Thomas Hatch: Oh, definitely. Usually we segment them along solution verticals.

Instead of looking at them from industry perspective, and there’s certain industry overlap relative to solutions, we look at the very much so as to what are the problems that they’re trying to solve.

And we’ve got customers that are solving IoT problems, that, you know, if our automation said, “customer,” you would probably think, “well, I wouldn’t think that they’d be the kind of customer that would be solving the IoT problem.”

And vice versa. I mean, when we come back, when we look at one of our customers is a major cruise line, and they use Salt to manage devices on cruise ships. So we categorize them as an IoT customer.

Other customers will do things like as this typical network automation or application deployment, all those generally automation platform use cases that you run into.

So, yeah, that’s mostly how we separate them.

Customer Interaction

Michael Schwartz: How do you interact with customers?

Thomas Hatch: We’ve got numerous tiers that we can sell, everything from allowing somebody to get us on the phone within a few minutes, all the way down to, they’re going to open up a ticket, we promise to get back to them in the next, say, 72 hours. All depending on which tier of support they want to buy with their licenses.

You look at what you offer, then you look at what people use, and it doesn’t always line up quite as nicely as you hope when you create set offerings.

Customer Size

Michael Schwartz: Is your business skewed more towards large or small transactions?

Thomas Hatch: Our software, by its nature, it tends to be more applicable to larger, more complex problems, from a platform perspective.

From a solution perspective, it’s a lot easier for us to sell into those smaller and mid-market size companies.

And then, again, I completely agree there, I mean, that is your sustainable revenue.

But, yeah, that has been something that’s been interesting, because earlier on, the vast majority of our revenue came from really large installations.

Or they would come back and say “we just have to have a relationship with you, guys.” We are using Salt to manage hundreds of thousands of systems and, yeah, we just have to have a relationship. Even if they would go months, and not talk to us or reach out to us, we’re reaching out to just say, “hey, is everything okay?”

That still turned out to be a significant chant, again, early on. But, yeah, as we’ve evolved and as we’ve been able to embrace these solution expressions in a cleaner way from our proprietary software, that has attracted more of the smaller and mid-market customers.

How Has Offering Evolved?

Michael Schwartz: That sounds like your offering has changed. What’s working well, or what hasn’t worked well, or what do you think is missing?

Thomas Hatch: Oh, I mean, what I think is missing is, you know, just more software. Always trying to get more products, build them to be more robust, but yeah, I mean, like you expressed, it’s a journey, so a lot of my assumptions about things have changed over the years.

A lot of my assumptions about the role and mechanism of how development happen, that’s changed.

Again, if we look back at the customer base, how that’s changed, we still have a lot of strength and really big customers.

We’re used by quite a few major banks, insurance companies, big tech companies, but yeah, the things have changed for the better, I think, more recently over the last year, so that improved traction in the mid-market.

And our change, I would say is very much so towards our ability to begin to shift from a pure platform support sale to say, “you’ve got a really complicated platform. Our Enterprise piece can help you manage it.” To being able to say “you’ve got a target of pain point, and the entirety of the software that we deliver can help you solve that pain point.”

When we talk in terms of pain points directly, that is more directly applicable to those smaller and mid-market companies, whereas the big platform play makes more sense to the larger ones.

Reducing Revenue Lumpiness

Michael Schwartz: Can you imagine serving the smaller pain points is more scalable, in terms of building the revenue?

Thomas Hatch: Definitely. I mean, as you are probably aware, you get the big guys and it’s up and flow.

You have two months that are fantastic, and then you hold your breath for another month and a half until the next giant deal comes in, but once we’ve got more of those pain point solutions in, they’re significantly easier to support, they are significantly easier to sell, you get a more consistent sales cadence, our revenue still looks a little lumpy, but it’s not lumpy in a bad way, like it was two or three years ago.

It’s lumpy in a way of saying, hey, there’s frosting on the top of this cake.

Channels

Michael Schwartz: Have you worked much with channel partners?

Thomas Hatch: Oh, I think channels are very, very useful. They can be tricky.

In the early days, we started with professional services channels. And we did, we struggled a lot to get those going. But more recently, the picture and painting is that our model has evolved a lot for the last few years and it continues to evolve.

And as it’s continued to evolve to be more solution-oriented and less supporting a complex platform-oriented as far as that mixture, then channel partners have started to be much more lucrative for us.

It’s a lot easier for a channel partner to execute a sale that has a more simplified delivery, as opposed to going in and dealing with that whole complex platform sale.

Resources

Michael Schwartz: Where do you feel like you’re the most constrained, resource-constrained?

Thomas Hatch: I don’t know, everywhere? That is admittedly a very challenging question. I definitely feel resource-constrained in engineering, both on the Open and Enterprise side.

Probably the place where I feel like I’ve heard the most resource-wise over the years honestly has been marketing.

I think that’s one of the things that we struggled more with, and I think we’re struggling more, going back again to that platform versus solution thing.

As we’ve rolled out our solutions, it’s been a lot easier to market, as opposed to platform play. It’s hard to market – “hey, we can do anything, tell us what your problem is?”

Underdog?

Michael Schwartz: Do you think SaltStack’s an Underdog?

Thomas Hatch: We get pigeonholed by the market in certain places, and so we have a very powerful configuration management platform. So the market likes to put us with Puppet, and Chef, and Ansible.

When our customers come back to us, they say, “yes, we use your config management, but not in the same way that the other guys use it.”

So it’s hard to categorize this as an underdog because I think we’re an underdog in the sense of the market thinks we belong there.

But that’s not where we want to be, as much as – I mean, don’t get me wrong, config is very important and I would still argue that we’ve got the best config platform – but so much of our focus is around automating tasks that are outside of that classical config management use case.

I don’t see us as much of like an application, deployment and management company, which is what you see a lot more from the config management players.

I see it’s more of an infrastructure maintenance and building company, and then, event-driven company, and a security out of remediation company.

And in those areas, I don’t feel as much like we are an underdog. When we go in, from command and control, and an execution position, we don’t really have a lot of competition there.

We’ll get compared to some of the other open source players, and we just say, “yeah, but they don’t do X, Y, and Z.”

So the nature of our offering is so different, and the approach that we’re taking to managing infrastructure has so many differences in it that I generally get pretty frustrated when somebody comes and they say, “well, you know, I use Ansible instead of Salt because of X,Y,Z.”

And I’m like, “I don’t care. Most of my customers are using also Puppet, Chef, or Ansible.” For many of them, they’re using Salt to orchestrate and manage those tools.

It’s frustrating along those lines, because we’re kind of still pigeonholed in that area when we really shouldn’t be.

And one of the things that I am excited about is that some of these products that we’re working right now, I feel like they’re really going to help us break out of that better and expose our functionality better, and the reasons why what we do isn’t just, you know, a copy of Puppet.

Salt In 5 Years?

Michael Schwartz: Where do you see SaltStack in 5 years?

Thomas Hatch: Especially with the newer security offerings that we’re producing an offering now, as well as the expansion of a number of other solutions, our sale’s on a positive trajectory, moving us towards going public.

And, I mean, again, we’re still many years away from that. I’ll avoid making some public statement. Ten years, I sure hope we make it there in ten years.

That’s definitely the track that we’re on from a macro business perspective. Yeah, figuring out what the revenue models look like is something that I feel is increasingly part of our past. And our future is more of executing along those models and growing out the pieces that we’ve learned to work really well.

Going Public?

Michael Schwartz: Do you think that going public would assure the continuance of the open source mission, in a way that another exit strategy would not?

Thomas Hatch: So I should probably temper my earlier statement, I believe very strongly that you should manage a company towards building the company in such a way that it would be viable as a publicly traded company, and that you’re gearing your company towards growth and stability.

I certainly agree that taking a company public is not necessarily always the best option any more. The markets are much more complicated than they used to be.

I also want to, you know, caveat that by saying I believe that managing a company in such a way to go public, and managing it responsibly, is probably one of the best ways to, either be acquired by another company, or private equity, or whatever.

The right tact that I would take would be that I am managing towards the goal of making SaltStack into a company that can be viable in the public market. But I’m not going to close myself off to potential opportunities.

Final Advice

Michael Schwartz: What are the biggest challenges facing the founders of open source software companies?

Thomas Hatch: The open source business model is viable. I certainly don’t want to say anything to disparage that, but being able to build internal culture, and a management mindset, that deals with open source, and profits from open source, and functions in a stable and responsible way with respect to open source, is in and of itself one of the big challenges that you’re going to face.

It’s one thing to make a piece of open source software and get people to use it. It’s another thing to realize that you now have to build another thing on top of that open source.

Two, your sale cycle and your marketing cycle, these are going to be complicated. You’re not selling candy, so you’ve got to work hard to make sure that your sales teams and your marketing teams, and a lot of these people that are more token and dollar-driven are able to see the benefit that they are receiving from the open source.

Otherwise, you’re bringing these people from other areas and they can eat you alive, and end up sending you down bad paths. And that’s another one of those things that I see all the time.

The other thing to warn against is the extreme on the other end. It’s really easy from an open source developer perspective, and I was very much this way, to get caught up in the virtue of Open Source and the idealism of Open Source.

I certainly believe in the idealism of open source, but my views have changed from what I feel was used to be more of a blind trust in open source into more of views that are along the lines of saying that I understand how open source is important and viable inside of a complex economy, instead of just looking at it and saying, “If I build it, they will come.”

Credits

Michael Schwartz: Well, that’s it for episode 7. Thanks to Tom for answering all my tough questions.

Transcription and episode audio can be found on opensourceunderdogs.com

Special thanks to the Linux Journal for co-sponsoring this podcast to the All Things Open conference for helping us publicize the launch.

Music from Broke For Free, by Chris Zabriskie and Lee Rosevere.

Production assistance from Natalie Lowe. Operational Support from William Lowe.

Thanks to the staff of SaltStack for scheduling support.

Next week, we are joined by Matt Mullenweg, Founder and CEO of Automatic, the company behind WordPress which needs no introduction.

Until then, thanks for tuning in.