Episode 29: Chef – Rapid Configuration Automation with Corey Scobie
Corey Scobie is the Senior Vice President of Product and Engineering at Chef, maintainers of the popular open source configuration management tool. In this episode, Corey discusses the “4 freedoms of open source software,” and various challenges associated with building a business around open source software.
Interested in learning more about Chef? We encourage you to check out The Changelog‘s podcast episode featuring Chef Co-founder Adam Jacob!
Michael Schwartz: Welcome back, Underdogs. This week, we have Corey Scobie, VP of Product and Engineering at Chef.
If you want to hear even more about Chef, you should check out Episode 353 of The Changelog, where Adam Jacobs, one of the co-founders, gives his perspective on “The war for the soul of open source” – it’s a really interesting contrast with this podcast.
Like WSO2, Cloudera, and you could say Red Hat, Chef has moved to an all open source code base. They are licensing what Corey calls “The post-production binary.”
I don’t want to spoil much more, so let’s just cut to the tape.
Michael Schwartz: Corey, thank you so much for joining us today.
Corey Scobie: Thanks, Mike, it’s great to be here.
Michael Schwartz: Can you just tell us a little bit about when you joined Chef and how your role has evolved since you’ve been there?
Corey Scobie: Sure, absolutely. I joined Chef in March of 2018, so a little shy of 18 months ago.
At the time, Chef had been through some really interesting expansion of the product portfolio. And so, I was brought in to lead the product and engineering team and vision overall around bringing together the parts of the product portfolio that we put on the table at that point, which included the traditional original Chef project, InSpec, which is our open source project around security and compliance, and then Habitat, which is our open source project around application, orchestration and deployment.
Michael Schwartz: Recently, Chef announced that it was open sourcing 100% of its offer under the Apache 2 license, and at the same time, announced the Chef Enterprise Automation Stack. I’m wondering what’s in the Enterprise Automation Stack, and how do you get people to pay for it if it’s provided under Apache2?
Corey Scobie: Sure, that’s a great question. In order to answer that, I need to sort of take you back a little bit in time.
Chef was founded in 2008 and started an open source project shortly after under an Apache 2 license, the original Chef project, and went through years of development.
Overtime, as the product portfolio evolved, Chef had become essentially what we described as a loose open core company, which meant that there was a core product which provided value to customers in the Chef Engine, the Chef infrastructure automation suite of tools.
And then, we had created some proprietary software on top of that to help fuel the monetization value of using the underlying open source project. For us, that was a product called Chef Automate, which was our enterprise access control plane.
Ultimately, at the end of the day, we went through sort of an existential discovery of our own last year to figure out how we would take the parts of the product portfolio.
We continued to take the parts of the product portfolio forward, honoring the open source nature of the way that Chef was started, and that most of the other projects, InSpec and Habitat were started, and yet, being able to effectively deliver value and monetize commercial customers in that realm.
We announced in the beginning of April that we were going to do exactly what you just said, Mike. We were going to open source the entirety of our kit, so for us, what that meant was, we were going to take any of the additional on top of the open source project software development that we were doing.
For us, that was Chef Automate, and about 50% of the company was of the engineering team anyway, was primarily focused on delivering Chef Automate, as a proprietary piece of commercial software on top of Chef and InSpec.
We decided to take all of the remainder of the product work that we were doing and open source it, to attach it to the Apache2 license, because we believe in the alignment and the permissiveness of Apache2, as a license vehicle for open source software development.
Today, if you look at today’s Chef, a 100% of the work that we do in product engineering, so a 100% of the work that we do that results in software distributions that customers use first order in their environments, is done in open source. It’s done with the Apache2 License, it’s done in an open and collaborative way that you would expect well-run, open source project, or multiple well-run, open source projects to be.
And for what it’s worth, we have somewhere around a thousand repos today in Github. So, that gives you some sense of the expansiveness of the product portfolio. Now, a lot of those repos are around specific components that we built for Habitat, but nonetheless, there’s still hundreds of sort of core repos that are the tools that make up the Chef ecosystem.
We open source everything, and in the belief that being transparent and having that software development process be accessible to our customers with value to everybody, we can build better software if we collaborate with the people that use the software.
The flip side of that is that we’re still venture-backed commercial entity, and of course, our goal is to get people to pay for the value that they get in Chef, and InSpec, and Habitat, and Automate, and all the rest of our product portfolio.
So, along with the announcement and the move to move all of our software development to open source, we also decided to make a change about how we distributed the software that we built and maintained on a regular basis.
The coupling of that was that, moving forward, we decided that all new builds or new releases of Chef software, of InSpec, of Habitat, and ultimately of Automate, would come with a commercial EULA attached to them.
The EULA basically sensed if you are a commercial customer, a commercial user of the software, and you’re using it for commercial gains. You’re obligated to license that distribution from Chef.
And so, the combination of those two things means that, essentially, we were able to resolve two core questions that were really hampering us in our relationship with customers, and with users and open source.
One of them was, “Explain to me what’s free and what’s proprietary.”, or “What’s open source and what’s proprietary?” Let’s call it that.
That was a very difficult question to answer, and we would have to draw across the various different projects, a very wavy line, that differed by project, on how much of it was proprietary, how much of it was open source, where did the features go, etc.
So, by moving to a 100% open source, the answer is very simple, 100% of it is open source and 0% of it is proprietary. And we’ve got a very simple way to answer that question to anybody in the Chef community, or ecosystem, or customer base.
The other question that customers often asked us was, what was free and what was commercial. That of course also had a very wavy line. It was very difficult to explain to anybody the difference between the free or the commercial parts of our offering.
And, of course, as an open source company that it’s trying to commercialize on it open core model, you’re motivated to try and put more of the feature functionality into commercial or proprietary software, to try and ensure a value proposition, by going to a 100% open source and going to a commercial distribution of the software that we use. So, distribution being the software that we produce after we go through our release engineering and testing cycles and everything.
The actual binary packages that we distribute to the world, those come with the commercial license attached to them. And so, the answer is 100% of the intellectual property is free, and you can do with it what you like, but if you want our post-production version of how to easily install turn-key software into your environment, that comes up with a commercial license, and we expect customers to honor that with a relationship with Chef.
Does No Binary Distribution Hamper Adoption?
Michael Schwartz: Do you think that not providing a binary for the community hampers adoption?
Corey Scobie: Well, interestingly, I think if you were starting out as an open source company, you really want to ask yourself, like, what are your goals, and why are you choosing open source as a software development methodology.
In many cases, if you’re starting out and you don’t have an established community, established user base, established brand, etc., one of the things that you can do is you can offer yourself for free to the world, free as in beer as opposed to free as an available intellectual property. That can definitely help you with adoption.
We have a relatively, you know, Chef goes back a decade now, and we have a strong base of users, what we’ve decided to do in order to try and maintain the grassroots adoption of our software is, make it as accessible as it was before.
So, in other words, anybody can land on Chef.io and download our binary software in the exact same way as they could before. So, the availability of this offer is not different.
The license terms that come with it simply say: if you’re using it – it doesn’t matter what kind of organization you are – if you’re using it in an experimental way, or you’re trying to learn about it, for your own benefit, or to evaluate, fit in your organization – that is for use, and you’re not obligated to have a license relationship with Chef.
If you were using it in a commercial way, you are obligated to have a license relationship with Chef to use that package of that software, and so in that regard, I suspect that we probably will turn down our grassroots adoption a little bit.
For us, it is a mature open source company – that’s a trade-off that we’re willing to make.
But if you are a brand new company, and you’re trying to get adoption and validation of the technology in the field – that probably could be a major hamper for sure.
Michael Schwartz: Enterprise Automation is a huge horizontal market, and I’m wondering if you segment the market in any way?
Corey Scobie: We don’t naturally. In other words, there’s not specific vertical industries that we try to focus on, although I will say that if you look at the demographics of our customer base, there are industries that naturally lend themselves to Enterprise Automation more than others. And those industries tend to be industries that are heavy in computes, environments, and transactionality, etc.
So, if you look at financial services, for example, financial services is a strong sector for us, it’s a strong sector for I think most companies in the Enterprise Automation space. We don’t specifically target financial services, but the characteristics of financial services is, they tend to have lots of compute resources in lots of hybrid environments, both on premise and in the Cloud, and the management of those environments is a material cost to them and also a material opportunity for efficiencies that get them to the next level of execution in their industry.
So, they tend to go together well. Other sectors that are strong for us are things like governments, where there’s a lot of computing in certain types of both civilian and DoD kinds of government environments, the hospitality industry, the logical places where you see companies, larger companies deploying larger fleets of computing resources.
Michael Schwartz: So, when you changed – and I realize this change is very recent – but when you changed to, I guess, tweak the business model, did you think that that affected the value proposition for the commercial offering?
Corey Scobie: What’s interesting is that I actually don’t think it really affected the value proposition that much, because before we changed our commercial licensing on the binary distribution, the reality was that the customers that were paying us money for Chef were paying us money for Chef because they wanted to get their software from us, they wanted to have assurance and support that somebody was there to stand behind the software when they needed it.
They wanted all of the things that are basically the core value proposition. And today, if we describe the value proposition of having a commercial relationship with Chef, it’s that you get the best word-class distribution of the software that’s available.
Currently, there are no other alternative distributions of the software available. However, our community is working hard at building some community editions of that, and we have been welcomed with open arms, and we’ve been collaborating with them to do that in the most constructive way possible.
There’s no other distributions of the software broadly available today, but I suspect that that will be a true statement in the future. Red Hat and CentOS are probably the historical example of that kind of thing.
But the core value proposition is that they get software from the Originators of the software, who have more people working on the open source project than anybody else. They get the insurance and support of having Chef to be able to stand behind that in an Enterprise Plus way, 7/24 support availability, SLA is on things like bug fixes and security updates, etc.
They also get access to the people that we have at Chef, many of whom are very, very skilled field operators that help customers with complex Enterprise Automation needs. And then, we are also working on building content that really couples well with our commercial distribution of the software and is available to commercial customers, that is, you know, there’s a world of Chef content available out there, cookbooks and InSpec’s profiles, etc.
We’re trying to build a small set of the best curated version of that content, and then manage that in the same kind of software development loop that we manage our core software engines in.
Michael Schwartz: I was just giving the press release, and one of the lines that stood out to me was, “Vendor-driven development and distribution models that closely align with FOSS principles,” and I’m wondering what are those FOSS principles that aligned with vendor-driven development and distribution?
Corey Scobie: When we talk about FOSS principles, we talk about the Four Freedoms of open source software. And it’s interesting because the Four Freedoms are age-old sort of documents around, you know, new project, and what are the essential freedoms that you need to really be able to claim that you’re an open source software project.
And the freedoms are really succinct, every time I read them, I read them like they are amendments to the constitution. They were written a long time ago by some people that really had a great vision of where it’s going. And I realize that today, this is super controversial because there’s a lot of conversation in the marketplace about what does actually conform to the Four Freedoms, or whether the Four Freedoms should really be the definition of open source going forward.
But the freedoms are, the freedom to run the program as you wish, for any purpose, to not have an assumption about what you step program might go to. The freedom to study how the program works and change it, so access to source code is a precondition for this, the freedom to redistribute copies, so that you can help others.
And for what it’s worth, we do have some restrictions in saying that the license is not transferable, but the binary distribution of our software is redistributable in many cases. And we use RubyGems and other distribution mechanisms in order to get broad distribution of the software out there.
And then, the freedom to distribute copies of modified versions of the software, which, for us, the example of that is that the community is actually building and modifying the software, to create a new open source and free as in beer version of the Chef client as an example.
They may not recreate the entirety of the ecosystem of the Chef client for companies that don’t want to use our commercial distribution, or for people that don’t want to use commercial distribution like ours. And there’s no problem with that.
The one rub that we have in our open source strategy is that Chef is unlike many other projects, which emanate from free software islands like an Apache project or something along those lines, we are Chef, we actually invented the project to begin with, Adam Jacobs is one of the founders, he founded the first open source project around Chef, and Chef, the company owns the trademark to that.
We are protective of our trademark in that other people shouldn’t be allowed to use our trademark, really just the same way that you can’t build running shoes and call them Nike.
But beyond that, beyond the trademark limitations that we have around the brand itself are open source projects that are available to anybody to use, and to modify, and to redistribute as they see fit.
Investing in Project
Michael Schwartz: The one and the best position to nurture the project are the ones who have invented and continued to drive innovation?
Corey Scobie: To continue to drive innovation and continue to invest. One of the great things about open source is that people show up and will invest their time and their effort, but there’s real capital required to make a lot of the open source projects that we really value broadly applicable and broadly valuable to the market at large.
And so, for us, one of the things that we bring to the table as vendors, we brought a hundred million dollars’ worth of VC money at the table to invest in the projects over time.
Where Does the Community Add Most Value?
Michael Schwartz: Where do you think the open source community makes its most valuable contributions?
Corey Scobie: To me, it’s really about the idea of identifying a problem. I’m going to take it to the existential line and say it’s really about the idea of identifying a problem or a gap in the market and building a collaboration around how to solve that.
And the reason I think that that’s really important is, because one of the ways that you build great products is by bringing multiple viewpoints into the picture, and then, collaborating with those viewpoints to come to sort of the most broadly commonly applicable aspect of the problem that you’re trying to solve.
And one of the great things about open source as opposed to proprietary software, particularly – you know, my history is largely with proprietary enterprise software – historically, is that those viewpoints, those different angles of the problem-solving often don’t come in in an open source software development life cycle until the very end, until you’ve done 95% of the solutioning and engineering problem solving.
The way that open source makes such a great contribution is, it usually takes the problem and sticks it right out on the table, right upfront, and then, people get to collaborate. And those opinions and the ideas of how to solve that problem are integrated right from the get-go. And I think that what results in that is just better quality solutions to the challenges that you’re trying to solve overall.
And so, it’s the super existential version of that question because there are many, many other versions of that answer that point at specific parts of the technology stack that are pervasive in the industry, etc., but for me, it’s really about trying to make the best of whatever problem you’re trying to solve, and that’s where open source shines.
Michael Schwartz: Do you have any numbers or thoughts about the percentage of open source deployments that convert into Enterprise subscriptions?
Corey Scobie: You know, I should. We’re definitely able to measure that better since our open source/license model changes this spring, but it’s very early days, and so we’ve seen some conversions and some returns on that front.
But what I would say is this, it is part of the challenge of being an open source company, particularly one that delivers enterprise software to proprietary computing environments in the enterprise – it’s hard to get real telemetry.
So, what I couldn’t tell you today is, how many companies, or how many versions of companies, or how many nodes of software I have deployed in the world, both are a combination of open source and proprietary.
I can certainly give you statistics on what I have in terms of commercial licensing on the one side, but the part of the equation that isn’t clear because we don’t have good product telemetry on that front is, how many nodes of software are actually deployed.
So, we don’t know what the conversion rate is, we don’t know how many people will come to a decision point, at a point in the future, where they have to decide whether they want to continue on the open source path or the commercial path and make an explicit decision there.
Unfortunately, Mike, today, I don’t have very good statistics about that. I can tell you that we suspect that we will see some uplift in companies deciding to follow the commercial path, now that we are driving that decision point at some point in the future, but it’s going to be difficult to calculate how that converts to the distribution of software in the past.
45 million downloads of Chef for over the last 10 plus years. So, there’s a lot of it out there.
Michael Schwartz: It could also become more challenging if there is another distribution that’s not distributed by you, so that the objectives of that other distribution may not be to include the telemetry or to do a deploy squeeze to get information on who’s deploying it – do you see any friction there?
Corey Scobie: Yes, it’s possible, for sure. I mean, one of the things that’s interesting, and I think somewhat unique about how things are shaping out in our community is that the alternative distributions are, as opposed to a group I’m coming along and hard forking the project.
What we’ve done with the community is that we’ve sort of collaborated on an additional distribution, or making it easier to create additional distributions of the software, by staying on the same core-source tree.
Just as an example, if we decided to put telemetry into the product, and that went into the core-source tree, a downstream distribution might choose to exclude that, or they might choose to stay compatible, in which case, they would have the opportunity to collect, and maybe show that telemetry as well.
So, I guess the answer is it could go either way. There, from a technology perspective, the path of least resistance is for us all to stay on the same source space. And that way, we can try and guarantee for the backward compatibility across the different distributions of the software, which is something that we, as a community, have to decide if it is important for our users, but it could go different ways too.
Sales and Marketing
Michael Schwartz: Switching tracks a little bit towards sales, are most of the leads inbound? And I’m wondering if – you haven’t been there that long – but do you have any visibility of how the sales teams evolved over time?
Corey Scobie: One of the things that we decided to do when we made our sort of our licensing changes, and what-not, is to really focus on the fact that we are a company that largely delivers to enterprise-class customers. So it’s the global 5000 industry players.
On that front, it’s a good mix. I mean, we still have lots of inbound interest, it comes at different levels, and it’s sort of more curated now than it’s been in the past, I think.
If you would have talked to Chef of 2015, the answer would have been, “Everybody and anybody is our target market.” With that, leads or inbounds, we do a lot of prospecting ourselves as a sales force. But where there’s no real opinion about whether we should be tracking down a lead, based on the potential size of a commercial opportunity, or what-have-you.
I think the Chef of 2019 is a lot more focused on really being outbound-driven on the enterprise-class customers. And then, inbound-driven on right-sizing the inbound leads into the appropriate bucket.
So, for us, we have partner offerings that are focused on sort of smaller scale, small footprint customers, often in a hosted environment, so we have a relationship with Amazon web services that has our product offering, a certain Chef product offering called OpsWorks for Chef Automate.
That’s often a place where small and medium customers gravitate to because it’s more of a consumption-based model. And then, the larger enterprise customers tend to want to have a relationship directly with Chef. Ourselves and those customers tend to be larger contract values in a longer-term commitment.
Michael Schwartz: The pricing is one of the hardest parts of tech entrepreneurship, and I think it’s really hard for infrastructure software when you say, “Well, what’s the value of this infrastructure? – Well, your company maybe couldn’t run without it.”
I’m wondering if you struggled with pricing, or change pricing, and sort of how you’ve approached, like how do you figure out what’s the right value?
Corey Scobie: So, what we didn’t do is, we didn’t change the market, the established market value of our suite of infrastructure automation security compliance. So, overall, the net value of that in the marketplace is exactly the same before and after the business model.
What’s interesting about what we did with licensing is that prior to changing the commercial terms on our distribution, customers really bought a license to Chef automate which was our proprietary management console. But they paid by the number of systems that they were connecting up to that management console.
It was sort of a proxy value to the real value that they were getting, which is, they put Chef on a server, and it does configuration management for them. But then, they paid us, by paying us, to foresee and manage the visibility of that in an Enterprise console on top.
That was a side effect of the open core proprietary open source software of the relationship that we had, all of those products. What we did host the license and packaging change as we said, “Chef itself has a value, and that value is the thing that customers experience the most value in.” So, we’re going to put a specific price point on Chef.
Same for InSpec, which is around security and compliance, and continued security and compliance there. And the aggregate of those two was the same value as they were paying us before on a per note basis for Automate.
We basically did unbundle the pricing to create distinct, unbundled pricing, as well as creating some easy-to-consume Enterprise skews on top of that. Because often, Enterprises don’t want to buy a bunch of part pieces but what they want to buy is the solution that does infrastructure automation, or security compliance, or for us, the ES bundle is all of the pieces of the puzzle together, into one pricing bucket, into one easy-to-consume skew.
So, the short answer is, we didn’t change the market value, but what we changed is how you count the pieces, the components that go into that market value, to make it easier to understand and see the real value of what you were licensing in our commercial relationship with Chef.
Michael Schwartz: I’m wondering about, if you have any channels, or have you built partner networks, and does that account for a meaningful amount of sales?
Corey Scobie: We definitely do have partner networks, and often, they take different forms. Like, most commercial software companies, some of our partner channel is about fulfilment, like how people actually create a contract vehicle and purchase things. And it’s easier for them to do that with – if it’s a new customer, for instance – it’s easier for them to do that with a partner who may have an established business relationship with them. And it might then bring on a new vendor.
We also have channels that are about, both software and services delivery, and building solutions channel is on top of that. Enterprise Automation, as you said, it’s a complex, horizontal landscape.
We have a number of partners, where they are sort of value-added partners on top of the software core. So, what they’ll do, they’ll bundle on services or other kinds of consulting engagements on top of that.
Overall, our partner channel does account for a meaningful, not a huge, but a meaningful part of our annual recurring revenue footprint as an example. And then, the last thing is, of course, we are strategic partners.
I mentioned Amazon and the relationship that we have with Amazon and OpsWorks for Chef Automate. We also have a relationship with Microsoft, relationship with Google, and of course, these platforms are the dominant platforms of the next generation of Enterprise computing, are important, have strategic relationships with as well.
So, yeah, we continued to build and invest in our integration there from a software and a valued perspective, and we also are building our commercial relationships with them over time as well.
Challenges of Open Source Software Startups
Michael Schwartz: What do you think are the biggest challenges facing new open source software vendors today?
Corey Scobie: The attractiveness of open source is that you get this great collaboration with the community of users that you’re trying to solve problems for. You probably get them uplift in terms of the community coming along and adding value to the amount of investment that you as a new open source company are able to put into that code base.
So, those are both positives. And plus, you get, depending on what you choose from a licensing perspective, so choosing to be an open source company and choosing a licensing path are sort of two different things, and they should be thought of as two separate decisions that come together to help reinforce your business strategy overall.
But if you look at it from a licensing perspective, whether you go on a freemium, or a truly free distribution to try and amp up acceptance and distribution of your technology, you can definitely get more eyes and more interest and more potential users to your door that way.
I think the real tricky part is to understand clearly the dynamics of the relationship between the value that you provide as a software company, and how you expect customers to realize and pay for that value in the long term.
I think that is a really tricky thing. And it’s not static for the lifetime. At least, in my opinion, it is not static for the lifetime of an open source company.
You may choose open source as a business strategy and a software development strategy early in your life cycle for one set of reasons, and then, choose to continue, or expand, or refactor that open source capability later in a lifecycle of your company, depending on where you’re at in that evolution. And it’s something that you should revisit on a regular basis.
I think the other thing that’s really hard for open source companies today is to choose – there’s a lot of political and other thoughts going on in the industry around open source, and the existential threat of Cloud vendors, and whatnot – that’s a whole sort of political hotbed of topics that I think has spawned a huge derivative of interpretations of sort of open source licensing and open source licensing strategy, etc.to try and protect the intellectual property based from something.
And gosh, I think, you got to go into it with the idea that you’re going to be the best at solving the problem that you’re starting right now, and let the market play out.
Michael Schwartz: Corey, that was really fantastic. Thank you so much for your sharing all your thoughts today.
Corey Scobie: Thanks for having me, Mike. It was great, and obviously something I’m passionate about is the evolution of open source, and how we can all be better open source stewards.
Michael Schwartz: And best of luck with Chef.
Special thanks to the Chef team for volunteering Corey.
Transcription and episode audio can be found on opensourceunderdogs.com.
For this episode, we also added a link to The Changelog #353, if you want to hear the interview with Adam.
Music from Broke For Free, Chris Zabriskie and Lee Rosevere.
Production assistance from Natalie Lowe.
Operational support from William Lowe.
Please leave a review or add Open Source Underdogs to your podcast favorites list. That helps us get the word out.
Our Twitter handle is @fosspodcast.
Next week, another bootstrapper, Peter Zaitsev, Founder and CEO of Percona, he’s one of the early engineers at MySQL. He’s had a ringside view of the open source database landscape, and he’s super interesting – don’t miss it.
Until next time, thanks for listening.