API Evangelist Conversation with Dave Shanley of Princess Beef Heavy Industries

Hey there, Ken. Hey, everyone. Thanks for having me back. My name is Qobix, or Dave Shanley. I’m the founder of Princess Beef Heavy Industries.

Great. Thank you. I think, I appreciate the opportunity. So, the OpenAPI Doctor is, it’s where I wanted to be when I first started on this journey. It’s what I imagined existing when I first started. And, uh, it’s really an accumulation of all the open source tools that already exist out there that, you know, are being used within these individual units of value. But the Doctor is bringing all that together combined with you know, a platform around it and APIs and then turning it into not only an API platform, so you can everything you see, you see visually, you can also do through an API, obviously, because I’ll be crazy not to do that. Um, you know, it goes against the whole point, the purpose of the, of the technology. So it’s really bringing it all together and making it available in a unified experience, something that [00:02:00] I care deeply about is unification of experiences. It’s just that. So when you mentioned it’s visual, what it’s doing is actually visualizing all those lower level tools like vacuum, for example, there isn’t a UI for vacuum. I mean, there is, um, there’s like a terminal UIs, and there’s an HTML report, but it’s not actually something you can use. It’s not like a tool that you can engage with, and it doesn’t kind of go further and beyond what you see today. So really, the doctor is the UI of all the tools put together Right now it’s just exposing vacuum and some new, you know, custom features like the Explorer, um, and like the tree navigation and visualization tools. Those, those are unique to the doctor, but it’s really just got a vacuum in there with the governance, which is what you were talking about with rule sets and being able to use the rule sets. It’s actually the first user interface for. Rules as far as I know, but there might be something from stoplight and spectral, but actual managing rules and having a UI for that the docs was actually the first one that I know of, I could be wrong in this, but, uh, probably I’m wrong, [00:03:00] but, but anyway, it’s, it’s how I would always imagine to do it. And it really, it’s just scratching the surface of it. So as I mentioned, it’s right now, it’s just the, it’s just vacuum really being exposed with a couple of custom features, new features, but what’s coming up next is things like open API changes as well as wire tap integration. So that means, um, but right now, every time you’re working on that open API contract in the doctor and you’re, you know, making changes. In the background, I’m actually tracking everything, every change being made everywhere. What I haven’t exposed yet is the OpenAPI changes experience where you can see alongside your contract, everything, every single change or everything that has changed, you know, with every edit you’ve made. And that will be not just, you know, during your workspace session, it will be over time as well. So for example, if you’re pulling in from a GitHub repo, which will be available soon. You can do the same thing there. So it’s, it’s all of that stuff is all mechanics are already there. I just haven’t wired them together. We’re exposed them the same with wire tap. You know, if [00:04:00] you’re working on your open API contract and you want to turn it into a mock server, right? The options are you go and use, you know, another mocking tool, like a micro X or a wire mock, um, uh, there’s a few, there’s a few great ones out there. But, uh, why can’t you just click a button and, you know, have, you know, have a mock service spun up because why it’s, I can do that at the same time. You might say, well, okay, I’ve got my, my contract here. Now I want to test if my UI and my backend are compliant with the contracts. You can do the same thing. You just spin up a proxy. It’ll then track all that stuff for you and show you where your violations are, you know, in the doctor during that experience. So having it all automated and have it all. You know, integrated and done for you is really the key because right now, different companies and different, uh, different teams, whoever’s using these tools, they’re all kind of building their own ways around it, how to use it. This would be a You know, a turnkey solution for all of that. And it will run locally or it will run in your own data center. So you don’t even need to depend on the service. You can take [00:05:00] it and run it yourself.

Yeah, it’s, it’s, it’s, so it’s, I’m trying to create an interface for every type of user. So you’ve got your developers, which is, that’s why all the open [00:07:00] source stuff is low level and it’s all APIs and SDKs and, you know, stuff you can bring into your apps and, and use programmatically. That’s the first and primary. Customer always is engineers and developers or hackers, as I call them. I call myself a hacker. Um, and then, you know, you’ve got that level and then you’ve got the next level up, which is the technical, uh, like technical project or operator who understands the technology, knows how it works, wants to be able to move with it a little bit quicker. So they need something like having vacuum pre compiled as a binary or run as a Docker image, or having this tool available as a drop and go, I don’t want to. Get too involved with it. I just want to pick it up and go so I can solve my problem. That’s like the, that’s the, that’s the biggest slice of the pie. It’s the, the folks that need a solution. They don’t want to get into the depths of it. They don’t want to learn too much about it. They don’t really care too much about it. You know, their business is something completely different, nothing to do with open API or, or anything like that. They just want to get. Uh, and those can usually either [00:08:00] developers, you know, who are moving through a chain fast or they’re, you know, the technical project managers or managers, things like that. So it’s fitting that, that use case. I just want to get in and get out and do the thing. I want it to work reliably. We’ll pay a premium price for that or, you know, service, et cetera. Then there’s the other side, which is actually where I feel is the most unloved, which is the. Less technical, more architect based, um, persona, which is wants to be able to design APIs, wants to be able to understand the process, but doesn’t necessarily have the time or the experience to be able to get into the nitty gritty of, you know, learning about JSON schema. Or learning about open API and the many, many, many different things you can do with it in very interesting ways, or anything really, any, any kind of standard they, they, but they want to be part of the design process. You know, they’re, they’re high level, they’re, they’re senior engineers. You may not get their fingers too dirty with code anymore, but they fully understand system design. They fully understand schema design and would add a lot of value, but they just don’t want to get down into it. Or [00:09:00] it’s, you know, non technical project managers who. What can understand a UML diagram, but doesn’t necessarily, I guess, loss when they look at 50, 000 lines of YAML. So it’s catering to those people that want to be able to take a more design first approach where they’re thinking about the architecture of their APIs in a more traditional fashion, using something like UML or visualizing the specification as UML and be able to manipulate it and edit it and operate it that way. That’s. That’s where I’m putting a lot of effort in because I feel like it’s the most unloved part of the market. I think one thing that stoplight has done very well and capture very large, a piece of people’s hearts in this space is stoplight studio. Um, I’m not sure what SmartBear are doing with stoplight studio or how they’re integrating moving forward. I’ve seen pieces of it, but I don’t see, you know, the, the level of investment going into that product is what there once was, and that’s. A lot of people love that because it’s got the low level stuff and it’s also got the high levels. It’s got a high level WYSIWYG experience [00:10:00] and it’s got an ultra low level developer experience and it’s got that middle piece where it’s just, I want to get the thing done, get my specs in, get my specs out. Um, I think they captured that whole stack very well. Um, but I haven’t really seen anyone. Continuing on with that with that approach. Um, so it’s kind of what I’m what I’m focusing on the moment.

Hmm. Yeah. The, the, the idea of having like a, the, the gov, the governance board or the compliance board or some, some, some word to describe some way for oversight and engagement, um, and a way to track it and measure it. That’s what the tooling does. And then you’ve got the execution. So, uh, yeah, four personas. That’s, that’s what I’m looking at. I’m not necessarily. Right now. And I don’t know if it would ever be the case. I’m not looking at the generation of output. So, uh, things like SDKs and the manufacturing of artifacts downstream. I’m going to leave that to, uh, other players in the industry. We’re doing an excellent job like speakeasy, um, and, uh, scalar. com. Do a great job over there. Uh, I’m [00:12:00] going to focus on the upstream. As I mentioned before, it’s design. It’s really the design part of API design or API first. I feel like is the most unloved part of our. Of our ecosystem, if you will. And, uh, it’s mainly because it’s, it’s really hard to get it right and to cater to enough different faces, right? Not everyone wants the WYSIWYG, not everyone wants a super technical. How do you blend them together in a way you can have your cake and eat it? That’s hard. And that’s a tough nut to crack.

Yeah, so, um, you know, everything that, everything that exists in the Doctor, anyone can build themselves from the source that I have published. Everything that you see visually is me taking my years of experience and trying to create, you know, an experience out of all of that open source software [00:14:00] on the top. So the Doctor is a rendering of, you know, bringing all this together in a platform. So, What I’ve built, anyone can actually go ahead and do the same thing, uh, which is what, you know, I want people to do. That’s why I’ve built it in this way. These are Lego pieces that provide these individual units of power. There’s lots of stuff in that open source that is designed for higher level functionality that just isn’t exposed in a way that is easily understood. But if you go looking, you’ll see all of these, these shortcuts, these super powerful little bits and pieces I’ve just made available, but I haven’t necessarily heavily documented because, you know, it’s part of the. Part of the magic of bringing it all together means doing a bit of work, you know, and figuring out for yourself. So, um, my goal is that you can either build any part of what, you know, I have available to service yourself, or you can build fragments of it, just the bits that you care about and the same with the actual tooling itself. It’s going to be. You know, here’s the whole thing or here’s just a bit or you can, you can pull it apart, you know, cause it’s, it’s the way it’s been built is a, [00:15:00] is a coin that’s becoming really popular now at the moment is a modular monolith. You know, we’ve kind of the pendulum swung towards microservices and, uh, Kubernetes, everything. And then it was like, Oh my God, that’s a lot of work. So let’s kind of, let’s go back to, you know, being, you know, one simple piece of software, but make it really, excuse me, I’m banging my cups here. Make it really easy to, uh, um. You know, uh, modulize and upgrade individual pieces and deploy independently without having to bring everything together at once. So that’s, that’s what I’m doing effectively is creating that, that modular monolith is all these pieces you stick together in yourself. I’ve done that for you in a way that I think is, is the right way, but there’s nothing stopping you doing that. Uh, and if you want to use what I’ve built, which is, you know, it’s a turnkey solution or will be a turnkey solution. It’s just a preview right now, then, you know, that becomes a premium cost. But it’s always open for open source. So if you’re doing what I’m doing. You get to use it for free forever. That’s, that’s one of the, one of the givebacks, but if you want to do it privately and you’re doing it commercially and you want [00:16:00] me to track it for you and do all the analytics for you and all that kind of good stuff, then yeah, there will be some form of commercial element to it. And if you look at the licenses of the libraries. Um, there’s, there’s actually a mix of them. Uh, I use a mix of MIT licensed stuff, like LiveOpenAPI, Vacuum. That’s all MIT. Use it, take it. I don’t, it’s, it’s yours. It’s free. But it does actually consume, um, a BUSL license that is called, it’s actually called the Doctor. So the Doctor is available as a license. So all that high level stuff that I built, like the charts and the graphs and the gra all available through that library. But that is a B B S U L. So anyone who’s using it internally, For nonprofit or like an internal tool chain, or they’re building their own internal products can use it for free. Just like, uh, um, Oh God, where did it come from? Uh, it’s like a database. Uh, was it not Mongo or something else? Anyway, uh, a really popular open source database, uh, uh, created it, the business source license. So I just took it from that. In fact, HashiCorp went through the same thing with all of their tooling [00:17:00] recently. I’m a fan of it. You know, it’s free for everyone to use for non commercial purposes, but if you start making money off of it, then you’re taking food out of our mouth. You’re, you’re stealing from us. So that’s where it changes, but if you’re not, please, it’s free. I agree with that. Not everything can be completely free. Some stuff can be free for, you know, just, you know, for non capitalist purposes.

Love it. Love to hear it. You know, let’s, let’s, let’s all make money together. It doesn’t mean that everything has, it’s not like all, all commercial, all free. There is the ability to blend, you know, the world is not black and white. It’s patches are great everywhere. So, um, yeah, I love that. I love that philosophy and I believe it’s the right way forward. You know, everyone can prosper together without one, one side being over unbalanced.

Great, thank you. Um, so LibOpenAPI is obviously a library for OpenAPI, but it’s not just a library, you know. When you think of a library, it’s like you can go to the simplest level, which is It’s a library I bring into my software that allow me to pass an [00:20:00] OpenAPI file and do something with it. You know, whether that’s, you know, generate my own, it gives me that structured model to be able to read in. And that’s as much as I need, right? And that’s, that’s what a lot of people need in a library for OpenAPI. It’s a read only process to read something in and do something interesting with it. And then they’re kind of done. So that, that’s, there’s already, there’s already a great library for doing just that out there already. It doesn’t support 3.

1. OpenAPI 3. 1, but it does support OpenAPI 3 and below and it’s called Kin OpenAPI, as I mentioned last time, but the problem with Kin OpenAPI, and it’s not that it’s a problem, it’s just there was never part of the design, um, is when I wanted to do something like, um, OpenAPI changes. Which is where it does a semantic diff on properties on schemas and everything, every single piece of the tree and the graph, really, it does, um, uh, like a semantic analysis. So if you were to completely jumble up all the properties and put them in two textiles side by side, you know, but it [00:21:00] was the same thing. It would come out with no diff. You know, it doesn’t look at, you know, like a, like a traditional diffing tool would do, you know, looking at lines, additions, removals, things like that. It actually looks at the properties and the values. Um, Can’t do that. Sorry, I went on a bit deep there. Uh, but the reason why I went so deep is because I wanted to do that and I needed a like a pre built tool to do it and uh, Kin OpenAPI couldn’t do it. Because it doesn’t, it loses all of that low level information about line numbers and column numbers and things like aliases and anchors that when you really get into the depths it didn’t support any of that stuff because it just was a much simpler, uh, uh, use case. So I realized the only way for me to do what I wanted to do was literally to build it myself. There was nothing available that would exist for me to be able to get this kind of data out. Um, now, I’ll pause there and go, you know, if I bring you back to the last conversation we had about me working at VMware. I actually built Lib OpenAPI already once inside VMware. It was much more specific. It was nowhere [00:22:00] near what I’ve built outside VMware as open source. And it was very limited in terms of its use cases and very specific for the internal business. But it was the same thing. It was a, it was actually called Lib Swagger originally. And, uh, you know, we, I changed it to Lib OpenAPI internally, but that’s the thing I wanted open source. And like you said. At the beginning is that, that’s our, you know, that’s our magic. That’s a, I’m like, no, this is, this is what people need. Let’s, let’s, let’s make this available for everyone. Um, and again, that wasn’t a possibility. So when I set about, about building it from scratch, not, not only do I want to go down to that lowest, lowest level, which is essentially building a compiler that knows every single node, every single position, every value, every type to the, to the T you need, you need to build something called an abstract syntax tree to be able to do that. Um. So I would do that, but I would create it as something that I would be thinking of as an open source library from day one. So, um, you know, it’s actually got 99. 96 percent code coverage [00:23:00] on that library, which there’s a few things in there we can’t test just because they’re like testing operating system failures, but it was designed to have near. Perfect code coverage. Every single line is covered and there’s about 30, 000 lines of actual code in there and like nearly a hundred thousand lines of tests, like the test way out way that the production code, but I built it as this, as if people are going to adopt it, if companies are going to use it, they need to trust it. The only way they can trust it is to see things like that is designed from the ground up to be built as a tool within this vein. Like you would normally build inside a company when you’re thinking about building a product, I needed to build the product as open source. So it was building a compiler, uh, something that would go super low, right down to the lowest level. And in order to build that, I had to build this really complicated model. And I thought, well, this, this isn’t going to work. You know, people are going to look at this model and go, this is awful. I can’t use this. It takes 55 calls to get this done. The piece of data I want, this is crap. Um, so then I took a bit of inspiration [00:24:00] from Git actually, and they’ve got this really nice, um, porcelain and plumbing architecture. You can use Git at the plumbing level, which exposes everything. Everything you could possibly need in every dimension for every kind of engineering possibility. And then you’ve got the porcelain layer, which is that simple, easy to use, really basic and get, you know, it’s a couple of commands and you’re up and running with, with, with Git. So I did that. So there’s actually two models. There’s a full set of high level models and those high level models are built by the low level models. The low level models, it kind of. Kind of go up. There’s a there’s a diagram on on pbc3f. io. If you’re interested in looking at it explains that that model architecture and I call it a DOM. So it represents this document object model. Very same word that we use in the browser to represent. You know, how we compose a webpage. Uh, it’s the same thing. It’s the same concept. And it’s also got, you know, some super powerful machinery underneath called a Rolodex. Which is what, you know, a compiler has a, you know, a linker and hash maps for [00:25:00] memorizing where all things are and objects and all that kind of stuff. This does the same thing. So really what it is, is an OpenAPI compiler. Um, and it’s read and write model. There’s still a bunch of functionality missing from it that I want to build. But, you know, it’s enough to give you What you basically 99 percent of what, uh, developers need from open API and most of those use cases, it’s got some things are missing, but most of the dimensions are not covered. It’s actually, we’re not too far off being a V1 release. To be honest, it’s actually stabilized pretty well over the last year. And the APIs aren’t changing at all, really. So, um, along with the addition of 3. 2 open API, 3. 2, and eventually 4. 0, um, I think we’ll probably cut 1. 0 before we hit. 4. 0. Anyway, I’m just rambling now. But anyway, yeah, the whole point behind this was Uh, that magic secret source I want to make available for everyone. And then it’s done. If anyone who wants to come along and redo it, good luck, have fun with it. It’s a great adventure. I know a [00:26:00] lot of people do it, but they, they get like, they get like three feet into a, you know, a 50 foot depth of pool and realize that, wow, this pool is really deep and they stop. Um, uh, so there’s, there’s a couple of great, like for example, scaler. com. They’ve actually created their own open API JavaScript parser that they’re using in their documentation tool. Really, really good. It doesn’t have that low level model, which I was talking about, which is, you know, what we need for building tools upon, but that high level, it’s like replace, it’s becoming the new JavaScript, uh, or it should be anyway, because it’s great. Um, you becoming the new way to pass open API using JavaScript. So I’m being really technical here. I apologize if I’m going too far into the weeds.

It makes perfect sense. Exactly. It’s, it’s solving these lower level bricks once and for, at least for a good long period. Um, there was a great comment that I saw in the open API, uh, slack today. Um, and, uh, somebody has said, like, I haven’t looked at tooling, uh, in the industry in a while, and it was terrible back then I’m looking now and it’s the landscape is a lot, lot better. It’s very exciting. So I think once, you know, everyone is like, it’s, and he’s right. And the last couple of years have been actually really exciting for the space because. People like realizing that, that this, there’s actually, there’s actually some real mess that we’ve got to clean up here and there’s lots of opportunity and directions. I’m with you in the, let’s not make money off the fragmentation. Let’s just, let’s build up, let’s build a series of tools and products that all work together. For example, uh, speakeasy use a lot of a princess beefs tools and libraries and their products. So every time that the community makes, you know. [00:30:00] my stuff better. Their stuff gets better. Uh, it’s things like that. They’re just, it’s, we need to do more of that and enable that and, and solve this lower level, effectively commoditize this level, which is when, when a technology has reached that maturity on that plateau, when it’s become that, you know, background noise, it’s concrete, it’s corn, it’s, it’s canola oil. You know, it’s something that is just an industrial artifact. That’s what we should get to. So we’re now, we’re not worrying about these problems anymore. They’re solved. What’s the next set of challenges, you know, And then we can, you know, like, I call them your problems. Um, uh, somebody told me that one self, it was a brilliant, brilliant way to describe it. a problem that you do, that you worry about once you’re sitting on the dock of your yacht or standing at the top of your yacht or the bridge, you’re looking at, you know, we should go over there or should we go for lunch over there? Or should we just, you know, make some more money in the stock market? You know, it’s like, it is a lovely problem to have when you’re at that point. Uh, when, when this, this, this foundational pieces, so that’s, that’s where I want to get to because then we can all work on the next set of problems.