Ian Jennings sat down with Stathis Georgakopoulos from SlashData to talk about Developer Experience. Be sure to find the full episode here. Transcript follows.

Hello, welcome to a new episode of under the hood of developer marketing I’m status. Your host, the cost is driven towards helping and empowering developers and also making their lives easier. So I’m excited to welcome today. A man who takes great care of this latter part. Well, this episode, I’m joined by Ian Jennings, founder of huckster developer experience. Welcome to the show. Thanks for having me. Will you please introduce yourself to our listeners? My name is Ian Jennings and I’m the founder of hacks or developer experience, and we are applying concepts of user experience research to developer experience. So do you want to walk us two year journey? What was the driving force or, yeah, the role model. When did you first the technology? I don’t think [inaudible], so my dad created utilities for the internet, especially windows 95. And I grew up seeing him sell and support software products.

And as I grew up, I started getting involved in the hackathon scene, which are a 24 hour coding events where developers come together to build something over a weekend. And early in that seeing that those advance, we started seeing companies show up like Twilio and Stripe, and [inaudible] these developer advocates showed up and they started hanging out with these students or young adults who were getting started. And I thought that was just the coolest job. So I was a developer advocate for six years. And in that time I took a really big interest in driving the developer experience. And again, this stuff is new and I became sort of the authority at the company I worked at called PubNub and what was a good developer experience and how could we make this easier for developers? So it’s sort of just came through interest partially through, you know, my father, but then also through just genuine passion.

And as I got deeper and deeper into that area, I noticed that there were no tools and there was almost no support or documentation are no books. It’s just a really new area. And I thought through all my experience, I really had something to offer into that space. Yeah. It sounds like your father was also one of the pioneers in the industry, no taking care of the things, and it’s a good thing that inspired you and all the way to create your own company. So one thing, would you say, was it a hobby that you picked up in your childhood, but you still carry to work today? That’s a really good question. Probably isolating myself in my room and, and coding for a really long time. It’s something I’ve been doing since I was really young. And it’s funny to see that turn from something that’s antisocial and nerdy into something that’s really celebrated.

Culturally, and if you look at something like that, the movie, the social network you see, like, for me, that’s like a turning point where like being, you know, this kind of closet nerd person who’s working all the time or building or engineering became really cool. That’s definitely something that I’ve carried with me over over the years. Yeah. The word there used to have a negative connotation, but I don’t see that way. You know, I say this, no, unless something very positive, you know, someone who’s committed to what they do and I’m well read then. Yeah, of course. If you want. Yeah, exactly. I think there’s, I think there’s something there with D with developer relations and developer experience too, because yeah, yeah. A lot of the people that are acting as developer advocates or out there and pushing for this stuff are usually more extroverted and it’s, it’s a rare role where that combines communication and it combines that kind of nerdiness too.

And I think that there’s something there’s some kind of special like merge or Venn diagram or something of, of nerdy people and extroverts who combined the big getting to this space. Right. Because they’re sort of like public figures or something. Yeah. I realize I’m also talking a lot about Mmm. I’m covering a lot about development relations, but I think it’s really closely related to it. The developer experience. Well, it doesn’t, it’s one of the reasons why we wanted to have you here on the podcast. I know what it is, but in the past in deceptions, we’ve heard developer marketing. We have developer relations with have heard advocacy. So for someone who doesn’t know it, what is developer experience?
So developer experience is the experience that a developer has as they’re working with [inaudible] API or technical product. And it comes from user experience, which is the idea that if we observe these people, working with the products and measure their experience, we’ll be able to improve our products, learning where they encounter friction or problems. But since developers are such a unique audience, we have a new term so we can focus solely on their needs. Versus that of a user users are typically more consuming, Hey yeah, investing as much time or effort into the product.

Developers have their own environments. They have their own tools. They’re builders and things are a lot more complex. So we’re treating it as a new field that can establish it’s own sort of it’s own best practice. The way I understand that these deliberate experiences social very closely linked to the onboarding process for new product.
Why is the onboarding process showing important technical product or an API? Well, onboarding is equivalent to any traditional marketing. Okay. So this is the first experience that somebody is going to be having with your API or tool. And it’s important for them to have a good experience. I have a really good answer. Total example here, where I, when I was getting started working for PubNub before I’d even applied, I was building product myself and, and I had actually tried five different API APIs to do what PubNub I did before using PubNub. But all five of those API, none of them worked on the first try. So rather than try to figure it out or contact the support of those companies, I went to the next API until I got to PubNub, which was like the sixth one. And when I put PubNub in, it just works.

And after working there for so many years, I know why it did. They use a different protocol than a lot of the other companies had been using. Right. That’s why developer experience is important because I was able to switch APIs quicker than I was able to solve my problems. I was able to switch API is quicker than I was able to solve the problems by looking at the, at those companies documentation. And in return, PubNub got my project. Well, they also got me as an employee. And so for six years, I was there working for this company just because they had a good developer experience. And that’s why it’s so important. These people are making these, he used decisions based on only one day or two days of working with an API or a tool. If you look at how much AWS spend is, for example, well, okay.

I signed up for AWS and I hate it. I might not be investing my entire companies infrastructure in AWS. And that’s, that’s why it becomes important to, to make sure that these experiences are really great. Why don’t you shave developer experience is also closely linked to adoption, right? Developer experiences is directly linked to adoption. Yeah. And I think going back to that question about a developer relations and advocacy, those are also closely related to adoption developer and marketing relations, advocacy. That’s all about getting those developers to that point where they’re going to have their first developer experience. They work in hand to hand and often you’ll see a lot of developer experience feedback coming from developer relations coming from developer. Is he, and the whole point is to drive adoption. Yeah. Well it makes a little sense. You’ve been will developer experience. What do you think has been your biggest challenge in these fields?

That’s a really great question. The biggest challenge is the hardest part has been demonstrating the value of optimizing developer experience. And one of the reasons for that is it’s really difficult to measure. So it’s really hard to show that investing and this comes from developer advocacy too. We see the same thing. It’s hard to say. It’s hard to show in numbers that investing in advocacy or developer experience is going to directly increase adoption. Now, luckily with developer experience, things are a bit more measurable and that’s where hacks or comes in. But even with the numbers, it’s hard to convince the company is that this is a place they should be investing. And we can see that yeah. In the market, the companies who are investing here are doing incredibly well, but it’s amazing to see the companies who, who still don’t believe that this is a place that they need help.

And I think it segues great into the great into the, into some of the trends. Yeah. Just makes perfect sense. And we’ve covered. Yeah, no, a lot of times how, how crucial it is to have a senior level sponsor. Were you developer programming? These also comes in this, wow. In this case with developer experience, can you walk us through about how Huxford does work? Doesn’t what you guys, Luke too, does she, if something will first a good developer experience. So at hacks, sir, we test and measure developer experience. And what that means is we take your onboarding process and we work with developers who record themselves while they get started. You can see a lot from when a developer gets started with your API. You can find out where they get frustrated, where they encounter bugs, you can find any typos they make.

And what hacks her does is exposes ways that you can optimize your experience too, to improve that process. So we work with hundreds of developers who sign onto your site and they go through your typical hello world. Yeah. Companies believe their hello world will take about 15 minutes. But in testing, we find that about half of the developers can’t even get started. And if they do get started, it usually takes them about an hour. So testing is a great way to see how your expectations meet reality and effectively measure how well you’re doing at optimizing your developer experience. So you can go back there and measure it every month, every quarter and see, is it getting better or is it getting worse as we make changes and fix bugs? That’s a huge difference. I think, you know, going from seeing, okay, you should be bringing your head a little world in 15 minutes and you know, you have half the people not even reaching that point and the other half taking four times more the diamond was supposed to be taking them.

Yeah. A lot of developer companies are currently measuring the time from a developer signing up in the portal until they make their first API call. When we record the, what I would call the full experience, which is about 30 minutes before that. And 30 minutes after that, we see that developers typically spend 15 or 20 minutes just reading the documentation before they even sign up, because they want to know it’s going to work. I consider that a part of the developer experience because that’s also an opportunity for them to, to move on or to someone else. Like, it’s amazing because we’ve worked with public, you know, fortune 500 companies and sometimes five out of five developers. I can’t even get through the onboarding because some bug or problem or a typo, and these are, these are massive companies and this is a major part of their strategy, but yet they’ve never seeing that perspective before.

Yeah. That’s a big issue, you know, the way you saved and yeah. Well, the good thing, the way you do listings is that you have, you know, the delivers the, to themselves gives you direct access the whole process from start to finish. Yeah. From what you’ve seen, what is the, the most common things that bring the original companies who do four, but the developer experience, or if you want, what are the, so things the companies are doing good regard with regard to developer experience, as far as commonalities go, every company as bugs. So all the documentation every year, all the companies have bugs and that’s just a guess a GoTo. That’s an E usually the low hanging fruit, they’re easy to fix. Just go have some developers sign on and find out where those typos are another common, but very difficult problem that companies have.

It’s the information architecture. Every company has made its own decisions on the languages to support on APIs. They want to prioritize the products they want to prioritize and organizing those in a way that makes sense for a developer. Getting started that aligns with the business goals is really, really tough. And we’re seeing all sorts of crazy things in, in the market right now, personally, I just was working with a API called open talk, which was bought by Vonage. And they have a tutorial navigator builder, meaning you enter in your use case and it okay. Determines the tutorials that you should be following. And to me, that’s another attempt at solving this problem, which is every year contrary to like a user experience where there maybe there’s a few paths with developer experience. The first person might be working with JavaScript on the web and the next person might be Java on mobile.

And those people both have to have a great developer experience, but it’s rare that a company is going to give them both quality experiences. And there’s a whole, a bunch of reasons I can go into why, but typically it comes down to what the company is prioritizing versus what a developer is trying to get done. So I think a large, a couple of things I’ll say is that a really common problem is information architecture. That a lot of these companies don’t have the information organized in a way. Yeah. It makes sense for the developer experience and a really, really common problem we see is that companies or onboarding documentation we’ll drop a developer off. Yeah. What I’ll call a cliff edge, for example, a download button and they’ll say, download the SDK, but then they don’t offer a guide or like the first step. And so there’s a lot of just dumping these people off in these one way streets where there’s no, there’s no step two.

So that’s a really common thing. And the next one is releasing SDKs, which are unsupported. So there’s, I mean, it’s, it’s an incredible amount of work release and support and document and SDK. And as companies play these numbers games, they’re trying to support, you know, 70, 80 SDKs and all different languages, you know, whatever. But typically it’s better to have a lower number of SDKs and a higher quality have support there and just really target those major platforms where most of your developers are going to be. Anyway, in summary, if you want frictionless onboarding and experience, you should speak to your architecture. We’ll make sure that there were things in place and don’t leave developer service, the case hanging. Was that a good summary? I would say, yeah. Try to, okay. Ways you, and in general, try to find ways that you could be helping the developer who’s onboarding.

So if there’s a relevant article, if there’s a guide that could be in that document that they’re on or that page, you’re looking at, put it in there and let them link them to it. You know, that’s, that’s really what we’re seeing is just, it’s not that the information is not there. It’s, it’s mostly that it’s, it’s not being referenced. If you haven’t some quick things for someone who wants to and better experience another onboarding experience, what would you say that, okay, here are the three or five things I should just, you should do. Let’s leave that. I think number one would be, get the developer’s perspective. Anyone, the company is unable to give you a perspective of somebody new. We have a saying, you can only be new once. Meaning somebody who is already familiar with your lingo in your vocabulary and the API is just not gonna be able to provide you accurate feedback about what it’s like to be a new user.

Number two would be, be consistent. So try to find places where vocabulary changes in between your marketing pages and your documentation. The marketing page might reference a variable is X while the documentation might, you’ve written by an engineer. And that’s where that says the variables called Y. So find those little inconsistencies created by your, your different teams. Next I would say is segment the audience by language and by platform. So try to get developers into a place where they’re working with the language that they’re working with and they don’t see anything else. So if I’m working with web, I don’t want to see anything iOS or Android. I don’t know about that. And that’s just, it’s just crushed. I think a lot of companies struggle with this, or do you see a lot of death portals where this stuff is just all over the place?

Number four would be take a look at that information architecture. So in the same way as your segmenting, look at the organization and make sure there’s a clear path. Okay. Zero to one. Typically we see tutorials being hidden as sort of an afterthought when often they’re the number one way people get started, make sure there’s a place to go after you clicked, download stuff like that. And then finally, I would say, just identify those bugs and typos, because those things can be really frustrating. If you can go through and just test the developer experience with a few developers and have them report when things are inconsistent or wrong, I’m finding bugs and almost all the documentation that we test, we find typos. So there you go. There’s five. I love how you connect the, you know, number one and number two, it’s for someone, you know, who comes across this product and it’s something completely new to them.

They need to first, you know, understand what it does show, as you said that you have your own lingo and you might be using it, but it’s very hard for someone yeah. Neutral at least to connect the dots. Chief, you know, the dots are different or it’s the same dot with two different names. So yeah, I can, I can see there how this could make a huge difference. That’s a really, really good point. Oh, overall, like I think what we’re dealing with is a problem where there’s simply too much information. A lot of, huh. I think that these developer experience teams are drowning on her, all the documentation information that they have, that they’re in charge of. And often these are small teams, so it’s really difficult for them to, to organize and, and deal with this information and curated in a way that makes sense for each experience.

I think that a lot of what we’re seeing is, and a general rule rule for usability is okay, simplicity, right? We want to have, we reduced the amount of information and a developer. I can see that’s irrelevant. So try to get them toward the most relevant information as quick as possible, and also remove the other stuff that’s irrelevant. So, yeah, I think that’s a really good point. It’s a, it’s a really overarching theme, which is we, I don’t think we’ve seen a landscape where there were so much information and so much possibility. And that’s why I think, yes. I think developer experiences just going to be a massive field. It’s unlike anything else where there’s so much Mmm. Every is essentially building their own experience. It’s it’s actually, we rarely have, we rarely have these environments where there’s so much information. Usually we’re desperate to produce information and try to fill in the blanks.

But here it’s like, don’t show me the iOS. Don’t show me the Android. Don’t show me the web. I’m trying to do something for Apple watch, you know, it’s an effort to initiate, to cover all the blanks, but you end up creating more noise than yeah. Being helpful. I mean, I think from, you know, bringing in, okay, mine seem relevant to that. Yeah. I’ve, I’ve seen, and that just, I’m just holding the incidents, but I’ve, I’ve seen some tests where the, you know, the no JS documentation was the landing page. And so even if you tell a developer, Hey, can you try this Java, SDK, they get stuck. They’re going to see that on the homepage as Java JavaScript, they’re going to go and try JavaScript. Cause I think that’s the recommended language it’s but it wasn’t, it just happened to be the one that was the first one.

So like, there’s, there’s all these kinds of things where the way you organize your information architecture, it’s actually recommending to the developer how they should work too. It’s kind of saying like, these are our priorities. And so you want to have your priorities aligned with what the developers, sorry. Yeah, that’s it, that’s a definitely to be the number one priority behind. Yeah, sure. What do you love most about the work you’re doing? That’s a, that’s a good question, too. What I liked the most about it is at the end of the day, we’re improving accessibility. So by making this stuff easier, it’s allowing developers who are the modern day. Creators are one of the, you know, the largest segment of creators right now, bill their own passions and achieve their goals for everybody who gets started with an API for some people that also might be the first time they work with that language.

So I’m getting started with, you know, Twilio and Java. I might have just started working with Java that day. These companies are a part of this learning experience now, and it’s essentially, Mmm. You know, it’s educational content. And so I think that for me, there’s a lot of satisfaction coming out, knowing that we’re saving people time and that these people are working, these developers are helping build their own passion and we’re helping them achieve their own goals together. Right. It’s like me and these companies, and we’re all helping these, these builders work on these dreams they have. And that’s, to me, that’s super cool. And I, the mission of Axar is, is to increase accessibility and reduce the time spent on these duplicate tasks that maybe every developer who’s onboarding is getting stuck on. We’re trying to help innovation. It happened faster, essential, improving accessibility.

I think it’s a noble cause by default too, even if they, you know, in this case that you you’re helping guys, as you said that you’re helping build a list, utilize, you know, the dreams they have, the ideas they have that yeah. Might be, I think that’s good. Change the future. As we know it, [inaudible] speaking of future, where do you see how, or how do you see developer experience growing into, what do you see in the future? I think in the future, we’re going to see a lot more standardization right now. There’s a lot of experimentation into what makes a good developer portal. How should we organize our documentation? How do we segment developers? Like I mentioned, Mmm, how do we measure this stuff? There’s just so much unknown right now. And I think over the next five years, we’ll see a lot more standardization.

And I think one big opportunity for that as a standardization of documentation architecture. So I think that there’s an answer to how to structure documentation. It’s something that I’m definitely researching, but for other things like measurement, I mean, that gets a little, it gets a little case by case basis. I think that we’ll see a lot more resources and I think we’ll see a lot more just to investment in this space. If you look at it the way API APIs went, I mean, API APIs were very small and there wasn’t any standards or practice for a long time for, you know, for like 10 or 10 or 15 years, APS were just, they’re being monetized, but there wasn’t really any, you know, we didn’t have like read me IO and all these other like sort of developer tools and platforms that popped up. But now we do, right.

We have, we have a swagger or, or open API and you just, you know, kind of specs coming out that standardized how things should be done. And those, those are very helpful for, yeah. Okay. We’re reducing the barriers to communication and, and creating standards. I think for developer experience, we’re going to start seeing certain documentation features become standard or certain organizational trends become standard. And I think that’s great because that’ll make things easier to use. Oh, he’s learning developers listening to this. The future cannot come soon enough. Yeah. Yeah. This is a utopic experience that you discussed there, but yeah, right. I think this is where we’re going to end where we, we should go. In our last episode, we introduced a new section in the, these podcast episodes. We called let’s talk. The, what we do is we visit them or legs are definitely, Portella where we fit your free resources. [inaudible]
Pick a graph that stands out to you from our trends page. Then we’ll talk about why it stands out to you. I will add the link to the actual description for our listeners. Have you picked your graph from derelicts.com/trends? I’ve actually two.

Yeah. I really like this report. It’s like, it’s, as I mentioned to you from the last question, it’s hard to find this information and the agencies and companies, and, and I can produce this sort of these sort of metrics and stuff are, are going to be incredibly helpful to making this stuff easier to two to navigate basically. Yeah. I just want to say that this is all these results come from our developer economic survey, which is a survey for developers. And we get around 40,000 tons per year from developers from 165 countries. So wow. What we showcase here and it’s the stuff that we want to share with the world, because we, we want to, wow, well, the word, understand developers and developers understand the world.

Yeah. You can use data too, to get to know the developers, what they want and I’ll address that. So this is why we created the trends page here, which is a free for everyone black says, but we also, you know, want to see how people interpret these data. So yeah. Which one did you pick out? You should? Yeah, that’s good. Sure. So the first one I picked is the developer program, benchmark, Google Mozilla, and Microsoft lead the pack. And this is a X, Y Mmm. The graph on the bottom. It says developer satisfaction. And it goes from negative 20 to 25. A weighted score of program features out of a hundred. And on the Y axis, it says engagement, the percent of program users that used resources, at least weekly. So on the Y we have engagement on the X, we have satisfaction and I’m the top or right.

Which is the most engagement with the highest satisfaction is Mozilla, Google, and Microsoft. When I look at this chart, aye, thank you. No, those are massive companies that have huge budgets to spend on this. So it’s great that they’re allocating some of that money toward developer satisfaction. They’re also companies where developer experience developer. I am helper tools and APIs. It’s a massive part of their strategy. You can see that Microsoft when Google, of course, I think the one thing that struck out to me about this graph overalls, it’s not surprising to me that those, this company is, are up there, but the rest of the company is here and to the far left, which means that unsatisfied, we have Twitter and 10 cent SAP, Alibaba, Baba, and Samsung IBM AMD. I mean, there’s just every other know a fortune 500 company you’ve heard that has, that does developer stuff.

They’re over there on the left. And overall though their range of this chart goes from, even though it’s surveyed out of a hundred to a hundred, it goes from negative 20 to 25. So in general, what it’s telling is most the engagement is really high because the engagement’s about 65%. But when we look at satisfaction, everybody’s leaning from like a rating of negative 22, bye, which is in general telling me that developers are just unsatisfied. And even if you’re Google or Mozilla and you have seven, you know, 70% engagement, you’re only going to get a satisfaction rating of 20. That’s just like, you know, I think that, like that shows you what we’re talking about here. It’s a small range. And I think these, there’s a lot of opportunity on the, to, to get better at this because yeah. A majority of these companies are just have negative satisfaction rating.

Does that make sense that I interpret that, right? Yeah. It makes perfect sense. You know, the data is there to show exactly what you said. And I think it’s a multi indicator of how critical the developer audiences. I found another charter it’s called how developer marketing experts segment their audience. And I really liked that because it shows where marketing experts are targeting there, budget the top number one rated segment is the industry vertical or app category. I think that makes sense. Right? Cause you want to, if you’re a payment processing API, you want to find developers who are payment processing. That makes sense. But then strange, we get geography as a number two segment. And as we go down, we get professional verse hobbyists and then type of development, target audience. So if we look at those three geography, hobbyist, or students and type of development, okay.
It shows these show, the marketing priorities. Right. I think it’s strange that geography is there because either that’s so general that it’s just a given or we’re, we’re seeing people only target developers in San Francisco. I don’t know. I don’t know exactly what that comes down to. Yeah. You know, developers are everywhere. The world is global. And when I work as a developer, I might use a SDK that’s written in Chinese because it was produced by the Ali Baba. And it’s true. I mean, I have used these SDKs and it’s like the domestic in Chinese. Right. So like geography, I think is it’s strange because I don’t, I don’t think it’s actually geography. I think it’s language or maybe it’s spend or something. That’s that one’s a little, I feel outdated. And then, and then professional verse hobbyists or students is interesting as well. Because if we look at the strategies of like Twilio, they were investing in students for years, it seems short term to me to look at only professionals.
I don’t think marketing experts are looking to target hobbyists or students often because they’re looking for people who are spending money. So I think that that’s a disconnect. I think that students and hobbyists are a great place to invest because students in four years, well leave university and there’ll become employees of a tech company where they’re in charge of making some decisions. So the payout might not be immediate. Like it would be with the professional, but there are companies who this is a valid strategy company size again. I mean these, like, I think a lot of these ratings are representing short term outlet because if we’re looking to, if, if I were to look at what the maximum would be for each of these ratings, it would be a large company that was full of professionals doing exactly my kind of development in my own geography.
Those companies are also going to be hard to sell to because they’re probably already invested in some other ones platform. Those, I mean, this could just Chuck and might as well say that Mmm developer marketing experts are targeting enterprise companies, but there’s a lot of opportunity on these other ends because people make technology decisions long before they become enterprise companies. And enterprise companies definitely are obviously huge deals, right. As we know, developer marketing is a grass roots effort. And what this tells me is that what most people are doing is a top down effort. Yeah. That’s how I, that’s what I would say about yes. Yeah. Let’s totally agree. There can just send it in that. Thank you for for this thorough analysis. Okay. Yeah. It’s been great talking to you for our listeners might want to learn more from you, where can they reach you? So you can find hacks there at H a X O R dot S H like a shell script. And you can find more about me@jnnngs.com. My name is Ian Jennings and it’s been great to be on this show. I’m just always excited to talk about developer experience. Okay. [inaudible],

You know, get to know these other agencies in the space, right? It’s been great having you for our listeners. Thank you for listening to under the hood of developer marketing, the podcast, devoted to developer, marketing and relations. If you want more episodes and resources, including industry news, or if you’re looking for a new job, make sure to visit Denver lakes.com and subscribe to our biweekly digest. Follow us on Twitter at SLAs data HQ. Thank you very much, Tim. Thank you.
[Inaudible].