ChatGPT has been the talk of the developer community ever since it was released to the public as a research preview at the end of last year.

The tool, developed by OpenAI and trained off its GPT-3.5 model, is an AI-based conversational chatbot that is actually quite impressive in its capabilities. 

It enables you to ask it nearly any question and then it will generate a response. I’ve made use of it in a number of ways since it came out, such as rewording text for me to share on social media, and even in coming up with interview questions for this very story. It’s easy to understand how it can take on some of the tasks I do every day as a writer; after all it is a chatbot. But where it’s really impressive is when you consider what it can do in the software development space.

For example, you can ask it to write a piece of code and then continue to ask it to refine what it comes up with until you’re happy with it, and then even ask it to transform that piece of code into another language. 

Another sample use case highlighted on OpenAI’s website is using it as a way to ask why code is not working as expected. 

 

Cody DeArkland, head of developer relations at feature management company LaunchDarkly, said that because it is so good at giving explanations it can be a really good learning tool. “One of my favorite parts of it is being able to get a deeper understanding about what code is doing,” he said.

ChatGPT remembers previous things you’ve said to it in a conversation, unlike other AI assistants where you ask a question and get a response and then the conversation is done. This enables you to get a response, then ask more questions. You can ask it to explain what it just told you, or even ask it to fix a part of what it gave you if it wasn’t quite what you were looking for. 

“I was writing a login component for a project the other night, and I just wanted a quick example  that I could then build off of and change for my own uses,” DeArkland said. “So I went into ChatGPT and said, ‘Can I have a login component written in React?’ … and it returns that back.” 

It may have required some customizations to make it fit his particular use case, but the foundation was there, he explained. 

DeArkland said that where this gets really interesting is when you’re looking for something in a framework you’re not familiar with. He could take the React component from the example above and ask it to switch it to Flask within Python instead, for example. 

“I’ve been writing code for 10 years now, but there’s still things I’m learning. And when I want to learn more, or I want to understand more about a different part of this, my existing workflow today is go and hit Google and look around and find interesting blog posts or find a StackOverflow guide about it. Being able to use ChatGPT to learn from is something I’ve seen a lot of peers in the industry doing.”

DeArkland sees this as a way to lower the bar to entry for those entering the field of development because it will not just provide you with the code but enable you to ask questions like why the component was chosen, why certain directives were used in the code, or ask it to explain what a line of code does. 

One limitation of this, however, is that according to OpenAI’s website, ChatGPT was “fine-tuned from a model in the GPT-3.5 series, which finished training in early 2022.” This means that it doesn’t always know the latest information about the technologies you’re asking about, and it doesn’t know about new technologies either.

According to DeArkland, asking about a technology it doesn’t know much about will result in limited answers because it simply doesn’t have enough data to learn from to give a really good answer. 

“I think the timing thing is going to be a challenge, especially in fast-moving industries, where new features, new concepts, and new innovations are coming out. It’s going to have to learn how to manage those. And maybe we have to learn how to manage our expectations for what it knows,” said DeArkland. 

ChatGPT isn’t taking your job

When you start playing around with ChatGPT and see what it is capable of you might start to worry if it could take over the role of developers. It seems that at this point in time, the answer is no. 

“There’s always going to be a place for us to understand what our context is, what our business logic that we need to have in there is, what design choices we have, all of those things. Humans are safe; we still have a place in a ChatGPT world,” said DeArkland. 

And while it is really good at writing code, that isn’t all a developer does all day. “When you go to college and you learn to write code, all you’re doing is writing code, and that’s all you’re ever evaluated on. But when you get into the real world, that’s only 20% of what you do. The rest is config, design, architecture, etc. ChatGPT can’t help with any of that,” he said. 

Marcus Merrell, VP of tech strategy at testing company Sauce Labs, believes that not having an API to access ChatGPT through is another limitation to its usefulness. 

He explained that at the end of the day it’s just a chatbot, meaning you still have to copy + paste whatever information it gives you to your code environment.  

“I can’t tell it, okay, you zip up all this code and send it to me in a GitHub repo, or go scan that GitHub repo and tell me how I can replicate that, fork it. It can’t do anything yet. And I think that’s what we’re probably all waiting for. That’s when it might really start to threaten some stuff,” said Merrell.

Another reason why it won’t be taking the role of developers anytime soon is because there still needs to be verification that what it’s giving you is what you intended. “ChatGPT is biased to provide an answer, not necessarily a correct answer,” said Kimen Warner, VP of product management at B2B marketing company Drift.

Speaking in the context of marketers, Warner predicts that while there may be adoption of ChatGPT for generating content options, there still needs to be a human in the loop to edit and approve its suggestions. 

“The way I think about it is it’s much easier to edit something rather than to create something from scratch,” she said. “And I think that’s what ChatGPT and other language models like it will let you do. So I think that’s probably where we’re going to see adoption for a while.”

It is turning heads though

While Merrell wasn’t worried about the idea of ChatGPT replacing developers, he said it was the first tool that he really felt he had to pay attention to. 

During his career he has evaluated 18 or 20 different low-code testing tools in the market, and he hasn’t felt the need with any of them to use to further his career, until now. 

“This one’s the first time I felt like I need to pay attention to this. It’s not going to replace my job. But it would help me in getting some things done,” he said. 

Merrell recounted a recent conversation with a colleague where he pointed out that the thing that “ChatGPT has generated the most of is hot takes about ChatGPT.”