"Can a Robot Replace Your Programmer? ChatGPT Is Trying—And It's Surprisingly Good at It"

2025-10-06

A New Era in Software Development?

Artificial Intelligence is no longer just a buzzword—it’s writing code. Researchers at Mälardalen University in Sweden and the University of L’Aquila in Italy have put ChatGPT, the popular AI chatbot, to the test: can it solve real programming problems as well as humans?

The answer? Yes—but with limits.

In a large-scale experiment, the team fed ChatGPT 240 programming challenges from the well-known coding platform LeetCode. These ranged from simple tasks to complex algorithmic puzzles, written in both C++ and Java. The goal was to see how well ChatGPT could perform compared to human programmers—not just in getting the right answer, but also in how fast and efficiently it could do it.

“We were curious to see how far ChatGPT could go in replacing or assisting human programmers,” says Dr. Alessio Bucaioni, lead researcher at Mälardalen University.

What Did They Find?

• Success Rate: ChatGPT solved 100% of easy and medium-level problems—for hard tasks the success rate was 93%.

• Performance: When it came to runtime and memory usage, ChatGPT lagged behind human-written code, especially for harder problems.

So, Can ChatGPT Replace Programmers?

Not quite. While ChatGPT shows promise as a coding assistant, especially for beginners or repetitive tasks, it still struggles with complex problems that require deep reasoning, optimization, or creativity.

The study also highlights the importance of prompt design—how you ask ChatGPT matters. Simple prompts led to faster code, while detailed ones improved accuracy but hurt performance.

Why This Matters to You

Whether you're a student, a developer, or just someone curious about AI, this research shows how tools like ChatGPT are reshaping the way we think about programming. It’s not just about automation—it’s about collaboration between humans and machines.

How This Research Supports the UN Sustainability Goals

Goal 4: Quality Education

ChatGPT can serve as a learning companion for students, helping them understand programming concepts and practice coding in a supportive environment.

Goal 8: Decent Work and Economic Growth

By automating routine coding tasks, AI tools like ChatGPT can boost productivity, allowing developers to focus on more creative and impactful work.

Goal 9: Industry, Innovation and Infrastructure

By exploring how AI can assist in software development, the research promotes technological innovation and efficient infrastructure for digital industries.

Source material