You Do Not Need a PhD to Become an AI Engineer: The Essential Skill Stack

Something strange happened on LinkedIn’s list of the fastest-growing jobs in the United States this year. The top spot did not go to a nurse practitioner or a renewable energy technician. It went to a job title that barely existed before 2023: AI engineer, ranked first for the second year in a row. The most common skills listed next to that title were LangChain, retrieval-augmented generation, and PyTorch, none of which show up in a typical computer science curriculum.

I have spent over 20+ years building systems that were not allowed to be wrong, including a loan accounting platform whose output fed directly into SEC-reported numbers for major U.S. banks, and Kafka-centric, event-driven platforms processing product data across 29 languages. The engineers I mentor who thrive as AI engineers are rarely the ones with the deepest math background. They understand the right order to learn things in, and why that order matters.

The Engine Builders and the Car Builders

The term “AI engineer” only became common usage in 2023, popularized by a widely read essay arguing that a wide set of tasks once requiring a dedicated research team could now be handled by a single builder with solid API documentation and a free afternoon. Andrej Karpathy, one of the most respected researchers in the field, publicly agreed with that framing.

“One can be quite successful in this role without ever training anything.”
Andrej Karpathy, post on X, 2023

That distinction tells you exactly where to spend your energy. A machine learning researcher trains foundation models from scratch, publishes papers on new architectures, and usually needs an advanced degree in a math-heavy field. An AI engineer takes a model that already exists, whether a frontier model from a major lab or an open-source release, and wires it into something people can actually use. If the researcher builds the engine, the AI engineer builds the car around it. Right now there are far more engine designers than reliable car builders, which is exactly why this role has outpaced almost everything else in tech hiring.

The Boring Foundation Nobody Gets to Skip

The engineers who struggle later almost always skipped the same starting point. They jumped into building an agent before understanding the plumbing underneath it, then spent months quietly relearning fundamentals while their project stalled.

Tier one has nothing to do with AI directly and everything to do with being able to work at all. You need enough Python to read what a coding assistant just wrote and catch it when it is wrong, since most of the libraries an AI engineer touches, from PyTorch to the client libraries for frontier models, run on Python underneath. You need git, because nothing you build stays on your own laptop for long. You need real comfort with the command line and Linux, since almost every model you will ever call is running inside a Linux container somewhere. And you need a genuine grasp of APIs, because this entire discipline is, structurally, a long chain of well-designed API calls between your application, a model, and a set of outside tools.

That same rigor carries straight over from finance. Years ago, on a loan accounting platform whose numbers fed SEC filings for major banks, a badly handled timeout or a silent edge case could corrupt a figure a regulator would eventually read. The model call in an AI system feels new, but the plumbing around it — the retries, the error handling, the logging — is the same unglamorous craft that has kept serious systems honest for decades.

Teaching a Model to Stop Guessing

Once the foundation holds, tier two introduces AI-specific skills, and the first is retrieval-augmented generation, usually shortened to RAG.

A large language model, or LLM, only knows what it saw during training. Ask it about your company’s return policy from last month or a contract that closed yesterday, and it will either admit it does not know, or worse, guess confidently and get it wrong. RAG solves this by giving the model somewhere to look before it answers, the same way a sharp new hire checks the employee handbook instead of bluffing through a question about vacation days.

The mechanics run through embeddings, which turn text into numerical vectors so a system can search by meaning instead of exact keyword matches. A document mentioning containers and orchestration will surface for a query about Kubernetes even though the words never match exactly. The system chunks documents, embeds them, and stores them in a vector database. When a question comes in, the system retrieves the most relevant chunks and hands them to the model alongside the original question, so the answer comes back grounded in something real instead of invented from nowhere.

This is not a fringe technique. The research behind it was published by Patrick Lewis and his co-authors in 2020, describing:

“models which combine pre-trained parametric and non-parametric memory for language generation”
Patrick Lewis et al., Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks

In plain language, that means combining what the model already learned with a live lookup into a verified outside source. In my own work on translation and product enrichment pipelines spanning 29 languages, that principle — ground the answer instead of trusting memory — is the difference between a system that clears 90 percent precision and one that quietly embarrasses the business in front of a customer.

From Answering Questions to Getting Things Done

RAG makes a model well informed. Agents make it useful in a different way, by letting a system move from answering a question to actually doing something about it.

There is a real distinction worth holding onto here, one Anthropic’s own engineering team drew clearly in its published guidance on building agents. A workflow follows a fixed sequence of steps decided in advance by whoever wrote the code. An agent behaves differently. It can look at what just happened, decide what to do next on its own, call a tool, observe the result, and loop again until the task is genuinely finished.

“systems where LLMs dynamically direct their own processes and tool usage”
Anthropic, “Building Effective Agents”

I think about this the way I think about event-driven systems built on Kafka. A workflow behaves like a batch job on a fixed schedule: step one runs, then step two, then step three, regardless of what the data looks like. An agent behaves more like a Kafka consumer sitting on a topic, reacting to whatever state the world is actually in and deciding what to do next. The value is not the intelligence of any single step. It is the reliability of the loop under real, messy conditions, at scale, without a human watching every cycle.

This is also, right now, the single most in-demand skill in Applied AI. Building a loop that holds up under pressure, rather than falling apart the moment a tool call fails or a model returns something unexpected, is what separates a good AI Engineer from someone who has only ever shipped a demo.

Getting It Off Your Laptop

None of this creates real value until it leaves your laptop and reaches an actual user, which is where tier three comes in.

Containerization and orchestration, usually Kubernetes, let you package a model or an agent so it moves across environments without breaking. Observability lets you see why an agent made a decision, which matters when something goes wrong in production and a stakeholder wants an explanation instead of a shrug. Monitoring keeps token spend and security exposure in check, since an agent with unmonitored access to tools and a live API key is a genuine risk, not a line item to worry about later.

I led a shift from waterfall delivery to Agile practice across teams in multiple countries, and the lesson that carried over cleanly into AI systems is that shipping fast without visibility is not shipping fast. It is borrowing speed from your future self, and the interest rate is brutal once an ungoverned agent causes an incident nobody can explain.

The Code Was Never the Hard Part

Here is the part that should reframe how you think about all three tiers together. Writing code is no longer the bottleneck it used to be. Eighty percent of new developers on GitHub now use Copilot within their first week on the platform, according to GitHub’s own Octoverse 2025 report, and that share keeps climbing.

What AI coding tools cannot do for you is decide how a system should be structured, what to build in the first place, or why one architectural approach beats another for your specific situation. That judgment is exactly what a classroom struggles to teach, because it only really develops by building real things, watching them fail in production, and fixing them under pressure. It is also exactly why these three tiers belong in this order. Skip the foundation, and you will spend a year quietly relearning it while your agent falls over in ways you cannot explain. Build it properly, and the RAG pipelines, the agent loops, and the deployment pipeline all become far easier to reason about.