Exploring Local AI: What self-hosted models can and can’t do for OpenEdge developers

At PUG Challenge Amsterdam 2026, I showed what happens when you run a language model on your own machine instead of in the cloud, and where its work still needs checking.

Most developers have used an AI assistant by now. ChatGPT, Copilot, Claude and Gemini have become part of everyday work, and with them comes a question many companies have not fully answered: what happens to the code we paste into them?

At PUG Challenge Amsterdam 2026, I explored one answer in my session “Exploring Local AI: Capabilities and Limits of Self-Hosted Models for Development and Documentation”. The idea is simple to state: run the model yourself, on your own hardware, so that prompts and code never leave your machine. The reality, as the session showed, is more nuanced. Local models are genuinely useful, but only if you understand what they need and where they fall short.

I prefer to own the tools I rely on. A language model should be no exception.

 

Same AI, different owner

 

A local model is not a different kind of AI. Like any cloud service, it is made of three pieces: the model itself, the application you talk to, and the machine that does the computing. With a cloud service, all three belong to the provider. With a local setup, all three belong to you.

The model is an open-weight file: a set of trained parameters that an AI lab has published for anyone to download. Meta (Llama), Google (Gemma), Alibaba (Qwen), Mistral and OpenAI (gpt-oss) all publish such models, most of them distributed through Hugging Face. The application is a desktop tool such as LM Studio or Ollama, which loads the model into memory and offers a chat window and a local API. The machine is your own laptop or desktop.

The trade-off is clear from the start. Open-weight models are smaller than the cloud flagships, so their answers are weaker on hard problems and slower on modest hardware. In return, they are private, carry no per-use fees, work offline, and stay exactly the version you chose.

 

Local, subscription, or both

 

I was careful not to present local AI as a replacement for subscriptions. In an honest comparison, subscriptions win most categories: answer quality, speed, setup and editor integration. Local wins decisively on one: privacy. For companies that handle customer or financial data, that single category often decides everything.

My recommendation was to use both: local models for code and data that cannot leave the building, a subscription for everything else. One practical pattern combines them. A strong cloud model plans a task as a series of small, precise steps; a local model implements each step; the developer and the compiler verify the result. Planning benefits from the best reasoning available, while small, well-specified steps are exactly where local models perform best. The caution is that the planner sees whatever it is given, so for sensitive projects it should receive specifications and function signatures rather than the code itself.

“Local” does not have to mean a laptop, either. A single GPU server inside the company network can serve a whole team with much larger open models than any laptop can run. Products such as GitLab Duo Self-Hosted and Tabnine Enterprise already work this way, and the University of Victoria has run a small internal Ollama server for its developers. For an OpenEdge shop that already hosts its own databases and application servers, this may be the most natural starting point.

 

The one rule: it has to fit

 

The most practical part of the session was about hardware. Everything follows from one rule: the model file, plus its working memory, plus everything else running on the machine, has to fit in memory. If it does not, the system starts swapping to disk and performance collapses. Three ideas make that rule usable.

Parameters. A model’s parameters are the billions of numbers it learned during training. A model with 35 billion parameters, stored at 16 bits each, needs about 70 GB: more than almost any workstation has to spare.

Quantization. To make models fit, their parameters are stored with fewer bits, much like rounding. A value such as 0.6872 might become 0.687 at 8 bits, 0.69 at 4 bits and 0.7 at 3 bits. Four bits is the usual compromise; below three, quality drops quickly. Code is less forgiving than prose, because a single wrong character breaks it.

Mixture of experts. Some models, marked with names such as “35B-A3B”, use only a fraction of their parameters for each word they produce. Such a model runs at roughly the speed of a 3-billion-parameter model, but all 35 billion parameters still have to be loaded. In short: you pay memory for 35B, but wait as if it were 3B.

Where the model runs matters as much as whether it fits. I tested the same model, OpenAI’s gpt-oss-20b, which takes about 12 GB, on two of his own machines.

 

Machine Memory for the model Result with gpt-oss-20b
Laptop: Intel i5, 32 GB RAM, no graphics card About 17–20 GB of system RAM, shared with everything else Fits; 2 to 3 minutes per answer
Desktop: Radeon RX 7900 GRE, 16 GB About 14 GB of fast graphics memory Fits entirely on the GPU; 10 to 14 seconds per answer

 

On the desktop, the model, its working memory and Windows together used 14.9 GB of the card’s 16 GB: it fits, but only just. On the laptop it fits too, but system RAM is many times slower than graphics memory, and every word the model writes means reading the model from memory again. Hardware sets the ceiling; knowing what fits is how you reach it.

 

What the demos showed

 

Rather than relying on live demos, Alexandru showed recorded runs of gpt-oss-20b on realistic OpenEdge tasks, marking each part of every answer as either correct or as wrong, ambiguous or borrowed from SQL.

Reading a schema. Given four tables from the sports2000 sample database, the model did well. Every description used only fields from the excerpt, and every relationship was right, from Customer to Order to OrderLine to Item, including OrderLine’s composite key. The one flaw was revealing: it labelled the links as primary and foreign keys. That is correct for a SQL database, but the ABL schema does not declare those links as foreign keys, and ABL does not enforce them. It was the first of several moments where the model’s SQL habits showed through.

Designing a REST API. In the same conversation, the model designed a read-only orders API for a customer portal. The structure was clean, and the security design, the part that mattered most, was right: the customer’s identity comes from the login token and is compared with the customer number in the URL, any mismatch is rejected, and a server-side filter adds a second line of defence. Yet the answer contradicted itself on paging. Its rules said pages start at zero, while its own example treated page one as the first page. Each half read well on its own; only a reviewer comparing the two would notice.

Documenting code. In a fresh session, the model received eight lines of ABL that total an order’s lines, and returned a 50-line header comment in about ten seconds. Everything it read from the code itself was correct. The mistakes appeared where it added general knowledge. It advised resetting an OUTPUT parameter before each call, one sentence after correctly explaining why that is unnecessary. It implied that runtime errors always reach the caller, which in ABL depends on the error-handling settings. And its description of NO-LOCK suggested that uncommitted changes are invisible, when in OpenEdge a NO-LOCK read can see them. All three were written with the same confidence as the correct parts.

For comparison, the same prompt went to a cloud model, Claude Opus 5.5, which got all three points right. That is the trade in a single example: the larger model knows more about a niche stack, but the code leaves the building.

Laptop versus desktop. The final demo ran the same three prompts on both machines. The desktop answered all three in 38 seconds. The laptop needed seven and a half minutes, about twelve times as long. Same model, same prompts, same settings: the only difference was the hardware.

 

Know the limits, and stay safe

 

The demos pointed to four limits that matter in daily work. Local models retain less context, so long procedures and large schemas fall out of view. Multi-step reasoning is where errors cluster. Niche languages such as OpenEdge ABL appear far less often in training data than Python or Java, so the model guesses more, without signalling that it is guessing. And larger models demand serious hardware.

From these, I proposed five rules before generated code touches a real project:

  • Compile and test everything. Generated code goes through the same pipeline as human code.
  • Review it like a junior developer’s work. A useful draft, never the final word.
  • Pin models and prompts. The same model and the same instructions give reproducible results.
  • Check licences and logs. Model licences vary, and local does not automatically mean private: applications can still log prompts.
  • Keep your own skills sharp. The goal is leverage, not dependence.

 

Getting started

 

For developers who want to try it themselves, the session closed with a plan for a first hour:

  1. Check your memory: graphics memory if you have a graphics card, system RAM if you don’t, minus what your OS, IDE and database need.
  2. Install an application such as LM Studio or Ollama, and keep it updated: new models often need a newer version.
  3. Pick a model that fits with a few gigabytes to spare, rather than trusting the “recommended” download.
  4. Set the basics: a 16K context, the publisher’s recommended temperature, and a system prompt naming your language version and conventions.
  5. Test with questions you already know the answers to, to learn where the model slips.
  6. Compare with a subscription by running the same non-sensitive task through both.

 

Final Thoughts: Drafting, yes. Deciding, no.

 

The conclusion was neither enthusiastic nor dismissive. Given good context, a local model can explain a schema, sketch an API and draft documentation, all on code that never leaves the company. What it cannot do is replace the compiler, the tests, or an experienced developer who knows the stack. Anything specific to OpenEdge, anything that sounds right for SQL, and anything that contradicts itself still needs a human to check it.

Or, in the session’s closing words: drafting, yes; deciding, no.

 

 

 


 

Author: Alexandru Trasca, Senior Developer

Alex is a Senior OE Developer with a strong passion for technology and problem-solving. Always ready to lend a hand, Alex is not only a talented developer but also a supportive mentor to his teammates. When he’s not coding or solving complex technical challenges, you’ll likely find him rewatching his favorite movie—Lord of the Rings.

SEE HOW WE WORK.

FOLLOW US