Mayur Patel
October 27 2025
There are already a number of blogs and posts describing how to set up an AI-assisted development environment. However, because things move so fast and there are so many options, I thought I'd take a moment to describe a solution that works well for me at the time of this writing. I am in no way an expert in any of this, but I hope this brief blog gets you moving in the right direction.
First, I'll describe the moving parts. Then, I'll walk through the installation process. I use Linux Mint 22 as my target platform, but I provide references so you can investigate the differences in process for other operating systems.
The Players
There are lots of options for AI-assisted coding and many employers make such solutions available to their engineers. However, if you're a student or perhaps working on personal projects, there is value in using an open source solution that does not incur a monthly subscription fee. Furthermore, there could circumstances where privacy may be required, so hosting the AI model locally would be one way of insuring that sensitive information does not leave the local development environment.
It is worth noting that there are some free-tier models available from for-pay services. For example, GitHub Copilot has a free-tier plan. Depending on your use case, this might be suitable; but also, you might find yourself hitting limitations on usage.
With all this in mind, I recently set up a new laptop with the following components running locally:
- Ollama, an open source server for hosting AI models.
- Qwen-2.5-coder:7B, an open source AI model that specializes in coding.
- Visual Studio Code, a popular IDE from Microsoft.
- Continue, an extension for VS Code that connects to Ollama to provide the developer with AI-assisted coding features.
Let's go through these one-by-one.
Hosting AI Models
Ollama has quickly become a standard platform for hosting open source AI models. Installation instructions can be found here; but below, I'll describe my process for installing it onto a new laptop running Linux Mint 22.
At the time of this writing, I was able to download and install Ollama with a piped command:
curl -fsSL https://ollama.com/install.sh | sh
I verified the installation with:
ollama -v
With that, I became able to pull new models and run them.
# install models:
ollama serve
ollama pull <model_name>
I could also introspect the server and models on the local host.
# CLI reference: https://docs.ollama.com/cli
ollama ls # list what's downloaded
ollama ps # what's running
ollama rm <model> # remove from local host
ollama stop <model> # stop a running model
With the server ready to go, I needed to decide which models to pull.
Choosing a Model
Thankfully, there are a lot of models available for Ollama. However, choosing one for your specific application requires a bit of study. Furthermore, models are constantly changing, so something that may have been a top performer a few months ago might already be displaced by a new generation of models.
Another consideration is that you need to select models that can be executed on your specific host. With 16GB of RAM, a model with approximately 7B parameters is probably a sweet spot for developers. Many models are available in multiple sizes, so you can select the variant that you can reliably execute with your available hardware. Keep in mind that, without a reasonable GPU, the execution of these models could be quite slow. It is important to have reasonable expectations.
That having been said, this is the research that came up for me for ~7B parameter models that specialize in coding:
| Model (Instruct Variant) | Parameters (B) | HumanEval (Reported) | EvalPlus (HumanEval+) | MBPP+ Score | |||
|---|---|---|---|---|---|---|---|
| Qwen2.5-Coder-7B | 7.0 | 88.4% | ~78.7% | N/A | |||
| CodeGeeX4-ALL-9B | 9.0 | 82.3% | N/A | N/A | |||
| DeepSeek-Coder-6.7B | 6.7 | 78.6% | 71.3% | N/A | |||
| Llama 3.1 8B | 8.0 | 68.3% | N/A | N/A | |||
| CodeGemma-7B-IT | 7.0 | 56.1% | N/A | 54.2% | |||
| CodeLlama-7B-Instruct | 7.0 | 55.0% | N/A | N/A |
From the available information, Qwen2.5-Coder-7B looked attractive to me. I chose to install that for my environment.
Looking through the repository for Ollama models, I came across gsxr/one, which is an implementation of Qweb2.5-Coder-7B. I installed it with:
ollama pull gsxr/one
Setting up the IDE
Microsoft Visual Studio Code is a popular IDE, and it is the one I chose to install for my environment.
Download instructions can be found for many platforms, but I followed the directions specific to Linux Mint.
# Downloaded the .deb file for AMDx64b instruction set.
cd ~/Downloads
sudo apt install ./code_1.105.1-1760482543_amd64.deb
Next, I needed to install the Continue extension to Code, which would allow me to connect to the Ollama server I installed earlier. Here are some detailed instructions. From inside of Code, I easily found and installed the Continue extension.
After the extension was installed, I needed to configure it.
- Select the Continue icon in the VS Code Activity Bar on the left.
- Select the gear icon to configure.
- Select the "Configs" tab.
- Select the gear icon to edit the local configuration.
- Edit the config.yaml to incorporate the model you pulled.
For example, my config.yaml ended up looking like this:
You might choose to configure your installation differently.
To verify the integration, I confirmed that I was able to select my model from the drop-down menu in the Continue chat panel.
What's Next
My experience over the last eight months or so has been mixed with AI-assisted coding. Some things it does very well. Some languages seem to be easier for an AI model than others. You'll need to explore for yourself to find your own path to success, but here are some things that work for me:
- Use simple and clear language, like speaking with a child. Don't be afraid to be redundant or verbose about relationships between entities. Don't worry about pleasantries like "please" or "thank you."
- For coding new features, it might be worthwhile to write a detailed markdown or text document and to have the AI agent read this from your source repository. Good comments also make a difference when working with established code rather than creating a new feature.
- I found that AI agents did really well at writing unit tests, but not as well at writing high quality code in the project itself. Read every line of generated code. Do a critical code review of AI-generated code and don't take anything for granted. AI doesn't test, or even review, its own work. You might be able to coax it into reviewing its own work by asking questions about the code or asking it to evaluate its own code for best practices. But there is no substitution for being thorough with your code review.
I hope this has been helpful and contributes to your productivity.