13 Using Ollama for Development

13.1 Overview

This chapter describes how to configure a local large language model (LLM) development environment using Ollama, Podman, Positron, Continue, and Aider.

The workflow described in this chapter allows developers to use AI-assisted development without connecting source code to external AI services. All inference is performed locally on the developer workstation.

This workflow is intended to support:

R package development
Python development
Shiny application development
Quarto documentation development
Software architecture analysis
API refactoring
Technical documentation generation

13.2 Prerequisites

Before continuing, complete the following chapters:

Positron Getting Started
Podman Desktop
Podman Desktop Getting Started

You should also verify that:

Podman Desktop is installed and functioning.
Podman can run Linux containers.
Positron is installed.
Git is installed and configured.

This workflow assumes the following platform for GPU-accelerated local inference:

Windows host workstation
NVIDIA GPU with current Windows NVIDIA drivers
Podman machine running on WSL2
Git Bash or another Bash-compatible shell for Bash examples

If the workstation does not provide an NVIDIA GPU, or if Podman GPU integration is not configured, Ollama may run in CPU-only mode and model responses may be significantly slower.

13.3 Architecture

The development environment consists of the following components:

Podman
└── Ollama
    └── Local LLM Models

Positron
├── Continue (VS Code Extension)
└── Local source code repositories

Terminal
└── Aider

Component Roles

Ollama

Ollama provides local LLM inference. Both Continue and Aider communicate with Ollama through a local HTTP API. Runs in a local container hosted in Podman Desktop.

No cloud services required
Models run locally
Direct file system access
Allows access to larger contexts (e.g., repo contents)

Positron

Positron serves as the primary Integrated Development Environment (IDE) for:

R development
Python development
Quarto documentation
Git repo maintanence operations

This approach depends on VS Code Extensions for Positron described in the next sections.

Continue

Continue - open-source AI code agent provides:

Code explanation
Code generation
Documentation assistance
Architectural discussions
Interactive development support

Continue operates directly in a chat window inside Positron.

Aider Composer

Aider Composer provides:

Multi-file refactoring
Repository-wide code modifications
Git-aware code changes
Feature implementation assistance

Aider is executed from a terminal within the repository.

13.4 Verify Host GPU Availability

Before configuring the container, verify that the Windows host can see the NVIDIA GPU.

In PowerShell, run:

nvidia-smi

A successful result should show:

the installed NVIDIA GPU,
driver version information,
available GPU memory.

If this command fails, resolve the host GPU driver installation before continuing.

13.5 Verify the Podman Machine

This workflow depends on the Podman machine running as a Linux environment on WSL2.

In PowerShell, run:

podman machine list
podman info

Confirm that the Podman machine is running before continuing.

13.6 Install NVIDIA Container Toolkit in the Podman Machine

GPU access inside Podman containers requires NVIDIA container tooling inside the Linux environment that backs Podman. In the current Fedora-based Podman machine environment, install the toolkit with:

In PowerShell, run:

podman machine ssh "dnf install -y golang-github-nvidia-container-toolkit"

Verify that nvidia-ctk is available:

podman machine ssh "command -v nvidia-ctk"

Expected output should include:

/usr/bin/nvidia-ctk

13.7 Generate the NVIDIA CDI Configuration

Podman uses Container Device Interface (CDI) device definitions to expose the host GPU to Linux containers.

Generate the CDI configuration:

podman machine ssh "mkdir -p /etc/cdi && nvidia-ctk cdi generate --output=/etc/cdi/nvidia.yaml"

Verify that Podman can resolve the GPU CDI device:

podman machine ssh "nvidia-ctk cdi list"

Expected output should include:

nvidia.com/gpu=all

If nvidia.com/gpu=all is not present, do not continue. Resolve CDI configuration before creating the Ollama container.

13.8 Create the Ollama Container

Remove any previous Ollama container before creating the GPU-enabled container.

In PowerShell, run:

podman rm -f ollama

In Git Bash, run:

podman run -d \
  --name ollama \
  -p 11434:11434 \
  -v ollama:/root/.ollama \
  --device nvidia.com/gpu=all \
  --security-opt=label=disable \
  --restart=always \
  ollama/ollama

In PowerShell, use:

podman run -d `
  --name ollama `
  -p 11434:11434 `
  -v ollama:/root/.ollama `
  --device nvidia.com/gpu=all `
  --security-opt=label=disable `
  --restart=always `
  ollama/ollama

Verify that the container is running:

podman ps

Example output:

CONTAINER ID  IMAGE                  STATUS
xxxxxxxxxxxx  ollama/ollama:latest   Up

13.9 Verify GPU Access Inside the Container

Confirm that the running Ollama container can access the GPU.

In Git Bash or PowerShell, run:

podman exec -it ollama bash -lc "nvidia-smi"

A successful result should show:

the NVIDIA GPU,
nonzero or available GPU memory,
and, during active inference, the Ollama inference process.

If nvidia-smi does not work inside the container, do not continue to model setup. Recheck the Podman machine CDI configuration.

13.10 Verify the Ollama API

Test that the Ollama service is available.

In Git Bash, run:

curl http://localhost:11434/api/tags

In PowerShell, run:

Invoke-RestMethod -Uri "http://localhost:11434/api/tags" -Method Get

A JSON response indicates that the service is functioning correctly.

13.11 Download a Model

Enter the container:

podman exec -it ollama bash

Download a model:

ollama pull qwen3:8b

Model downloads may take several hours depending on network conditions.

List installed models:

ollama list

Example output:

NAME       SIZE
qwen3:8b   ...

13.12 Test Model Inference

Verify that the model can generate responses.

In Git Bash, run:

curl http://localhost:11434/api/generate -d '{
  "model":"qwen3:8b",
  "prompt":"Write an R function that buffers sf geometries.",
  "stream":false
}'

In PowerShell, run:

$body = @{
  model  = "qwen3:8b"
  prompt = "Write an R function that buffers sf geometries."
  stream = $false
} | ConvertTo-Json

Invoke-RestMethod `
  -Uri "http://localhost:11434/api/generate" `
  -Method Post `
  -ContentType "application/json" `
  -Body $body

To confirm that inference is using the GPU, monitor GPU activity on the host while a prompt is running:

nvidia-smi

During active inference, GPU memory usage should increase and the Ollama inference process should be visible. A successful response confirms that local inference is functioning.

13.13 Install Continue

Open Positron, open the Extensions panel, and search for Continue.

If the extension is available in the Extensions panel, install it there.

If the extension is not available in the Extensions panel, use the manual installation method below.

This workflow has been validated on a Windows workstation using a portable user-level Positron installation suitable for IT-managed environments without administrator access.

## Install Positron Extensions
latest_version_url <- "https://open-vsx.org/api/Continue/continue/win32-x64/1.3.38/file/Continue.continue-1.3.38@win32-x64.vsix"
latest_version_filename <- basename(file.path(latest_version_url))

downloaded_vsix <- tempfile(latest_version_filename, fileext = ".vsix")

download.file(
  url = latest_version_url,
  destfile = downloaded_vsix,
  method = "libcurl",
  mode = "wb"
)

system(
  paste("positron --install-extension", shQuote(downloaded_vsix))
)

On some Windows workstations, certificate revocation checks may prevent the download from succeeding. If this occurs, run:

Sys.setenv(R_LIBCURL_SSL_REVOKE_BEST_EFFORT = "TRUE")

Then retry the download.file() step above. Restart Positron if prompted. Continue may take about a minute to initialize the first time Positron starts after installation.

13.14 Configure Continue

Configure Continue to use the local Ollama service.

On the validated Windows workstation configuration, the Continue configuration file is located at:

C:/Users/<username>/.continue/config.yaml

A minimal local Ollama configuration that has been validated for this workflow is:

name: local-ollama

models:
  - name: qwen3
    provider: openai
    apiBase: http://localhost:11434/v1
    apiKey: dummy
    model: qwen3:8b

This configuration uses Ollama’s local OpenAI-compatible API endpoint.

The Continue configuration format may change over time. If the schema differs in a future release, consult the current Continue documentation and adapt the equivalent local Ollama settings.

13.15 Test Continue

Open Continue inside Positron.

Confirm that:

Continue opens successfully.
The configured local model is available.
No network or authentication errors are displayed.

Start with a simple prompt to confirm that Continue can reach the local Ollama endpoint.

For example:

Reply with the single word: connected

Then test Continue against the active file context.

In a source file, ask Continue:

Describe @Current File

A successful result should confirm that:

Continue can communicate with the local Ollama service.
Continue can access editor context.
The configured local model responds correctly inside Positron.

13.16 Install Aider

Install Aider using the method appropriate for your workstation.

Verify installation.

aider --version

13.17 Configure Aider

Configure Aider to use the local Ollama endpoint.

Example configuration:

model: openai/qwen3
openai-api-base: http://localhost:11434/v1
openai-api-key: dummy

Refer to the Aider documentation for the current configuration format.

13.18 Test Aider

Open a terminal within a Git repository.

Start Aider.

aider

Test a simple refactoring request.

Example:

Rename function calc_buffer() to compute_buffer() throughout the repository.

Verify that:

Aider identifies affected files.
Proposed changes are displayed.
Git changes are correctly generated.

13.19 Recommended Workflow

Continue

Use Continue for:

Understanding existing code
Reviewing architecture
Drafting documentation
Planning refactors
Exploring implementation options

Aider

Use Aider for:

Multi-file edits
API migrations
Large refactorings
Coordinated code changes
Repository-wide updates

Ollama

Use Ollama as the shared local inference service for both tools.

13.20 Troubleshooting

Ollama API Not Available

Verify that the container is running:

podman ps

Verify that the API is reachable.

In Git Bash:

curl http://localhost:11434/api/tags

In PowerShell:

Invoke-RestMethod -Uri "http://localhost:11434/api/tags" -Method Get

Restart the container if necessary:

podman restart ollama

GPU Device Not Available in Podman

Verify that the host GPU is available:

nvidia-smi

Verify that the Podman machine can resolve the CDI device:

podman machine ssh "nvidia-ctk cdi list"

Expected output should include:

nvidia.com/gpu=all

If the CDI device is missing, regenerate the CDI configuration:

podman machine ssh "mkdir -p /etc/cdi && nvidia-ctk cdi generate --output=/etc/cdi/nvidia.yaml"

Verify GPU visibility inside the container:

podman exec -it ollama bash -lc "nvidia-smi"

If the container cannot run nvidia-smi, recreate the Ollama container with:

--device nvidia.com/gpu=all
--security-opt=label=disable

Model Not Found

List installed models:

podman exec -it ollama ollama list

Download the required model if it is not present.

Continue Cannot Connect

Verify API compatibility:

In Git Bash:

curl http://localhost:11434/v1/models

In PowerShell:

Invoke-RestMethod -Uri "http://localhost:11434/v1/models" -Method Get

Review Continue configuration settings.

Aider Cannot Connect

Verify API compatibility:

In Git Bash:

curl http://localhost:11434/v1/models

In PowerShell:

Invoke-RestMethod -Uri "http://localhost:11434/v1/models" -Method Get

Review Aider configuration settings.

Bash and PowerShell Command Differences

Several commands in this chapter use Bash syntax.

On Windows workstations:

Use Git Bash for Bash-style curl examples and multi-line commands with \.
Use PowerShell-specific examples where provided.
Do not assume that curl in PowerShell behaves the same as curl in Bash.

13.21 Current Limitations

This workflow is currently under evaluation.

Known areas requiring additional testing include:

Continue integration details
Aider integration details
Repository-scale refactoring performance
Large-model performance
Validation on additional Windows workstation configurations
Validation on non-Windows host operating systems

GPU acceleration with Podman has been validated for a Windows workstation using:

WSL2-backed Podman machine
NVIDIA GPU
NVIDIA container toolkit installed in the Podman machine
GPU visible inside the Ollama container via nvidia-smi
CDI device configuration generated with nvidia-ctk