AI at Your Desk: A Practical Setup for Daily Work

Most people using AI at work are stuck copy-pasting into a chat window in a browser tab, switching context every time they want help. That works for occasional questions, but it falls apart once you're using AI several times an hour across coding, writing, and research. This guide covers a setup that keeps AI close to where you already work, with an eye on cost, privacy, and speed.

Pick the right model for the right job

Not every task needs GPT-4-class reasoning. If you're summarizing a meeting transcript or rewriting an email, a smaller, faster model does the job in a fraction of the time and cost. Run something like Llama 3.1 8B or Mistral 7B locally through Ollama for quick, low-stakes tasks:

ollama pull llama3.1:8b
ollama run llama3.1:8b "Summarize this in 3 bullet points: ..."

Reserve the bigger hosted models (GPT-4o, Claude 3.5 Sonnet) for tasks that need real reasoning: debugging a gnarly stack trace, drafting a technical proposal, or reviewing architecture decisions. Splitting work this way cuts your API bill noticeably if you're paying per token, and local models respond in milliseconds versus seconds for round-trip API calls.

Wire AI into your editor, not your browser

If you write code, the highest-leverage move is putting AI directly in your IDE. VS Code with the Continue extension lets you point at either a local Ollama model or a cloud API key, and you can select code and ask questions inline instead of retyping context into a chat window.

{
  "models": [
    {
      "title": "Local Llama",
      "provider": "ollama",
      "model": "llama3.1:8b"
    }
  ]
}

For terminal work, tools like aichat or a simple shell function wrapping the OpenAI CLI save you from opening a browser at all:

function ask() {
  curl -s https://api.openai.com/v1/chat/completions \
    -H "Authorization: Bearer $OPENAI_API_KEY" \
    -H "Content-Type: application/json" \
    -d "{\"model\":\"gpt-4o-mini\",\"messages\":[{\"role\":\"user\",\"content\":\"$1\"}]}" \
    | jq -r '.choices[0].message.content'
}

Call it with ask "explain this regex: ^(?=.*[A-Z]).{8,}$" and get an answer without leaving the shell.

Keep sensitive data off third-party servers

Before you paste anything into a cloud model, ask whether it contains customer data, credentials, or proprietary code your company doesn't want leaving its network. Many teams set a simple rule: anything that touches internal codebases or PII goes through a local model or an enterprise-tier API with a zero-retention agreement, and everything else can go to a consumer-tier tool.

Ollama running on a machine with a decent GPU (a single RTX 4070 or better handles 7B-13B models comfortably) covers most day-to-day drafting and code review without a single byte leaving your laptop. For heavier local work, quantized models (Q4_K_M or Q5_K_M) trade a small amount of accuracy for speed and lower memory use.

Build small scripts instead of repeating prompts

If you find yourself typing the same prompt structure over and over, turn it into a script. A daily standup summarizer that pulls from your git log is a good starting point:

git log --since="yesterday" --author="$(git config user.name)" --oneline | \
  ask "Turn these commits into a 3-sentence standup update"

This kind of small automation compounds. Five minutes saved per day across a team of ten adds up to real hours over a month, and it removes the friction that makes people avoid using AI tools in the first place.

Watch for the failure modes

AI at your desk still gets things wrong in specific, predictable ways: it fabricates function names that don't exist in your codebase, misreads indentation-sensitive code, and confidently gives outdated library syntax. Always run generated code before trusting it, and treat any factual claim (a CVE number, a library's default behavior, a deprecation date) as something to verify independently rather than take at face value.

The goal isn't replacing your judgment with a model's output. It's cutting the time between having a question and getting a useful first draft of an answer, so you spend more of your day on the parts of the work that actually need a human.

If you want to go further with local models, prompt design, or scripting your own tools around them, check out the AI and Scripting segments over at Korra Studio.