All videos
0:00 / 0:00
ai

When to choose CPU vs GPU: Databricks AI Runtime Explained

Databricks16 June 2026Watch on YouTube

Description

The conversation around GPUs has shifted this year. It used to be about training models from scratch. Now it is about token economics. Here is the simple mental model: → CPU for the data work. ETL, feature engineering, SQL, classic ML. → GPU for deep learning. Fine-tuning LLMs, computer vision, recommenders, neural networks. Calling a frontier proprietary model on every request adds up fast at production scale. A lot of teams are realizing they can fine-tune a strong open-weights model like Kimi K2 or Qwen on their own data, run it on GPU, and get a system that is cheaper per token and often better at their specific task. That is where on-demand GPUs start to matter. You pick your accelerator, A10 or H100, attach it to your notebook, and fine-tune the open model that fits your workload.

What you'll learn

  • Use CPU for data work such as ETL, feature engineering, SQL, and classic machine learning.
  • Use GPU for deep learning tasks: LLM fine-tuning, computer vision, recommenders, and neural networks.
  • Fine-tuning open-source models on your own data is cost-effective compared to calling proprietary model APIs at production scale.
  • Token economics now determines the choice between CPU and GPU, rather than training models from scratch.

Frequently asked questions

When do you choose CPU instead of GPU for AI workloads?
CPU is suitable for data work such as ETL, feature engineering, SQL queries, and classic machine learning. GPU is not needed for these tasks and CPU is more cost-effective.
Why are teams now considering open-source models instead of proprietary APIs?
Fine-tuning open-source models on your own data can be cheaper per token and often deliver better results for specific tasks, compared to calling proprietary models at production scale.
What is the role of token economics in the GPU versus CPU choice?
Token economics is now the central factor in the decision. Instead of training models from scratch, teams look at cost per token and choose GPU to fine-tune open-source models when this is cheaper than proprietary API calls.
How do you use on-demand GPUs for fine-tuning models?
You select an accelerator such as A10 or H100, attach it to your notebook, and then fine-tune the open-source model that fits your workflow.

Topics

gpu-computingfine-tuningllm-economicsdatabricksopen-weights-models