Can Gemini CLI use Gemma 3 or Gemma 3n models instead of the Gemini API?
Can google-gemini/gemini-cli be set up to use Gemma 3 or Gemma 3n models instead of the Gemini API, for offline use or to save API quota? A user reported that when gemini-cli loads Gemma 3 and 3n models, they fail, and they suspect function calling is not enabled for them.
Originally asked on GitHub: https://github.com/google-gemini/gemini-cli/issues/1957
5 answers
The issue is closed (COMPLETED), but the thread does not include a maintainer explanation, a fix, a config flag, or documentation showing how to run Gemma in gemini-cli.
What the thread reports: gemini-cli will load Gemma-3 and Gemma-3n models served from Google, but they fail. One commenter suspects this is because function calling through the API is not available for those models. Another commenter notes that Gemma 3 supports function calling in general, and asks whether the limit is server-side or a configuration issue in gemini-cli.
Ideas raised in the thread, none confirmed to work:
- Use a CLI coding assistant that already supports local LLMs. Commenters listed Aider, MyCoder, Goose, Devseeker, Codai, oli, Open Codex, and OCode.
- Look at config.ts and models.ts in the gemini-cli source, where models are defined. One commenter suggested changing these files, but the thread does not say this works.
- Expose gemini-cli's tools to Gemma through an MCP server. This is only a proposal.
- A commenter linked a separate prototype, sairambokka/ollama-coding-agent, which is a different application. The commenter said they had forked gemini-cli and might later build a Gemma version. The thread does not say this repo runs Gemma.
- A commenter linked IllFil/gemma3-ollama-tools for Gemma 3 function calling with Ollama. Another commenter said it does not apply here, since it is for local models and this issue concerns gemini-cli.
General advice: if you need an offline coding agent, a tool built around local LLMs is probably a simpler starting point than patching gemini-cli.
Did this work for you?What @cosmicdreams tried on GitHub (Outcome unclear)
The reporter requested an option to use Gemma as a local fallback in gemini-cli for offline use and to reduce quota use. The issue is marked closed (COMPLETED), but the thread shows no implementation.
Source: https://github.com/google-gemini/gemini-cli/issues/1957 (opens external site)
Did this work for you?What @GhostArchitect01 tried on GitHub (Didn't work)
Loaded Gemma-3 and Gemma-3n models in gemini-cli. They failed, and the commenter suspected function calling via the API is not available for them.
Did this work for you?What @GhostArchitect01 tried on GitHub (Outcome unclear)
Suggested that models are defined in config.ts and models.ts in gemini-cli and that changing them is worth trying to enable Gemma.
Did this work for you?What @GhostArchitect01 tried on GitHub (Outcome unclear)
Proposed an MCP server that would let Gemma 3 or 3n use gemini-cli tools, since the models load but lack function calling. No result was reported.
Did this work for you?
Related questions
- gemini-cli 400: "only 'enum' and 'date-time' are supported for STRING type"
- Gemini CLI: API Error 'Model stream ended with an invalid chunk or missing finish reason' after ReadFile or Edit tool calls
- gemini-cli stuck on "This is taking a bit longer" after Esc cancel — is there a fix?
- Is there a way to get codebase indexing in gemini-cli?
- How can I use voice input instead of typing with Gemini CLI?