ramalama-convert - Man Page
convert AI Models from local storage to OCI Image
Synopsis
ramalama convert [options] model [target]
Description
Convert specified AI Model to an OCI Formatted AI Model
The model can be from RamaLama model storage in Huggingface, Ollama, or a local model stored on disk. Converting from an OCI model is not supported.
Note: The convert command must be run with containers. Use of the --nocontainer option is not allowed.
Options
--gguf=Q2_K | Q3_K_S | Q3_K_M | Q3_K_L | Q4_0 | Q4_K_S | Q4_K_M | Q5_0 | Q5_K_S | Q5_K_M | Q6_K | Q8_0
Convert Safetensor models into a GGUF with the specified quantization format. To learn more about model quantization, read llama.cpp documentation: https://github.com/ggml-org/llama.cpp/blob/master/tools/quantize/README.md
--help, -h
Print usage message
--network=none
sets the configuration for network namespaces when handling RUN instructions
--type=raw | car
type of OCI Model Image to convert.
| Type | Description |
| car | Includes base image with the model stored in a /models subdir |
| raw | Only the model and a link file model.file to it stored at / |
Example
Generate an oci model out of an Ollama model.
$ ramalama convert ollama://tinyllama:latest oci://quay.io/rhatdan/tiny:latest Building quay.io/rhatdan/tiny:latest... STEP 1/2: FROM scratch STEP 2/2: COPY sha256:2af3b81862c6be03c769683af18efdadb2c33f60ff32ab6f83e42c043d6c7816 /model --> Using cache 69db4a10191c976d2c3c24da972a2a909adec45135a69dbb9daeaaf2a3a36344 COMMIT quay.io/rhatdan/tiny:latest --> 69db4a10191c Successfully tagged quay.io/rhatdan/tiny:latest 69db4a10191c976d2c3c24da972a2a909adec45135a69dbb9daeaaf2a3a36344
Generate and run an oci model with a quantized GGUF converted from Safetensors.
$ ramalama --image quay.io/ramalama/ramalama-rag convert --gguf Q4_K_M hf://ibm-granite/granite-3.2-2b-instruct oci://quay.io/kugupta/granite-3.2-q4-k-m:latest Converting /Users/kugupta/.local/share/ramalama/models/huggingface/ibm-granite/granite-3.2-2b-instruct to quay.io/kugupta/granite-3.2-q4-k-m:latest... Building quay.io/kugupta/granite-3.2-q4-k-m:latest... $ ramalama run oci://quay.io/kugupta/granite-3.2-q4-k-m:latest
See Also
History
Aug 2024, Originally compiled by Eric Curtin ecurtin@redhat.com ⟨mailto:ecurtin@redhat.com⟩