How can I serverlessly generate vector embeddings for images I upload at a relatively cheap price for the case of an e-commerce startup to particularly fuel an image search functionality?
I have my product’s embeddings in qdrant. I generated them using my pc locally. I want to implement an image search where I upload an image, it goes through a pipeline, an embedding of it is generated and then compared with products in the qdrant database.…
I have my product’s embeddings in qdrant. I generated them using my pc locally. I want to implement an image search where I upload an image, it goes through a pipeline, an embedding of it is generated and then compared with products in the qdrant database. I’ve tried cloudflare workers but it can’t, task is too large for it. Google cloud run can’t start , times out before the model is loaded into memory. I’ve tried railways too but the free tier limits me to 0.5gb of memory and the model can’t load either. This maybe the wrong sub for this question and if so, it’d be helpful if you pointed me to communities that fit my use case. Umm But what I want exactly is a platform that can solve my problem on the free tier. I have no problem paying after I’m sure that everything works. That’s why I didn’t pay for the railways to increase my memory limit. I have no guarantee that the next tier can solve the issue. Thanks for reading and have a good day
Collected discussion
I used qdrant/clip-vit-B-32-vision
Your post/comment has been determined to be a low-effort post or comment. This includes title-only posts, easily searchable questions, vague/open-ended discussion prompts, LLM generated posts or comments, and posts/comments that do not provide enough context for meaningful replies or discussion.
Hey u/Available_Fondant_11, Qdrant team member here 😄 qdrant/clip-vit-b-32-vision is available through Qdrant Cloud Inference, so you can upload an image, have Qdrant generate the embedding, and search it in the same API call. Nothing runs in your function, so it works fine on Workers and most free tiers. In Python, set cloud_inference=True and pass Image(image=..., model="qdrant/clip-vit-b-32-vision") instead of a vector. There's also a matching qdrant/clip-vit-b-32-text model, so you can search the same collection with text too. Docs: https://qdrant.tech/documentation/inference/cloud-inference/ If you want to self-host, Cloud Run is probably timing out because of cold starts, not model size. The ONNX model is only ~340 MB. Bake it into the container image instead of downloading it at startup, and give it ~2 GB RAM. Railway's 0.5 GB was never going to cut it.
Yay qdrant. I’m such a fan. Thanks I’ll try that.
What model, and what framework/engine do you use to generate an embedding? Maybe just use a really cheap API? Might be cheaper than rolling your own. Also, r/localllama might be able to help you
Qdrant API Error (500): {"status":{"error":"Service internal error: Authentication failed for inference service (401 Unauthorized): {\"error\":\"This model: qdrant/clip-vit-b-32-vision is not allowed in free tier\"}"},"time":0.023853889}
I'd stop trying to load the model inside the function and call a hosted embedding API instead (Qdrant Cloud Inference, Jina, or Replicate's CLIP endpoints) — at startup volume per-image pricing is basically free, and it eliminates the cold-start memory problem entirely.
This comment was deleted.
It is probably not the platform, it is how the model is being loaded. CLIP through torch + transformers pulls in 2GB+ of dependencies before the weights even load. Convert it to ONNX and quantize (fastembed does this out of the box, or optimum-cli from huggingface) and clip-vit-b-32 runs in about 350MB of RAM. That fits Railway's 0.5GB and cold-starts well inside Cloud Run's default timeout. If you stick with Cloud Run, also set min instances to 1 so you stop paying the cold-start cost on every request. The free tier usually absorbs one idle instance.
This comment was deleted.