Batch Upload Crop Images to Qdrant with n8n Automation

What This Workflow Does

This workflow takes images of crops stored in Google Cloud Storage and uploads them to Qdrant, a vector database. It creates image embeddings using Voyage AI’s API in batches. This helps easily find similar images for crop anomaly detection. It checks if the Qdrant collection exists and creates it if needed. Then it fetches images, filters some out, generates embeddings, and uploads data with unique IDs.

You save time by automating batch embedding and upload. Errors go down because data is handled carefully step by step. Your Qdrant database becomes ready for fast image similarity search for AI tasks.

Who Should Use This Workflow

• Data scientists who build crop anomaly detection models using image data.
  
• AI engineers needing to upload large crop image sets to vector DBs like Qdrant.
  
• Farm tech developers working with agricultural image datasets stored on cloud storage.
  
• n8n users wanting automated batch image embeddings and uploads without manual coding.
  

Tools and Services Used

• n8n: Workflow automation platform that runs the job.
  
• Google Cloud Storage: Holds raw crop images in cloud buckets.
  
• Qdrant Cloud: Vector database for storing image embeddings and metadata.
  
• Voyage AI API: Creates multimodal image embeddings from URLs.
  

Inputs → Processing Steps → Outputs

Inputs

• Crop image files in Google Cloud Storage bucket folder.
  
• API credentials for Qdrant and Voyage AI embedded in n8n.
  

Processing Steps

• Check if Qdrant collection "agricultural-crops" exists.
  
• Create collection and index on crop_name if missing.
  
• Fetch crop images filtering prefix and exclude tomatoes.
  
• Assemble public URLs and crop names from file paths.
  
• Split images in batches (4 per batch), generate UUIDs.
  
• Format batch payloads for embedding API and Qdrant upload.
  
• Call Voyage AI embedding API to get vector embeddings.
  
• Upload batches with embeddings and metadata to Qdrant.
  

Output

• Batched, uniquely identified crop images uploaded as vectors + metadata to Qdrant.
  
• Ready for fast similarity searches and anomaly detection.
  

Beginner step-by-step: How to build this in n8n

Step 1: Download and Import Workflow

1. Download the workflow file using the Download button on this page.
   
2. Open your n8n editor.
   
3. Use the Import from File option to load the workflow.
   

Step 2: Configure Credentials and IDs

1. Add your Qdrant Cloud API Key in the n8n credentials manager.
   
2. Add your Voyage AI API Key the same way.
   
3. Check the Google Cloud Storage node and update your bucket name or folder prefix if needed.
   
4. Update variables like collection name, batch size in the Set node named "Qdrant cluster variables" if required.
   

Step 3: Test the Workflow

1. Run the workflow manually by clicking the Execute button.
   
2. Watch the progress and ensure batches upload correctly to Qdrant.
   
3. Check for any error messages in each node.
   

Step 4: Activate for Production

1. Enable the workflow in n8n to run on schedule or trigger as needed.
   
2. Monitor runs using n8n’s execution logs.
   

For more control or privacy, consider self-host n8n on your server.

Customization ideas

• Change the batch size number in the "Qdrant cluster variables" node to upload bigger or smaller groups.
  
• Switch the Google Cloud Storage prefix to pick different image folders.
  
• Edit the filter node to exclude or include other crop types.
  
• Update the Voyage AI embedding model in the HTTP Request node to use newer or specific models.
  
• Add more metadata fields like timestamps or camera tags in the "Get fields for Qdrant" node.
  

Common Problems and Fixes

"Qdrant collection create call returns error 409 Conflict"
This means a collection already exists with that name. Check your "Check Qdrant Collection Existence" node and If node to make sure creation runs only when collection is missing.

"Voyage AI API returns embedding error or empty response"
Watch the format of your request body. Verify your Google Cloud Storage URLs are public and valid. The API needs the image URL inside a content field with image_url keys.

"Upload to Qdrant fails with authentication error"
Double-check the API keys for Qdrant in n8n credentials. Keys may be expired or invalid.

Pre-Production Checklist

• Make sure Qdrant and Voyage AI API keys are set inside n8n.
  
• Verify the Google Cloud Storage bucket has the correct crop images with access permissions.
  
• Test the "Check Qdrant Collection Existence" node alone to confirm detection works.
  
• Run a small batch first to check URLs and embeddings.
  
• Backup existing Qdrant data before replacing or updating collections.
  

Deployment Guide

After testing, turn on the workflow in n8n for live use.

Monitor executions in the workflow log for errors or partial failures.

Consider adding notifications or alerts for status updates if processing large datasets regularly.

Keep API keys safe and update them on schedule.

Summary of Results

✓ Save many hours by automating batch image embedding and upload.

✓ Avoid errors by systematized checks, batching, and filtering.

✓ Get a Qdrant vector database ready with unique IDs and indexed metadata.

✓ Enable fast similarity search and AI detection for crop images.

→ This workflow makes crop image processing simpler and faster for agricultural AI.