Automate Upwork Job Tracking with n8n and MongoDB Integration

Opening Problem Statement

Meet Sarah, a freelance developer who spends hours every day manually browsing Upwork for new Python and Java job posts. Sarah struggles to keep track of the latest relevant gigs, often missing out on opportunities because she can’t refresh the search frequently enough, and duplicates clog her notes. She wastes precious time copying data and later sorting which postings are new or expired. Manually doing this multiple times a day dissipates her energy, makes her prone to errors, and reduces her responsiveness to new job offers.

This inefficiency translates into lost potential revenue since quick responses to fresh job postings drastically improve a freelancer’s chances of landing contracts. Sarah’s challenge is common among freelancers aiming to maximize time spent on actual work rather than tedious job hunting.

What This Automation Does

This n8n workflow acts as Sarah’s personal job-hunting assistant. Once set up, it automatically performs these key actions every 10 minutes during working hours (3AM to 3PM server time):

• Queries Upwork for new job posts related to Python and Java by sending specified URLs to an API crawler.
• Checks for duplicates by comparing job title and budget against entries stored in a MongoDB collection to filter out already seen job posts.
• Stores unique new job posts in MongoDB to maintain a growing, deduplicated database of relevant gigs.
• Notifies Sarah via Slack by sending detailed messages in a designated Slack channel with job title, publication date, budget, skills, and links.
• Runs only during working hours to avoid unnecessary API calls and notifications overnight.
• Supports proxy usage for location-specific scraping to better mimic local browsing behavior and reduce IP blocks.

This setup saves Sarah hours by automating tedious search, filtering, and notification tasks, enabling her to focus on applying and working.

Prerequisites ⚙️

• n8n account (cloud or self-hosted) with workflow editor access ⚙️
• MongoDB database account with a collection named n8n and read/write permissions 🔐
• Slack workspace and a bot/token authorized to post messages in the #general channel 💬
• Apify API token to use the Upwork scraping actor (set as HTTP Query Authentication credential) 🔑
• Optional proxy service configured for country-based scraping (here set to France – FR) 🔌

Step-by-Step Guide to Build This Workflow

1. Setting the Schedule Trigger Node

First, open your n8n editor and add a Schedule Trigger node:

• Click + Add Node → search for “Schedule Trigger” → select.
• In the node parameters, set the interval to 10 minutes to run the workflow every 10 minutes.

You should see the rule active showing “Every 10 minutes.” This ensures the workflow polls Upwork frequently.

Common mistake: Forgetting to set an appropriate frequency which can either overload your API quota or miss timely jobs.

2. Filtering Working Hours with If Node

Add an If node after the Schedule Trigger to restrict workflow runs to between 3 AM and 3 PM server time:

• Click + Add Node → search “If” → select.
• Configure two conditions on the $json.Hour field (derived from current time by n8n automatically):
• Hour > 2
• Hour < 15

This avoids unnecessary API calls outside working hours.

Common mistake: Misconfiguring the hour range, causing jobs to be missed or spam notifications.

3. Assigning Parameters Node (Set Node)

Add a Set node named Assign parameters after the If node:

• Set two fields:
• startUrls: an array of job listing URLs, in this case Upwork URLs querying for Python and Java jobs, each with method GET.
• proxyCountryCode: the proxy country code string, here set to “FR” (France).

Example value for startUrls:

[
  { "url": "https://www.upwork.com/nx/search/jobs/?nbs=1&q=python", "method": "GET" },
  { "url": "https://www.upwork.com/nx/search/jobs/?nbs=1&q=java", "method": "GET" }
]

You should see the node ready to pass these parameters for the scraping API call.

Common mistake: Entering invalid URLs or formats here will cause scraping failures.

4. Querying Upwork Job Posts with HTTP Request Node

Add an HTTP Request node named Query For Upwork Job Posts after the Assign parameters node:

• Set Method to POST.
• Set URL to the Apify actor endpoint: https://api.apify.com/v2/acts/arlusm~upwork-scraper-with-fresh-job-posts/run-sync-get-dataset-items
• Enable Send Body and enter body parameters for startUrls and proxyCountryCode by referencing values from the previous Set node using the expression editor:
• ={{ $json.startUrls }}
• ={{ $json.proxyCountryCode }}
• Use HTTP Query Authentication with your Apify token configured in n8n credentials.

Trigger the node manually to test and see job posts data returning as JSON.

Common mistake: Forgetting to configure authentication or body parameters causes empty or error responses.

5. Checking for Existing Job Entries in MongoDB

Add a MongoDB node named Find Existing Entries connected to the HTTP Request node’s main output:

• Set operation to Find.
• Use a query matching title and budget fields from the incoming JSON to avoid duplicates:

{
  "title": "{{ $json.title }}",
  "budget": "{{ $json.budget }}"
}

• Select your MongoDB credentials with read access.

This filters out jobs already stored.

Common mistake: Not matching all relevant fields will let duplicates slip through.

6. Merging New Entries to Exclude Existing Jobs

Add a Merge node named Output New Entries:

• Set mode to combine with join mode keepNonMatches.
• Set fields to match on title, budget to only keep jobs not found in MongoDB.

This step derives only fresh unique job posts.

Common mistake: Incorrect match fields that don’t reflect actual unique job identifiers.

7. Inserting New Jobs into MongoDB

Add another MongoDB node named Add New Entries To MongoDB:

• Set operation to Insert.
• Map fields to insert like title, link, paymentType, budget, projectLength, shortBio, skills, publishedDate, normalizedDate, searchUrl.
• Select the same MongoDB credentials with write access.

This builds your persistent job post database.

Common mistake: Missing required fields or permission issues cause insert failures.

8. Sending Notifications to Slack Channel

Finally, add a Slack node named Send message in #general linked to the MongoDB insert node:

• Choose the Slack channel #general.
• Use a message template with job details merged via expressions such as:

Job Title : {{ $json.title }}
Published : {{ $json.publishedDate }}
Link : {{ $json.link }}
Payment Type: {{ $json.paymentType }}
Budget: {{ $json.budget }}
Skills: {{ $json.skills }}
Bio: {{ $json.shortBio }}

• Select your Slack credentials configured to post messages.

You should see notification messages in Slack nearly instantly for new job posts after workflow activation.

Common mistake: Wrong channel name or missing Slack API token will block messages.

Customizations ✏️

• Modify job keywords: In the Assign parameters node, change or expand the startUrls array URLs with any other Upwork search URLs for different tech stacks or job types.
• Adjust working hours: Update the If Working Hours node’s hour range to suit your timezone or working preferences.
• Use multiple Slack channels: Add additional Slack nodes to notify different teams or channels based on job categories.
• Proxy location changes: Update the proxyCountryCode in the Assign parameters node to scrape job posts for specific countries to bypass geo restrictions.
• Additional data fields: In MongoDB nodes, map more fields from the API response to keep richer data about each job post.

Troubleshooting 🔧

Problem: No new jobs appear in Slack notifications

Cause: The Merge node configuration may be filtering out all job posts because all are detected as duplicates.

Solution: Review MongoDB query logic and merge fields to ensure uniqueness is correctly identified by matching fields. Also, confirm fresh jobs exist by triggering HTTP node manually.

Problem: MongoDB insert operation fails

Cause: Insufficient permissions or missing required fields in the insert node.

Solution: Verify your MongoDB user role, and ensure fields mapped for insertion match those required by your database schema.

Problem: HTTP Request returns authentication error

Cause: Incorrect or missing Apify token in HTTP Query Credential.

Solution: Double-check the configured API token value in your n8n credentials and make sure the correct credential is selected in the node.

Pre-Production Checklist ✅

• Verify the Schedule Trigger runs every 10 minutes as configured.
• Test the If node time condition by running the workflow within and outside defined hours.
• Test HTTP Request node independently with valid credentials and see live Upwork job posts fetched.
• Confirm MongoDB connection with right credentials and test Find and Insert operations.
• Ensure Slack node posts correctly using a test message to your channel.
• Validate proxy country code setting if used, to avoid connectivity issues.
• Check workflow logs for any errors or failed executions before full deployment.

Deployment Guide

Once tested, activate this workflow in n8n by clicking “Activate” at the top right of the editor. The automation will then start running every 10 minutes during the set working hours.

Enable workflow execution logs in n8n settings to monitor operation and quickly identify any failures or performance issues.

This automation can run indefinitely, making it a reliable job scout that requires little maintenance.

FAQs

Can I add other job search URLs besides Python and Java?

Yes, just edit the Assign parameters node’s startUrls array to include additional Upwork search URLs for other languages or roles.

Does this workflow consume API credits or cost money?

The main API usage is via Apify’s Upwork scraping actor, which may require an Apify plan depending on your usage. n8n itself is free or subscription-based for cloud hosting.

Is my data on MongoDB secure?

Ensure you use proper credentials with minimum necessary permissions and secure your MongoDB instance using IP whitelisting and strong passwords.

Conclusion

By building this n8n workflow, you’ve automated the tedious part of freelance job hunting on Upwork, ensuring you get notified of fresh Python and Java gigs without lifting a finger.

This will save you hours every day and improve your application speed and success chances significantly.

Next, you might explore adding automatic proposal submissions, integrating job filtering by budget or skills, or expanding the workflow to other freelance platforms using similar APIs.

Happy automating your freelance journey!