Backend Engineer Project:

File Import, Mapping, and API Development

πŸ“Œ Project Overview

Welcome to the Backend Engineering challenge for Sustainity! πŸš€

This project will test your ability to handle file imports, data processing, API design, and database interactions

Your task is to develop an API in Node.js that allows users to:

               1.            Upload a CSV file containing sustainability-related data.

               2.            Map the file’s columns to predefined database fields.

               3.            Process and store the data in a database.

               4.            Retrieve stored records via API..

πŸ› οΈ Instructions

1. Download Sample Data

  • You can download the sample CSV file here (Provide a real link).

⏳ Time Estimate

  • This project is designed to be completed in 3-4 hours.

  • If you’re unable to complete all features, focus on core functionality and code quality.

πŸ“₯ Submission Guidelines

  • Push your code to GitHub (public repo) or send a ZIP file.

  • Include a README.md with setup instructions.

  • API should be Dockerized (optional but preferred).

2. Build a Backend Service

  • Use Node.js with Express.js (or Fastify).

  • Store data in PostgreSQL or MongoDB.

5. Data Validation & Error Handling

  • Handle missing fields, incorrect data types, and invalid mappings gracefully.

  • Log errors properly using Winston or Bunyan.

3. Implement API Endpoints

File Upload & Processing

  • POST /upload – Accepts a CSV file upload (multipart form-data).

  • POST /process – Reads the file, maps columns to database fields, and saves valid data.

Data Retrieval

  • GET /data – Returns all stored records (paginated).

  • GET /data/:id – Returns a specific record.

4. File Mapping System

  • Allow users to specify how CSV columns map to database fields.

  • Example mapping: "Transaction Date" β†’ "date", "COβ‚‚ Emissions (kg)" β†’ "emission_kg".

  • Save mapping configurations in a database..

⭐ Bonus Tasks (Optional, if Time Permits)

βœ… Asynchronous Processing – Use a queue system (e.g., Bull.js, RabbitMQ) for large file handling.

βœ… CSV Preview API – Let users preview the first 10 rows before processing.

βœ… Unit Tests – Implement basic API tests using Jest or Mocha.