Practice 4 — Node.js Express Backend

Build REST API for Review Aggregator with Database Integration

Practice
Node.js
Express
Backend
MySQL
Auteur
Affiliations

Emmanuel BRUNO

Université de Toulon

LIS UMR CNRS 7020

Date de publication

2025-10-21

Résumé

Course lectures and practices for JavaScript full‑stack web development with AI‑assisted workflows.

Practice Overview

Prerequisites: Node.js Express lecture completed
Project: Multi-Source Product Review Aggregator backend
Objective: Build Express REST API with in-memory data storage

What You’ll Build

Complete Express backend for the review aggregator project:

  • ✅ Express server with proper middleware and routing
  • ✅ REST API endpoints for product reviews (CRUD operations)
  • ✅ In-memory data storage using JavaScript arrays
  • ✅ Review aggregation logic (statistics calculation)
  • ✅ External scraper integration (simulated)
  • ✅ Error handling and input validation

Technologies: Node.js, Express.js, JavaScript arrays, async/await
Foundation: Using fullstack-minimal-app/backend as starting point

Note: This practice focuses on Express fundamentals. In Session 5, you’ll upgrade this API to use MySQL database for persistent storage!

Part 0: Initialize Project and Scripts

Run these commands from the parent folder to create and prepare the backend project.

# Create and enter project directory
mkdir my-review-aggregator-backend
cd my-review-aggregator-backend

# Initialize package.json
npm init -y

# Create src structure
mkdir -p src/routes src/services src/data src/middleware

# Install runtime deps 
npm i express joi cors morgan dotenv

# Install dev deps to Enable auto-restart on code changes
npm i -D nodemon

Add scripts to package.json:

{
  // DO NOT replace entire package.json, just add the scripts section
  // DO NO TOUCH FROM "dependencies" TO "devDependencies"
  "dependencies": {
    "cors": "^2.8.5",
    "dotenv": "^17.2.3",
    "express": "^5.1.0",
    "joi": "^18.0.1",
    "morgan": "^1.10.1"
  },
  "devDependencies": {
    "nodemon": "^3.1.10"
  }, // ADD THE COMMA IF MISSING and THEN ADD:
  "name": "my-review-aggregator-backend",
  "version": "1.0.0",
  "type": "commonjs",
  "scripts": {
    "dev": "nodemon src/server.js",
    "start": "node src/server.js"
  }
}

Create the server entry (loads env and starts the HTTP server):

// src/server.js
require('dotenv').config();
const app = require('./app');

const PORT = process.env.PORT || 4000;
app.listen(PORT, () => {
  console.log(`API listening on http://localhost:${PORT}`);
});

Note:

  • Do not run node src/app.js. Use npm run dev or npm start.
  • Ensure src/data/mockData.js exists before starting to avoid “Cannot find module ‘../data/mockData’”.
  • Ensure you mounted the products router before the 404 handler:
    • app.use(‘/api/products’, require(‘./routes/products’));
    • app.use(‘/api/reviews’, require(‘./routes/reviews’));
    • then the 404 handler, then error handler.

Part 1: Project Setup

Step 1.1: Navigate to Backend

# If not already inside
cd my-review-aggregator-backend

Step 1.2: Create In-Memory Data Store

Create src/data/mockData.js:

// In-memory data store for Session 4
// In Session 5, we'll replace this with MySQL database

let products = [
  {
    id: 1,
    name: 'Wireless Bluetooth Headphones',
    description: 'Premium noise-canceling headphones with 30-hour battery life',
    price: 199.99,
    image_url: '/images/headphones.jpg',
    created_at: new Date('2024-09-01')
  },
  {
    id: 2,
    name: 'Gaming Mechanical Keyboard',
    description: 'RGB backlit mechanical keyboard with blue switches',
    price: 149.99,
    image_url: '/images/keyboard.jpg',
    created_at: new Date('2024-09-02')
  },
  {
    id: 3,
    name: '4K Webcam',
    description: 'Ultra HD webcam with auto-focus and noise reduction',
    price: 89.99,
    image_url: '/images/webcam.jpg',
    created_at: new Date('2024-09-03')
  }
];

let reviews = [
  {
    id: 1,
    product_id: 1,
    source: 'amazon',
    external_id: 'AMZ-12345',
    reviewer_name: 'TechReviewer01',
    rating: 5.0,
    title: 'Excellent sound quality!',
    content: 'These headphones exceeded my expectations. The noise cancellation is fantastic and battery life is as advertised.',
    review_date: '2024-10-01',
    verified_purchase: true,
    helpful_votes: 15,
    fetched_at: new Date('2024-10-02')
  },
  {
    id: 2,
    product_id: 1,
    source: 'amazon',
    external_id: 'AMZ-12346',
    reviewer_name: 'BudgetBuyer',
    rating: 3.5,
    title: 'Good but pricey',
    content: 'Sound quality is great but I think they are overpriced for what you get.',
    review_date: '2024-10-02',
    verified_purchase: true,
    helpful_votes: 8,
    fetched_at: new Date('2024-10-03')
  }
];

// Auto-increment IDs
let nextProductId = 4;
let nextReviewId = 3;

module.exports = {
  products,
  reviews,
  getNextProductId: () => nextProductId++,
  getNextReviewId: () => nextReviewId++
};
About In-Memory Storage

We’re using JavaScript arrays to store data in this practice. This lets us focus on Express fundamentals without database complexity.

Limitations of in-memory storage:

  • ❌ Data lost when server restarts
  • ❌ Not suitable for production
  • ❌ No concurrent access protection

In Session 5, you’ll learn to replace these arrays with MySQL database for persistent, production-ready storage!

Step 1.3: Environment Configuration

Create .env file in backend root. This file will hold environment variables to configure the server.

# Server Configuration
PORT=4000
NODE_ENV=development

# Note: No database configuration needed for Session 4
# We'll add MySQL configuration in Session 5

Part 2: Products API Routes

The routes are the entry points for the API to interact with products and reviews. Use AI assistance (like GitHub Copilot) to speed up coding and to get explanations.

Exercise 2.1: Create Products Routes

Create src/routes/products.js:

const express = require('express');
const router = express.Router();
const { products, reviews, getNextProductId, getNextReviewId } = require('../data/mockData');
const { fetchReviewsFromSources } = require('../services/mockScraper');

// GET /api/products - Get all products
router.get('/', (req, res) => {
  try {
    res.json({
      success: true,
      data: products,
      count: products.length
    });
  } catch (error) {
    res.status(500).json({
      success: false,
      error: 'Failed to fetch products'
    });
  }
});

// GET /api/products/:id - Get single product
router.get('/:id', (req, res) => {
  try {
    const productId = parseInt(req.params.id);
    const product = products.find(p => p.id === productId);
    
    if (!product) {
      return res.status(404).json({
        success: false,
        error: 'Product not found'
      });
    }
    
    res.json({
      success: true,
      data: product
    });
  } catch (error) {
    res.status(500).json({
      success: false,
      error: 'Failed to fetch product'
    });
  }
});

// POST /api/products - Create new product
router.post('/', (req, res) => {
  try {
    const { name, description, price, image_url } = req.body;
    
    // Basic validation
    if (!name || !price) {
      return res.status(400).json({
        success: false,
        error: 'Name and price are required'
      });
    }
    
    const newProduct = {
      id: getNextProductId(),
      name,
      description: description || '',
      price: parseFloat(price),
      image_url: image_url || '',
      created_at: new Date()
    };
    
    products.push(newProduct);
    
    res.status(201).json({
      success: true,
      data: newProduct
    });
  } catch (error) {
    res.status(500).json({
      success: false,
      error: 'Failed to create product'
    });
  }
});

// POST /api/products/:id/fetch - Fetch reviews from external sources for a product
router.post('/:id/fetch', async (req, res) => {
  try {
    const productId = parseInt(req.params.id);
    const product = products.find(p => p.id === productId);
    if (!product) {
      return res.status(404).json({ success: false, error: 'Product not found' });
    }

    const scrapedReviews = await fetchReviewsFromSources(productId, req.body || {});
    let added = 0;
    let skipped = 0;

    for (const s of scrapedReviews) {
      const exists = reviews.find(r =>
        r.product_id === productId &&
        r.source === s.source &&
        r.external_id === s.external_id
      );
      if (exists) {
        skipped++;
        continue;
      }
      reviews.push({
        id: getNextReviewId(),
        product_id: productId,
        ...s,
        fetched_at: new Date()
      });
      added++;
    }

    res.json({
      success: true,
      message: `Fetched ${scrapedReviews.length} reviews`,
      data: { added, skipped, total: scrapedReviews.length }
    });
  } catch (err) {
    res.status(500).json({ success: false, error: err.message || 'Failed to fetch reviews' });
  }
});

// GET /api/products/:id/reviews - List reviews for a product
router.get('/:id/reviews', (req, res) => {
  const productId = parseInt(req.params.id);
  const productReviews = reviews.filter(r => r.product_id === productId);
  res.json({ success: true, data: productReviews, count: productReviews.length });
});

// GET /api/products/:id/aggregate - Aggregated stats for a product
router.get('/:id/aggregate', (req, res) => {
  try {
    const productId = parseInt(req.params.id);
    const productReviews = reviews.filter(r => r.product_id === productId);

    if (productReviews.length === 0) {
      return res.json({
        success: true,
        data: {
          overall: { average_rating: 0, total_reviews: 0, min_rating: 0, max_rating: 0 },
          by_source: [],
          rating_histogram: { "5": 0, "4": 0, "3": 0, "2": 0, "1": 0 }
        }
      });
    }

    const totalRating = productReviews.reduce((sum, r) => sum + r.rating, 0);
    const avgRating = totalRating / productReviews.length;
    const minRating = Math.min(...productReviews.map(r => r.rating));
    const maxRating = Math.max(...productReviews.map(r => r.rating));

    const bySource = {};
    productReviews.forEach(r => {
      bySource[r.source] ??= { total: 0, count: 0 };
      bySource[r.source].total += r.rating;
      bySource[r.source].count += 1;
    });

    const sourceBreakdown = Object.keys(bySource).map(source => ({
      source,
      average_rating: Math.round((bySource[source].total / bySource[source].count) * 10) / 10,
      review_count: bySource[source].count
    }));

    const histogram = { "5": 0, "4": 0, "3": 0, "2": 0, "1": 0 };
    productReviews.forEach(r => {
      const bucket = Math.round(r.rating).toString();
      if (histogram[bucket] !== undefined) histogram[bucket]++;
    });

    res.json({
      success: true,
      data: {
        overall: {
          average_rating: Math.round(avgRating * 10) / 10,
          total_reviews: productReviews.length,
          min_rating: minRating,
          max_rating: maxRating
        },
        by_source: sourceBreakdown,
        rating_histogram: histogram
      }
    });
  } catch {
    res.status(500).json({ success: false, error: 'Failed to calculate aggregate statistics' });
  }
});

module.exports = router;

Exercise 2.2: Update App Configuration

app.js is already set up with basic middleware. Now, integrate the products routes.

Update src/app.js to use the products routes:

const express = require('express');
const cors = require('cors');
const morgan = require('morgan');



const app = express();

const errorHandler = require('./middleware/errorHandler');
const requestLogger = require('./middleware/requestLogger');


// Middleware
const corsOptions = {
  origin: 'http://localhost:5173',
  credentials: true,
  methods: ['GET', 'POST', 'PUT', 'DELETE', 'OPTIONS'],
  allowedHeaders: ['Content-Type', 'Authorization']
};
app.use(cors(corsOptions));
app.use(morgan('dev'));
app.use(express.json());
app.use(express.urlencoded({ extended: true }));
app.use(requestLogger);

// Routes
app.use('/api/products', require('./routes/products'));
app.use('/api/reviews', require('./routes/reviews')); // general reviews endpoints

// Health check
app.get('/health', (req, res) => {
  res.json({
    status: 'ok',
    timestamp: new Date().toISOString(),
    storage: 'in-memory'
  });
});

// 404 handler
app.use((req, res) => {
  res.status(404).json({ 
    error: {
      message: `Route not found: ${req.method} ${req.originalUrl}`,
      status: 404
    }
  });
});


// Error handling (must be last)
app.use(errorHandler);

module.exports = app;

Part 3: Reviews API Routes

Exercise 3.1: Create Review Routes Structure

Now we’ll create routes to manage reviews directly. We separate these from product routes for clarity. But we still link reviews to products via product_id.

Create src/routes/reviews.js:

const express = require('express');
const router = express.Router();
const { reviews, products, getNextReviewId } = require('../data/mockData');

// GET /api/reviews - Get all reviews (with optional filtering)
router.get('/', (req, res) => {
  try {
    const { product_id, source, min_rating } = req.query;
    
    let filteredReviews = [...reviews];
    
    // Filter by product_id
    if (product_id) {
      filteredReviews = filteredReviews.filter(r => r.product_id === parseInt(product_id));
    }
    
    // Filter by source
    if (source) {
      filteredReviews = filteredReviews.filter(r => r.source.toLowerCase() === source.toLowerCase());
    }
    
    // Filter by minimum rating
    if (min_rating) {
      filteredReviews = filteredReviews.filter(r => r.rating >= parseFloat(min_rating));
    }
    
    res.json({
      success: true,
      data: filteredReviews,
      count: filteredReviews.length
    });
  } catch (error) {
    res.status(500).json({
      success: false,
      error: 'Failed to fetch reviews'
    });
  }
});

// GET /api/reviews/:id - Get single review
router.get('/:id', (req, res) => {
  try {
    const reviewId = parseInt(req.params.id);
    const review = reviews.find(r => r.id === reviewId);
    
    if (!review) {
      return res.status(404).json({
        success: false,
        error: 'Review not found'
      });
    }
    
    res.json({
      success: true,
      data: review
    });
  } catch (error) {
    res.status(500).json({
      success: false,
      error: 'Failed to fetch review'
    });
  }
});

// POST /api/reviews - Create new review
router.post('/', (req, res) => {
  try {
    const {
      product_id,
      source,
      external_id,
      reviewer_name,
      rating,
      title,
      content
    } = req.body;
    
    // Validation
    if (!product_id || !source || !rating || !content) {
      return res.status(400).json({
        success: false,
        error: 'Missing required fields: product_id, source, rating, content'
      });
    }
    
    // Check if product exists
    const product = products.find(p => p.id === parseInt(product_id));
    if (!product) {
      return res.status(404).json({
        success: false,
        error: 'Product not found'
      });
    }
    
    // Check for duplicate review (same product, source, external_id)
    if (external_id) {
      const duplicate = reviews.find(r => 
        r.product_id === parseInt(product_id) && 
        r.source === source && 
        r.external_id === external_id
      );
      
      if (duplicate) {
        return res.status(409).json({
          success: false,
          error: 'Review already exists',
          code: 'DUPLICATE_REVIEW'
        });
      }
    }
    
    const newReview = {
      id: getNextReviewId(),
      product_id: parseInt(product_id),
      source: source.toLowerCase(),
      external_id: external_id || null,
      reviewer_name: reviewer_name || 'Anonymous',
      rating: parseFloat(rating),
      title: title || '',
      content,
      review_date: new Date().toISOString().split('T')[0],
      verified_purchase: false,
      helpful_votes: 0,
      fetched_at: new Date()
    };
    
    reviews.push(newReview);
    
    res.status(201).json({
      success: true,
      data: newReview
    });
  } catch (error) {
    res.status(500).json({
      success: false,
      error: 'Failed to create review'
    });
  }
});

// DELETE /api/reviews/:id - Delete review
router.delete('/:id', (req, res) => {
  try {
    const reviewId = parseInt(req.params.id);
    const index = reviews.findIndex(r => r.id === reviewId);
    
    if (index === -1) {
      return res.status(404).json({
        success: false,
        error: 'Review not found'
      });
    }
    
    const deletedReview = reviews.splice(index, 1)[0];
    
    res.json({
      success: true,
      message: 'Review deleted successfully',
      data: deletedReview
    });
  } catch (error) {
    res.status(500).json({
      success: false,
      error: 'Failed to delete review'
    });
  }
});

module.exports = router;

Exercise 3.2: Create Mock Scraper Service

We will not connect to real external services like amazon to get the reviews. Instead, we’ll create a mock scraper that simulates fetching reviews. A mock is a simplified version of a service that mimics its behavior for testing purposes. It is free and simple to use. For real-world applications, you would replace this with actual API calls or web scraping logic.

Create src/services/mockScraper.js:

// Mock external scraper service
// Simulates fetching reviews from external sources (Amazon, BestBuy, Walmart)

const SOURCES = ['amazon', 'bestbuy', 'walmart'];

const SAMPLE_REVIEWERS = [
  'TechEnthusiast', 'HappyCustomer', 'CriticalBuyer', 'GadgetLover',
  'ValueSeeker', 'QualityFirst', 'SmartShopper', 'ProductTester'
];

const SAMPLE_TITLES = [
  'Great product!', 'Exceeded expectations', 'Good value for money',
  'Disappointed', 'Amazing quality', 'Could be better', 
  'Highly recommend', 'Not worth the price', 'Perfect for my needs'
];

const SAMPLE_CONTENT = [
  'This product works exactly as described. Very satisfied with my purchase.',
  'Quality is excellent and shipping was fast. Would buy again!',
  'Had some issues initially but customer service helped resolve them.',
  'Not what I expected based on the description. Returning it.',
  'Absolutely love this! Best purchase I\'ve made in a while.',
  'Decent product but there are better alternatives available.',
  'Works well for the price point. Good value overall.',
  'Build quality could be better but functionality is solid.'
];

function getRandomElement(array) {
  return array[Math.floor(Math.random() * array.length)];
}

function getRandomRating() {
  // Weighted towards higher ratings (more realistic)
  const rand = Math.random();
  if (rand < 0.4) return 5.0;
  if (rand < 0.7) return 4.0;
  if (rand < 0.85) return 3.0;
  if (rand < 0.95) return 2.0;
  return 1.0;
}

function getRandomDate() {
  // Random date within last 6 months
  const now = new Date();
  const sixMonthsAgo = new Date(now.setMonth(now.getMonth() - 6));
  const randomTime = sixMonthsAgo.getTime() + Math.random() * (Date.now() - sixMonthsAgo.getTime());
  return new Date(randomTime).toISOString().split('T')[0];
}

async function delay(ms) {
  return new Promise(resolve => setTimeout(resolve, ms));
}

/**
 * Simulate fetching reviews from external sources
 * @param {number} productId - The product ID to fetch reviews for
 * @param {Object} options - Options for scraping
 * @param {number} options.count - Number of reviews to generate (default: 5-10 random)
 * @param {Array<string>} options.sources - Specific sources to use
 * @returns {Promise<Array>} - Array of review objects
 */
async function fetchReviewsFromSources(productId, options = {}) {
  // Simulate network delay (1-3 seconds)
  const delayMs = 1000 + Math.random() * 2000;
  await delay(delayMs);
  
  // 10% chance to simulate network error
  if (Math.random() < 0.1) {
    throw new Error('Failed to fetch reviews from external source: Network timeout');
  }
  
  const count = options.count || (5 + Math.floor(Math.random() * 6)); // 5-10 reviews
  const sourcesToUse = options.sources || SOURCES;
  
  const reviews = [];
  
  for (let i = 0; i < count; i++) {
    const source = getRandomElement(sourcesToUse);
    const rating = getRandomRating();
    
    reviews.push({
      source,
      external_id: `${source.toUpperCase()}-${Date.now()}-${i}`,
      reviewer_name: getRandomElement(SAMPLE_REVIEWERS),
      rating,
      title: getRandomElement(SAMPLE_TITLES),
      content: getRandomElement(SAMPLE_CONTENT),
      review_date: getRandomDate(),
      verified_purchase: Math.random() > 0.3, // 70% verified
      helpful_votes: Math.floor(Math.random() * 50)
    });
  }
  
  return reviews;
}

module.exports = {
  fetchReviewsFromSources
};

Exercise 4: Middleware and Error Handling

Now we’ll create middleware for error handling and request logging. Logging is the process of recording information about requests and responses for monitoring and debugging purposes. It helps track the behavior of the application and identify issues.

Exercise 4.1: Create Error Handling Middleware

Create src/middleware/errorHandler.js:

const errorHandler = (err, req, res, next) => {
  console.error('Error:', err);

  // Joi validation errors
  if (err.isJoi) {
    return res.status(400).json({
      success: false,
      message: 'Validation error',
      errors: err.details.map(detail => detail.message)
    });
  }

  // MySQL duplicate entry error
  if (err.code === 'ER_DUP_ENTRY') {
    return res.status(409).json({
      success: false,
      message: 'Duplicate review - already exists in database'
    });
  }

  // MySQL foreign key constraint error
  if (err.code === 'ER_NO_REFERENCED_ROW_2') {
    return res.status(404).json({
      success: false,
      message: 'Product not found'
    });
  }

  // Default server error
  res.status(500).json({
    success: false,
    message: 'Internal server error',
    ...(process.env.NODE_ENV === 'development' && { stack: err.stack })
  });
};

module.exports = errorHandler;

Exercise 4.2: Create Request Logging Middleware

Create src/middleware/requestLogger.js:

const requestLogger = (req, res, next) => {
  const start = Date.now();
  
  res.on('finish', () => {
    const duration = Date.now() - start;
    const { method, originalUrl } = req;
    const { statusCode } = res;
    
    console.log(`${method} ${originalUrl} ${statusCode} - ${duration}ms`);
  });
  
  next();
};

module.exports = requestLogger;

Exercise 4.3: Update App Configuration

Update src/app.js to use middleware:

// ...existing code...
const errorHandler = require('./middleware/errorHandler');
const requestLogger = require('./middleware/requestLogger');

// Middleware
app.use(cors());
app.use(morgan('dev'));
app.use(express.json());
app.use(express.urlencoded({ extended: true }));
app.use(requestLogger);

// Routes
app.use('/api/products', require('./routes/products'));
app.use('/api/reviews', require('./routes/reviews')); // general reviews endpoints

// Health check
app.get('/health', (req, res) => {
  res.json({
    status: 'ok',
    timestamp: new Date().toISOString(),
    storage: 'in-memory'
  });
});

// 404 handler
app.use((req, res) => {
  res.status(404).json({ success: false, error: 'Endpoint not found' });
});

// Error handling (must be last)
app.use(errorHandler);

module.exports = app;

Part 5: Testing & Integration

Exercise 5.1: Test API Endpoints

To test the API endpoints, you can use the vscode extension “REST Client”.

Create a file api-tests.http in the project root with the following content:

### Fetch reviews for product 1
POST http://localhost:4000/api/products/1/fetch
Content-Type: application/json

{
  "sources": ["amazon", "bestbuy"],
  "count": 8
}

### Get all reviews for product 1
GET http://localhost:4000/api/products/1/reviews

### Get aggregate statistics for product 1
GET http://localhost:4000/api/products/1/aggregate

### Get reviews filtered by product and source (use /api/reviews)
GET http://localhost:4000/api/reviews?product_id=1&source=amazon&min_rating=4

### Delete a specific review (replace :id with actual review ID)
DELETE http://localhost:4000/api/reviews/1

You can run each request individually in VSCode by clicking “Send Request” above each request.

Exercise 5.2: Test Complete Flow

  1. Start the server:

    npm run dev

  2. Test sequence:

    • POST to fetch reviews → Should return success with count
    • GET reviews → Should return the stored reviews
    • GET aggregate → Should return calculated statistics
    • Test error cases (invalid product ID, malformed requests)

Réutilisation