Duplicate requests are one of the silent killers of production APIs. A user double-clicks a “Pay” button, a mobile client retries after a timeout, a webhook fires twice, and suddenly your customer is charged twice. The fix has a name: idempotency keys. In this practical tutorial, we’ll build a battle-tested implementation of an idempotency key REST API pattern using Node.js, Express, and PostgreSQL, including the exact SQL schema, middleware code, and concurrency handling that most tutorials skip. An in-depth look at it is worth the time.

What Is an Idempotency Key in a REST API?

An idempotency key is a unique client-generated identifier (usually a UUID) sent in the HTTP Idempotency-Key header. The server stores the result of the first request associated with that key, and if the same request comes in again, it returns the cached response instead of executing the operation twice.

This pattern is used by Stripe, PayPal, Shopify, AWS, and pretty much every serious payment provider. It is now a proposed IETF standard and documented on MDN for POST and PATCH requests.

When Should You Use Idempotency Keys?

• Payment endpoints (charges, refunds, transfers)
• Order creation and checkout flows
• User signup / account creation
• Sending emails, SMS, or push notifications
• Any POST or PATCH that has side effects and could be retried

GET, PUT, and DELETE are already idempotent by HTTP spec. You mostly need this for POST and PATCH.

How the Idempotency Flow Works

1. The client generates a UUID v4 and sends it in the Idempotency-Key header.
2. The server checks if that key already exists in the database.
3. If it does not exist, the server locks the key, executes the operation, stores the response, and returns it.
4. If it exists and is completed, the server returns the stored response without re-executing.
5. If it exists but is still processing, the server returns 409 Conflict.

The PostgreSQL Schema

The database is the single source of truth. Redis is faster but does not give you the transactional guarantees you need for payments. Here is the schema:

CREATE TABLE idempotency_keys (
  key             VARCHAR(255) PRIMARY KEY,
  user_id         BIGINT NOT NULL,
  request_method  VARCHAR(10) NOT NULL,
  request_path    VARCHAR(500) NOT NULL,
  request_hash    VARCHAR(64) NOT NULL,
  response_status INTEGER,
  response_body   JSONB,
  status          VARCHAR(20) NOT NULL DEFAULT 'processing',
  created_at      TIMESTAMPTZ NOT NULL DEFAULT NOW(),
  completed_at    TIMESTAMPTZ,
  expires_at      TIMESTAMPTZ NOT NULL DEFAULT NOW() + INTERVAL '24 hours'
);

CREATE INDEX idx_idempotency_expires ON idempotency_keys(expires_at);
CREATE INDEX idx_idempotency_user ON idempotency_keys(user_id);

Why Each Column Matters

Column	Purpose
request_hash	SHA-256 of the request body. Prevents key reuse with a different payload.
user_id	Scopes keys per user so two users cannot collide on the same UUID.
status	Tracks processing, completed, or failed state.
expires_at	Keys should not live forever. 24h is the Stripe standard.

The Express Middleware

Here is a production-ready middleware. It uses pg and handles concurrency with a database transaction plus SELECT ... FOR UPDATE.

const crypto = require('crypto');
const { pool } = require('./db');

function hashBody(body) {
  return crypto
    .createHash('sha256')
    .update(JSON.stringify(body || {}))
    .digest('hex');
}

async function idempotencyMiddleware(req, res, next) {
  const key = req.header('Idempotency-Key');

  if (!key) {
    return res.status(400).json({
      error: 'Idempotency-Key header is required for this endpoint'
    });
  }

  if (!/^[a-zA-Z0-9-]{8,255}$/.test(key)) {
    return res.status(400).json({ error: 'Invalid Idempotency-Key format' });
  }

  const userId = req.user.id;
  const bodyHash = hashBody(req.body);
  const client = await pool.connect();

  try {
    await client.query('BEGIN');

    const existing = await client.query(
      'SELECT * FROM idempotency_keys WHERE key = $1 AND user_id = $2 FOR UPDATE',
      [key, userId]
    );

    if (existing.rows.length > 0) {
      const record = existing.rows[0];

      if (record.request_hash !== bodyHash) {
        await client.query('ROLLBACK');
        return res.status(422).json({
          error: 'Idempotency-Key reused with a different request body'
        });
      }

      if (record.status === 'completed') {
        await client.query('COMMIT');
        return res.status(record.response_status).json(record.response_body);
      }

      if (record.status === 'processing') {
        await client.query('ROLLBACK');
        return res.status(409).json({
          error: 'A request with this Idempotency-Key is already in progress'
        });
      }
    } else {
      await client.query(
        `INSERT INTO idempotency_keys 
         (key, user_id, request_method, request_path, request_hash, status)
         VALUES ($1, $2, $3, $4, $5, 'processing')`,
        [key, userId, req.method, req.path, bodyHash]
      );
    }

    await client.query('COMMIT');

    // Intercept the response to persist it
    const originalJson = res.json.bind(res);
    res.json = async (body) => {
      try {
        await pool.query(
          `UPDATE idempotency_keys 
           SET status = 'completed', response_status = $1, response_body = $2, completed_at = NOW()
           WHERE key = $3 AND user_id = $4`,
          [res.statusCode, body, key, userId]
        );
      } catch (err) {
        console.error('Failed to persist idempotency response', err);
      }
      return originalJson(body);
    };

    next();
  } catch (err) {
    await client.query('ROLLBACK').catch(() => {});
    next(err);
  } finally {
    client.release();
  }
}

module.exports = idempotencyMiddleware;

Using the Middleware on a Payment Endpoint

const express = require('express');
const idempotency = require('./idempotencyMiddleware');
const router = express.Router();

router.post('/payments', idempotency, async (req, res) => {
  const { amount, currency, source } = req.body;

  const charge = await stripe.charges.create({
    amount,
    currency,
    source
  });

  res.status(201).json({
    id: charge.id,
    status: charge.status,
    amount: charge.amount
  });
});

Handling Concurrent Requests Safely

The trickiest part of any idempotency key REST API is when two requests with the same key hit your server at the same time. Here is how our implementation handles each scenario:

Scenario	Behavior
Same key, same body, first request finished	Returns cached response instantly
Same key, same body, first request still running	Returns 409 Conflict
Same key, different body	Returns 422 Unprocessable Entity
Two concurrent inserts (race condition)	Second one blocks on FOR UPDATE, then sees processing status

The SELECT ... FOR UPDATE is what makes this bulletproof. Combined with the PRIMARY KEY constraint on key, PostgreSQL guarantees only one transaction can insert or read the row at a time. Make Your API Idempotent, Avoid Ruining Clients Lives covers this in more depth.

Cleaning Up Expired Keys

Add a scheduled job that runs every hour to purge old entries. A simple pg_cron job works, or you can use node-cron:

const cron = require('node-cron');

cron.schedule('0 * * * *', async () => {
  const result = await pool.query(
    'DELETE FROM idempotency_keys WHERE expires_at < NOW()'
  );
  console.log(`Purged ${result.rowCount} expired idempotency keys`);
});

Common Mistakes to Avoid

• Storing keys only in memory or Redis for payment endpoints. If the cache is lost, you lose replay protection.
• Not hashing the request body. A client could reuse a key with a completely different amount.
• Not scoping by user. Two different users could generate the same UUID (unlikely, but possible with bad clients).
• Persisting the response before the operation completes. If the operation fails mid-flight, you must not cache a partial success.
• Forgetting to expire keys. Your table will grow forever.

Testing Your Implementation

Here is a quick test with curl to verify the behavior:

KEY=$(uuidgen)

curl -X POST https://api.example.com/payments \
  -H "Idempotency-Key: $KEY" \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"amount": 5000, "currency": "eur", "source": "tok_visa"}'

# Run the same command again. You should get the exact same response,
# and no second charge should appear in your payment provider dashboard.

FAQ

Should the client or the server generate the idempotency key?

The client generates it. This is critical because the client is the one that needs to retry safely after a network failure. If the server generated it, the client would not know what key to use on retry.

How long should an idempotency key be valid?

24 hours is the industry standard (Stripe, PayPal). It is long enough to cover retries and short enough to keep the table manageable.

Can I use Redis instead of PostgreSQL?

For non-critical endpoints, yes. For payments and financial operations, use a durable relational database. You need transactional guarantees and durability that Redis in default mode does not provide.

Do I need idempotency keys on GET requests?

No. GET requests are already idempotent by the HTTP specification. You need this pattern for POST and PATCH primarily. Related reading: What Is API Idempotency? A Practical Guide.

What HTTP status should I return for a replayed request?

Return the exact same status and body as the original response. Some APIs add an Idempotent-Replayed: true header so the client knows the response came from the cache.

What if two requests arrive at the exact same millisecond?

The PRIMARY KEY constraint on the key column plus SELECT ... FOR UPDATE ensures PostgreSQL serializes the transactions. Only one will succeed at inserting; the other will read the existing row.

---

Implementing an idempotency key REST API correctly is one of those small pieces of engineering that separates hobby projects from production systems. If you are building payment flows, order processing, or anything with real-world side effects, this pattern is non-negotiable. At Box Software, we implement this pattern by default on every critical endpoint we ship for our clients. If you need help auditing or building a resilient API, get in touch with our team.