Built a Primer-orchestrated subscription billing engine holding 91% recurring-payment success across 50k subscriptions

I led the backend for a new subscription management system built on Primer, a payment-orchestration platform that gives one interface to many processors (PayPal, Stripe, dLocal, Braintree, Adyen) and routes each charge to whichever one clears in a given market. My focus was the billing engine on top of it: configuring subscription plans, running recurring charges across every provider through that single interface, recovering failed payments, and cancelling cleanly, at a scale of roughly 50k subscriptions.

It is a Laravel service backed by PostgreSQL for plan and subscription data and MongoDB for fast-moving counters, with Horizon and Redis queues driving the charge and notification work.

Before Primer, each payment provider (Stripe, PayPal, Braintree) was integrated on its own, so adding another one meant a full, bespoke integration. Coverage was uneven too: in some countries certain providers simply do not clear payments. Braintree 3DS, for example, failed for customers in Denmark, who had to be routed to Adyen instead. I needed a payment layer that made adding providers cheap and routed each charge to one that actually works for the customer's country and configuration, and then a subscription billing engine on top of it that charges recurring payments reliably, recovers the ones that fail, and never double-charges.

Building on Primer meant one interface for every processor, so a single billing code path drives them all and onboarding a new provider is a configuration change, not another integration. On top of that, recurring billing runs as an hourly job that selects every subscription whose next charge is due (its ends_at is in the past and it is not cancelled) and charges it through Primer. A success rolls ends_at forward a period and clears the retry counter; a failure enters a dunning flow that pushes ends_at forward a grace period and increments a retry counter kept in MongoDB, so the schedule itself becomes the retry mechanism. The retry policy is config-driven per provider and per payment method (Primer retries 3 times 10 days apart, Adyen twice 30 days apart) because different providers serve different markets, and a hard decline or an exhausted counter gives up and cancels.

Two things keep it safe. A double-charge guard reconciles Primer's succeeded-payment count against ours before every charge and refuses to bill if they diverge, so a missed webhook can never cause duplicate billing. On the plan side, a plan's locked state is computed from whether it has active subscribers rather than stored as a flag, so it can never drift out of sync, and a locked plan cannot be deleted. Cancellation stops the cron from ever charging again and fans a cancellation event out to the rest of the platform.

The new system runs roughly 50,000 subscriptions at a 91% recurring-payment success rate, with the dunning engine recovering a large share of soft declines instead of churning them. Because the billing engine sits on Primer's single interface it stays provider-agnostic, so reaching a new market or adding a processor is a config entry rather than another integration. Charges never double-bill thanks to the reconciliation guard, plan lock state cannot drift because it is derived, and the whole engine is decoupled behind queues, so a slow processor never blocks billing.