Outbox Messaging Pattern for performance and message reconciliation - (Gary and Andrew)

Submitted by Andrew Young H Chaa

Talk Abstract:

You check the telemetry of your APIs. Running each stored proc on SQL server takes 1 ms. You have 5 SPs to run and it only takes 5 ms collectively. Now it raises a Topic message on Service Bus to notify other systems of the change. Then, surprisingly, it takes 100 ms. Wait a minute! This can't be true. Why does it take so long? You check every API calls. The performance is not reliable. Sometimes, it takes as low as 10 ms and other times, it's 100 ms. It's because Service Bus connection is pooled and messages are batched.

You want your APIs to be blazingly fast and not to be dragged down by messaging system. Use Outbox table and raise messages from a separate worker process. I'll show you how we benefit from Outbox Messaging Pattern at Clear Bank.

The talk will be given by Gary Day (Senior Tech Manager / Lead Architect at Clear Bank) and Andrew Chaa.

About Andrew Young H Chaa

Experienced API and Cloud platform Developer with a demonstrated history of working on large scale, high capacity distributed systems. Skilled in message and event driven Microservice Architecture, CQRS, and Functional-flavoured Programming. Having worked at Just Eat and currently working at Clear Bank as Principal Engineer.