Meek bank: Idempotency for API and scheduled jobs
Felix van Hove7 min read·1 hour ago--
Meek bank is a banking application for low income environments. It exposes a REST API and comes with standard features like different account types, a job scheduler, ledger based accounting etc. Written in Go, with Postgres as its database and Linux as its host, it is closely related to a separate project: its browser front-end based on Svelte and Tailwind. The combined work of the two is meant to provide a simple, fast and flexible banking application. (Source code repositories here and here.)
The following text and its sequels are not meant as an introduction. They want to shed light on specific aspects of the two Meek bank projects. The present text focusses on idempotency and concurrency in HTTP and then idempotency in the scheduler implementation. These areas are key safety features of every banking system and it is important to get them right on every level.
When do we need API idempotency and concurrency controls?
HTTP requests arriving at the API layer can result in unintended data for two important reasons:
- The same request has already been processed or is currently being processed. An obvious cause is the accidental double click on a button, but network issues can have the same effect (e.g. misconfigured proxies or CDN retries). This problem must be addressed by guaranteeing idempotency: The same HTTP request sent repeatedly is not allowed to have any additional effect. (If idempotency is solved, it also enables the client to repeat the same request in case of network problems.)
- The request can be based on stale data. Think of user A who updates account details, when user B has already done the same and even changed more since A started. The update of A is based on obsolete data; it might overwrite valuable, more recent changes of B. This problem must be addressed by optimistic concurrency control: A’s request is sent, but blocked, and A is told why.
How can idempotency and concurrency control be guaranteed? Most of all, it depends on the HTTP method. Both afore reasons for unintended data require an HTTP request ultimately impacting data in the database. Meek bank utilizes a number of different HTTP methods for triggering database changes. For each we need to ask: Is idempotency automatically given or can a repeated request duplicate data? Is concurrency not a problem or can a request be based on obsolete data?
POST requests
Many POST requests are sent to create new data. We can ignore the problem of concurrency for such requests. However, idempotency is not given: If the same click on a button creates two accounts for the same person, this might well cause considerable problems later.
There is also a class of POST requests that are not primarily meant to create resources, but to initiate an action, e.g. transferring money from one payment account to another. In these cases, we need idempotency (e.g. only one money transfer is to happen), but also concurrency control (if the source account got closed in the meantime, no transfer is to happen).
PATCH requests
For almost all update operations the API uses PATCH (exceptions are POST requests like change-password). It never uses PUT. (It implements JSON merge patch as opposed to JSON patch and follows rfc7396.) The API only expects those fields in the requests that are supposed to overwrite existing data (which includes overwriting with null, if a field value should be reset). The API never designs updates additive. Idempotency is therefore automatically given: The new value can be set as often as it arrives at the API — it will always have the same effect. On the other hand, often concurrency is a problem — a user might overwrite a more recent change of another user. In this case we need a concurrency control.
DELETE requests
DELETE requests — by definition — can not delete multiple times: one resource can only be deleted once. In that sense, idempotency is guaranteed out-of-the-box. Concurrency control is a necessity for deletions however. E.g. we should not allow a user to DELETE a resource, when another user has — in the meantime — changed important attributes of it. (Some resources — e.g. a subscription to notifications — can not be updated. In such cases a concurrency control is unnecessary.)
How are idempotency and concurrency control guaranteed by the API?
Let’s start with concurrency: In the Meek bank API, the response to every GET request targeting a single resource contains an ETag header. The HTTP header I just got for a GET-group request
http://localhost:5173/v1/groups/cd7e5e34-10d9-4261-8191-01567e647d7dlooked like this:
Etag: “lm:1776098785143561”The value on the right side of lm represents in fact the last_modified_at timestamp audit field existing on most table objects, its time rendered as microseconds since the Unix epoch. It should not matter, however. The client is not asked to decompose the value. There is no guarantee the last_modified_at date is actually the basis of the response header. What the client must do: pass the value on to the server in an If-Match header, whenever a concurrency sensitive action is attempted. If the deletion for the afore group is done with the wrong (mind the last digit) etag
If-Match: “lm:1776098785143569”this is the result:
The server answers with a HTTP 412. (The current browser front-end still forwards most server side error messages 1:1 to the UI. This will soon be corrected.)
In order to facilitate deletions, many list endpoints send responses that include etag information too, e.g. a list of groups may look like this:
[
{
“id”: “cd7e5e34–10d9–4261–8191–01567e647d7d”,
“name”: “Together”,
“status_id”: 3,
“office_id”: 1,
“created_at”: “2026–04–13T18:46:25.143561+02:00”,
“last_modified_at”: “2026–04–13T18:46:25.143561+02:00”,
“created_by”: “8315c79f-0835–4eb9–8ed3–8d49a5327883”,
“last_modified_by”: “8315c79f-0835–4eb9–8ed3–8d49a5327883”,
“meta”: {
“etag”: “lm:1776098785143561”
}
}
]Therefore the browser client can send a delete request straight away, based on a rendered list, without the need to send an extra GET request to retrieve the specific group. And the browser knows that the request will be rejected, if the client’s data is stale.
Idempotency on the other hand — when not implicit in the request method as such (Meek bank PATCH requests) — must be a conjoint effort of client and server. The server can’t send any information in response headers to be passed on e.g. in POST requests that insert database rows.
In order to guarantee idempotency the client must send an Idempotency-Key header, e.g.
Idempotency-Key: 8f93008c-9d69–45a4-b711-e863713147ddThe UUID value on the right side is randomly generated by the (browser Javascript) client at page load. This key will be saved at least 2 minutes by the server. (With the current configuration, the Go background job might need 30 seconds on top to remove it from the cache.)
The UUID must be generated in an early stage of the request. It must not be generated as part of the execution context of a button click, but exist prior to it. Otherwise a double click would send two different idempotency keys and both would trigger the same action.
Most pages of the UI obey the following rules:
- The UUID is generated, when the user navigates to the page.
- When the user clicks on “submit”, the form is disabled and the submit button can’t be clicked anymore.
- At the end of a successful submit action, the browser navigates to a view of the created object, which can not directly be modified.
- At the end of a failed submit action, the browser stays on the (editable) page and the submit button is enabled. The UUID is not re-initialized, because the client can trust the server does not cache UUIDs of failed actions.
For the more likely case “request already processed” idempotency means “no error”. The server lets the client know nothing was created, — but hardly — can you spot the difference?
For the rarer case “request currently being processed” the server answers with a HTTP status code 409 instead.
Idempotency for scheduled jobs
At the time of writing Meek bank comes with 7 scheduled jobs out-of-the-box.
These jobs process accounts transactional one-at-a-time. If set up as recommended, the scheduler is triggered on an hourly basis. So how come, for example, saving interests for a single account are not accrued 24 times a day?
Let us first distinguish between jobs that are idempotent by definition and jobs that need extra care: Of the 7 jobs, nightly_loan_account_maturity can be executed repeatedly without danger of creating duplicates. This job transitions loan accounts with outstanding balance from “active” to “matured”. An inactive account is not subject to this job. Therefore we can execute the job as often as we want — it is idempotent. All the other jobs are not.
As the catch-up policy column might indicate (see above screenshot) — how job runs are organized is (in particular considering different error paths) a complex subject. This needs to be discussed in a separate article. But on a very elementary level we can answer the question by looking at the general design of an isolated account transaction: Every type of account (Meek bank currently has loan, payment and saving accounts) is represented in the database by a subledger table: loan_account_transactions, payment_account_transactions, saving_account_transactions. The subledger tables all have their own external_reference field. This field is coerced to be unique per account ID. All 6 jobs requiring idempotency write to this field a standardized text:
<prefix>:<YYYY-MM-DD>The prefix is a static name of the job. For loan account installments (where we iterate over loan_schedules, not directly over loan_accounts) the format is slightly larger, because the specific installment is added to the string:
<prefix>:<installment>:<YYYY-MM-DD>If writing the subledger row fails due to a unique exception, the job continues with the next (account or loan schedule). This guarantees idempotency for scheduled jobs on the level of individual accounts resp. installments.
