The Strange World of Bank Python

In 2021, Cal Paterson published an oral history of 'Bank Python' – a term for the proprietary forks of Python used by major investment banks. These systems are so different from standard Python that people often dismiss descriptions as 'ravings of a swivel-eyed loon'. But they power trillions in financial instruments.

Barbara: The Global Object Database

At the core of Bank Python lies Barbara, a key-value store that uses Python's pickle and zip to serialize objects. It's not a traditional database; it's a global, hierarchical key-value store where everything – trade data, instrument data, market data – lives in a single namespace.

import barbara
db = barbara.open()
my_gilt = db["/Instruments/UKGILT201510yZXhhbXBsZQ=="]
current_value: float = my_gilt.value()

Barbara supports multiple 'rings' (namespaces) that can be overlaid for reads. Writes are strongly consistent within an instance, but eventually consistent across replicas. The soft limit for object size is ~16MB (compressed pickle). It's simple, robust, and a far cry from SQL or kdb+ for large datasets.

Dagger: The Directed Acyclic Graph of Valuations

Financial instruments have dependencies – derivatives derive value from underlying assets. Dagger tracks these dependencies in a directed acyclic graph (DAG) and automatically revalues instruments when underlying data changes.

class CreditDefaultSwap(Instrument):
    def __init__(self, bond: Bond):
        super().__init__(underliers=[bond])
        self.bond = bond

    def value(self) -> float:
        return ...  # cached valuation

When a bond's credit rating is downgraded, Dagger recalculates all affected derivatives. Positions and books (sets of positions) also auto-revalue. This is essentially Excel's dependency graph, but in Python with version control and tests.

Walpole: Bank-Wide Job Runner

Source code in Bank Python is stored in Barbara, not on disk. Walpole is a single, bank-wide job runner (like Jenkins + systemd) that executes code from the database. It handles periodic jobs, long-lived services, and builds. Anyone can deploy a job with a small ini-style config – no Kubernetes or Terraform required.

MnTable: The Ubiquitous Table Library

Bank Python includes a built-in table library (MnTable) that avoids the hash-table obsession of standard Python. Tables are columnar, support joins and aggregations, and are used everywhere for risk calculations and reporting.

Why It Matters

Bank Python is a fascinating example of extreme pragmatism: a globally shared object database, code stored in the database, and a DAG-based computation engine. It works at scale for a demanding industry, but its esoteric nature means little documentation exists outside these oral histories. For developers, it's a reminder that 'best practices' are context-dependent – sometimes a global pickle store is the right answer.

Technical Details

  • Barbara uses pickle and zip for serialization, with a 16MB soft limit.
  • Dagger automatically revalues derivatives when underlying instruments change, using a DAG.
  • Walpole stores source code in a special Barbara ring called 'sourcecode'.
  • MnTable is a columnar table library that replaces hash tables for financial data.

Conclusion

Bank Python is not something you'll encounter in open source, but its ideas – global object databases, DAG-based computation, code-as-data – challenge conventional wisdom. If you're building a system that needs automatic dependency tracking and a single source of truth for data and code, consider the lessons from high finance.

Read the full oral history at calpaterson.com/bank-python.html.