A couple million lines of Haskell: Production engineering at Mercury

Mercury is a fintech startup built to serve the businesses of tomorrow. They provide banking services – checking accounts, debit cards, and more – specifically tailored to startups. What’s less known is how they've built this infrastructure. The core of Mercury's banking platform isn't built on Java, Python, or Ruby on Rails; it's built on Haskell, a purely functional programming language. And not just a little Haskell – a couple million lines of it. This article delves into why Mercury chose Haskell, the engineering challenges they’ve faced, and the benefits they’ve reaped from this often-unconventional choice in the finance world.

§Why Haskell for Finance?

The financial industry demands exceptional reliability, security, and correctness. A single bug can have significant financial consequences, not to mention reputational damage. Traditional programming paradigms, while widely adopted, can be prone to errors. This is where Haskell shines.

• Purity and Immutability: Haskell enforces pure functions – functions without side effects. This means a function always returns the same output for the same input, making reasoning about code far easier. Immutability (data that cannot be changed after creation) eliminates a large class of bugs related to shared mutable state.
• Strong Static Typing: Haskell's type system is incredibly powerful. It catches errors at compile time that would often slip through in dynamically typed languages. This significantly reduces runtime errors.
• Concurrency and Parallelism: Finance applications often involve handling many concurrent transactions. Haskell's built-in support for concurrency, thanks to its purity, makes it easier to write safe and efficient concurrent code.
• Formal Verification Potential: While Mercury isn't yet employing full formal verification across its codebase, Haskell’s mathematical foundations make it a natural fit for systems where proving correctness is paramount.

Mercury’s engineering team recognized these advantages early on. They weren't simply looking for a trendy language; they were searching for a technology that could meet the stringent requirements of building a financial platform from the ground up. They needed a tool that would help them avoid bugs, not just find them later.

§The Initial Decision & Building the Core

The decision to adopt Haskell wasn't taken lightly. It meant facing a steeper learning curve for the team and a smaller pool of available Haskell developers compared to more mainstream languages. However, the long-term benefits were seen as outweighing these challenges.

Mercury began by focusing on the core banking engine – the system responsible for processing transactions, managing accounts, and ensuring the integrity of financial data. This was a conscious choice; if Haskell could prove itself in the most critical part of the system, it would be a strong endorsement for wider adoption.

They initially leaned on existing Haskell libraries for cryptography, database access (PostgreSQL being a common choice), and web frameworks. However, they quickly realized that they needed to build much of their domain logic from scratch. This was deliberate. They wanted to have complete control over the most critical parts of the system, and existing libraries often didn’t quite fit their specific requirements.

§Engineering Challenges and Solutions

Building a production-scale financial platform in Haskell hasn't been without its hurdles. Here's a look at some of the key challenges Mercury faced and the solutions they implemented:

§1. Tooling and Debugging

Haskell's tooling ecosystem, while improving, historically lagged behind that of more popular languages. Debugging pure functional code can also be different from debugging imperative code.

• Solution: Mercury invested heavily in improving their internal tooling, building custom debuggers and linters specifically tailored to their Haskell codebase. They also embraced technologies like stack and cabal for dependency management and build processes. Profiling tools are critical for identifying performance bottlenecks. https://example.com/ – A good resource for learning Haskell is "Learn You a Haskell for Great Good!".
• Monitoring: Comprehensive monitoring and logging were crucial. They integrated Haskell with existing monitoring infrastructure (e.g., Prometheus, Grafana) to track key performance indicators and identify potential issues.

§2. Developer Onboarding & Skill Gap

Finding experienced Haskell developers is challenging. The language requires a different way of thinking than many developers are accustomed to.

• Solution: Mercury implemented a rigorous onboarding program for new Haskell developers, focusing on functional programming concepts and the specifics of their codebase. They also encouraged internal knowledge sharing and mentorship. They actively contribute to the Haskell community and participate in conferences to attract talent.
• Gradual Adoption: Instead of forcing a complete rewrite, they adopted Haskell incrementally, starting with the core banking engine and expanding its use over time.

§3. Performance Optimization

While Haskell’s purity often leads to elegant and maintainable code, it can sometimes come at the cost of performance if not carefully optimized. Garbage collection, in particular, can be a concern.

• Solution: Mercury’s engineers became experts in Haskell performance optimization techniques. This included:
  • Strictness: Carefully controlling when and where expressions are evaluated.
  • Data Structures: Choosing appropriate data structures for the task at hand.
  • Code Fusion: Optimizing code to reduce intermediate data structures.
  • Profiling: Regularly profiling their code to identify performance bottlenecks and guide optimization efforts.
• Database Optimization: Optimizing database queries and indexing strategies is also essential.

§4. Interoperability

Integrating Haskell with existing systems and services (e.g., third-party APIs) can be complex.

• Solution: Mercury leveraged Haskell’s Foreign Function Interface (FFI) to interact with C libraries and other systems. They also used tools like http-client for making HTTP requests to external APIs. They carefully managed the boundaries between Haskell code and external systems to minimize the risk of introducing impurities.

§The Benefits: A More Reliable Platform

Despite the challenges, Mercury has seen significant benefits from their Haskell-based approach.

• Reduced Bug Count: The strong typing and purity of Haskell have resulted in a significantly lower bug count compared to systems built with more traditional languages. This translates to fewer outages and a more reliable platform for their customers.
• Increased Developer Productivity (Long-Term): While the initial learning curve is steep, experienced Haskell developers are often more productive in the long run. The language's expressiveness and powerful abstractions allow them to write concise and maintainable code.
• Enhanced Security: The inherent security features of Haskell, combined with Mercury's careful attention to detail, have resulted in a highly secure platform.
• Scalability: Haskell's concurrency features have enabled Mercury to build a highly scalable system that can handle a growing number of transactions and users.

§The Future of Haskell at Mercury

Mercury remains committed to Haskell. They continue to invest in the language, expand its use within the company, and contribute to the Haskell community. They are exploring areas like formal verification to further enhance the correctness and reliability of their platform.

They are also sharing their learnings with the broader fintech community, advocating for the use of functional programming languages in financial applications. They believe that Haskell has the potential to revolutionize the way financial systems are built, making them more robust, secure, and reliable. You can learn more about their engineering philosophy on their blog and through their open-source contributions. https://example.com/ – Check out books on software architecture for understanding how to design scalable systems.

§Disclaimer

This article contains affiliate links. If you purchase a product through one of these links, we may receive a small commission. This helps support our website and allows us to continue providing valuable content. We only recommend products that we believe are helpful and relevant to our audience.