Hot Topic: Multi-Cloud

Large enterprises and regulated global institutions are increasingly tackling the subject of utilizing multiple cloud providers - AWS, Microsoft Azure, Google Cloud Platform. This is a complex area with many competing positions and interpretations.

Why Multi-Cloud?

As investment into cloud-native implementations is ramping up, many large enterprises are resolving to tackle the question of multi-cloud adoption. Key concerns typically noted:

1. Avoiding concentration risk - i.e. ability to redistribute workloads across providers, initially or rebalancing in the future, meeting prudential regulators’ concerns around the dependence of our systems on a single provider
2. Retaining ability to utilize unique innovations from competing providers without migration that is too costly
3. Retaining possibility of commercial leverage and arbitrage for commodity services
4. Avoiding single vendor lock-in

Notably, achieving higher levels of resilience is not typically among these, as utilizing multiple availability zones and regions can in many geographies achieve availability targets without multi-cloud.

Dimensions

The biggest challenge in multi-cloud is avoiding having to descend to the lowest common denominator (VMs or IaaS-only) and losing the benefits of higher-level PaaS services (cloud databases, object stores, functions, event queues, etc.)

Still, it's worth outlining different angles from which multi-cloud can be approached:

1. Code-level portability - avoiding having to rewrite substantial amount of business logic or data transformation code in order to enable it to run in alternate cloud provider infrastructure
2. IaaS portability - deployment and run concerns, from utilizing provider-independent images and container configurations to networking (e.g. BYOIP), load-balancing and block storage 
3. Control plane portability - the hardest of them all. Notably includes monitoring/APM, alerts and end-to-end tracing, security & access management

A Measured Approach

I believe that code-level portability is the sweet spot for multi-cloud approach for new services and applications.

IaaS-only portability, e.g. forklifting whole VMs into cloud and eschewing the use of native/PaaS services cloud providers offer is fine for moving legacy workloads off-premise without refactoring (if you must), or for packaged apps where one doesn’t maintain source code and therefore has no control over code portability. However, for custom-developed, new code applications this approach would significantly reduce the value of the cloud proposition.

Control plane portability is today still quite difficult. Some of it can certainly be accomplished by utilizing common toolsets - e.g. OpenAPM, ELK, etc. across providers, but truly seamless control plane across multiple cloud providers is still either too difficult or too costly to achieve. Security and access management models vary across providers with few exceptions, e.g. BYOK has potential to broker root keys across cloud providers.

Code-level portability should be achievable more readily: after all, most PaaS dependencies - object stores, DBaaS, event queues and even functions/FaaS are being increasingly commoditized. It's possible to focus on common API subsets that eschew individual vendor tie-ins yet retain most of the power those native PaaS services offer. After all, why should the business or data transformation logic know whether they're publishing an event to Kafka, AWS Kinesis, Azure Event Hubs or Google Pub/Sub?

Even if common API subsets prove elusive in some cases, it should be possible to isolate dependencies into vendor-specific libraries/adapters, which can be shared across engineering teams. Dependency-injection and vendor-specific configurations added within build pipelines are another viable approach.

Historical Parallels

This in some ways parallels the path our industry travelled with respect to databases - from fully embracing vendor-specific features by writing business logic in stored procedures (with many good arguments marshalled in its favour) to common SQL standard and DB abstractions such as ORM that made cross-DB migrations much more feasible.

I fully expect multi-cloud to be an increasingly active area in open-source and open-core world with focus on common API abstraction for commodity PaaS.

It’s time to build the bank of the future

Original article published on NAB Tech Blog

How can large retail banks tackle the monumental challenge of modernising core systems architecture without sacrificing their role as economic guardians?

When automatic teller machines (ATMs) started appearing in the late 1960s, there was a sense that we’d taken our first real step into the future of electronic banking.

As people discovered the freedom to access cash at all hours on demand and the convenience of card payments began to gradually replace cash, a golden era of mainframe computing helped banks manage the rising volume of financial transactions.

Jump forward to 2018 and most of our dollars are now dispensed digitally (through shopfronts real and virtual), as trips to the ATM rapidly become a thing of the past.

As banks, we’ve successfully evolved our systems and technology to meet customers’ fundamental needs, and they now feel very comfortable with the digital representation of money. But customer expectations have evolved far beyond these basics.

The next revolution

The truth is that many banks are only just waking up to the next revolution in their sector, and I predict that developments in coming years could well be just as disruptive as the rise of electronic banking.

Some drivers of this are evident in the rise of fintech start-ups developing innovative financial tools and services increasingly relying on real-time data. Of course, examples of cross-subsidization and regulatory arbitrage can be readily found as well in the fintech sphere.

Many fintechs do not attempt to replace the core services traditional banks provide, but augment their value proposition. These can only succeed, however, if bank platforms are open enough to allow API-based integration, event-based pub/sub data exchange and other modern approaches to creating flexible, extensible ecosystems.

The question is, how do system architects (who’ve focused so much of their work on the protection of customers’ financial assets) now set about opening these systems up to the wider world?

The scale of the task is significant.

The foundation of a lot of today’s banking architecture is built on a giant network of tightly coupled legacy systems, each purpose-built for very specific business needs, with layers of bespoke integration and security, risk management and control implementations embedded throughout them.

Many of those systems took years to evaluate and implement, but now the race is on to replace them with far more open, flexible and agile ecosystems — all while retaining the robustness and security needed to beat ever-more-sophisticated financial crime and fraud.

Defending our customers’ assets is a massive job in itself — and not one we can afford to fail. So, how do we transform our banking systems without sacrificing our role as economic guardians?

A robust, coherent and ambitious architectural vision is crucial in the transformation.

Thinking big

Modern architectural approaches are not new but sometimes take surprising time to become well-established within our industry as best practices. Services replacing monolithic applications, improving loose coupling via event pub/sub and avoiding distributed monoliths, employing eventual consistency where appropriate to improve availability and resilience to partial failures, reactive patterns, establishing coherent API ecosystems are hardly new but require experience to get right and avoid pitfalls. Increasingly, CQRS, event sourcing and real-time stream processing provide powerful ways to move the needle still further.

Systems, software and enterprise architecture should be about how internals of the systems work, translating into how resilient, performant and scalable they are, not just what functional features they provide. It’s very much about what’s under the hood. However, in many institutions the practice has been focused on buying and integrating technologies, systems and applications rather than designing and building them.

Starting with a coherent strategy and set of architectural principles aiming to maximize flexibility, maintain resilience and security, yet accommodate increasing velocity of change and easier experimentation the business demands at the edge, is key. These principles are translated into a concise set of patterns, technology choices and reference implementations providing reliable guardrails to individual delivery teams aiming to maximize their velocity.

We should evaluate all systems — existing or new — according to these key principles and patterns, so none becomes an obstacle in achieving a flexible technology ecosystem.

Anything blocking or impeding movement toward our agreed target states definitionally becomes technical debt, to be managed proactively. Applied consistently, it also gives us a model to phase out legacy estates where it’s an obstacle to our goals.

People power

As is true for any organization — whether a startup or a large firm — the people we choose to drive our technological journey are as important as the architecture that supports it.

Financial services are experiencing a truly seismic shift, as we seek to solve some fundamentally complex problems with innovative ideas and best-of-breed tools.

And technology as a field is continuing to undergo rapid evolution, offering us new tools and choices, whether it be stream processing underpinning the transition from Big Data to Fast Data, machine learning utilizing GPU compute, or quantum computing promising solutions to previously computationally intractable problems at a price of upsetting the status quo in cryptography. It is a great time to be a practitioner in our field.

Distilling the right set of architectural principles and patterns and picking our technology building blocks wisely, tying these together into a coherent architectural vision in our complex, fast-evolving ecosystem — where failure simply isn’t an option — represents an exciting career challenge indeed.

If you’re interested in joining us, you can learn more about working in technology at NAB here.