Head over to our on-demand library to view sessions from VB Transform 2023. Register Here
As cloud architects, we work toward continual improvement, both in meeting our daily tasks and toward the bigger goal of accelerating our organization’s journey to the cloud. We must create both immediate and long-term value.
I’ve spent a good deal of time in cloud systems architecture — and I know that our job is essentially applying tech knowledge to real-world business problems. Not all the problems we face are technical, though, nor are the solutions.
Although the companies I’ve worked for were varied, they shared similar challenges. So, I’ve come up with three key tips for optimizing the impact cloud architects can use to meet complex business challenges, whatever the context.
Use what you’ve got
When faced with a fresh software problem, we often go right to the whiteboard and start scribbling. We fool ourselves into believing that each problem we face is unique. Often, it’s not. This is the time to stand on the shoulders of the wise developers who preceded us and to build a bridge from the past to the future.
VB Transform 2023 On-Demand
Did you miss a session from VB Transform 2023? Register to access the on-demand library for all of our featured sessions.
Cloud architects must frequently work with tight limitations— time, budget, team skill sets — while getting things done ASAP. We must design with the constraints we’re given.
The best way to manage these constraints is to take advantage of existing software services, including cloud application design and architecture development as well as cloud management and monitoring, to see what’s working the best. This may require some institutional forensics into prior cases, to unpack why the successes worked and where the failures fell short.
Look for patterns in both success and failure – the technical as well as managerial and organizational. Then pick out the most solid building blocks to meet your current challenge, and put your mental energy into other parts of the problem that don’t lend themselves to off-the-shelf solutions.
It’s a basic rule of economics that any company, or individual, strives to get the most done with the least effort. So, start with what you have and build on it. There’s no need to reinvent the wheel. Use the wheels you have and start driving today. You’ll be able to get moving faster.
Reframe the problem
Probably the key consideration in any kind of software development project is time to market. The clock is always ticking. The quicker you develop a tech solution to the business problem, the happier everyone is. You want the best outcome with the least amount of complexity because complexity bogs down projects.
A common complaint from cloud architects is that they believe that the tools they have on hand are not sufficient to build the solution that’s needed. In most instances, however, these are the only tools you’ll have to solve that problem. You may assume that means you’ll have to sacrifice functionality. Instead, think differently.
Let me give you an example. In an earlier job, I was tasked with looking for malicious software in the organization’s systems, which numbered millions of machines over many networks. A giant task. It wasn’t impossible, but close.
Then I realized that hunting down malicious software was like managing a system. You’re looking for code or systems that are doing something they’re not supposed to be doing. With malware, it’s intentional and destructive; with simple system management, it’s not.
So, we reframed the problem with a simple question: What are the machines doing that’s different from what they normally do? We began tracking these differences — the deltas — over time. Delta encoding is a way of storing or transmitting data in the form of differences between sequential data, rather than storing complete files. (More generally, this is known as data differencing.)
This narrowed the amount of data we collected as well as how quickly we could analyze it. And it was scalable. An analysis of trillions of events done at millions of calculations per second was manageable. If we had collected and analyzed every bit of code, it wouldn’t have scaled. By looking only for the anomalies, we were able to design a solution.
As we cloud architects approach thorny problems, we must be able to listen, understand, solve and improve – in that order. Sometimes that involves reframing the problem.
Solve by simplifying
This principle unifies the first two points.
I believe that if you poorly invest your resources (time, talent, budget) your outcome can fall short. And if you don’t reframe the problem, you may attempt to develop solutions that are overly complex.
With these in mind, consider this: What could you sacrifice to build a simpler solution with existing software blocks that yield the most tangible results? What requirements for the system you’re trying to build could you essentially cheat on — not in a substandard way, mind you — to achieve your results?
You’re not trying to cobble together an inferior product, of course. You’re breaking complex problems into solvable blocks.
Many organizations are already on this path when they use containerized code. Containers are packages of software that contain all the necessary elements to run in any environment. By virtualizing the operating system, containers can be used anywhere — whether a private data center, the public cloud or even on a developer’s laptop. These blocks simplify and break apart that problem.
This approach has another benefit. There’s a tendency for those of us who build software to work with multiple abstract constraints, like the number of machines you can work with, the software version, and so on. But when you’re developing for the cloud, many of these boundaries disappear.
Your first question should no longer be “How am I limited?” but rather, “How can I use the tools I have to build great things?” Leave the old mental straightjacket behind. Your new cloud-based abilities frees you up to create shortcuts that lead to innovation.
Cloud architects are an essential part of any growing business. To provide our full value, we must be able to listen, understand, solve and improve. Sometimes this requires rethinking what we do and how we do it. That way, we can have a much bigger impact on our organization’s journey to the cloud.
Chad Verbowski is the senior vice president of engineering at Confluent
Welcome to the VentureBeat community!
DataDecisionMakers is where experts, including the technical people doing data work, can share data-related insights and innovation.
If you want to read about cutting-edge ideas and up-to-date information, best practices, and the future of data and data tech, join us at DataDecisionMakers.
You might even consider contributing an article of your own!