David Xiang

S3 Express One, Value-Less LSM Trees, ShardStore

by dxiang 2024-02-04

It’s early 2024 and S3 continues its ubiquity in computing. The often touted adjectives “serverless” or “cloud-native” usually mean—”we’re using S3 for something.” Amazon continues to push the boundaries of scale and further solidifies itself as the foundation for countless applications, platforms, and companies.

Recently, Amazon released their S3 Express One Zone Storage Class, advertised as the “fastest cloud object storage for performance-critical application.” In this post, I’ll highlight three key features of Express One and share some guesses regarding their implications on S3’s internal architecture.

Management Software

Raising The Alarm

by dxiang 2022-12-22

When was the the last time you realized a project was doomed? That time you knew the plug should have been pulled, but you kept things quiet and watched the work painfully fizzle out.

If you care enough about the work, you probably have these feelings often.

Maybe there’s context you’re missing and you’re being too sensitive. Maybe the implementation is fine and you’re being too pessimistic. But—maybe you’re right and you have the opportunity to save the company years of toil and unmeasurable amounts of opportunity loss.

Realistically, you probably can’t affect the situation. Who are you to call off an important project? Who is another person to question your work? Everyone, from managers to executives, has a hard time with this. Nevertheless, your opinion matters; you might be able to nudge the ship and prevent a terrible outcome. At the very least you can try.

Why bother? When is this appropriate? How do you even communicate this?

Management Software

Software Project Planning

by dxiang 2022-07-30

Software project planning is extremely difficult. In every step of the process—design, build, test, release—a multitude of things will go wrong.

For all the senior engineers, team leads, and engineering managers embarking on a challenging software project, here is a cheat sheet of activities to ensure your work runs as smoothly as possible. Every situation will be different, but common challenges frequently re-occur.

If you are leading such a project, keep these 3 themes in mind as they may easily run away from you:

Expectations—are the timelines and goals still reasonable?
Alignment—is leadership properly aligned and sponsoring this work?
Feasibility—given the resources, can this thing be built properly?

Best of luck.

Life Management

Art or Science?

by dxiang 2022-04-25

Art is subjective, soft, human, unique.

Science is objective, formulaic, algorithmic, process-heavy.

Think about a recent problem you faced and its surrounding context. Was your approach more artistic or more scientific? Did your perception change as you solved the problem? How did it compare with your colleagues?

As with anything difficult to define, let’s begin with principles:

Nothing is all art or all science; everything is on a spectrum.
Any positioning on this spectrum is subjective.
A common interpretation is that science leads to art. A scientist invents after decades of study. A dancer expresses after decades of practice. Said another way—science at a master level feels like art.

With respect to software development, which day-to-day activities lean one way or the other? What is art and what is science?

Life Management

Guidelines For Criticism

by dxiang 2022-03-09

#1 — Do not criticize others while being blind to your own faults.

Do not seek fault in others while staying blissfully unaware of your own.

When others code slowly, it’s because they’re inefficient and don’t manage their time well. When I code slowly, it’s because I’m being thorough and have to deal with a difficult system.
When others hold their technical opinions strongly, it’s because they’re stubborn. When I hold my technical opinions strongly, it’s because I’ve done my homework and need to be resolute.
When another engineer PRs into my codebase, they’re overstepping their bounds. When I PR into another codebase, I am exhibiting initiative.

#2 — Be flexible with others. Be strict with yourself.

This is the explicit reversal of the previous point.

Be flexible with others and offer them grace. Be strict with yourself and stay thoughtful in areas of self-improvement.

By default, examine your own weaknesses before looking for fault in others.

#3 — Do not presume you can teach others.

Do not presume you have a superior understanding. You have some amount of knowledge, wisdom, and experience. Be willing to share it.

This mindset helps avoid condescension and increases the overall effectiveness of criticism.

#4 – Follow through with your commitments.

Criticism and feedback is only heard when there is trust.

In any relationship, you contribute to trust by following through with your commitments.

Management Software

Monolith To Microservices Vs. Your Organization

by dxiang 2022-01-24

Microservices are nothing new. After being deployed successfully by mega-companies like Netflix, many growing engineering organizations gravitate towards this architecture as an aspirational symbol of success. This journey is known to many as the “monolith to microservices” migration.

A few companies (Spotify) have the clairvoyance to adopt microservices from the get-go and may never have to endure such arduous re-architecting. Other companies (Etsy) double down with their monoliths (a.k.a founding architecture) and scale into multi-billion dollar organizations. Many others opt for the long journey from monolith to microservices with roughly the following scenario:

We’ve seen immense success with our monolithic application and we are ready turn up the dial on growth. We need to explore new product lines, hire more developers, and increase velocity. To enable all of these wonderful things, we’ll start transitioning our monolith to microservices. This architecture will enable autonomous workstreams, failure isolation, flexible scaling, and {insert other microservices benefits}.”

I am not a fan of the term “microservice architecture” because it has become increasingly difficult to define. In regards to software, your system is either distributed—or not distributed. With many of us working in the context of distributed computing, I would venture to guess that you’ll find it impossible to use a textbook definition (microservices vs. service-based vs. event-based) to describe your architecture. You may describe your system as a “hybrid” or “ball of mud.”

While microservices are wonderful in principle, many software engineers are intimately aware of their challenges in practice. Parodies like this are a little too close to reality.

When pursuing grand endeavors, engineering leaders often forget to explicitly address organizational concerns. These constraints have the potential to block progress more so than any other technical issue.

Management Software

Use More 1-1-1s

by dxiang 2021-12-30

1-1s are core to management. Ad-hoc feedback delivery, bi-weekly check-ins with your directs, monthly meetings with your skip-levels, annual performance reviews. Their use cases are well-defined and well-understood.

1-1-1s, on the other hand, can be much more interesting. Significant progress can be made if you’re able to get the three right people in a room together. 1-1-1s are versatile and streamline communications for a variety of situations.

While managers are usually responsible for 1-1s, any leader can call for a 1-1-1.

So, when should you host one?

Your intuition will serve you best. Here are a few questions that will help you exercise these muscles:

A new role is being integrated into team—how’s it going?
A new project has caused something to change—are expectations clear?
A team is visibly disengaged—is there conflict?

In this post, we’ll review 5 effective 1-1-1 formats that deserve a spot in your leadership toolbox.

Management Software

4 Questions To Ask Yourself

by dxiang 2021-12-17

As a Junior IC

How comfortable am I developing code and debugging issues?
Could I give a presentation about my team?
Do I know who to contact if I’m not sure how to solve a problem (besides my mentor)?
Did I release a project end-to-end that required me to do more than coding?

As a Senior IC

Has another team around the company changed what they’re working on based on my actions?
How strong is my opinion regarding my team’s product goals and technical investments?
Do I have context around the challenges of my peers (even though it’s not my responsibility)?
Has another engineer on the team surprised me by taking care of a tricky incident without my involvement?

As a Manager

Did my reports demonstrate an expanded scope of impact? Is it easy to justify their promotions?
Did my level of ownership and responsibilities increase in the last year? If so, was it situational or did I seek out those opportunities?
Is there a clear choice for an individual who could take over my responsibilities?
My group drove a lot of impact these past two quarters—is it because my teams have grown or is it because I’ve driven innovations to make my teams more effective?