Designing More Effectively

Over the last year at Luminous Computing, I've worked as a product manager and AI lead defining system-level architectures for our AI chip. My approach to design has evolved substantially over the last year, and developing the below principles has exponentially increased my productivity.

1) Design at the system level.

It's important to consider how an entire system, or product, solves a user's problem, rather than focusing on features. Systems come with design tradeoffs; by focusing on one feature, there may not be engineering resources or time to focus on another, and by improving one metric as an organization, another might take a hit. Think about system level priorities or bottlenecks when picking which features to ideate on, and vice versa: when identifying a new feature, evaluate how it'll impact the overall system.

Computer architecture is always about system level tradeoffs and finding the right balance. The canonical textbook on computer architecture explains on its first page that its goal is to "demystify computer architecture through an emphasis on cost-performance-energy trade-offs." For example, at a high level, given a fixed total power budget, by expending more power on memory, a system has less power remaining for computing! If designs focus on optimizing just one feature (for example, just the memory in the above example), they may ignore the impact that those features have on other aspects of the system. Designing at the system level is also important in the presence of bottlenecks. For example, if the goal is to reduce the cost of a system, then effective design would involve breaking down the system costs. If the cost is $900 for the processor and $100 for the memory, then all of the focus should be on improving the cost of the processor by 10x! However, if the cost is $100 for the processor and $100 for the memory, then improving the processor by 10x, while sounding impressive, is not nearly as valuable as improving the processor by 2x and the memory by 2x. Focusing on designing at the system level can help avoid these pitfalls.

2) Communicate with writing.

Conversations are effective for brainstorming, but it's difficult to iterate on ideas orally after a couple of meetings. By writing all ideas down in detail, and in particular stating all assumptions for those ideas, it's possible for other people to identify their disagreements and brainstorm improvements to those ideas. Writing is broadly critical for team collaboration: any information kept in your head is useless to anyone else! It's also helpful to clarify your own thoughts; often, by writing, I've identified holes in my own argument and have understood next steps much more clearly.

An emphasis on written documentation has dramatically changed my personal productivity and that of the Luminous team as a whole. Over the last three months, I have written ~100 pages of design documents, explaining in detail my ideas, assumptions, and implications for engineering strategy. The huge volume of writing comes from the progression in the team's thoughts: every time I wrote out an idea, people could pinpoint exactly where they thought that idea could be improved (e.g. an implementation strategy, or a bad assumption, or an optimization). Moreover, by building a culture that requires written design proposals for ideas to be adopted, other team members have written out arguments that allow me to give more effective feedback. In addition to design documents, written notes on customer conversations are critical for an organization. The written takeaways allow everyone on the team to understand in just five minutes many of the details of a two hour conversation. By writing down all customer conversation notes in detail, other team members have then accurately referenced both facts and opinions from those conversations in subsequent product discussions.

3) Prioritize your time.

Working on design can be abstract, so working on the most important parts of the problem first is critical. If working on one part of the problem changes the problem itself, then all other tasks related to that problem would be rendered useless! A wonderful principle for doing this is prioritizing tasks that maximize the rate of information acquisition, outlined by Jacob Steinhardt in this post about research as a stochastic decision process. Another guiding principle for prioritizing time is trying to figure out why, what, and then how for a product: don't figure out how to implement something until you know what you're building, and don't commit to what you're building until you know why you're building it!

This is a harder one, and it's much more of an art than a science, but improving (and continuing to improve) this skill led to dramatic gains in my productivity. When focusing on design ideation, my goal would be to put forth an idea (often in a 10 page written document!) and then try to kill the idea as quickly as possible__with the goal of using that killed idea to inspire a better one. The way in which I would focus on killing the idea depended on the problem and was a prioritization decision; sometimes it would be by focusing on system design tradeoffs (e.g. if the cost of photonics in a proposed system was 10x the cost of other important components), other times it would be by evaluating an implementation strategy (e.g. maybe a certain approach is unlikely to succeed due to mechanical integrity issues), and on occasion it would be by running a red-team exercise (e.g. how could someone who's not at Luminous do this even better?). Each of these holepunching questions, notably, leads to an improved idea: e.g. how can we use different implementation strategies to reduce the cost of photonics?; how can we solve mechanical integrity issues with better packaging?; and how do we adapt the problem we're working on to make Luminous a unique winner. Picking the right direction to look for killing and reincarnating subsequent ideas is something that I'm constantly working on improving, and is one of the most difficult parts of my work. Prioritizing correctly, though, can lead to the next idea in two days rather than a week, making a designer thrice as effective.

At a company level, prioritizing time translates to focusing. It's hard to make every feature of your product better than all your competitors, but you can focus and make a handful of features truly shine. This fits in with designing at the system level; focus on the aspects of the system that are most important, and make those features the best in the world. At the same time, this also avoids stacked risks: if ten things need to go correctly in order for your system to be valuable, one of them can fail pretty easily, and Murphy's law will kick in. For example, if the cost of a system is determined by five components that each cost $100, and you wish to improve the system by 100x, then you run into issues with stacked risks: if you fail to improve any one of the five components by 100x, then your system can improve by at most 5x. This implies five independent failure points. Instead, by focusing on making a couple of specific aspects of the system shine, a large number of independent failure modes can be avoided.

Focusing will require deciding quickly with incomplete information. You'll never have more than 80% of the information you need, and waiting for more than 60-70% means you're too late in making the decision. Using heuristics to make decisions is critical, and knowing the (rare) occasions to reject conventional wisdom and be contrarian is equally important.

4) Combine top down and bottom up design.

When designing at the system level, there are two ways you could design a system. On one hand, you could identify a user's problem, and then design a system that targets that very problem. On the other hand, you could look at your tools, and propose different systems that you could build, afterwards examining which ideas are the best at solving people's problems. These two approaches can be divided into a top-down vs. a bottom-up approach to system design. Top-down design, i.e. the canonical five step design thinking process, really focuses on making sure the problem is correctly identified. However, it's also important to brainstorm ideas first, since some solutions may help identify new problems that people face, or even create new markets for problems that people didn't realize they had today. It's important to use the best of both of these approaches, since only using one of them may leave you blindsided.

Luminous, as a hardware company, is a case where using both design approaches is especially important. We engage in top-down design when doing customer interviews, understanding AI practitioners' workflows, and understanding what bottlenecks they face in the hardware they use today. This allows us to identify the biggest system-level bottlenecks (for example, more compute power, or more networking?), and design so that we can directly address a customer's needs. On the flipside, many hardware innovations come from a bottom up process. We often ask questions about how much juice we can get out of our matrix multiplier units, or how much networking bandwidth we might be able to supply. These are solutions, and they can inform the types of prospective customers we reach out to and the areas in which we search for problems. For a software company, combining the two design approaches would involve combining empathy on customer pain points with feature suggestions from engineers who might know how to build something cool based on the tools they have available.

5) Rely on yourself, but work with a team.

Designing requires system level thinking, and you can't hand that off; a single person has to have the entire system in their head to understand the tradeoffs incurred by focusing on different features. Understanding each part of the system, however, requires relying on others for details and working collaboratively to answer all the necessary design questions. You will have to work with a variety of different personalities that each approach design differently; for example, some team members may naturally be top-down design thinkers, and others may be bottom-up. Combining those perspectives is critical to being effective.

Relying on yourself and working closely with your team are two central aspects of Luminous's company culture statement. This has manifested in a variety of different ways. For example, engineers on our team realized the importance of gathering market data, and started reaching out to friends directly and then a broader network via cold emails and second degree connections. Having these conversations as team members with engineering expertise made them especially productive, since the goal was to understand the hardware engineering challenges that algorithm developers were experiencing, a problem we could uniquely empathize with. Instead of waiting for introductions to prospective customers, we relied on ourselves and reached out to friends, and after learning from them, worked with other team members' second degree connections to accelerate the process. On the system design side, AI engineers need to rely heavily on the photonics and digital teams to know what's possible, but can combine information about what is possible themselves to define useful products.

Acknowledgements – Thank you to Tanay Kothari and Shruti Ramanathan for invaluable feedback on this post, and many Luminous team members, including Mitchell Nahmias, Dave Baker, Tom Baehr-Jones, and Marcus Gomez for helping shape my perspectives on how to approach design.