Run Your Data Projects Effectively with The Right Design Docs

A good design docs is inseparable from A Good Data Scientist and Engineer — Vincent Tatan, Google ML Engineer

In most cases, Engineers spent 18 months contemplating and writing documents on how best to serve the customer. — Eugene Yan, Amazon Data Scientist

Last month, I presented the undeniable importance of Design Docs which explained design docs as the conceptual lighthouses to build and run machine learning systems.

As design docs are hugely important, I would love to share my principles to create design docs to strongly execute your data projects.

Why?

Design docs provide conceptual lighthouses to guide your data projects.

Design docs conceptually guides you in every step to understand your goals, impacts, and executions to benefit stakeholders. Design docs ensure your projects land with impacts.
Design docs save you time to design to highlight implementations and alternative solutions before executing them.
Design docs host discussions among teams to brainstorm best solutions and implement data projects.
Design docs serve as permanent artifacts to solidify your ideas for future collaborations.

Who?

Your audience is the key reason why you write design docs. In every design docs writeup, you must understand your audiences such as:

Yourself: To identify learning journeys, brainstorm ideas and future impactful projects.
Team members: To identify collaboration points, escalations, and system specific impacts. In the design docs, You need to align your assumptions to team members’ prior knowledge.
Cross departments: To identify cross departmental collaborations. Your design docs need to communicate prior knowledge and success metrics.
Executives: To make decisions. You need to provide solid recommendations to move the needle on high level metrics (e.g: user adoptions, revenue, and goodwill)
External: To foster professional reputations and network. You need to deliver solid takeaways and avoid using jargons .

Finding the Right Design Docs Types for The Right Space (Context)

I would like to highlight three different contexts which require various types of design docs. For terminology, I would highlight these contexts as solutions spaces: Architecture Space, Implementation Space, and Idea Space.

Architecture Space (Stable)

Design Docs Characteristics

Objectives: Document high level architecture on systems which solve complex problems and leads to direct user impacts.
Main Audience: Executives, Tech Leads
Time: Slow and stable. Ideally, it rarely changes except due to disruptions.

Types of Design Docs

Architecture Doc: Document high level system architectures with clear objectives For example, project Google Loon aims to solve the scarcity of reliable internet infrastructures.
North Star Metrics: Identify critical metrics to measure success/failures for executive communications. For example, in customer facing apps, the metrics will be user adoptions while it will be protecting users in abuse fighting apps.

Implementation Spaces (Launch)

Design Docs Characteristics

Objectives: To facilitate system designs implementations for example data storage, ML Ops, data privacy access, etc. This document ensures data products are launched, scaled, and evaluated properly.
Main Audience: Tech Leads, Cross departments (especially up/downstream applications)
Time: Moderate changes

Types of Design Docs

System Design: Highlight system implementation flowcharts, up/downstream interactions, data storage, appeals, etc.
Timeline Launch Documents: Highlight progress and timeline for a system to launch. In Google, we have Standard Operating Procedures (SOP) to ensure each launch is properly maintained and scaled.
Privacy Documents: Manage confidential data regarding users or other sensitive agencies.

Idea Spaces (Experimental)

Design Docs Characteristics

Objectives: To experiment minor tweaks, idea brainstorm, and quick feedback gathering. Idea spaces allow data professionals to seek ideas to deliver big impacts quickly.
Main Audience: Everyone including cross department
Time: Highly dynamic. One pagers are drafted, analysed and discarded on a daily basis. Your goal is to fail quickly and move on.

Types of Design Docs

One pager: Fast moving design docs to facilitate early idea reviews. As ideas grow to proven concepts, the one pager will be promoted into two pagers and system design docs.
Learning Journeys: Identify learning journeys in terms of past presentations, design documents, models launched. In big companies, the learning journeys are necessary to keep track of changes that happen very quickly in cross department and regional collaborations.
Pre Execution Evaluations: What are the expected impacts if we launch a product (e.g: models / tweaks)?
Post Execution Evaluations: What are the impacts after the past launched product (e.g: models / tweaks)?
Pre Mortem: What could go wrong when the product is launched?
Post Mortem: What has gone wrong after the past launched products?

“If you don’t know what you want to achieve in your presentation, your audience never will.” — Harvey Diamond

Five principles To Manage Your Design Docs

These are simple guides on how you can manage design docs.

Start from Ideas: Always start your experiment on one pager (idea spaces). Create a thought experiment and brainstorm quickly before investing further time into the idea.
Invest Time in Lower Level Spaces: The higher the space (Idea → Implementation → Architecture), the more time you should invest. Spend at most 1 week on one pager (idea space), 1 month on implementation/analysis space, and 1 quarter/semester on architectural space. Of course this depends on the scope, but you get the gist.
Prioritize Promising One Pagers: Promote and scale your one pagers based on impacts. If the idea is intended for cross collaboration, spend more time deliberating how your goals align the high level spaces (North Star Metrics, System Design, etc).
Land Into North Star: For general success metrics, focus on ideas with lowest time investments (e.g: small tweaks on machine learning), with high impacts to North Star Metrics. This helps you to build solid foundations to land in higher spaces.
Point Directly to Golden Nugget: Your design docs need to point the audience to the golden nugget as direct as possible. The higher the space is, the more direct this golden nugget should be.

Conclusion

In general, by knowing these principles, you will create design docs to help you:

Navigate dynamic, ambiguous or not well understood projects through all conceptual spaces.
Generate high impact projects that could be promotable for executive communications.
Optimize time investments for the best impacts which highlight the North Star Metrics.

I hope this article helps you create design docs and run your data projects effectively.

Soli Deo Gloria.

About the Author

Vincent fights internet abuse with ML @ Google. Vincent uses advanced data analytics, machine learning, and software engineering to protect Chrome and Gmail users.

Apart from his stint at Google, Vincent is also a featured writer for Towards Data Science Medium to guide aspiring ML and data practitioners with 1M+ viewers globally.

During his free time, Vincent studies for ML Master Degree in Georgia Tech and trains for triathlons/cycling trips.

Lastly, please reach out to Vincent via LinkedIn, Medium or Youtube Channel