Software Development Lifecycle: The Development Stage

‍

Goal #

In this post, I will walk through how to break down the software development lifecycle (SDLC) into subparts, and go on the discovery process of figuring out the set of companies that are working in each area. I will put emphasis on the development stage, since that’s the area that DevZero focuses on.

The next posts will cover how these companies fit into the SDLC, and if it makes sense for a new project to be spun up, targeting a specific problem space.

How does the engineering process work? #

Usually, if hacking on something small, you’d end up writing a small POC to gather feedback first. If it’s something substantial within a company (obviously depending on culture), you would write a brief document detailing the problem, the solution, and proposed implementation/architecture and how it would interoperate with existing systems. I have used LucidChart, Google Docs and Miro to achieve this, but Figma and Statewize look awesome too.

Either after or during the design process, some code will start to get written. This is where we start seeing a heterogeneity in the approaches employed.

1. For most companies, software is written on the local computer.
2. If there are some security, access-control, etc requirements, the software may be developed on cloud-hosted virtual machines that the company has an internal policy/platform around.
3. Some engineers also develop in docker containers, which resembles the production environment fairly well.
4. People working on infrastructure projects either develop inside Vagrants (since docker containers have some limitations) or bare-metal hosts or inside a virtual machine (depending on how the company manages their production infrastructure).

Then the engineer will pick a programming language and (usually) a framework that their project will be built on. This is influenced by the engineering culture of either the company, or more commonly, the team/organization that the engineer sits on. With the movement towards containerization, there has also been a movement towards using strongly-typed languages (there are multiple benefits to this, including from a security perspective). The larger companies predominantly use Java/Go/C#, but depending on requirements (data science, machine learning, frontend web), a lot of code is also written in Python/Node.js. Recently, there has been increased interest in using Rust as well {StackOverflow survey}.

Companies also have internal frameworks and general patterns on how to structure projects — code search tools are very helpful in this area too (think, Sourcegraph, OpenGrok, your editor, GitHub, etc). Buck, bazel, etc are usually used for general project tooling, but it’s quite common to wrap some tooling with Makefiles, Fabric, Poetry, etc.

Then you build your feature/project, add/fix tests, put it up for code review, land, and then hopefully CI/CD is responsible for deploying your project to appropriate environments, based on integrations with monitoring tools.

The visualization below is a subset of products we researched to develop a better understanding of the current state of SDLC, to get a better sense of the direction that the industry is heading in.

Future of local development #

Although I have not worked at Google or Facebook, the development experience there is very well respected. The tools used in these two companies, as well as in some of the larger tech companies, while really good, are usually quite tied to the internal infrastructure/practices. Therefore, they are not easy to clone outside without boiling the ocean.

We take some lessons on the basic requirements from those companies.

1. Developers want fast computers, and for their projects to “just work”, and keep working
2. Security/IT/Compliance needs to run software on computers, which often adversely impacts developer productivity
3. Developers don’t update/upgrade computers often out of fear of environments breaking/having to be set up

Compliance/security requirements will continue to evolve, so there’s not much room for us to change that. Yes, there can be better monitoring/log-collection agents, but those will continue to adversely impact the developer tooling in some way or the other — most usually, this happens around competing for local resources (which are obviously not elastic).

Project usually have “set up” steps and general requirements — these don’t change very frequently, but are critical to (eg: repository is tied to specific Java version or library, but you either have a different one on your local laptop, or have a computer with a processor where it doesn’t compile anymore). Docker containers are pretty good for this, but what would be even better was if a computer was built around the repository or collection of repos.

If we can make the concept of a dev environment disposable in nature, developers can simply get a new computer whenever they want, built off of a template that is kept updated at all times.

Internal infra/security/compliance/IT teams can piggyback off of the disposable nature to keep delivering new software, but an agent present inside the managed host can also be responsible for more critical requirements.

Getting this computer needs to be fast, and the overall new experience needs to be similar to the one the developer is already familiar with. Yes, a few years down the line, a lot of development is going to happen inside a browser, but there is a better path towards introducing developers to that world.

Looking ahead #

At DevZero, we look at projects as not just a source code repository, but as a source code repository together with the environment that it expects to be run in. This environment is either a complete computer or a container, and the developer or template owner (together with IT/Security, etc) has full control over how the environment is configured. We will reach a world that is cheaper, faster and better than it is today where developers have all their requirements met, and as does the business.

References:

• An ex-Googler’s guide to dev tools
• GitHub — jhuangtw/xg2xg

‍