Mixed Thoughts on the Software Development Life Cycle
I have used this week to collect my thoughts on the challenges involved in software development. Some stem from personal experience, others I only heard in passing. Just by writing down this conscious stream, I became aware of how interwoven many aspects of software development are.The software development life cycle (SDLC) describes a series of steps an organization can follow to develop and deploy applications on scale. Many roads lead to Rome, but even metaphorically speaking taking a shortcut here may not yield the best result in the long run.
As a project grows in size and complexity, ensuring consistency and coherence in documentation and code becomes increasingly more difficult to maintain. In the open-source software community, projects often start out as small proof of concepts that serve a very specific purpose, e.g. to fill in the gaps in an existing framework ecosystem or to develop a program that can work around the constraints of proprietary software solutions.
The choice of SDLC therefore depends not only on the project requirements, but also on the human resources available: especially community driven projects are susceptible to a high fluctuation in the developer team. Therefore, good documentation and rapid feedback is key to encourage outsiders to make their first contribution. Many different factors are at play here that determine how a community can grow and foster something together, though this is a topic for another time.
In the early stages of a project it might actually be detrimental to the progress to enforce strict processes at every stage of the life cycle, particularly if many new and unfamiliar technologies are involved. For this reason, it's up to the person in charge (in the broadest sense of the word for lack of better terms) to initiate the transition to an agile methodology.
Because FOSS projects are typically not being worked on full time, it might be a good idea to replace sprints with open-ended milestones. While it is still desirable to deploy frequent test builds for black box testing purposes, the reality of everyday life is that these types of projects don't have the highest priority, which is usually one of the two major reasons why progress can slow down after some time. However, this only goes to show you that the responsibility for creating release builds should sit on more than one pair of shoulders. Collecting feedback from clients and end-users alike early and regularly is one of the cornerstones of agile philosophy.
It is a common misconception that agile is anything more than an arbitrary set of values which only makes suggestions about certain philosophical aspects of project management. Scrum, on the other hand, is a process framework that implements this thought experiment for real. In the past decade, agile-like project management methodologies have started to make their rounds in software communities around the globe, though its principals are poorly understood by a vast amount of people. At times it feels like we moved back to a waterfall model with sparse documentation and a loosely defined set of requirements. This sometimes can also result in unnecessary feature creep that blows up the code base. From my observations, what often comes too short is the iteration part of the sprint process. Refactoring odd code is being shunned on if the test coverage is low or nonexistent for fear of breaking things that are difficult or tedious to test. In fact, writing the test itself is sometimes half the work and adds to the ambiguity that besets time estimates; after all, nothing beats experience. Therefore it is important to streamline operations (test and deployment infrastructure) in a way that makes it accessible to new members of the team. However, reaching that state in the first place requires an overall good developer experience. A project that is hard to debug can put a huge burden on even the simplest of changes, but one should notice that even new toolchains can be problematic at times. Designing a healthy setup is a balancing act where there are many hurdles to overcome.
While all of this might seem overwhelming at first, one should not aim for perfection at every step of the way. The secret ingredient to success is to try to improve gradually over time. There is an old Japanese proverb that says “Even monkeys fall from trees”, and I think that's a beautiful way to illustrate that it is unreasonable to assume that everything has to work out fine every time.