Introduction to the Fishbowl Process

Over time Zope has become a leading Open Source application server with a large community of users and developers. In the past Digital Creations has been a bottleneck for active participation by the community in developing the Zope platform. One of the reasons for this was a desire by Digital Creations to ensure a sound design focus for the platform. As the community continues to grow it is imperative to have an open development process that allows DC and the community to work in concert while retaining sound software development practices.

Digital Creations uses a mixture of key ideas of the Unified Process championed by Rational Software, plus best practices from extreme programming and real-world Open Source development. We are proposing to formalize a lightweight version of these practices into the "Fishbowl Process" that DC and the community will use to manage the development of the Zope platform and other software projects (such as the PTK).

Fishbowl Motivations

The Fishbowl Process is the way the community, including Digital Creations, will design and develop Zope. The name "fishbowl" comes from the idea brought to prominence by Mozilla.org that there are no internal conversations; rather, all communication, decisions, arguments, and progress is visible as if everyone were working "in a fishbowl".

These are the goals and motivations of this process:

1. Lightweight. The process should be as simple as possible and should not require a great deal of learning or ceremony to use. A process that is too heavy will eventually be ignored.

2. Consistency. When approaching projects, potential Zope developers should be able to quickly come up to speed on requirements and other project information. Likewise, people running new projects should have a consistent body of existing projects to emulate when running their own.

3. Thinking about thinking. If code is the only artifact, we're in trouble. The requirements, design and decision-making that lead to code are as important as the code. Equally important, it means that non-developers can participate in the process by commenting on (and contributing to) non-code artifacts.

4. Formalizing arguments. It is hard to engage a proposal when it isn't clear what is proposed, or to debate an architecture when you don't know what the architecture actually is. At the same time, it is frustrating for would-be developers to be told things such as, "You didn't write the requirements right" when there are no examples of good requirements.

5. Ensuring progress. Projects that are mushy and poorly defined are easily sidetracked and difficult to keep moving forward. Ensuring that projects are well defined and using milestone-based "hill-to-hill" planning as a part of the process can ensure that projects make steady progress.

Fishbowl Structure

A project using the Fishbowl Process progresses through the same four phases as the Unified Process: Inception, Elaboration, Construction, and Transition. If you don't know what these mean, that's OK. What these four phases mean in the context of the Fishbowl Process is described below.

Inception

We have a great idea for a new capability in Zope. Where do we start? The Inception phase is all about fleshing out our idea, determining the scope of effort and deciding whether to proceed.

In the Fishbowl we get our great idea in play by writing a proposal that includes:

• What is the problem being solved?

• What (broadly) is the proposed solution?

• What are the major project risks?

• What is the scope of the project?

• What are the deliverables of the project?

Zope.org members can create proposals in the Proposals wiki of the DevHome site. The Proposals wiki provides a common collaborative environment where people can work on developing, reviewing and providing feedback on proposals. A proposal should be a single wiki document at this point. A proposal is "ready" for consideration as a project when it addresses the above points and there is someone willing to move the proposal forward as an active project.

When the proposal is ready and has a leader, the leader should contact (zope-dev@zope.org for now) to ask that the proposal move to the next phase and become an active project. At this point, the maintainer of the software product in question will do two things:

• editorial review of the proposal - was the process followed?

• technical review of the material in the proposal

This is a recurring pattern that you'll see each time you move into a new phase. At transition points there will be an editorial review where we ensure that we've done things consistent with the process and a technical review focused on the ideas of the proposal itself.

For example, if we were to write a proposal containing only:

        DTML is ugly, non-intuitive and hard. I propose to fix that.

then ask to become an active project the Zope product manager is going to tell us that our proposal is not up to snuff from an editorial point of view. Our proposal at this point is far too broad and doesn't cover the points a proposal needs to address.

On the other hand maybe we have a great idea to embed video game emulation functionality into Zope and we've created a proposal that follows the process to the letter. In this case, the Zope product manager will probably have technical objections to the proposal.

At the end of Inception we have:

• Fleshed out our idea in a proposal that captures the problem being solved, the proposed solution, risk factors, a statement of the scope of the project and expected deliverables

• Incorporated feedback from the community

• Identified a leader for the project

• Gotten editorial and technical review from the maintainer of the relevant software product

Now we're ready to go on to the next phase: Elaboration.

Elaboration

Now that we've staked out the basic ground in our proposal, it's time for the really hard, frustrating, but critical part. Not writing code, but fleshing out the requirements and capturing the major design decisions so that people can really critique the project and understand its implications. Elaboration is all about making sure that our project is solving the right problem and solving the problem in the right way.

When we move into Elaboration we get a project area in DevHome. Project areas in DevHome consist of a Folder containing a Wiki (where most of the project work is done) and possibly a Tracker and other tools for managing the project. Project Wikis have a basic structure to organize the few main artifacts that are used in a Fishbowl project. The core artifacts are the Vision Statement, Risk List and Use Cases.

Once you have a project area, the first thing to do is put a copy of your proposal in the Wiki for historical reference. Next you work on the Vision Statement, which will probably be largely drawn from the material in the proposal (though you'll often want to go into a little more detail in the Vision Statement).

Every project implies risks, both inside the software and in external forces. It is important for the project owner to recognize these risks, and perhaps plan accordingly. For instance, if the architectural choice might imply hotspots in the Catalog that need to be worked around, the project owner might whip up a "spike solution" (to borrow extreme programming jargon) to prototype workarounds. The Risk List captures these risks associated with the project and the proposed solutions to those risks.

The broad outlines of the project captured in the Vision Statement are then fleshed out in Use Cases. The Fishbowl Process focuses on a pragmatic, simple approach: do as little as possible while still achieving the main goal of truly understanding the requirements of the project. Use Cases in the Fishbowl are not meticulously specified with fifteen fields or so as in other processes. A Use Case in the Fishbowl must at least:

• State the goal from point of view of the user

• Name the actor (kind of user) performing the use case

• Provide enough descriptive text to ensure that you've thought things through.

A good way to handle the description is to write it in a user documentation style as if it were to be consumed by the actor. This will often give you a head start on user documentation :^)

If you've used Use Cases in an organization before then you know that there are endless opinions on how they should be done. The main goal is that they provide a common understanding of the requirements. Community review is the best way to make sure that this goal is achieved. A good way to get started with Use Cases is to find a project fairly similar to your own and use the way that that project's use cases were done as a guide for your own.

As you flesh out the requirements in use cases, you will also be able to begin defining the "architecture" for the project. The architecture is simply a partial design that addresses the major risks of the project. Sometimes these key design decisions are captured in the course of architectural discussions that go on in the project Wiki. Other times it may be better to capture them more formally. Again, look to other projects for guidance.

Finally in Elaboration you begin planning the implementation (at least the first few milestones). The manager of a Fishbowl project is not expected to have an all-encompassing blow-by-blow plan to the end of the project. The planning you do should make sure that everyone knows what the next milestone is and what needs to be done to hit it. This "hill-to-hill" planning keeps everyone on the same page and ensures that the project keeps moving forward.

Once we understand the requirements, have made the key architectural decisions and have an implementation plan it is time to have the project manager for the software you are contributing to (Zope core, PTK, etc.) to do editorial and technical review on the work so far.

At the end of Elaboration we have:

• come to a good common understanding of the requirements through use cases

• made the key architectural decisions to deal with risks

• developed a basic plan for implementing and working on other project deliverables such as user documentation

• Gotten editorial and technical review from the maintainer of the relevant software product

If everything is in order, a CVS branch will be created for the project team to start working on. Now we can start following our implementation plan in the Construction phase.

Construction

The time in the fishbowl up to this point has been focused on thinking. Now it's time to start producing some code. What's the right way to do this?

The most imporant piece by far is to continuously plan. Without planning, you're just a one-person show. Isn't it important to communicate the status with people and set expectations? Isn't it important to accept and track input?

The first Fishbowl Process approach to planning is to work on a milestone to milestone basis. These milestones should be frequent, and you should be specific about what is coming in the next milestone. It is important to keep the plan up to date - people working on the project (and especially new people who would like to jump in and help) should always know what is happening now and what the next milestone is.

A good way to handle this is a "status" area of the project Wiki where people can go to quickly find out what's going on now and what the next milestone is. For larger projects, it might be a good idea to give the status of your project wider distribution via the Zope Weekly News. ZWN is a facility for collecting the status of Zope projects, packaging this up neatly, and distributing it far and wide. The Zope.org is more than happy to do much of the hard work of distribution if you're willing to do a good job of giving them status reports to include.

Next, use the Tracker as a facility for accepting bug reports, feature requests, and dumb (meaning, not your :^) ideas. You can then schedule these issues to give people a constant idea of where you stand and what is on your radar.

The Wiki can be your friend as well. "Water cooler" conversations are the enemy of every project, as invariably something critical gets communicated, and someone leaves with expectations that are lost in the ether. Wiki pages can help collect these unstructured tidbits.

In some cases a mailing list will be setup for substantive, large-scale projects. Otherwise existing mailing lists can be leveraged (for now, use zope-dev for this).

The goal of Construction is to deliver the deliverables committed to in the Vision Statement. This often includes things beside code - help system pages, API documentation, user manuals and other supporting materials are often part of the deliverables.

These deliverables must be completed (and tested!) before the project will be able to come off of the CVS branch and be merged into the main codebase. Well-run projects should plan so that the deliverables are worked on in parallel - you don't want to have everyone cram to produce final code and have to wait for API or user documentation to be done.

So at the end of Construction we have:

• completed and tested the promised deliverables

• Gotten editorial and technical review from the maintainer of the relevant software product and gotten agreement that the deliverables are ready.

If everything is in order, you're ready to move on to Transition, where the Zope community owes you a beer, where you get your net.fame and where someone else gets to do all the hard work for a change. :^)

Transition

Getting off your branch and onto the trunk can be a traumatic activity. Your stuff might break some stuff, other stuff might break your stuff. Have pity on the soul who has to sort all of this out :^)

In transition the relevant product manager will work with you to resolve any conflicts and make sure that all of the deliverables go where they need to go to get into official releases.

What happens after transition? Your work goes into the next product release and the world rejoices. More importantly, there is a strong record of how the code came into being for people to rely on later and another good example for future projects to follow.

Conclusion

That's an overview of the activities for the four phases. Being a loose approach, the Fishbowl Process relies on one thing for success: adoption in spirit as well as form.

Created by Brian. Last modified on 2000/07/06.