FrontPage WikiNG / backlinks

Wiki And Zope

Owner - KenManheimer

Note - this document is still a work in progress. Please do not make inline comments in the proposal - use WikiNGDiscussion for comments.

Problem

The classic wiki (eg, WikiWikiWeb) is an exceptionally low-impedance arrangement for collaborative authoring and presentation of interlinked web content. Zope is an unusually versatile framework for managing and deploying interlinked web content and systems. We're excited here at digital creations about improvements that the wiki way of doing things may offer for our content management story. With Zope we can broaden wikis benefits and the contexts where wiki can be used, ultimately yielding more streamlined, easy to use, yet comprehensive content management capabilities.

Our challenge is to introduce these features in a way that preserves wikis incisive economy - simple yet sufficient - and extend wiki to include capabilities necessary for scaling to larger and more general contexts.

Immediately below we cover a sample of conventional wiki's limitations. In the next section we'll detail several refinements we can see to extend wiki to larger contexts, while preserving wiki virtues.

Classic Wiki Limitations

The wiki's economical interface is extremely effective in certain contexts. See WikiWikiWeb:WelcomePage for the canonical wiki site, and WikiWikiWeb:WhyWikiFails for some evidence of the limitations.

"WhyWikiFails" is probably more about how users need to behave for wiki to work in the community context than it is about wiki being insufficient. However, we do feel that these requirements present obstacles to scaling use of the wiki, in size as will as in the range of contexts where can managebly be used. We want to fill in some substantial pieces, to convey the power of the wiki approach to larger and more general contexts.

Here are some representative limitations of the convention wiki:

• Wiki depends heavily on self-policing for management of chaos:

  • Anyone can change any text, and it's up to them to record who did it

  • Anyone can visit unresolved links to create pages

  • People involved in discussions need to monitor pages of interest to notice changes and to identify the substance of the changes

  This means that the burden is on the users to play fair, and take it on themselves to notice and track changes. Uncooperative users can easily disrupt an endeavor. Even for cooperative users, there are serious limits on the number of things they can track using conventional methods like revisiting nodes of concern, and/or checking RecentChanges pages.

• Conventional wiki depends on manager intervention to perform reorganization activities - deletion of pages marked obsolete, page renaming, etc. This centralized responsibility is a bottleneck, and further, changing the organization of a wiki can be painful and disruptive, because it's generally hard to arrange for incoming links to track the changes.

• It's up to the wiki visitor to navigate links in order to infer the structure of the wiki - the relationships between pages. The difficulty in getting a comprehensive sense of a wiki - in envisioning its "table of contents" - increases fast as the wiki's size increases.

• Wiki's simple approach to discussion about existing content - typically, including comments directly in the subject text, or creating links to new comments - become cluttered quickly in the former case (making it increasingly hard to identify the content under discussion), or making it hard to keep track of the connection between the content and the comments in the latter.

Solution - What Zope Can Offer Wiki

The section above sketches several ways that we see need for extending or changing wiki for use in wider contexts. We can see deploying some (existing or planned) Zope capabilities to enhance the wiki foundation, without sacrificing its essential strengths - in many cases, changing the wiki only to expose a native feature of all Zope objects:

• StructuredText - StructuredText provides an eminently readable and writable format for expressing document structure. With the addition of WikiName and [] forced wiki references it offers a powerful and coherent extension of wiki's economical simplicity.

• Membership - associate actions with members. This provides a basis for discrtion:

  • About who can do what (associated with Security, see below)

  • Attribution of changes for tracking

  • Registration of interest, to enable users to place the burden of tracking which changes they have or haven't seen on the system.

• Security - offer control over which members can do what. Zope has security intrinsic to any objects and actions. The challenge is to engage it in a way that doesn't complicate common wiki actions. We plan to organize it around the Organization/nesting plans, below.

• Through-the-web object management - Zope already offers comprehensive facility with creation, deletion, renaming, and other user-interface content management. Currently those features are offered though the Zope "management interface". They could be exposed less awkwardly through the wiki interface (eg, on or near the edit page), as long as they are exposed according to the user's privileges.

  We would keep common management choices - eg, browsing and then editing and creation of wiki pages - as the immediate, prominent choices, and add options for selecting among other document types, in the create-a-page form, and offer options for deletion and rename during edit. (WikiNGDiscussionAltTypes has more details on this topic.)

• Organization/nesting - page relationships and namespaces. One enhancement we've already made to wiki innards is to register the page from which a wiki page is created as its "parent" - and to provide things like family tree info in key places in the page presentation. This "nesting" info provides natural, comprehensive table-of-contents orientation in a wiki, regardless of its size. This topical organization also conveys the relationships along which the proposed paths for notification and security can propagate.

  Implementing explicit tracking of the organization structure also would facilitate such things as smart automatic revision of incoming links according to changes (renames or deletions) of their targets, and general optimization of searches, backlinks assessment, RecentChanges, etc.

• History - All Zope objects are versioned. Recent Zope changes are exposing this versioning info for all objects, indicating the owner of the revision and a facility for comparing arbitrary versions in the history. Zope wikis can utilize these facilities without any fundamental changes to the objects, and the functions could be employed in a higher level, more user-controlled checkin style operation.

  One underlying Zope change still pending, in this vein, is the ability for an object to be optionally exempted from packing. (We currently maintain wiki version info by not packing - on zope.org, by maintaining a mounted storage separate from the main storage, so the main storage can still be packed.)

• Notification - Zope wikis will be able to leverage another generic (planned) Zope service, the Observer-pattern based Notification service (ZopeInterfacesWiki:ObserverAndNotification). This will implement a general system for notifying user of changes according to their registered interests, with added benefits of tracking exactly the substance of those changes for the user (see History and the last item in Membership, above).

  Notification begins to enable some mailing-list style collaboration qualities, with finer-grained content-based control.

• Integration with other network access modes. Eg, FTP and WebDav^x browsing and editing of wiki pages. Zope ZWiki already provides for ftp access, though it's rudimentary, and it's not clear how to provide for greater discretion about metadata like alternate document types, permissions setting, and so forth. It will probably be easier to integrate these modalities with WebDav^x - and WebDav^x may be the basis for a good revision management model, with which Zope may well be reconciling its revision model...

  Email is another possibly very interesting access mode - at least for appending to wiki discussions. Email is already clearly an important mode for delivery of notifications. By providing for emailed responses to the notifications, as followups to threads within discussion-style wiki documents, we could have content-based mailling lists. The question would be whether we'd want/need to go further, to allow for submission of new pages/issues via email. This would allow for roundup style interactions, with the wiki pages constituting built-in archives - really, full fledged content-based mailling lists. Roundup may provide lessons that make the email-based user interfaces obvious and manageable, in which case this approach could be a win.

• Review and other workflow-based features - The observer object could also be instrumental for generic workflow-based activities with wikis - including review and approval of changes.

• Provisions for customizable wiki-document metadata - which allows views on collections of wiki documents according to classification, not just by topical nesting but also metadata configurable settings assigments by document authors and users.

• Provisions for structuring documents to organize discussion and commentary.

  There's a number of thrusts, here.

  For discussion, i can see two applications of a wiki document option that allows commentators (non-authors) to only add text, not change existing text. (Zope would diff the new revision and reject it if it contains changes to existing text.) Simplest application would be accepting changes only at the end - weblog style. Next simplest would be one that allows insertions anywhere, for comments next to subject lines. (Zope could offer readers knobs for controlling visibility of annotations.) The document authors could have the privilege of editing any text, to consolidate and refine. Both applications would be useful for different kinds of discussions.

  For more elaborate editorial and commentary annotations, i can see layered documents, using mixin objects that provide a tailored view on other or contained objects. The mixin would be a layer by which annotations are associated with text passages in the rendered subject document, like the crit system does for arbitrary web pages.

  Overall, document authors could use a particular annotation structure according to their needs. Eg, discussion objects for points which can be discussed, or brief editorial passages to give feedback, and author checkmarks for when they've satisfied or refute the suggestions, etc.

Risks:

• Perhaps the chief risk is that we wind up cluttering, overloading, or otherwise complicating wiki controls and operational flow, and wind up sacrificing the intuitive ease which is its primary virtue. This would defeate the benefits we seek.

  The response to this is to be careful about what and how we incorporate new functionality, so that it is evolutionary and well suited to the wiki way. This will involved a few keys:

  • Incorporate new features that are very non-intrusive - for example, by exploiting functionality native to all Zope objects, such as the new history facilities, and exposing the functionality only where it would be needed

  • And/or extend wiki in a way that is consistent with the existing foundation.

    For example, incorporation of nesting controls - a page's default parent is the page from which it was created, and the setting involves zero user intervention. The controls for changing a page's parents is integrated with the backlinks page - this fits well, since a parent must be among the backlinks, and presentation of the backlinks already is an intrinsic part of the wiki foundation.

• Another risk is biting off more than we can chew. The features detailed in "What Zope Can Offer Wiki", above, are extensive, and will take a lot of work.

  Weighing against that is that much of the work is in general Zope infrastructure of general value, intended to be done not just for wikis, and the effort to deploy those features in the wikis will often be relatively small, ie just to expose the functionality in suitable ways.

• One smaller, more contained risk is adequate reconciliation of the text rendering processes, ie StructuredText versus WikiNames versus dtml versus URI nuances. Eg on the last:

  • can we do references to named tags using wiki names?

  • how about relative references across wikis? how will these interact with StructuredText and basic WikiNames?

  This risk would probably best be addressed by explicitly spelling out the behavior of each treatment, making suitable choices for the situations where they interfere, and then seeing how the organization of the processing is affected. It's likely that StructuredTextWiki:StructuredTextNG will make it easy to organize the processing however we need.

Project Scope

This project will involve reimplementing wiki as a ZClass^x test bed for elaboration and implementation of the above features. We would collaborate with central Zope developers so that emerging features like history and the observer/notification system are well tailored to our purposes, and sometimes contribut to the development of those features.

Deliverables

• ZClass-based wiki as a test bed for development and exploration of these features (which jim refers to as "micro-WikiNG"). It would probably incorporate StructuredTextWiki:StructuredTextNG, extensible/configurable metadata, and by ZopeInterfacesWiki:ObserverAndNotification hookup, be organized using...

• WikiOrganization object(s) that represent the organizational info of a wiki namespace, and is hooked up to observer for its subject wiki pages to track updates. (We would need to consider and design how wiki namespaces are structured, probably in the light of Use Cases about how wiki pages might refer across wiki, and whether or not pages should be able to be included in multiple wikis.)

• Elaboration of wiki-tailored security, notification, and revision navigation systems, and implementation of those systems.

• Design and implementation of good support for annotation that makes it easy to see the current version of the content while also making it easy to see the comments associated with the text. (One possible implementation is with an annotation layer document mix-in, and optional incorporation of that layer to wiki documents.)

• Elaboration of a plan for Observer-keyed workflow, implementation of a simple rules system (like i did for the tracker), and hookup with the wiki to support collaborative authoring and document-processing workflow.

• Design and implementation of a structurable wiki document.

• Design and implementation of a few modest (or not so modest) applications which employ many of these features. Some possible examples are:

  • A collaborative proposal authoring system, with author/editor/commentator roles/priveleges, etc

  • A content-oriented discussion system which supports novel features like moderator options for consolidating discussions in addition to traditional option of reviewing contributions, and on-the-fly voting for points raised in the course of the discussion.

  These first two may overlap, in that proposal development involves feedback and discussion. Perhaps they're two parts of a single effort. Some more examples:

  • A patch manager

  • Longer term - an issue tracking system (maybe not a far reach from a cross between the first three, though)

  • Longer term - a souce code revision control system. ! There are many ways that the facilities of such a system would be instrumental for other systems - eg, explicit versioning and branching for collaborative authoring. We'll be implementing at least some of these pieces, and may find ourselves within shooting distance of the whole shebang.