The big news
- Sphinx-based docs
- Storage API
- Storage Layers: stores, backends, middlewares
- Mimetype items
- Item OO user interface
- Code cleanup
- Tree based transformations
- Processing Instructions
- GUI editor update and different approach
- Themeing with Jinja2 templating engine
- HTML / CSS / Images / ...
- New plugin system
- Framework / Libs / Python
- Google App Engine (GAE)
- How to help
- Release date
- The big news
MoinMoin 2.0 (short: moin2) will be a big step in moin development, we are rewriting and refactoring some core components of moin heavily.
Documentation: moin2 online docs or docs/ in the repo.
Issue Tracker: http://bitbucket.org/thomaswaldmann/moin-2.0/issues
the 27c3 Lightning Talk Slides (en)
the Europython 2010 Lightning Talk Video (en)
The big news
We are working on documentation using the Sphinx documentation system, which is also what python.org uses.
See the moin2 docs on readthedocs.org or docs/ in the repo.
Docs are far from finished, help is appreciated.
moin 1.x: the storage code was mixed into Page, PageEditor and AttachFile modules (yes, quite messy).
moin 2: storage code will be in MoinMoin.storage package and misc. storage backends will be supported, all conforming to the same "storage API".
Storage Layers: stores, backends, middlewares
- there are simple "stores" that really store key/value pairs somehow:
- fs - filesystem store
- sqlite - sqlite3 database, which comes with python
- sqla - supports all sorts of SQL databases supported by SQLalchemy, such as PostgreSQL, MySQL, sqlite
- Kyoto Cabinet and Kyoto Tycoon store
- GAE (Google App Engine) ("gae" branch in the repo)
- memory - memory store, only for special applications, like testing/debugging
- on the layer above these stores, there are backends that offer metadata/data storage:
- stores - tie together 2 stores to form a backend
- fileserver - expose parts of server filesystem (read-only)
- there are some kinds of middlewares / layers above them:
- routing middleware - namespace-based routing to the backend(s)
- indexing middleware - do powerful / complex stuff with the help of Whoosh, including search and indexing
- protecting middleware - checks permissions with access control lists (ACLs)
- validation code - checks data types/values whether they are acceptable
Namespaces can be used to keep stuff separate (on the frontend/logically as well as on the backend, potentially accessing different backends for different namespaces):
- new namespaces can be defined
- map namespaces to backends
- map external wiki namespaces
More info: MoinMoin2.0/Namespaces
In moin 1.x, there were 3 main types of stored objects in the wiki: pages, attachments and user profiles.
In moin 2, there will be only 1 type of stored object: an "Item".
- metadata can be any key/value pair
- data can be of any type: text, binary etc
- a revision is made from metadata and data
- an item is made up by all revisions sharing same ITEMID in their metadata
user_dir with user profiles
items with mimetype: application/x.moin.userprofile
data_dir with pages and attachments
item with mimetype: text/x.moin.wiki in metadata
item with mimetype: image/jpeg
items with appropriate mimetype
attachments have no metadata
revisions with arbitrary metadata
attachments "inherit" ACLs from page
all items/revisions may have own ACLs (in their own metadata)
As seen above, moin2 does much more with (own) metadata and even enables storage of user-defined metadata.
To speed up processing, metadata of items and revisions will be indexed (using whoosh).
Index can be used for:
- selecting wiki items/revisions by some criteria
- by current item name, by uuid, ...
TitleIndex like item list
- more difficult than in moin 1.x as we have all sorts of item mimetypes
- but we either want to show just some specific mimetypes (== "wiki pages")
- or we only want to show toplevel items?
- theme: when showing some item, list sub-items in some panel
- mimetype-specific listings
RecentChanges == items/revisions changed since date
- "Items edited by me/by some editor"
- Items linking to some specific item
- Items tagged with some tag (tags [metadata] will replace categories [data])
- selecting user items by some criteria
- by uid, by user name, by email, by jid, by ldapdn, ...
- users subscribed to some wiki item
- sorting such lists (by name/date/...)
moin-internal usages (like e.g. name -> itemid/revid lookup)
Item OO user interface
moin 1.x: pages and attachments had separate user interfaces: deleting a page worked differently than deleting an attachment, uploading an attachment worked differently than uploading page content, pages had revisions, attachments not.
moin 2: unified user interface for items:
- show item (render wiki page, render image/video with some additional data - all done in wiki content area)
- item infos / history
- modify item (create, edit, upload)
- rename (change name metadata)
- delete (remove name? TODO)
- download item content (raw wiki page, raw image)
Page, PageEditor, AttachFile code modules are dead.
- UI still needs much polishing / themeing before end user can use it, see themeing.
Lots of poor code was deleted, still some left.
Tree based transformations
(See /Converters for more information)
moin 1.x: parse wiki markup line-by-line, create html on the fly, which caused problems: poor html, no xhtml, poor include, poor TOC.
moin 2.0: input -> converter -> DOM tree -> converter -> output
wiki -> DOM
creole -> DOM
reST -> DOM
pygments -> DOM
docbook-xml -> DOM
image / audio / video -> DOM
DOM -> html5
moin 2.0: reverse direction (e.g. for GUI editor)
html -> DOM
DOM -> wiki
DOM -> reST
- generating well-formed x(ht)ml(5) output is easy now
- we use emeraldtree, which is a fork of elementtree
- operations on the DOM tree make some stuff much cleaner and easier:
- section edit (TODO)
html -> DOM -> html filters out dangerous stuff
- more practical testing of dom code, fix bugs
HelpOnProcessingInstructions have been replaced by
in text, same functionality
#format specifier and optional-arguments
"contenttype" in meta and currently no optional-arguments
? should be a meta key/val
? should be a meta key/val
? can be replaced by key/value meta parameter. value can be a list
GUI editor update and different approach
- ckeditor 3.x
- currently just works with html items (editing html, storing html)
- no conversion problems
- but: no collaboration between gui editor and text editor users on the same item
- conversion is WIP (see Tree based transformations)
Themeing with Jinja2 templating engine
moin 1.x: Output generation was mostly done by python code with embedded html. Some of this was within pluggable themes, some was hardcoded.
moin 2.0: use Jinja2 templating engine for theme and user interface.
HTML / CSS / Images / ...
- html5 (xmlish "polyglot" type), we now can really do that
- maybe svgweb for supporting svg stuff on crap browsers
- reorganize / cleanup css and images
- use stylus css meta language to have a more comfortable way to deal with css
- make css / js / html / images more extensible, modular, pluggable
Storage contents can be serialized / unserialized to a custom, simple format that is basically a stream of json (revision metadata) and binary data (revision data).
- backup / restore
- import from / export to other apps
- import from moin 1.9
New plugin system
The old plugin system is not powerful enough and too complicated.
Just rough ideas, nothing final:
- use WSGI for action-like stuff (actions have full control over http response)?
- WSGI won't be enough as a general plugin mechanism (it is too low level, only for web stuff, not usable for internals like macros, converters, etc.
- plugins should be able to have own i18n, static files (css, js, html, templates)
Framework / Libs / Python
- we require Python 2.7 now, so no need to test with older pythons any more
- We use Flask now as the framework.
- restful URLs
- werkzeug for WSGI stuff (since moin 1.9, will get even more and better used in moin2)
- jinja2 for templating
- using flask-themes for themes
- flatland (+ j2 macros) for forms generation / validation of entered data
- use flask-babel, babel, pytz for i18n/l10n
- using flask-script for scripts
- using flask-cache for caching
- use whoosh for indexing / selecting / searching
use xstatic to have static files in extra pypi packages
moin2 got rid of all the big 3rd party stuff that was bundled with moin 1.x and uses python packaging now, so you can use pip to install it (e.g. into a virtualenv) and it'll fetch all requirements from pypi.
We also have some scripts to build a standalone or GAE version that doesn't require using pip / setup.py.
Google App Engine (GAE)
moin2 now basically works on GAE (see "gae" branch in the repo). Test site: http://moin2-test.appspot.com/
There is still some todo for GAE in the issue tracker (search for GAE there), contributions and feedback are welcome!
How to help
As you see above, all this is quite a big task and we need help:
- people with a good artistic skills and taste for theme development
How to start:
Talk with us on MoinMoinChat.
- Read and try the new code.
- Expect brokenness/incompleteness and help fixing it.
- Bring in your ideas about how to improve. Some stuff is not specified yet.
As moin2 is still in heavy development, we can't give any release date yet.
If you want it to happen sooner rather than later, think about helping.
But we can give some criteria that help you to choose:
choose moin2 if ...
- you are a developer and
- you want to hack it / fix it / improve it
- you want to write a wiki in python (don't start from scratch, we already have quite some useful stuff and there's lots to play with)
you plan to do a bigger software development project related to moin that won't go into production soon anyway. if you would do it for moin 1.9, you'ld likely have to rewrite it for moin2. while you develop your stuff, moin2 will get nearer to release and moin 1.9 will get more outdated.
- you are a user and
- you like to play with cool stuff, even if it is partly broken
- you like less, but more powerful features
choose moin 1.9 if ...
- you are a user and
- you need something stable for "production"
- you need functionality that is not (yet?) present in moin2
- you need something that is highly compatible with moin 1.9
- have lots of important data in moin 1.x format
- you are a developer and
- you want to do no or only small changes or extensions for moin (note that you can use some stuff we also use for moin2, like jinja2, flatland, sqlalchemy, ...)
Synchronisation of 2 wikis?
How about the synchronization of 2 wikis, is that included in moin2, and if, does it include also the attachments?
A: If we implement sync, it likely will include all types of items, but it is not implemented yet.
There is a GoogleSoc2012 project working on branching/merging/syncing.
For navigation it is very useful to have a navigation tree for the pages (preferred left column). Is that possible now in moin2?
-- RudolfReuter 2011-02-22 13:40:10
A: Not implemented yet, but everything is possible - it may be just not in the first 2.0 release. If you want to help with a great theme just jump on it.
How would one go about porting extensions to Moin 2.0? Right now, I can see lots of dependencies on the formatter and parser APIs in Moin 1.x that I would need to replace with something else. Will there be equivalents in Moin 2.0, and if so, are these things the kind of contributions that Moin 2.0 needs? If not, which APIs would I need to target? -- PaulBoddie 2012-02-25 18:22:46
moin2 is a rewrite of moin1 and works quite differently at many places. So likely a moin2 extension will be a rewrite of a moin1 extension. Please note that there is currently no plugin system. -- ThomasWaldmann 2012-02-25 22:17:16
Note: Some old content can be still seen here: here