More ZODB Addons

Here are listed 9 ZODB addons.

ZEO

ZEO stores ZODB data in file storage on a server, and serves those objects to ZEO clients, usually web application servers.  ZEO caches data in memory on the server, caches objects in memory on the ZEO clients, and invalidates those caches, when the objects are updated.  

Relational Storage

RelStorage uses a traditional relational database and their client software to store ZODB data.  Relstorage can either store objects with history, so called versioned objects, or just do traditional transactions on objects, not preserving history.  Relatinal storage outperforms ZEO.

Z3C.blobfile

z3c.blobfile allows one to store large binary data such as images, or documents on the file system.  Blobs are properly versioned as different files, and cleaned up during ZODB packing.  It is even possible to cache blobs on ZEO clients.  

NEO Database

Neo provides a ZODB interface across a network of relational databases.

S3 Blob Server

This manages the caching and movement of data from the ZODB servers to  Amazon S3 storage. The links document the architecture. 

S3 Blob Storage Utilities

This package provides Python storages that work with the S3 blob caching server, This project provides:

  • A new "flat" blob layout that stores blobs in a single directory.
  • A file-storage subclass that:
    • Uses the flat layout.
    • Doesn't garbage-collect blobs, but optionally records information about blobs to be garbage collected in a file in S3, to be used by a separate blob garbage remover.
  • A ZEO client storage that downloads blobs from HTTP servers (like the blob server).

ZEO Raid

Zeoraid goes between the ZEO clients, and a number of identical interchangeable ZEO servers to provide Raid like reliability.  

Zope Replication Services

Sope replication services allow one to create copies of a ZODB continuously in real time.  One server is the master, transactions happen on it.  The other servers are the slaves, adn get copies of those transactions.  Zope Replication services makes it very easy to have high performance read applications.  There are multiple servers to service reads.   ZRS is also great for reliability.  If the master server fails, promote one of the slave servers to become master.

And finally Jim mentions that ZRS can be used to create an object index, on the slave servers, also increasing performance.  

Pickled Trees

Zopache Pickled trees look like a branch of the ZODB greph, but they are treated by the ZODB as a single object.  The root of a pickled tree is a Persistent Container.  (grok.container) The other nodes are pickled containers, and pickled objects or their subclasses.  In the ZMI a pickled tree looks like a tree.  To the developer a pickled tree looks ike a tree of python objects, but the ZODB pickles the tree in with the pickled tree root.  The ZODB reads and writes the whole tree at one time, and also treats the whooe tree as a single object for caching purposes.

Caveats: A pickled tree can refer to other ZODB objects.  Other objects cannot directly refer to the contents of a pickled tree because they do not have a permanent object id.   A pickled container cannot contain persistent ZODB objects, because thei  persistent object's parent references would keep around old versions of the pickced tree. 

A pickled tree is a great example of it is possible for the average developer to create new data distribution models on top of the ZODB. 

 




I invite you to Register and then link to your own blog postings and software packages..

Powered by Zopache, Grok, Zope and ZODB

Robots Crawl This