12. Pyro Change Log

Pyro versions before 3.0 can be found in the archive.

Pyro 3.0

• Major new version! Not compatible with older versions!
• Pyro is now under the MIT software license. This license is much easier and less restrictive than the previous (GNU LGPL). Please read the file "LICENSE"!
• Major speedup, up to twice as fast as before! (Thanks to new threading code) This makes Pyro in multithreaded mode as fast as singlethreaded mode :-)
• Removed last traces of sockets in core.py. It only works with the more abstract connection objects.
• Added SSL support (based on a patch by Holger Brückner, thanks!). Requires M2Crypto module.
• Simplified objectID encoding, now uses hexlified guids everywhere. No more converting from/to binary strings.
• Changed threading in protocol.py: Pyro used to hang when bad clients connected but didn't write their full message on the socket. Now Pyro splits off a new thread directly, as soon as it detected a new connection or request.
• Pyro no longer spawns a new thread for each request. Instead, it spawns a single thread for each connection, and hands control to that thread. The thread processes requests sequentially, and dies when the connection closes. This causes a major speedup when processing lots of small requests.
• Fixed some obscure potential socket/threading bugs and made protocol version mismatch handling slightly better.
• Connection denied reasons added to constants.py
• Fixed cpu-bound server in multithreading example
• Implemented XML pickling support using Gnosis_Utils' xml pickle, PYRO_XML_PICKLE config item to select it. Server automatically answers in correct pickle format.
• No longer dumps config twice when program uses both init client and init server.
• Fixed some cyclic references. Object-daemon reference is now a weak reference if available (Python 2.1+). Daemon/tcpserver now appears to clean up nicely.
• Added some extra cleanup methods to nicely close socket connections.
• Authenticated connection validation. You can now require the Pyro client to give a valid authentication ID (password) while connecting. Control this using the improved connection validator. Not used by default, so old code still works without authentication. See the "NS_sec_plugins" example.
• Configurable timeouts on sending and receiving of messages. Due to changed connection procedure, this also works for new connections. So Pyro can now also timeout when a new client just connects and blocks without sending anything (the port is not occupied forever).
• Added a new chapter about security to the manual.
• The Daemon is no longer directly remotely accessible. Instead, there is now a DaemonServant object with a limited interface that is the actual Pyro object and that calls the daemon.
• PyroURI objects have two new methods to directly get a proxy from an URI object: getProxy and getAttrProxy.
• Proxies can now be 'passivated' if they are not needed for a while (they release their network connection) proxy._release()
• Transient server objects can be automatically cleaned up after a specified period of inactivity. daemon.setTransientsCleanupAge(timeout) Also added this to the Bank2 example, so have a look there.
• Mobile agents can now import other modules that don't already exist on the server! (But only from within the agent class methods, not from within the defining module scope) They will also be loaded from the client, if the codeValidator allows this.
• Special base class for callback objects; CallbackObjBase. It will raise local exceptions also, not only silently passing them back to the server.
• Can now create PyroURI instances from PYROLOC: and PYRONAME: uris. The instance will be the actual URI, after performing the lookup. Not very useful actually, because we used to do the equivalent proxy=Pyro.core.getProxyForURI('PYRONAME://.......') Now you can also do proxy=Pyro.core.PyroURI('PYRONAME://.......').getProxy()
• Added two minimalist examples in the manual, to show how easy Pyro is :-)
• Name Server, Logger and listdir are now thread-safe (oops, they weren't before).
• Event Server no longer requires threads, checks if it is already running, and has new '-i' command line option to specify the required authentication passphrase.
• Name Server checks at startup if it is not already running in the network segment. Unless you use the new '-m' command line option, it refuses to start if another NS is detected.
• Name Server has new '-i' command line option to specify the required authentication passphrase.
• setup.py script can now be run unattended (for automated installs) by editing setup.cfg
• Added stresstest example (stresses the Name Server and the Event Server). Good for stability check.
• Pyro objects can now access Thread Local Storage by calling self.getLocalStorage(). This is an empty class to store your stuff into. It has one attribute, caller, that is the TCPConnection of the calling object.
• Renamed "Mobile Agent" to "Mobile Object" where appropriate. Mobile Agents are a special kind of Agent, and an Agent is a very special piece of software, and most mobile code that we're talking about isn't an agent.
• Added process ID and thread name info to logging messages.
• Fixed setup script, creation of RPM should now work (python setup.py bdist_rpm).
• Added "circle" example to show circular calling patterns.
• Added "quickstart-noNS" example, like the "quickstart" example but this one doesn't use the NS.
• Improved "callback" example and documentation about Callbacks.
• Promoted remote.py and remote_nons.py (from the "quickstart[-noNS]" examples) into the Pyro library: there is a new package Pyro.ext that contains them. They still have to be documented in this manual though.
• PyroURI objects are now correctly hashable, so you can use them as keys in dictionaries. The uniqueness is determined by the string representation of the oject.
• Documented the fact that MyObject.attribute.subattribute won't work in some cases, and extended the attributes example a bit.
• Fixed problem with deleting DynamicProxy in case of invalid URI.
• More troubleshooting tips added.
• Documented deadlock with object conversation problem (in Features chapter).
• When mobile code is not enabled, the remote invocation doesn't use the import wrapper class (5% speed improvement)
• Fixed shell scripts to allow quoted arguments such as: ns -i "the id string"
• URI parsing is stricter (could get in loop with certain invalid URI strings)

Pyro 3.1

• Finally fixed the Event Server crash problem with the "stresstest" example; the ES now uses worker threads and Queues per subscriber.
• Two new config items for the ES: PYRO_ES_QUEUESIZE and PYRO_ES_BLOCKQUEUE.
• Small fix in installation docs about where to put the module files.
• Documented the fact that proxies can be pickled when not connected, and added _release method to NameServerProxy.
• Documented the serious 'local object' issue with nested attribute access, in the Features chapter.
• Support added for PyXML's xml-pickler. Config item PYRO_XML_PICKLE changed: now chooses what implementation is wanted.
• Docs now contain links where referred to other parts of the docs. Various other documentation fixes.
• ns script now has '-v' option to enable verbose output. Normal output is slightly more compact.
• Proxy objects are now automatically pickleable, no need to call _release() first.
• New SynchronizedObjBase base class which provides threadsafe method calls (every method call is synchronized on the object).
• Fixes for CallbackObjBase error reporting, added another callback example for this.
• CallbackObjBase is better documented.

Pyro 3.2

• Big improvements in the mobile code department, thanks to Ulrich Eck:
  • 2-way mobile code now possible! Client can transparantly download missing code from the server, no longer only upload it to the server!
  • CodeValidators now work for both up- and downloaded code.
  • Remote code downloading from server is also recursive. watch out! there are no sanity checks yet.
  • downloaded modules no longer overwrite existing modules
  • Updated the "agent2" shopping example to use 2-way mobile code.
• Big improvements in the connection validation logic, thanks to Luis Caamano and Scott Leerssen: The connection validation logic has been condensed in a single class (DefaultConnValidator). It is now much more flexible. By providing a specialized ConnValidator object you can now completely control the logic that Pyro uses on both client-side and server-side to authenticate new client connections. The change is backwards-compatible and the default implementation still uses the md5 hashing of identification phrases for authentication.
• Added a WXPython GUI name server control utility: wxnsc.py, and a batch file wxnsc. Thanks to Jan Finell for this nice GUI tool with a tree view of the namespace.
• Fixed some path issues (mac) in the setup.py script.
• Fixed some Python 2.1 incompatibilities that had crept in (foo in dict instead of foo in dict.keys())
• NS broadcast server now binds on '' on Windows. '<broadcast>' doesn't work on Windows (???)
• Disconnecting objects from a Daemon no longer crashes when no NS is used.
• Scripts changed to pass correct sys.argv arguments to actual Python modules. Ugly '-c' argument removal code is gone.
• Daemon now tests if your hostname is valid and doesn't resolve to the loopback address 127.0.0.1 (which is often invalid). A warning message is logged in that case. NS and ES also print the message on the screen at startup. The test is done in the validateHostnameAndIP method of the Daemon.
• Name Server now also accepts non-absolute names and does the expansion itself, if needed. This enables simplified access to the NS trough other means than the default static proxy obtained from the locator. (suggested by Luis P Caamano).
• Replaced to-be-deprecated apply with func(*args) syntax. Small speed increase too.
• NS no longer logs error when detecting duplicate NS.
• Added Windows NT Services to Pyro (run NS / ES as NT service). Including batch files nssvc.bat and essvc.bat to install/remove them.
• Added Unix daemons to Pyro (run NS / ES as initd daemon). Including batch files nsd and esd to run them from init scripts.
• Relocated Event Server chapter in the manual after the Name Server.
• Fixed race condition in dir check in configuration.py
• Fixed race condition in _grimReaper in core.py
• Fixed mobile code load/compile issue on windows. Pyro now supplies .pyo, .pyc, .py in that order. On Windows, Python can't compile downloaded .py files if they have CR LF in them... :-(
• A few Jython compatibility improvements (GUID, config init).
• Fixed maxclient example and ConnectionDeniedError for maxclients.
• Thread Local Storage has been improved (docs + added initTLS in Daemon)

Pyro 3.3

• The NS tries "<broadcast>" first, this fails on some systems (windows) but it now automatically tries "" next (which should work).
• Pyro now enforces you to initialize it correctly by calling Pyro.core.initClient() or initServer(). Failure to do so will result in a PyroError very soon.
• Name Server now has Paired Mode: run two NS instances that synchronize with eachother. Use for fail-over.
• nsc tools have new options to deal with new NS features (meta info, paired NS).
• Name Server lookups automatically try to use the second NS if the first is unreachable.
• NS proxy automatically tries to find the NS again (or its twin) when the connection has been lost.
• Three new config items for the paired-NS mode: PYRO_NS2_* (see configuration chapter).
• NS ports slightly changed: NS now uses 9090 TCP and 9090 UDP by default (was: 9090 TCP and 9091 UDP). The second -paired- NS uses 9091 TCP and 9091 UDP.
• You can now run clients on read-only systems: PYRO_STORAGE is no longer created and checked for access when initializing clients. When you're initializing a server, it is still checked, unless you explicitly disable the check by setting the new optional argument storageCheck to zero: Pyro.core.initServer(storageCheck=0).
• More emphasis in the docs on the drawbacks of single-threading mode; you cannot use callbacks (Pyro will freeze)
• Rewrote the Event Server chapter in the manual.
• objects connected using connectPersistent() are no longer removed from the NS when the Daemon is removed.
• Fixed bug in pickling of DynamicProxy (it didn't work in the regular pickle module, and Jython). Also, you can now do this: proxy.method(proxy) i.e. pass the active proxy to a methodcall on itself. That didn't work before.
• Passing DynamicProxies no longer cause the socket connection to be disconnected. So on a new method call on the proxy that was passed, it is no longer needed to do a reconnect.
• Detailed tracebacks (with dump of the local variable's values) can now be enabled by setting PYRO_DETAILED_TRACEBACK to 1. Contributed by Richard Emslie.
• Pyro now deals correctly with obsoleted string exceptions (raise "blahblah"). (jf Gosset)
• Pyro exceptions now also automatically print their remote traceback info because the __str__ method has been customized in the PyroExceptionCapsule. No longer necessary to use exception handlers and call getPyroTraceback. Contributed by jf Gosset. This is turned off by default, turn it on by setting PYRO_PRINT_REMOTE_TRACEBACK to 1.
• New-style logging using the logging module can be enabled with the use of two new config items PYRO_STDLOGGING and PYRO_STDLOGGING_CFGFILE. The logging API is unchanged so old code still runs. With an appropriate logging configuration file, it should be possible to log to any destination (handler) that logging supports. This further enables clients (with logging!) on a read-only system: just log to syslog or a socket, no need to write logfiles to the local disk.
• "simple" example is now completely standalone instead of using testclient/server.
• Objects in the Name Server can now have user-defined meta information attached to them.
• nsc (and the others) can show the meta information.
• documented that the auth challenge string has to be exactly 16 bytes long, and Pyro now checks this.
• New config item PYRO_TCP_LISTEN_BACKLOG that specifies the socket listen backlog size (used to be a measly 5, is now default 200).
• Reordered a bunch of try..finally regarding mutex locking/unlocking. Should better prevent possible deadlocks.
• Added missing __copy__ method to DynamicProxyWithAttrs. (This bug caused the Bank2 example to crash, for instance).
• Added stdin/stdout redirection to NUL for NT service in BasicNTService.py, thanks to David Rushby.

Pyro 3.4

• Provided new SSL demo certificates, valid until january 2005. SSL example works again.
• Implemented PYROLOCSSL:// protocol which works just like the existing PYROLOC:// protocol, but uses SSL instead. (compare with PYROSSL:// versus PYRO://). This means you can now use Pyro in SSL mode without using the Name Server to look up objects. (PYROLOC didn't work over SSL)
• For Pyro clients, it is no longer required to explicitly call Pyro.core.initClient(). Pyro will do this automatically if it is not yet initialised.
• For Pyro servers, it is no longer required to explicitly call Pyro.core.initServer() if you are creating a Pyro Daemon as a first action. If you have not yet initialised Pyro explicitly, Pyro will do this for you when you create a Pyro Daemon. If you do other Pyro operations before that, this will obviously not work.
• You can now configure command line arguments for the Windows NT services (essvc, nssvc) in the Registry. The keys are: HKLM\System\CurrentControlSet\Services\PyroES and ...\PyroNS . The value under that key is: PyroServiceArguments (REG_SZ, it will be asked and created for you when doing a fresh nssvc remove ... nssvc install ... essvc remove ... essvc install.
• The win NT services (nssvc, essvc) now write their logging to a logfile in the root of your system drive (usually C:\), named "Pyro_NS_svc.log" and "Pyro_ES_svc.log" .
• Documentation improvements about the Event Server.
• Documentation improvements about Connection Validators.
• Documentation added about nssvc and essvc NT-Service scripts for the NS and ES.
• Documentation HTML cleanups.
• Added a "user_passwd_auth" example that shows how to do connection validation (authentication) based on username + password.
• Added a "filetransfer" example that shows how to do file transfers using Pyro.
• Added a "rserve" example that shows tuple-space like distributed communication. It can't get simpler than this! Contributed by John Wiegley.
• New wxnsc.py that deals with meta info.
• Fixed daemonizer.py to really detach the daemon.
• Fixed socket buffer/memory problem in protocol.py socket recv code, that seems to occur in certain situations.
• ResolvePYROLOC raises NamingError instead of KeyError if object name doesn't exist.
• Deamon now always prints and logs a warning if bound on localhost/127.0.0.1.
• Added a config item PYRO_SOCK_KEEPALIVE (default enabled) to control the SO_KEEPALIVE socket flag (to detect broken connections).
• Changed config item PYRO_BINARY_PICKLE into PYRO_PICKLE_FORMAT, which defaults to the best pickle format available. (on Python 2.3+ this is pickle.HIGHEST_PROTOCOL (2), otherwise it's 1). This means that on Python 2.3+ the new pickle format is automatically used that is much smaller and faster for new-style classes.
• Pyro.naming.NameServerStarter now raises exceptions when it cannot start the name server.
• Pyro.naming.NameServerStarter and Pyro.EventService.Server.EventServiceStarter now have a threadsafe way to wait until the server has been started: call the waitUntilStarted(timeout) method. This is used in the updated "AllInOne" example.
• Improved GUID generation a bit (threadsafe, tweaks).
• Fixed some issues with Pyro.conf config file.
• This is the last Pyro version that will work on Python 2.1, future versions require Python 2.2 or even 2.3.

Pyro 3.5

• Starting from this version, Pyro is only tested and supported on Python 2.3+. Python 2.2 may work, but no guarantees.
• PyroC -the static proxy compiler- has gone. Only dynamic proxies now.
• Removed the 127.0.0.1-error print statement from the Daemon (the warning in the log is still done ofcourse).
• When verbose flag is given, NS now also does the deamon loopback test (and prints warning) just like ES does.
• Remote method intercept code improved; it now uses a RemoteMethod stub object instead of the _name methodname-stack. You can now use bound method references to remote methods on the proxy (meth = proxy.someMethod), this did not work before.
• fixed some small documentation errors
• a few tweaks to the comparison methods (__eq__ etc) of the proxy object. Only eq and ne are defined now and work fine if a proxy is compared to a non-proxy object. You can now safely do prox in aList even when the list contains other objects.
• added the three missing name server methods fullName, setMeta and getMeta to the NS chapter in the docs
• Updated/clarified some things in the manual, such as the proxy-per-thread rule, and shutting down the server and daemon.
• daemon shutdown() is now honored by requestLoop()
• daemon shutdown() now takes an optional boolean disconnect parameter, if set to True (default=False) the daemon will unregister its objects directly (and shut down)
• rebindURI raises a more specific TimeoutError instead of plain ConnectionClosedError
• small fix in Pyroc so that it creates proxies for new-style classes too, but no guarantees that they work as intended (better to use dynamic proxy). Pyroc is a dinosaur and will likely be removed from Pyro.
• protocol code now uses socket.sendall() instead of a home grown loop.
• added support for MSG_WAITALL flag on socket recv for more efficient network transfers (especially large ones), on platforms that support this flag (such as Linux. Windows doesn't support it). I measured a 20% speed increase with the 'hugetransfer' example on Linux.
• added support for my recvall patch (it is used if detected). See Python patch #1103213.
• Fixed the NameServer when using paired mode (-1/-2) and identification (-i).
• NameServerLocator now sends the two broadcast messages to locate the NS (possibly running in paired mode) in parallel, so the initial delay for it to figure out that one of the two is down, is now gone.
• Added the offending name to almost all NamingError exception args.
• Fixed the persistent name server when it stores names on the filesystem with special characters (they are now escaped).
• Fixed persistent NS metadata-handling.
• Finally implemented persistent NS paired mode.
• The nsc tools now all have a setmeta command or button.
• Fixed various pychecker warnings and other assorted code cleanups.
• Fixed BasicNTService stream redir problems caused by recent pywin32 changes.
• New certificates in the ssl example that are valid until januari 2006.
• Tested with new M2Crypto (0.13+), had to implement a work-around for MSG_WAITALL-bug in M2Crypto API.
• Default connection validator createAuthToken now uses hmac-md5 instead of 'plain' md5. Updated the "user_passwd_auth" example accordingly.
• Fixed mobile code exception issue.
• Fixed ConnectionDeniedError exception issue.
• Changed md5.md5() calls to md5.new() (likewise hmac.new), the latter is the documented form
• Fixed mobile code pass-trough (as demonstrated by the new "agent3" example)
• Fixed mobile code issue that didn't allow multiple agent modules in the same package
• Added some more mobile code caveats in the docs.
• Fixed nsc shutdown command when not using broadcast.
• Fixed Oneway calls: they are now executed in their own thread, so that the caller is no longer blocked when using the same proxy again for the next regular call. Added a new "oneway" example that shows how that fix works.
• Added a very interesting "distributed-computing" example that 'cracks' md5, or sorts a huge array, but can easily process other tasks (if they can be partitioned)

Pyro 3.6

• a PyroException is now raised when Pyro detects you're sharing a connected proxy among different threads.
• sre module is deprecated, import re instead (core)
• socket reuseaddr is no longer set on cygwin, to fix connection issues on cygwin/windows (protocol)
• added a new distributed computing example (distributed-computing2) that uses pull model (workers pull new chunks of work)
• added some mobile code test scenarios: testmobile examples
• documented float NaN/Inf problems in troubleshooting chapter
• fixed timeout bug (send/recv), added timeout example
• a lot more documentation on multithreading issues with proxy objects
• added some explicit remarks about using the same major Python version to avoid pickle dataformat mismatches
• removed support for PyXML pickling format because PyXML's marshaler is buggy and no longer maintained
• removed the 'any' setting for PYRO_XML_PICKLE, you must explicitly name the implementation you want
• Added PYRO_GNOSIS_PARANOIA config item to be able to manipulate Gnosis pickler's paranoia (security) setting
• Added support for new Gnosis utils version 1.3.x
• added warning in guidlines in manual about attribute access to large data structures
• reformatted manual HTML, manual now has a nice CSS style and optimized print output as well
• improved Nameserver exceptions a bit
• added PYRO_BROKEN_MSGWAITALL config item for use on systems that have broken socket MSG_WAITALL support. When set to 1, Pyro will use a different piece of code to receive data (slower, but working on these systems as well).
• improved the PYRO_BC_RETRIES and PYRO_BC_TIMEOUT config items: you can now set them to a negative value to mean 'infinitely'.
• added PYRO_HOST and PYRO_PUBLISHHOST config items
• automatic remote traceback printing has been removed, and the config item PYRO_PRINT_REMOTE_TRACEBACK as well. Changes in Python's exception class implementation no longer allows to change the way exceptions print themselves.
• renamed (internal) setDaemon method to avoid name clash with method on Python's Thread object
• replaced select() calls with a safer utility method (suggested by Oliver Gathmann)
• fixed very old frame GC-leak in Pyro_dyncall (SF bug 1407267)
• when running on localhost, Pyro will no longer start a NS broadcast server (this avoids not-working-NS addresses on other machines in the network)
• the NameServerLocator will now try to contact a Name Server on the current machine and localhost if the broadcast lookup fails. This allows standalone Pyro to work on Windows without having to configure PYRO_NS_HOSTNAME etc.
• threading fixes in event service and core pyro code
• locks around code doing mobile code imports, to avoid import race conditions
• mobile code is now able to compile source code with strange line endings such as CR LF (now opens source files in 'rU' mode)
• fixes in copying and pickling of proxies, also includes identification data now (if any)
• removed rserve example
• changed forking client in multithreading example to use threads instead, so it now works on Windows as well
• fixed ns thread issue in distributed-computing example

Pyro 3.7

• Fixed typo for bdist_rpm target in setup.cfg
• Batch scripts have been renamed, all now start with a 'pyro-' prefix. This avoids name clashes. This was already done in the Debian package for instance.
• NS broadcast retry is now 1 (was 2) and timeout is now 0.75 (was 2)
• Pyro.core.SynchronizedObjBase now correctly handles string exceptions (deprecated anyway)
• the NS nt service won't respond to shutdown requests anymore
• wxnsc updated to recent WxPython API, deprecation warning is gone