I’ve been working in Plone, so I’ve seen
zope.component. I’m
also thinking of making a (potentially) networked game in Python, and
for that I was looking at things like
Twisted and
Kamaelia. Unsurprisingly, I’m not really
satisfied with any of these.
Let me try to articulate what I’m looking for when I say “component
system” (perhaps something more nebulous than “content management
system”).
Obviously I want to divide my game into components that implement
defined interfaces. For example, there’s a component that handles
the network communication with players, another that handles
streaming the game events to “observers” (people that watch but do
not participate), a component to handle physics, a component to
perform (for example) validation on incoming player commands, etc.
-
I want to define dependencies between components. These
dependencies are then used to acquire a suitable implementation of
the component’s interface.
For example, when I start the game maybe I “start” the front end
component that handles communications with player; lets call that
the “player server.” Now, the player server generates events such
as “player connected,” “player moved,” etc. Somewhere (in code, or
maybe even in something like ZCML) I’ve defined that this
implementation of the component needs a component implementing
IPlayerEventConsumer. The component system then finds (using my
configuration) an implementation for that interface and makes it
available to the player server.
Assuming my components are written correctly, I want to be able to
have a component execute in the same thread as other components
(e.g. in the “main thread” as a microthread/coroutine), or in a new
thread, or in a new process. For example, if I have lots of
CPU-intensive physics code, maybe I want to run that on another
processor, so I need threads. Of course, maybe I’m running on
CPython, where I might need separate processes to bypass the GIL
(ignoring for the moment the question of efficient IPC). Or maybe
I’m working with something like a GUI where I need to have that run
in a separate process (of course, doing a GUI can cause even bigger
headaches).
-
I want the ability to implement components in other languages (see
also: component executes in separate process, above). This means I
want a standard protocol for communication between components.
XML-RPC comes to mind. Something else easy enough to implement
comes to mind. There’s things like pickling in Python, but I don’t
know how much fun that would really be to implement in a non-Python
language; maybe something more language-agnostic?
For fun, I’ll add here that I might like to be able to communicate
with components using a variety of methods: pipes, Unix sockets,
TCP/IP, and so on. This is desirable.
Now, I think I’ve just described a billion other attempts at
“component systems”: COM/DCOM, maybe EJB, CORBA, maybe KDE’s DCOP.
Let me add a couple more requirements that should narrow the field a
bit:
The “component system” must be platform-independent–or at least
support including Linux, *BSD, OS X, and Win32 (actually, I could
give up Win32 if I had to).
I want the component system to be mostly transparent to me, the
coder. I expect to have to configure the bindings between
interfaces and implementations, specify dependencies, configure the
manner in which a component will execute
(microthread/thread/process; subject to the “execution style” the
component is prepared to run in), and configuring the location
(e.g. host/port) of other components in the system. I don’t want to
have to manually write up a proxy class for a remote component, for
example. As much as is humanly possible, I don’t want my code to
care whether a component is running in shared memory or on a box
1,000 miles away.
COM/DCOM is basically going to be Windows, right? DCOP might not be,
what with KDE 4 supposedly running on Windows (right?). EJB has
notorious boilerplate (though it has been a while for me), when I
think CORBA I think IDL, etc. Those aren’t “mostly transparent.”
Kamaelia looks interesting. The system of “wiring” components
together feels right to me; I was first exposed to this in
NesC. However, the implementation
needs to be updated to support the new features of generators in
Python 2.5, as the current syntax strikes me as rather ugly. In fact,
it looks like Kamaelia needs a recent release, period: the last one I
saw was from 2006.
(As a side note: everything should be easy_installable. Kamaelia
and Twisted are not, though Twisted has ongoing work to this end.)
Kamaelia also fails to offer multi-process operation, as far as I can
tell (you could write it yourself without too much pain), and it
needs a way to generalize its “wires” to support communication with,
e.g., remote components. You could actually combine the marshalling
component with the framing component and a TCP client/server
components and make this work; but that might have shot straight past
“configuration” into “programming.”
Twisted is a much, much, much larger framework than I ever
realized, and a lot of it seems pretty good. Nonetheless, the
centerpiece of Twisted (if you believe the docs) still seems to be
their “reactors” which require you to handle concurrency by defining
things like Protocol subclasses that receive event messages
(i.e. method calls) like connectionMade and dataReceived. This
programming model might feel a little strange. They’ve got this neat
looking inlineCallbacks
decorator,
which looks like it might lend itself to a coroutine kind of style.
Then you start to realize that you’re not sure what you can use it
for. I actually started writing something like:
class HelloWorldProtocol (LineReceiver):
def connectionMade(self):
self.transport.write("Hi, who are you?\n")
# Now I'll read their name:
line = yield self.readLin # ... hey, uh, there isn't a read method
I’ve seen several IRC logs where people try to figure out similar
things. For what I’m doing, inlineCallbacks doesn’t seem like
something I’m going to be able to use much.
I’ve considered building something like Kamaelia’s style of wiring up
components inside Twisted. Twisted has some kind of support for
things like processes and threads. I haven’t determined if these
really meet my needs, and I keep reading scary things about them being
deprecated.
If you believe Twisted’s finger
tutorial
they’ve also drank the “Zope Component Architecture” Kool-Aid, though
thankfully I didn’t see any ZCML (yet…). Look at the final product
of that
tutorial
and notice all the interfaces and “adapters” flying around. I don’t
really feel like I gain enough for that extra code.
The question I ignored earlier, the one of performance, is still
outstanding in my mind: can this kind of system be done [in Python]
efficiently? I’m afraid I’m dreaming of a “component system” that’s
going to be so slow as to be unpractical.
And, finally: do I really need these features?