Folks,
As a way to introduce how the EnhydraME project came
into existence I thought I'd share an initial email exchange
that
started the whole concept off back in June. There is a lot
of information here that would be useful to keep in the
archive.
I'd love to hear your comments and ideas...
- Paul.
<--
<-- Email from Paul Morgan to John Beaty & Stephan Haustein
<--
Stephan, to bring you quickly up to speed - you are
certainly aware that John Beatty gave a well attended
presentation and demo at JavaOne of Web Services running on
J2ME devices. His work is of course leverages your kXML and
kSOAP projects. Keith Bigelow (Lutris wireless visionary)
and myself had a conversation with John Beatty on Monday to
see if he would be interested in joining forces and
contributing his work as enhydra.org open source projects to
create an eco-system of J2ME projects...
The concept/vision is create the world first Micro
Application server for J2ME devices as open source software.
Perhaps "Application Server" is the wrong name, but it does
give the project high aims and scope. The motivation behind
this work is that the capabilities of micro J2ME devices are
expanding rapidly and, as John demonstrated at JavaOne, are
capable of acting as providers of Web services especially in
a peer-to-peep manner with other J2ME platforms. It would be
fun to discuss the composition of such a server engine in an
open source forum, but I envision at least the following
components/projects (all names are placeholders!):
* kHttp - A micro http server, precisely the technology I
believe you built. This would probably require a server side
proxy as in John's implementation.
* kBox - A lightweight component container for executing
downloaded application components or web services. This
would be something analogous to a lightweight servlet
engine.
* kSoap - Implementation of the SOAP protocol, furthering
the work of Stefan Haustein.
* kXml - Lightweight XML parser presumably leveraged in
multiple places in this proposed micro application server.
This again would further the excellent work of Stefan's.
* kJms - You guessed it a lightweight JMS implementation
hopefully leveraging SMS as one possible delivery protocol.
* kDB - Micro (VERY small footprint) relational database.
* kSync - An implementation of SyncML for synchronization of
both database AND application session state. Obviously this
would require a server side component that Lutris may
interested in building as an Enhydra plug-in service.
* kEnhydra - A project responsible for the integration of
each of the pieces into this micro application server
engine. This would also be a great place to house demos and
test applications.
The envisioned platform would be very modular and thus
each technology project would be individually useful and the
integration of the pieces would be taken on by the kEnhydra
project. I imagine somebody to be able to run such a server
and to download application components (say a chess game) to
run inside the container in an off-line manner. Such a user
might then be able to play chess with another user via SMS
messages (note this is free to similar subscribers with a
carrier like Nextel). Another idea is that the users
calendar is exposed as a Web service and is interrogated by
another business user trying to negotiate a mutually
agreeable meeting time. [I'm sure you have way more creative
use ideas than me ;-)]. Incidentally, we are close to
releasing a new open source build tool called Aleutia. This
effort is being lead by Brett McLaughlin and is primarily
focused on allowing an community member to assemble a
collection of individual open source projects into a whole
based on an XML description. Think of it as a client/sever
tool that somebody would use to interact with any project on
enhydra.org. Eventually it will be able to pull specified
binary releases, act as a front end to cvs and create
patches to be sent back to projects for easy integration. I
digress, but I wanted to introduce this as a tool that could
be used to coordinate such a set of projects.
The proposal is to create a
new area of Enhydra.org dedicated to the "k" series of
project. John, in case you are not familiar with the
enhydra.org
infrastructure, it is pretty automated these days with tools
for project owners to manages their CVS trees, web sites
(via WebDAV), push content live, check for broken HTML
links, manage email lists (MailMan web interface) and
archives, etc.
Aside for the work I would commit to doing in getting
this initiative rolling, I'd propose that John release his
work as open source projects and take responsibility for the
general management of these projects (my guess is the kHttp
and potentially the kEnhydra where the actual Web Service
demos could be hosted). I'd also propose that Stephan, break
out the kXML and kSOAP into two initiatives - something that
we discussed some time ago I believe. Keith is currently
tacking down a couple of people that have shown interest in
kDB and kJMS to join the consortium.
- Paul.
<--
<-- John Beaty's response
<--
I have several high-level thoughts, as well as a few
thoughts on the
proposed projects.
High-level thoughts:
- In developing J2ME apps, targetting the various
configurations, profiles,
and devices will be a large challenge. Of course, there is
a massive schism
between CDC and CLDC (APIs and device characteristics
mainly). And the
storage sizes and heap sizes vary dramatically, as do the
CPU speeds (there
is already an order of magnitude difference between a
Dragonball-based Palm
and a StrongARM-based iPAQ in terms of CPU & memory (and
inversely, battery
life!)). In some cases we will be able to effectively
compenentize and
layer the solutions such that the smallest devices have the
core
capabilities and the more powerful devices have additional
capabilities. In
other cases, we may have to relegate to having two separate
projects. What
do we feel the priority is? There will be probably 100
million CLDC-class
devices in deployment within two years. At the same time, we
probably don't
want to ignore the CDC opportunities either (considering
it's a much more
capable platform and there's a lot of interesting work to be
done).
- The 'k' moniker is presumably from 'KVM', correct? If so,
it's a moniker
which some people may presume limits our solutions to CLDC.
I'm not saying
I'm opposed to it, but rather, just that some people may
perceive it that
way.
- Java Embedded Server
(http://www.sun.com/software/embeddedserver/). At
least for historical context, it's good to know what's
already been done in
the embedded server space. Note that JES is not open source
(I'm not sure
if it's even SCSL). In short, it provides: HTTP server,
SSL/TLS, Servlet-ish
API, remote management, and a log service. Version 2 has
been retrofitted
for J2ME-CDC-Foundation Profile. Primarily designed for
home gateways, JES
would also be appropriate for the StrongARM-class of PDA. I
am unclear of
the viability JES has and the traction it is getting. JES
does have an
interesting component bundling model that may be good to
learn from. (Note
that these devices' networking environments will typically
have the same
issues of IP addressability I've been talking about, so my
HTTP Server Proxy
still has relevance.)
- The networking environments will be horribly diverse.
There are a number
of proposed solutions to dealing with NAT gateways. We will
need to pick our
targets and keep aware of what's coming down the pipe
(there's nothing more
annoying that developing a solution for an almost-obsolete
problem!).
Here's a quick summary:
- RSIP, a replacement for NAT
(http://www.usenix.org/publications/library/proceedings/ine2
000/full_papers/
borella/borella_html/)
- Univeral Plug-and-Play (UPnP)-enabled NAT gateways.
UPnP defines
protocols for programming NATs, including port mappings.
- firewall piercing via exploitation of TCP's
simultaneous open, which
allows very cheap connection brokering (3rd party falls out
of the picture
after connection is 'spliced'). Unfortunately, MS never
fully implemented
TCP and left this feature out! However, some people are
working on slim new
TCP stacks that would run in application space that would
overcome the MS
limitation.
- Gatewaying and application-level protocols to work
with currented
NAT'ted environments. The HTTP Server Proxy solution falls
into this
category. Also, for the JXTA project, I'm starting to design
a new transport
layer that aims to be as efficient as possible for both
asynch and synch
messaging (i.e., there are more creative solutions that the
HTTP Server
Proxy's brute-force approach, esp in the asynch case).
- As a point of interest, Microsoft is supposedly
working on some
solution that leverages your existing connection with MS
using Messenger.
This falls into the 'gatewaying' category.
- The Peer-to-Peer Working Group (originally started by
Intel, but since
escaped their grip) is working on this problem as well.
- IPv6 :)
Here are my project-specific thoughts:
> * kHttp - A micro http server, precisely the technology I
> believe you built. This would probably require a server
side
> proxy as in John's implementation.
There's a couple ways of tackling this one. We can optimize
my solution in
the case of CDC. In the CDC case, the most efficient thing
to do may be
using what I call a Generic Server Proxy rather than an HTTP
Server Proxy.
The Generic Server Proxy provides splicing of sockets rather
than HTTP
connections and is more efficient. There are some
downsides, however
(primarily that for the person hosting the server proxy,
virtual hosting
cannot be used any more). This can be explored further.
Also in the CDC case, because the runtime environment is
more flexible
(daemon threads, multiple VM instances, etc), we can write a
small piece of
software that remains resident that talks to the HTTP Server
Proxy. This
small piece of software becomes a 'virtual client' that
makes HTTP requests
to a completely stock HTTP server also running on the
device.
The upshot here is that we have more headroom on CDC and can
provide some
interesting solutions. Also, we can use stock HTTP servers
that already
exist or are currently being developed, if we can convince
someone to donate
to the cause. For CLDC, my Micro HTTP Server (with the HTTP
Server Proxy
protocol implementation built-in), is the way to go.
If this is all confusing, don't worry--it took me a while to
figure it out!
:)
> * kBox - A lightweight component container for executing
> downloaded application components or web services. This
> would be something analogous to a lightweight servlet
> engine.
Yes, absolutely. I even like the name :)
> * kSoap - Implementation of the SOAP protocol, furthering
> the work of Stefan Haustein.
The total coup with kSoap is to get a code generator that
generate the stubs
and skeletons (I'm sure I wouldn't be the best person to do
this). I'm sure
Stefan has a list of things to do for this as well.
I'd like to see where we're at as far as SOAP compliance and
supporting the
full SOAP feature set. In some ways I wonder if for the
CDC, somehow
leveraging Apache SOAP would work (I somehow doubt it, just
because of the
basic different between kXML and a JAXP/DOM parser)
> * kXml - Lightweight XML parser presumably leveraged in
> multiple places in this proposed micro application server.
> This again would further the excellent work of Stefan's.
Absolutely. Same thought for kXML w.r.t. spec compliance
and also CDC
devices. I wouldn't be surprised to see other lightweight
XML parsers pop
up in the CDC-class space.
> * kJms - You guessed it a lightweight JMS implementation
> hopefully leveraging SMS as one possible delivery
protocol.
I'm a huge fan of asynch messaging, esp. in the wireless
case. You can even
do effective request-response stuff over asynch transports.
So, we could
also provide some nice frameworks to make the asynch
programming a bit
easier (I wrote a simple framework for
asynch-transport-based
request/response interactions for the JXTA binding of
XML-RPC).
> * kDB - Micro (VERY small footprint) relational database.
Cool.
> * kSync - An implementation of SyncML for synchronization
of
> both database AND application session state. Obviously
this
> would require a server side component that Lutris may
> interested in building as an Enhydra plug-in service.
Cool again.
> * kEnhydra - A project responsible for the integration of
> each of the pieces into this micro application server
> engine. This would also be a great place to house demos
and
> test applications.
Sounds good. Is there a bit of a fuzzy line between kBox
and kEnhydra? This
will probably disambiguate itself for me pretty quickly.
> The envisioned platform would be very modular and thus
> each technology project would be individually useful and
the
> integration of the pieces would be taken on by the
kEnhydra
> project. I imagine somebody to be able to run such a
server
> and to download application components (say a chess game)
to
> run inside the container in an off-line manner. Such a
user
> might then be able to play chess with another user via SMS
> messages (note this is free to similar subscribers with a
> carrier like Nextel). Another idea is that the users
> calendar is exposed as a Web service and is interrogated
by
> another business user trying to negotiate a mutually
> agreeable meeting time. [I'm sure you have way more
creative
Funny you should mention chess: My russian friend just
finished up a
Java-based chess engine for a JXTA demo :)
> use ideas than me ;-)]. Incidentally, we are close to
> releasing a new open source build tool called Aleutia.
This
> effort is being lead by Brett McLaughlin and is primarily
> focused on allowing an community member to assemble a
> collection of individual open source projects into a whole
> based on an XML description. Think of it as a client/sever
> tool that somebody would use to interact with any project
on
> enhydra.org. Eventually it will be able to pull specified
> binary releases, act as a front end to cvs and create
> patches to be sent back to projects for easy integration.
I
> digress, but I wanted to introduce this as a tool that
could
> be used to coordinate such a set of projects.
Pretty interesting. Is this something that can't be done
elegantly through
defining additional Ant tasks? It sounds like you may be
working at one
level higher than Ant (app assembly versus compiling a
specific app). Just
curious--I would bet others will have the same question
considering Ant has
so much traction.
> Aside for the work I would commit to doing in getting
> this initiative rolling, I'd propose that John release his
> work as open source projects and take responsibility for
the
> general management of these projects (my guess is the
kHttp
> and potentially the kEnhydra where the actual Web Service
> demos could be hosted). I'd also propose that Stephan,
break
> out the kXML and kSOAP into two initiatives - something
that
> we discussed some time ago I believe. Keith is currently
> tacking down a couple of people that have shown interest
in
> kDB and kJMS to join the consortium.
Okay. To kickstart it most quickly, I can get my initial
stuff cleaned up
just a bit and we can release it. Just to re-emphasize, my
Micro HTTP
Server is optimal for CLDC-MIDP only (and perhaps PDAP, but
they haven't
released the damned spec yet!). The HTTP Server Proxy is
completely generic
and has relevance to things outside the J2ME and wireless
space (e.g., it
actually works for the peer-to-peer space in general). What
would be your
positioning of that? I'm just a bit concerned that it will
be unnecessarily
pigeon-holed into j2me/wireless.
- John.
--
Paul A Morgan
Chief Technology Officer
Lutris Technologies, Inc.
1200 Pacific Avenue, Suite 300
Santa Cruz, CA 95060 USA
831.460.7307; 831.471.9754 (fax)
http://www.lutris.com
http://www.enhydra.org
|