Simple SOAP posting from the command line

From the linux command line, you can do use curl to submit a soap file to a URL.

curl -H "Content-Type: text/xml; charset=utf-8" \
     -H "SOAPAction:" \
     -d @soap.txt \
     -X POST http://localhost:18181/httpWSDLService/httpWSDLPort

where the @ specifies a file as input and soap.txt is the file.

Microformats: A DoD Use Case

Microformats have been around awhile, but I have just recently took the time to find out what they are and what they can do. A good starting place I can recommend is microformats.org. Basically, using microformats means adding attributes to existing html elements to enhance the meaning of the content.

The physical address presented on this blog is marked up, for example, as a hCard.

<div class="vcard">
  <div class="adr">
    <div class="org fn">
      <div class="organization-name">Gestalt-LLC</div>
    </div>
    <div class="street-address">320 East 4th Street</div>
    <span class="locality">Joplin</span>, 
    <abbr class="region" title="Missouri">MO</abbr>
    <span class="postal-code">65801</span>
  </div>
</div>

This could be thought of as a lightweight semantic web enabler. And the beauty of course is the simplicity of it. It is suitable for use in marking up HTML, RSS, Atom or XML.

So now the question: what could be accomplished with microformats? What might be a scenario in a military domain?

1. A soldier is observing and recording enemy aircraft taking off from a hostile airfield. He microblogs about it on a ruggedized PDA marking up the aircraft information in a military airframe microformat (date, time, aircraft type, payload configuration, number of aircraft in formation, observed tail numbers, for example).
2. This is an unanticipated observation, so the data may not be immediately useful or actionable.
   
3. But, Google will find it (and yes, the DoD has Google appliances running on their classified networks). Not necessarily based on the microformating, but based on its normal searchable content. Technorati, however, does have a microformat search engine.
   
4. These search results are microformat parsed and fed into a Temporal Analysis System (TAS) whose job in life is to predict the future by comparing recent events with similar historical events chains and their outcomes.
   
5. It turns out that what the soldier observed, perhaps 10 cargo aircraft, all with extra fuel pods, bearing NNW, from grid reference 18SUU8401, is a 90% predictor that the hostile battalion will be mobilizing and deploying in 72 hours.
6. Useful information to know derived from someone outside the normal chain of command and information flow!
   

Powerful and simple.

Source Control Tips for Netbeans Projects

This post is specifically for SOA projects centered around JBI composite applications and service units, but I think 90% would still apply to any project type. If you create the projects inside of NetBeans and use an external source control tool, like SmartSVN, these are the directories you'll want to exclude and ignore.

For a service unit project, like a BPEL module, exclude

• build
• nbproject/private

For a composite application, exclude

• build
• dist
• nbproject/private
• src/jbiasa
• src/jbiServiceUnits

After you successfully commit the projects, make sure the above directories stay out of subversion by setting the ignore property. In SmartSVN, do this by clicking on the the project name and using the menu bar to select Properties > Edit Ignore Patterns.... Enter the directory names, save and don't forget to commit!

Procrastination

Hard work often pays off after time, but laziness always pays off now.

Using BPELs doXslTransform() function

As a improvement to my previous design of aggregating RSS feeds from various continuous integration systems, I wanted to do a simple XSL transform to make my post-processing easier. If I could get each element to be on it's own line, then things become much easier for the later parsing that I have to do.

Given that I am using OpenESB and JBI to accomplish my goals, I have two choices of how to perform the transform. 1) Use the XSLTSE or 2) Use the BPEL function doXslTransform(). I choose the latter option because it was the simplest thing that worked. (I will have another post about using the XSLTSE though. Stay tuned!)

To start, I created the XSL stylesheet that takes input and produces output that validated to the same schema (rssbcext.xsd in this case). Then in my Netbeans BPEL editor, I clicked on the Assign1 operation.



This brings up the BPEL Mapper. Add the doXslTranform functoid (and no, I did not come up with that term).



The Netbeans tooling is not yet complete for this function, so we have to add the stylesheet file-name as a URN string literal. In my case it was 'urn:stylesheet:citransform.xsl'. Drag the RSSConsumerWSDLOperationIn.part1 to the node-set and drag the return-node to the FileWSDLOperationIn.part1.

Clean and Build, then deploy to Glassfish. That's all there is to it. My RSS feeds are now in a compact format, making my processing easier and saving valuable disk space. :)

Using JBI To Keep An Eye on Continuous Integration

I'm a big fan of Continuous Integration. We thrive on it at work to get feedback for our code integration on a constant basis. As part of a bigger company effort, we wanted to be able to create team dashboards showing CI health (server up, building, not broken too long, etc). The teams here mostly use CruiseControl, but we also a few teams utilizing Hudson and Luntbuild.

So what's an easy way to keep tabs on 3 different build systems? RSS of course!

CruiseControl publishes out an RSS feed, Luntbuild publishes an ATOM feed and Luntbuild recently added RSS and ATOM feeds (committed, but not distributed yet, as of 1 Aug 2007).

And I don't want to write any code to aggregate these feeds together.

Enter JBI, Open-ESB, and the RSS Binding Component (BC).

Start by downloading the latest Open-ESB/Glassfish bundle. Start up Netbeans. To subscribe to multiple RSS feeds via the RSS BC, we need an RSS provider and an RSS consumer composite application.

Create the provider BPEL module by creating a new Netbeans project (New Project > Service Oriented Architecture > BPEL Module). Name it CIProviderBpelModule. Now we need to import two xml schemas into our project (rssbcext.xsd and wsaext.xsd). Follow the steps outlined here to do the imports.

The WS-Addressing extension schema is used to have access to the element EndpointReferenceList, which we'll use to feed the RSS feed URLs into the system via a SOAP request.

Create two WSDLs, one for http and the other for rss, with the New WSDL Document wizard.

On the Name and Location step, name the rss WSDL "rssciprovider" and import the rssbcext.xsd schema, By File, with a prefix of "rssbcext". On the Concrete Configuration section, make sure RSS is selected as the Binding Type.

On the Name and Location step, name the http WSDL "httpci" and import the wsaext.xsd schema, By File, with a prefix of "wsaext". On the Concrete Configuration section, make sure SOAP is selected as the Binding Type.

At this point both WSDLs should validate (Alt+Shift+F9) correctly. This will make sure all schemas are imported correctly.

Open the httpci.wsdl and navigate to the request message part1. Click the element and then in the Properties pane of Netbeans, change the element attribute from type="xsd:string" to element="wsa:EndpointReferenceList" Do the same for the reply message part1. (Make sure you pick element and not type. Thanks James!).

Open the rssciprovider.wsdl and navigate to the request message part1. Change the type to an wsaext element EndpointReferenceList as above. You can also remove the reply message and the output from the operation and binding as this will be an In-Only message. For the binding operation, change the input to . For the service port change the rss:input to . This correlationId is important and will match up to a correlationId in our rssciconsumer.wsdl.

Validate the WSDLs and then Process Files > New > BPEL Process. Give it a name "rssProviderBpelProcess". Drag and drop the two WSDLs into the process flow diagram; this will create partner links. Name them "httpPartnerLink" and "rssProviderPartnerLink". Swap Roles for the rssPartnerLink to "Partner Role".

From the Palette pane, drag and drop Receive, Assign, Invoke, Assign and Reply operations onto the BPEL flow. Edit Receive1 to point to the httpPartnerLink and create an input variable. Edit Invoke1 to point to rssProviderPartnerLink and create an input variable. Edit Reply1 to point to httpPartnerLink and create an output variable. Click Assign1 and using the BPEL Mapper pane at the bottom of Netbeans drag a line from HttpciOperationIn.part1 to RSSciOperationIn.part1 (ignore data types don't match warning).

For the SOAP response, we will just hardcode something to acknowledge the RSS provider is subscribed. Click Assign2 and using the BPEL Mapper create a String Literal with a value of "Done.". Drag a line from the String Literal to HttpciOperationOut.part1.

Validate the BPEL file.

Create a new Composite Application project "CIProviderCA" and add the JBI Module project CIProviderBpelModule to it. Clean and build.

Halfway there!

Create the consumer BPEL module by creating a new Netbeans project named CIConsumerBpelModule. Import the rssbcext.xsd schema into the project.

Create a rssciconsumer.wsdl with the rssbcext.xsd schema imported as before. Make sure "RSS" is selected as the Binding Type. Edit the wsdl and change the message part element to element="rssbcext:EntryList". Change the operation input to . Change the service port address to .

Create a fileci.wsdl with the rssbcext.xsd schema imported as before. Make sure "File" is selected as the Binding Type. Edit the wsdl and change the message part element to element="rssbcext:EntryList". Remove all output references; In-Only again. Change the operation input to .

Validate the WSDLs and then create a new BPEL Process named "rssConsumerBpelProcess". Drag and drop the two WSDLs into the process flow diagram; this will create partner links. Name them "filePartnerLink" and "rssConsumerPartnerLink". Swap Roles for the filePartnerLink to "Partner Role".

From the Palette pane, drag and drop Receive, Assign, Invoke operations onto the BPEL flow. Edit Receive1 to point to the rssConsumerPartnerLink and create an input variable. Edit Invoke1 to point to filePartnerLink and create an input variable. Click Assign1 and using the BPEL Mapper drag a line from RssciconsumerOperationIn.part1 to FileciOperationIn.part1 (ignore data types don't match warning).

Validate the BPEL file.

Create a new Composite Application project "CIConsumerCA" and add the JBI Module project CIConsumerBpelModule to it. Clean and build.

Start glassfish and deploy both JBI Composite Applications to it.

In the provider CA project, create a test a new test case, pointing it to the httpci.wsdl and the httpciOperation. Sweeeeeeeeet. Edit the test case input. Each Endpoint reference only needs the Address element to be valid. Add as many as EndpointRerefences as you need to the EndpointRerefenceList. Run the test.

Look in C:\Temp (or whatever directory the file service port referenced) and you should see a ci-feeds.xml file with an aggregation of all the continuous integration RSS/ATOM feeds in it.

Accountability and the Knowledge Worker

Are you a Knowledge Worker? Fred Nickols really sets the framework for how companies can really make a difference when they are drive by a new breed of accountable individual (any he did this back in the '80s)

• I seek out and I set direction.
• I resist supervision; I welcome support.
• I work at my own pace within the constraints posed by the situation.
• I put my name on what I do -- and the names of those who contribute in any way to any endeavor for which I am accountable.
• I promote, praise, recognize and reward accomplishment and service.
• I contribute more than I consume.
• I seek control over myself and influence over events about me.
• I care about my work -- it is a reflection of me.

• I care about the enterprise of which I am a part and I will do my best to protect it against all threats -- whether they are external or internal.
• I care about the people with whom I work; they are not just co-workers or colleagues -- they are also friends, allies and comrades.
• I refuse to simply "obey orders" and "go along with the program."
• I will hold my superiors as accountable for their actions as they hold me accountable for mine.
• I will not turn a "blind eye" or a "deaf ear" to wrongdoing at any level of any organization of which I am a part.
• I will do my best to share what I know with those who are interested in learning from me and I will do my best to learn from those who are skilled, knowledgeable or competent in areas I am not.
• I will at all times strive to advance and improve the knowledge base that undergirds my skills, abilities and accomplishments.

This ties in nicely with the theme of accountability. What it's not: imposed on you by others. What it is: self-directed behavior.

Testing Karma

I think Earl said it best when he said "Do good things, and good things happen." What kind of good things? How about good software engineering practices? Use Case modeling, sequence diagramming, continuous integration and automated testing are all good starts.

Diagrams and models are good because they clearly communicate intent. The picture is worth more than a thousand words if a stakeholder (or peer) can talk to the diagram and say something as simple as "delete that line and add this one", or "change the arrow to point the other way on the sequence flow". Then everyone goes, "ah... now I get it". Practices like these definitely pay dividends over waiting to fix any architectural flaw later in the lifecycle of the system.

Creating automated tests, too, are equally valuable. And not hard. And can help you develop better code. Sometimes writing unit tests after the fact can be a real head-scratcher. Thinking about the code and the test as a singular unit can help structure the code in better ways. "When you think of code and test as one, testing is easy and code is beautiful."

Also from artima :

• The perfect is the enemy of the good. An imperfect test today is better than a perfect test someday.
• Rejoice when they pass; Rejoice when they fail.

Be creative and flexible when writing your tests. Write that ugly test for the "untestable" code. Take that first step and good things start to happen. And when you're walking, you won't notice the steps because they will have become habit.

Unit testing advice

Good stuff from the guys over at the Google Testing Blog.

I had never really given much thought to naming the methods in a unit test class. But if you give it a little though, the method names by themselves can read like a list of low-level functional requirements.

Why write test methods like

• testClient() or
• testURI() or
• testRun()

when you can name the methods

• testTryToInvokeServiceBehindFirewallDirect() or
• testCompiledInputCommandIsConvertedAndDispatched() or
• testWriteDataToFile()?

The rest of the world is so smart

North America really needs to get with the world program. Witness:

Paper Sizes:

http://www.cl.cam.ac.uk/~mgk25/iso-paper.html

Date and Time:

http://www.cl.cam.ac.uk/~mgk25/iso-time.html

and of course the Metric System:

http://www.cl.cam.ac.uk/~mgk25/metric-system-faq.txt

Words from the guy who was responsible for the Google SOAP API

Nelson Minar doesn't work for Google anymore and bears no ill will that Google yanked the SOAP API that he was responsible for. If fact, he has a pretty negative view of SOAP, its interoperability problems and brittleness.

But I particularly liked the last sentence of his post, "Just do something that works". It ties in nicely with my last post, Solving The Problem in a Very Good (Not Perfect) Way.

http://www.somebits.com/weblog/tech/bad/whySoapSucks.html

Perfect is the enemy of Very Good

I found this a good, quick read and helpful as a reminder on how to tackle hard problems. My takeaway was to take action and not wait until "all the data is in" before making a decision. In short, deliver the Very Good as opposed to Perfection. As the CEO of the company I work for said recently, working in a vaguely correct direction is better than standing still. [Update: In the software business, the remark about "working in a vaguely correct direction is better than standing still" works because of Scrum. If you are talking to your client and demonstrating your solution every 30 days, your can have that meaningful conversation where both parties refine the problem and the solution together.]

The full discussion for the quotes below is here: http://pliantalliance.org/?p=34

Never let perfect stand in the way of very good.

It may be tempting to wait until you have the time, the knowledge, or the inclination to design the “perfect” solution, especially if you aim to solve the real problem. However, you must temper this instinct with practicality. While it may look like you are holding out for “the perfect solution” (that solves the whole, real problem), what you are really doing is preventing “the very good solution” that can be applied in the interim. We must recognize that perfection can only be approached asymptotically through evolution of design and implementation: in essence by refining our deployed very good solutions to make them more perfect. Solving the real problem should be applied where scope allows and it should guide our path to tell us where perfect is, but we are allowed to get there in more than one step. Very good solutions have value, and value delayed is value lost.

The same site had a good collection of quotes regarding change and flexibility:

http://pliantalliance.org/?page_id=5

"He that will not apply new remedies must expect new evils; for time is the greatest innovator.."