tag:blogger.com,1999:blog-8313820002440171722024-02-06T19:45:24.611-08:00The CodeBeneath0C:0D:3B:3N:3A:7H >> Software engineering, continuous integration in practice <br>
and thoughts on getting things done in the workplaceJeffhttp://www.blogger.com/profile/17053090306859709462noreply@blogger.comBlogger46125tag:blogger.com,1999:blog-831382000244017172.post-46759605473366111052018-10-22T20:09:00.000-07:002018-10-22T20:09:29.100-07:00Running Apache Geode in a Docker Container<br />
Apache Geode is a distributed, in-memory database with strong data consistency. It's also the basis for a powerful commercial implementation that runs in Cloud Foundry.<br />
<blockquote class="tr_bq">
"Pivotal Cloud Cache (PCC) is a specialized implementation of Pivotal GemFire, Pivotal’s commercial caching product based on Apache Geode." This quote comes from a <a href="https://content.pivotal.io/blog/acid-cap-and-pcc">good article</a> covering the balancing act between <i>Availability </i>and <i>Consistency </i>of this technology.</blockquote>
In any of it's three variations, it can provide a strong foundation for a <a href="https://12factor.net/">12 Factor</a> application that needs a persistence layer (Geode also supports read-through, write-through and overflow techniques).<br />
<br />
<h2>
Start Geode in a Docker container</h2>
To get started playing around with Geode, it's easy to spin up a Docker container. There is one gotcha, that I'll describe a workaround for. Ideally, someone more Docker savvy can let me know a better method to access Geode from the host OS.<br />
<br />
<span style="font-family: Courier New, Courier, monospace;">docker run -p 8080:8080 -p 10334:10334 -p 40404:40404 -p 1099:1099 -p 7070:7070 -it apachegeode/geode</span><br />
<br />
<h2>
Start a Locator and a Server</h2>
Once the container starts, you'll have an interactive shell active from which you can start Geode services. The "gee-fish" prompt is a shortened name for the GemFire Shell.<br />
<br />
<span style="font-family: Courier New, Courier, monospace;">gfsh > start locator </span><br />
<blockquote class="tr_bq">
Note the hostname it starts on. It will be output from the previous command in a format like: "<span style="font-family: Courier New, Courier, monospace;"><b>d26f1ea42d39</b>[10334]"</span>.</blockquote>
<br />
<span style="font-family: Courier New, Courier, monospace;">gfsh > start server</span><br />
<br />
<h2>
Setup a Geode Region</h2>
<div>
Next, create a region that will host your application data.</div>
<div>
<br /></div>
<span style="font-family: Courier New, Courier, monospace;">gfsh > create region --name=hello-world-region --type=REPLICATE</span><br />
<h2>
<br /></h2>
<h2>
Access the Geode from a Windows OS Host</h2>
To reach the Geode services controller from outside of the container, I had to implement a small networking hack. Edit your Windows "C:\Windows\System32\drivers\etc\hosts" file to give the address "127.0.0.1" an alias matching the locator host address noted above.<br />
<br />
For the location host shown above, the hosts entry would look like this:<br />
<br />
127.0.0.1 localhost <b>d26f1ea42d39</b><br />
<br />
<h2>
Running a Simple Java Client</h2>
I created a simple GitHub repo containing a Java client application to verify that the container is up and accepting data flow to the defined region: <a href="https://github.com/thecodebeneath/geodeclient">https://github.com/thecodebeneath/geodeclient</a><br />
<br />
To run the simple Spring Boot client that writes some data to Geode, run the Maven command:<br />
<br />
<span style="font-family: Courier New, Courier, monospace;">> mvn spring-boot:run</span><br />
<br />
Query the Geode region for data by running this command from the container prompt:<br />
<br />
<span style="font-family: Courier New, Courier, monospace;">gfsh > query --query='select * from /hello-world-region'</span><br />
<br />
<h2>
Summary</h2>
I hope this is a helpful bootstrap to get up and running with a Geode instance.<br />
<br />
If you know of a way to avoid the Windows hosts file edit, please let me know - I'm always ready to learn something new!Jeffhttp://www.blogger.com/profile/17053090306859709462noreply@blogger.com1tag:blogger.com,1999:blog-831382000244017172.post-18143387428347664322018-10-17T22:03:00.003-07:002018-10-17T22:06:52.897-07:00Pushing a Java Project To Cloud FoundryPushing a self-contained Java artifact to Pivotal Cloud Foundry is very straightforward. However, I found the instructions to push an existing legacy application this is not self-contained sparse and not as clear as they could be. This blog post detail what I found out and how to do it.<br />
<blockquote class="tr_bq">
As a code reference, you can look at this Java project and scripts in my GitHub repository here: <a href="https://github.com/thecodebeneath/cfpushdir">https://github.com/thecodebeneath/cfpushdir</a></blockquote>
The project in this repo can be built with Maven and pushed to Cloud Foundry two ways:<br />
<h3>
</h3>
<h3>
1. Standard CF push pointing to a jar file</h3>
This is the easiest method and mirrors a simple uber-jar that the Java buildpack can easily upload, detect and start.<br />
<div>
<br /></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">> cf push -f manifest-from-jarpath.yml</span></div>
<div>
<br /></div>
<div>
Where the manifest file looks like this:</div>
<div>
<br /></div>
<div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">---</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">applications:</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">- name: carservice-from-jarpath</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> path: target/release/lib/carService-1.0-SNAPSHOT-shaded.jar</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> buildpacks:</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> - java_buildpack</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> memory: 1G</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> instances: 1</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> random-route: true</span></div>
<div>
<br /></div>
<h3>
2. CF push a directory of files that contain the jar</h3>
<div>
This is the method that caused me a lot of trouble trying to figure out. The key is understanding that the Java buildpack can handle three separate push styles: 1) jar/war, 2) zip file or 3) nested directories. The last two styles end up both being handled by the "distzip" capability of the Java buildback. This requires an explicit set of named directories, in addition to any that you might also need for your application. The two required directories are "bin" and "lib", as documented here: <a href="https://github.com/cloudfoundry/java-buildpack/blob/master/docs/container-dist_zip.md">Dist Zip Container</a>.</div>
<div>
<ul>
<li>The "bin" directory must contain your start script</li>
<li>The "lib" directory must contain your main Java artifact</li>
</ul>
</div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">> cf push -f manifest-from-dirpath.yml</span></div>
<div>
<br /></div>
<div>
<div>
Where the manifest file looks like this:</div>
<div>
<br /></div>
<div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">---</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">applications:</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">- name: carservice-from-dirpath</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> path: target/release</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> command: "$PWD/bin/startCommand"</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> buildpacks:</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> - java_buildpack</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> memory: 1G</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> instances: 1</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"> random-route: true</span></div>
</div>
</div>
</div>
<div>
<br /></div>
<div>
The target/release directory on your development machine contains the "bin" and "lib" directories, and the startCommand script in the "bin" directory has a command that references the jar as a relative path inside of "lib": </div>
<div>
<br /></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;">> $HOME/.java-buildpack/open_jdk_jre/bin/java -jar lib/carService-1.0-SNAPSHOT-shaded.jar 'manifest-from-dirpath'</span></div>
<div>
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span>
<span style="font-family: "courier new" , "courier" , monospace;"><br /></span></div>
<h3>
Summary</h3>
<div>
I hope my instructions and git project help someone else avoid the time and confusion I had while trying to figure out this solution.</div>
Jeffhttp://www.blogger.com/profile/17053090306859709462noreply@blogger.com0tag:blogger.com,1999:blog-831382000244017172.post-37842817853471006702018-08-21T12:11:00.000-07:002018-08-21T12:15:17.214-07:00Articles You Should ReadHere are my current favorites articles, in no particular order. I've shared them all at work and now I'm sharing them with you.<br />
<br />
<br />
<a href="http://randsinrepose.com/archives/the-guard/" target="_blank">The Guard</a>: I've read this article <i>at </i><i>least </i>nine times now. I identify most with being the Old Guard, which makes me cringe helplessly every time I read it. It is thoughtfully written and makes some excellent points about how a team starts to trust themselves. Key excerpt:<br />
<div>
<ul>
<li>The Old Guard: "I feel empowered to fix everything."</li>
<li>The New Guard: "I don’t know how to fix anything."</li>
</ul>
<div>
<a href="https://m.signalvnoise.com/teaching-iteration-89f384d5c4ab" target="_blank">Teaching Iteration</a>: High schools around the world need to start teaching this topic now and it should be a mandatory course. The cool thing is that the core concept can be applied to just about any subject. Let the <b>students </b>pick the thing they have an interest in and teach them skills for "how to make it better". </div>
<div>
<br />
<a href="http://www.benorenstein.com/blog/dont-get-blocked" target="_blank">Don't Get Blocked</a>: This may be the closest thing I have to a personal mantra. The anti-pattern is people who like putting up roadblocks. Those people should get (symbolically) run over.</div>
</div>
Jeffhttp://www.blogger.com/profile/17053090306859709462noreply@blogger.com0tag:blogger.com,1999:blog-831382000244017172.post-82142085367870886442016-12-15T09:57:00.000-08:002016-12-22T20:14:38.296-08:00Sprint Review Best PracticesThe sprint review is a critical checkpoint in agile development. It is the opportunity for the stakeholder to see your solution to their problems. Their feedback during the presentation leads to future work being defined and provides a common understand of thier pain points to the development team.<br />
<br />
From personal observation, I noticed a few things that detract from the effectiveness of a sprint review. Particularity, new team members don't know what to expect, so I think having a short set of guidelines helps set expectations.<br />
<br />
The best case scenario is that you are sitting in the same room with your client. However, some of the points below consider the challenges of a remote team conducting a sprint reviews via telecon and screen-sharing.<br />
<br />
<span style="font-size: large;">Presentation</span><br />
<br />
<ul>
<li>Have a compelling narrative with demonstrable stories flowing into each other</li>
<ul>
<li>Use a common scenario or theme as much as possible</li>
</ul>
<li>Role-play as a specific stakeholder role, demonstrating new functionality within context of their daily tempo</li>
<ul>
<li>Use their team names, role titles and domain language to inspire confidence that you know their processes and how the implemented stories help them</li>
<li>Shun words and phrases like "lorem ipsum" and "foo/bar" as they show you don't understand the problem or care enough to use stakeholder specific terms</li>
</ul>
<li>Focus on new functionality, unless a review of previous functionality helps understand the context</li>
<ul>
<li>Minimize UI components that don't contribute to demonstrating the new story</li>
<li>Maximize the UI so that background elements don't detract from the story focus</li>
</ul>
<li>Know the audience and understand if they have previously seen the functional area you are talking about. There could be new people listening in or specially invited VIPs.</li>
<ul>
<li>Give reminders of what acronyms stand for before using them</li>
</ul>
<li>Know your project and its capabilities so that you can react to customer confusion and answer questions during the live demonstration</li>
<li>Allow "space" during the presentation for feedback. Remember, a good sprint review is not a monologue, it is a conversation.</li>
</ul>
<span style="font-size: large;">Etiquette</span><br />
<br />
<ul>
<li>The stakeholder always has precedence during conversations; don't over-talk them. Keep side conversations at a low volume so everyone can hear stakeholder feedback.</li>
<li>Mute your telecon microphone if you are not actively involved in a conversation. What seems like low background noise to you can come through loud and clear over the telcon.</li>
<li>Turn off cell phone ringers</li>
<li>Presenter should close all applications, such as email, IM and team chat apps, that could display popup notifications during the presentation</li>
</ul>
<br />
<br />
I hope these points help you during your next sprint review.<br />
<br />
And if you think these are all common sense, stay tuned for my next blog post!Jeffhttp://www.blogger.com/profile/17053090306859709462noreply@blogger.com2tag:blogger.com,1999:blog-831382000244017172.post-43443089096151801832016-11-16T15:24:00.000-08:002016-12-22T20:09:16.993-08:00Agile Story Done Criteria Template<span style="font-family: inherit;">As an agile development team lead, I assisted our solution owner by explaining the intent and direction of our backlog stories to the team. Normally this happened during a weekly team "backlog grooming" session. The purpose of the session was to fully understand the story, scope out the done criteria to match a fairly solid implementation plan and to estimate a story WAG. </span><br />
<span style="font-family: inherit;"><br /></span>
<span style="font-family: inherit;">I found that the extra planning and team discussion before the story was accepted at backlog selection a valuable time investment. To help focus the grooming session discussion, I created a standard template for the story Done Criteria section. The concept of the template is to encourage team participation, scope the full expectations of the story, and document important discussions and ideas that will prevent confusion during the implementing sprint.</span><br />
<span style="font-family: inherit;"><br /></span>
<span style="font-family: inherit;">I'll present my template below and then follow that with a more detailed explanation for each section of the done criteria.</span><br />
<span style="font-family: inherit;"><br /></span>
<br />
<h3>
<span style="font-family: inherit;">Story Done Criteria Template:</span></h3>
<span style="font-family: inherit;"></span><br />
<hr />
<span style="font-family: inherit;">User Acceptance:</span><br />
<br />
<ol>
<li>User can save a new map scene composed of a map background layer, a lat/lon center point and a zoom level.</li>
<li>User can only delete a map scene if the user was the creator.</li>
<li>...</li>
</ol>
<br />
<span style="font-family: inherit;">Team Definition of Done (derived requirements, quality, standards):</span><br />
<br />
<ol>
<li>Approved by UI team</li>
<li>Must create a supporting release notes task documenting manual installation steps</li>
<li>Remove unused database entries for "system.map.*" properties.</li>
<li>...</li>
</ol>
<br />
<span style="font-family: inherit;">Out-of-scope:</span><br />
<br />
<ol>
<li>Users cannot rename map scenes. This is covered by a future backlog story.</li>
</ol>
<br />
<span style="font-family: inherit;">Discussion Answers:</span><br />
<br />
<ol>
<li>TBD</li>
</ol>
<br />
<hr />
<h3>
<span style="font-family: inherit;">Template Explanation:</span></h3>
<span style="font-family: inherit;"><br /></span>
<span style="font-family: inherit;"><b>User Acceptance:</b></span><br />
<span style="font-family: inherit;">User-facing requirements that must be complete for the solution owner to accept the story. Typically thought of as hard requirements that would necessitate direct negotiation with the solution owner (or users) if an problem were discovered during implementation or testing.</span><br />
<span style="font-family: inherit;"><br /></span>
<span style="font-family: inherit;"><b>Team Definition of Done:</b></span><br />
<span style="font-family: inherit;">Derived requirements that influence the team's implementation. These could represent functionality that a user would notice, but that does not impact the user's intent of the story. This category also encompasses criteria that the implementation team values for internal quality, consistency or project standards reasons. Items in the section can be contributed by anyone on the team: developers, designers or quality control, for example.</span><br />
<span style="font-family: inherit;"><br /></span>
<span style="font-family: inherit;">Note that these specific criteria typically represent areas of high risk and otherwise important tasks that have not yet become a normal habit of a high performing team. For example, we don't call out typical story tasks such as unit testing, peer reviews or required documentation updates.</span><br />
<span style="font-family: inherit;"><br /></span>
<span style="font-family: inherit;"><b>Out-of-scope:</b></span><br />
<span style="font-family: inherit;">This important category complements the positive criteria of the story by documenting good ideas for future backlog stories. Having this section seemed to encourage team members to contribute ideas on stories while allowing a place to record the ideas without bloating the scope of the story being discussed.</span><br />
<span style="font-family: inherit;"><br /></span>
<span style="font-family: inherit;">The other important reason for this section is that by being explicit here, we can avoid mid-story confusion when someone asks: What is the expected functionality when the user does X? Did we forget to consider this functionality or did we talk about it and arrive at a decision that it was not covered by this story?</span><br />
<span style="font-family: inherit;"><br /></span>
<span style="font-family: inherit;"><b>Discussion Answers:</b></span><br />
<span style="font-family: inherit;">This section is for use AFTER the story has been accepted by the team and is under development. If any mid-sprint issues are discovered where the expectations are not clearly covered by user criteria, the team will work out a plan with the solution owner AND record the decision here. All such answers should be recorded here as the source of the requirements. This avoids having important story criteria being discussed in email threads, or on the phone, but not transcribed here for full team visibility.</span><br />
<span style="font-family: inherit;"><br /></span>
<span style="font-family: inherit;">Typically, these answers will be translated into criteria for user story acceptance tests (USAT) and become part of the official application specification.</span><br />
<span style="font-family: inherit;"><br /></span>
<span style="font-family: inherit;"><b>Miscellaneous Helpful Ideas</b><b style="font-family: inherit;">:</b></span><br />
<ul>
<li>The criteria should use numbered bullets for quick, unambiguous verbal reference during the team grooming session.</li>
<li>Out-of-Scope items should have a POC assigned so that the ideas get translated into new stories or suggested to the users during the sprint review presentation.</li>
</ul>
<br />
<span style="font-family: inherit;"><br /></span>Jeffhttp://www.blogger.com/profile/17053090306859709462noreply@blogger.com0tag:blogger.com,1999:blog-831382000244017172.post-58047816279966350962016-04-27T23:12:00.000-07:002016-04-27T23:17:01.574-07:00Workplace Strife Advice<br />
This post is a second chance. A respected coworker asked me for advice on a heated personal office conflict, and while I think I gave a good answer, he was unconvinced. In hindsight, I did not do a good job explaining some good reasons for the answer I gave, which summarized, would be:<br />
<blockquote class="tr_bq">
<b>Take the easy out and move on as quickly as possible.</b></blockquote>
While I suspect there is professional pride involved in the disagreement, it is in the interest of everyone in the office to get over it and put it all in the rear view mirror so that work can progress.<br />
<br />
My second chance advice on the nature of the conflict derives from what I know about my coworkers:<br />
<br />
<ul>
<li>They are both experienced team leads</li>
<li>Both teams work on the same software project; developers and quality control</li>
<li>They work in geographically separated offices </li>
<li>Neither has met the other in person</li>
<li>They do work together on a daily basis via <a href="https://slack.com/">team chat</a>, emails and morning scrum</li>
</ul>
<br />
I have an long drive home from work, so I naturally started to reflect on that conversation and the difficult situation that my friends were in. I came up with the following points that may help someone else who is experiencing a difficult personal interaction that is seemingly at an impasse.<br />
<blockquote class="tr_bq">
<b>The very best solution, of course, would be for both people to meet (or call because of being on a distributed team) and work through the differences so they can continue together as a high functioning team.</b></blockquote>
But short of that, I believe all of the points below are helpful. They are not meant to be used all at once as a big fire-hose solution to extinguish a workplace fire. Rather, I'm hoping one or more of these register as a well made point that makes sense and helps someone through a similar conflict.<br />
<br />
I also hope these points are not taken as being callous, as any one of them would resonate with me if I was in the same situation (seemingly stuck without chance of agreement nor common ground reached), and someone offered me an outside view of my situation.<br />
<br />
<ol>
<li>Your future self does not care about this argument as much as you do. Look back on this situation in 10 years and realize that it simply was not that important.</li>
<li>As a team lead, you need to consider that time spent extending a conflict is time taken away from your leadership responsibilities owed to your junior team members</li>
<li>Being wrong can be a drastic side effect of choosing <a href="http://codebeneath.blogspot.com/2007/08/perfect-is-enemy-of-very-good.html">your Perfection over their Very Good</a>.</li>
<li>The thing you are arguing over may be of-value, but the argument itself is valueless. You were hired for your life experiences and professional skill set, neither of which are being put to use to solve the problems you should be working on. You expect to be paid for excessive time spent arguing?</li>
<li>So, you've been challenged to a High Noon showdown and you're asking for advice on which ammunition to use instead of how to avoid a public gunfight?</li>
<li>If you're mutually blocking each others progress, does it matter who gets credit for standing their ground the longest? There is no absolute right or wrong in matters of opinion.</li>
<li><span id="goog_2094293656"></span><a href="http://www.benorenstein.com/blog/dont-get-blocked">Don't get blocked!</a><span id="goog_2094293657"></span> It <b>IS</b> within your power to move forward. </li>
<li>Is your professional ego more important than your human interactions? What is the point of being "right" if you are causing, or prolonging, a painful situation when both of you are literally losing sleep over it? Conflict is natural, but suffering is not.</li>
</ol>
<br />
What advice would you give a coworker in a heated, ongoing situation?<br />
<br />
<br />
<br />Jeffhttp://www.blogger.com/profile/17053090306859709462noreply@blogger.com0tag:blogger.com,1999:blog-831382000244017172.post-8041814629337085352013-11-13T20:33:00.003-08:002013-11-13T20:33:52.944-08:00Always, Never and ContextIt sure would be nice to work for a client that knew exactly what they wanted and why they wanted it. Then again, if that were the case, I probably wouldn't have a job because well defined problems are easier to solve and don't pay as much.<br />
<br />
The more challenging case is when I'm helping them by delivering an iterative solution that works for them in two important ways: 1) It alleviates pain and 2) It actually clarifies the domain problem. It is the software equivalent of a picture is worth a thousand words. Since we deliver software every 30 days, the relative cost is low enough to have confidence to take a step in generally right direction instead of standing still waiting for a concrete requirement to materialize.<br />
<br />
A logical consequence of being in this situation is that when architecting the system to support the "known knowns", there is simply not enough information to declare "Always" or "Never". The reason being that the problem itself or a solution to the problem is not yet fully understood. It is the context that is missing. This is actually OK because an implementation can evolve as does the business case. One example of this could be that users are "misusing" your application in unanticipated ways, leading to an an opportunity to <a href="http://www.w3.org/2007/06/eGov-dc/papers/PavingTheBareSpots">"pave the bare spots"</a>.<br />
<br />
During each iteration of the solution, there should be enough of a cost/benefit analysis to help arrive at an appropriate implementation. Part of this analysis could take into account concepts like "simple as possible, but no simpler" and Behavior-Driven Development to define the progression. I can say with more confidence, however, that the one input that must always be present is domain Context.<br />
<br />
Which leads me to a more concrete example: "An N+1 query is never correct". This is certainly a well known performance antipattern.
But the problem here is one of context. This statement has no insight into the domain problem it is part of solving. There is no business driver inputs that allow such a statement to be true in all cases.<br />
<br />
Consider that your stakeholder has provided you with what they know to be currently true that lets you derive:<br />
<ul>
<li>N maxes out at 2.</li>
<li>N, at an order of magnitude bigger than the largest known business case, executes < 1ms.</li>
<li>An AOP slice needs access to each N for required non-repudiation auditing.</li>
<li>A sql solution is not testable at a necessary confidence level that the DAO language can provide.</li>
<li>Only one developer on your team has expertise to write and maintain an optimized query.</li>
<li>As you build out a backlog of functionality, it is very likely that a near future refactor will negate time spent developing a more performant solution. </li>
</ul>
Without such inputs setting the stage, you are flying blind in a way where absolutes tend to become negative attributes. Solve the problem given what you know, stay loose and don't be afraid to refactor when the context changes.<br />
<br />
To borrow a phrase, Context is King.Jeffhttp://www.blogger.com/profile/17053090306859709462noreply@blogger.com1tag:blogger.com,1999:blog-831382000244017172.post-34355129466234794332013-01-02T10:20:00.000-08:002013-01-02T10:20:08.917-08:00The Marketplace Of Ideas Should Be BrutalI think this wisdom applies to everything in life, beyond software development. You really don't know if your idea is superior until honest feedback is received and it is compared to competing ideas. I thought this was summarized nicely as:
<blockquote>"Ideas are meant to be attacked, torn apart, and put back together again. You may well want to shield your idea from the harsh sunlight at first, but by the time it’s ready to meet the world, it should also be ready for rain or shine. Bad ideas are supposed to wither under the stress of criticism."</blockquote>
-- Via <a href="http://37signals.com/svn/posts/3376-everything-is-not-equally-good">David's post at 37Signals</a>Jeffhttp://www.blogger.com/profile/17053090306859709462noreply@blogger.com0tag:blogger.com,1999:blog-831382000244017172.post-52554298043225767022010-11-09T18:35:00.000-08:002010-11-09T22:17:13.126-08:00Mutual PKI Authentication With a Java-based ApplicationI recently added certificate based authentication to an application I have been working on for awhile. It took longer to get done than I would have thought primarily because the number of moving pieces and most advice and guidance I found online was incomplete. I hope that documenting the complete setup I used will save someone else some time and frustration.<br /><br />The requirements were:<br /><ul><li>Wiring together two Java applications via SOAP, using CXF framework</li><li>The producing application needed a valid server certificate</li><li>The consuming application needed to present a valid client certificate to the server</li><li>These certificates needed to be signed by a trusted root and share the same certificate authority</li><li>For authentication purposes, no custom code within the application itself</li><li>Use the Java keytool program to create keypairs and generate signing requests</li></ul><br />The broad strokes:<br /><ol><li>Create the necessary certificates</li><li>Setup tomcat on the producing side to accept only https connections and require client authentication</li><li>Setup the consuming application to present client certs that can satisfy the secure authentication handshake</li></ol><br /><span style="font-size:130%;"><span style="font-weight: bold;">Creating and signing the certificates</span></span><br /><br />I have previously created a CA setup using strictly openssl, but for this project, I needed to lean more heavily on <code>keytool</code> to originate the keypairs and openssl to do the certificate signing. The <code>keytool</code> program comes with the JDK. I used a windows based openssl binaries from <a href="http://www.slproweb.com/products/Win32OpenSSL.html">SLP</a>.<br /><br />Setup a directory structure as recommended by openssl installation and set an environment variable, <code>%SSLDIR%</code>, to point to your CA directory.<br /><br /><span style="font-style: italic;">Create CA and RA</span><br /><pre class="brush:js"># Create the CA's keypair and self-signed certificate<br /># -x509 means create self-sign cert<br /># -keyout means generate keypair<br /># -nodes means do not encrypt private key.<br /># -set_serial sets the serial number of the certificate<br />openssl req -verbose -x509 -new -nodes -set_serial 1234 -subj "/CN=codebeneathCA/OU=project/O=company/ST=MO/C=US" -days 7300 -out %SSLDIR%/certs/cacert.pem -keyout %SSLDIR%/private/caprivkey.pem -config %SSLDIR%/openssl.config</pre><br /><br /><pre class="brush:js"># Create the Root Authority's keypair and Certificate Request<br /># without -x509, we generate an x509 cert request.<br /># -keyout means generate keypair<br /># -nodes means do not encrypt private key.<br />openssl req -verbose -new -nodes -subj "/CN=codebeneathRoot/OU=project/O=company/ST=MO/C=US" -days 7300 -out %SSLDIR%/certs/csrra.pem -keyout %SSLDIR%/private/raprivkey.pem -config %SSLDIR%/openssl.config</pre><br /><br /><pre class="brush:js"># Have the CN=codebeneathCA issue a certificate for the CN=codebeneathRoot<br /># We need -extfile exts -extenstions x509_extensions to make sure<br /># CN=TheRA can be a Certificate Authority.<br />openssl ca -batch -days 7300 -cert %SSLDIR%/certs/cacert.pem -keyfile %SSLDIR%/private/caprivkey.pem -in %SSLDIR%/certs/csrra.pem -out %SSLDIR%/certs/ra-ca-cert.pem -extfile exts -extensions x509_extensions -config %SSLDIR%/openssl.config</pre><br /><br /><span style="font-style: italic;">Create Server Cert</span><br /><br /><pre class="brush:js"># Create keypairs and Cert Request for a certificate for server<br /># This procedure must be done in JKS, because we need to use a JKS keystore.<br /># The current version of CXF using PCKS12 will not work for a number of<br /># internal CXF reasons.<br />keytool -genkey -alias codebeneath-server -dname "CN=10.9.5.210,OU=codebeneath-server,O=company,ST=MO,C=US" -keystore C:/certs/server-keystore.jks -storetype jks -storepass password -keypass password -validity 365<br />keytool -certreq -alias codebeneath-server -keystore C:/certs/server-keystore.jks -storetype jks -storepass password -keypass password -file codebeneath-server-csr.pem</pre><br /><br /><pre class="brush:js"># Have the CN=codebeneathRoot issue a certificate for server via the Certificate Requests.<br />openssl ca -batch -days 7300 -cert %SSLDIR%/certs/ra-ca-cert.pem -keyfile %SSLDIR%/private/raprivkey.pem -in %SSLDIR%/requests/codebeneath-server-csr.pem -out %SSLDIR%/requests/codebeneath-server-ra.pem -config %SSLDIR%/openssl.config</pre><br /><br /><pre class="brush:js"># Rewrite the certificates in PEM only format. This allows us to concatenate them into chains.<br />openssl x509 -in %SSLDIR%/certs/cacert.pem -out %SSLDIR%/certs/cacert.pem -outform PEM<br />openssl x509 -in %SSLDIR%/certs/ra-ca-cert.pem -out %SSLDIR%/certs/ra-ca-cert.pem -outform PEM<br />openssl x509 -in %SSLDIR%/requests/codebeneath-server-ra.pem -out %SSLDIR%/requests/codebeneath-server-ra.pem -outform PEM</pre><br /><br /><pre class="brush:js"># Create a chain readable by CertificateFactory.getCertificates.<br />type %SSLDIR%\requests\codebeneath-server-ra.pem %SSLDIR%\certs\ra-ca-cert.pem %SSLDIR%\certs\cacert.pem > %SSLDIR%\requests\codebeneath-server.chain</pre><br /><br /><pre class="brush:js"># Replace the certificate in the server keystore with their respective full chains.<br />keytool -import -file codebeneath-server.chain -alias codebeneath-server -keystore C:/certs/server-keystore.jks -storetype jks -storepass password -keypass password -noprompt</pre><br /><br /><pre class="brush:js"># Create the server Truststore file containing the CA cert.<br />keytool -import -file cacert.pem -alias codebeneathCA -keystore C:/certs/server-truststore.jks -storepass password -noprompt</pre><br /><br /><span style="font-style: italic;">Create Client Cert</span><br /><br /><pre class="brush:js"># Create keypairs and Cert Request for a certificate for client (codebeneath)<br /># This procedure must be done in JKS, because we need to use a JKS keystore.<br /># The current version of CXF using PCKS12 will not work for a number of<br /># internal CXF reasons.<br />keytool -genkey -alias codebeneath -dname "CN=10.9.5.105,OU=codebeneath-client,O=company,ST=MO,C=US" -keystore C:/certs/client-keystore.jks -storetype jks -storepass password -keypass password -validity 365<br />keytool -certreq -alias codebeneath -keystore C:/certs/client-keystore.jks -storetype jks -storepass password -keypass password -file codebeneath-csr.pem</pre><br /><br /><pre class="brush:js"># Have the CN=codebeneathRoot issue a certificate for client via the Certificate Requests.<br />openssl ca -batch -days 7300 -cert %SSLDIR%/certs/ra-ca-cert.pem -keyfile %SSLDIR%/private/raprivkey.pem -in %SSLDIR%/requests/codebeneath-csr.pem -out %SSLDIR%/requests/codebeneath-ra.pem -config %SSLDIR%/openssl.config</pre><br /><br /><pre class="brush:js"># Rewrite the certificates in PEM only format. This allows us to concatenate them into chains.<br />openssl x509 -in %SSLDIR%/requests/codebeneath-ra.pem -out %SSLDIR%/requests/codebeneath-ra.pem -outform PEM</pre><br /><br /><pre class="brush:js"># Create a chain readable by CertificateFactory.getCertificates.<br />type %SSLDIR%\requests\codebeneath-ra.pem %SSLDIR%\certs\ra-ca-cert.pem %SSLDIR%\certs\cacert.pem > %SSLDIR%\requests\codebeneath.chain</pre><br /><br /><pre class="brush:js"># Replace the certificate in the client keystore with their respective full chains.<br />keytool -import -file codebeneath.chain -alias codebeneath -keystore C:/certs/client-keystore.jks -storetype jks -storepass password -keypass password -noprompt</pre><br /><br /><pre class="brush:js"># Create the client Truststore file containing the CA cert and server cert.<br />keytool -import -file cacert.pem -alias codebeneathCA -keystore C:/certs/client-truststore.jks -storepass password -noprompt<br />keytool -import -file codebeneath-server-ra.pem -alias codebeneath-server -keystore C:/certs/client-truststore.jks -storepass password -noprompt</pre><br /><br /><span style="font-style: italic;">Create trust for the server to know the client</span><br /><br /><pre class="brush:js"># Import the client cert into the server Truststore.<br />keytool -import -file codebeneath-ra.pem -alias codebeneath -keystore C:/certs/client-truststore.jks -storepass password -noprompt</pre><br /><br /><span style="font-style: italic;">Export client cert from keystore for browser import </span><br /><br />The <code>keytool</code> program cannot export private keys in any format, so use <a href="http://portecle.sourceforge.net/">portecle</a> to do these steps so that we can verify the setup using a browser without involving the actual client application.<br /><ol><li>Start portecle: <code>java -jar ./portecle.jar</code></li><li>File... Open keystore file... select the <code>./client-keystore.jks</code> file. Type the password.</li><li>Right-client the "codebeneath" certificate name. Select Export...</li><li>Export Type: Private Key and Certificates. Export Format: PKCS #12. Click OK. Type the password.</li><li>For the PKCS #12 password, it must be non-blank value or else the browser will not import it correctly.</li><li>In the browser, import the certificate as a "Personal" certificate.</li><li>In the browser, import the CA certificate as a trusted authoritative certificate.</li></ol><br /><span style="font-style: italic;">Double check the results so far</span>...<br /><br />On the server machine:<br /><br /><code>keytool -list -keystore ./server-keystore.jks</code><br /><ul><li>one signed certificate chain, PrivateKeyEntry, alias="codebeneath-server"</li></ul><code>keytool -list -keystore ./server-truststore.jks</code><br /><ul><li>CA certificate, trustedCertEntry, alias="codebeneathca",</li><li>client certificate,trustedCertEntry, alias="codebeneath"</li></ul><br />On the client machine:<br /><br /><code>keytool -list -keystore ./client-keystore.jks</code><br /><ul><li>one signed certificate chain, PrivateKeyEntry, alias="codebeneath"</li></ul><code>keytool -list -keystore ./client-truststore.jks</code><br /><ul><li>CA certificate, trustedCertEntry, alias="codebeneathca",</li><li>server certificate, trustedCertEntry, alias="codebeneath-server"</li></ul><span style="font-weight: bold;font-size:130%;" ><br />Server setup using tomcat</span><br /><br />To accomplish the required mutual authentication (server is trusted https:// and client presents a valid certificate) requires setup within tomcat and the server war web.xml file. There is no code involved, it is strictly the setup of the servlet container to handle the security handshaking.<br /><br /><span style="font-style: italic;">Tomcat server.xml file</span><br /><br />Note that the <code>clientAuth</code> attribute must be "true" for things to work properly, even though the specification says that each war may individually be setup to require/not require client auth.<br /><pre class="brush:xml">...<br /><Connector port=\"8443\" protocol=\"HTTP/1.1\" SSLEnabled=\"true\"<br /> maxThreads=\"150\" scheme=\"https\" secure=\"true\"<br /> clientAuth=\"true\" sslProtocol=\"TLS\"<br /> keystoreFile=\"C:/certs/server-keystore.jks\"<br /> keystorePass=\"password\"<br /> keyAlias=\"codebeneath-server\"<br /> truststoreFile=\"C:/certs/server-truststore.jks\"<br /> truststorePass=\"password\"<br /> /><br />...</pre><br /><span style="font-style: italic;">Tomcat tomcat-users.xml file</span><br /><br />Each username must match a client certificate subject line. Use portecle to open the client cert and cut-and-paste the "Subject:" line to this file's username value. So, even though the client certificate is valid given the truststore requirements, each client must also be specifically listed in this file to have access to the server application.<br /><pre class="brush:xml">...<br /><role rolename=\"codebeneathRole\"><br /><user username=\"CN=10.9.5.43, OU=project, O=company, ST=MO, C=US\" password=\"password\" roles=\"codebeneathRole\"><br />...</pre><br /><br /><span style="font-style: italic;">Server war web.xml file</span><br /><br />Both "role-name" elements must match the tomcat-users.xml defined rolename. The "auth-method" value of "CLIENT-CERT" will require a client certificate for authentication. The "transport-guarentee" value of "CONFIDENTIAL" will redirect all requests<br />...<br /><pre class="brush:xml"><login-config><br /><auth-method>CLIENT-CERT</auth-method><br /></login-config><br /><security-role><br /><role-name>codebeneathRole</role-name><br /></security-role><br /><security-constraint><br /><display-name>CodeBeneath</display-name><br /><web-resource-collection><br /> <web-resource-name>CodeBeneath</web-resource-name><br /> <description></description><br /> <url-pattern>/*</url-pattern><br /> <http-method>GET</http-method><br /> <http-method>POST</http-method><br /> <http-method>HEAD</http-method><br /> <http-method>PUT</http-method><br /> <http-method>OPTIONS</http-method><br /> <http-method>TRACE</http-method><br /> <http-method>DELETE</http-method><br /></web-resource-collection><br /><auth-constraint><br /> <description><br /> <role-name>codebeneathRole</role-name><br /></auth-constraint><br /><user-data-constraint><br /> <description><br /> <transport-guarantee>CONFIDENTIAL</transport-guarantee><br /></user-data-constraint><br /></security-constraint><br />...</pre><br /><br /><span style="font-style: italic;">Debug SSL</span><br /><br />In the event that the mutual authentication fails, we can enable specific debugging on the server side. In the <code>catalina.bat</code> file, add the switch <code>-Djavax.net.debug=ssl,handshake</code><br /><br /><span style="font-weight: bold;font-size:130%;" >Client Setup<br /></span><br />Since the project used SOAP WS-*, including WS-Security, on the client side we used the CXF project to handle the wsdl-to-java code generation and to create the client port. My project uses CXF version 2.2.11, with the additional "cxf-rt-ws-security" dependency jar.<br /><br /><span style="font-style: italic;">Client java code</span><br /><pre class="brush:java">package com.codebeneath;<br /><br />import org.apache.cxf.endpoint.Client;<br />import org.apache.cxf.frontend.ClientProxy;<br />import org.apache.cxf.jaxws.JaxWsProxyFactoryBean;<br />//import org.apache.cxf.interceptor.LoggingInInterceptor;<br />import org.apache.cxf.interceptor.LoggingOutInterceptor;<br />import org.apache.cxf.transport.http.HTTPConduit;<br />import org.apache.cxf.configuration.jsse.TLSClientParameters;<br />import org.apache.cxf.binding.soap.saaj.SAAJOutInterceptor;<br />import org.apache.cxf.ws.security.wss4j.WSS4JOutInterceptor;<br />import org.apache.ws.security.handler.WSHandlerConstants;<br /><br />import java.net.URL;<br />import java.security.GeneralSecurityException;<br />import java.security.KeyStore;<br />import java.security.KeyStoreException;<br />import java.security.NoSuchAlgorithmException;<br />import java.security.UnrecoverableKeyException;<br />import java.security.cert.CertificateException;<br />import java.util.HashMap;<br />import java.util.Map;<br />import java.util.logging.Logger;<br />import javax.net.ssl.KeyManager;<br />import javax.net.ssl.KeyManagerFactory;<br />import javax.net.ssl.TrustManager;<br />import javax.net.ssl.TrustManagerFactory;<br />import javax.xml.ws.BindingProvider;<br /><br />/**<br />* Client a SOAP WS client for the service. Handles logging support, WS-Security and PKI certificates<br />*/<br />public final class Client {<br /><br />private static final String DEFAULT_PASSWORD = "password";<br />private static final String KEYSTORE_FILENAME = "/client-keystore.jks";<br />private static final String TRUSTSTORE_FILENAME = "/client-truststore.jks";<br /><br />private Client() {<br />}<br /><br />/**<br /> * Create a new WS client<br /> * @param serviceUrl, the service URL endpoint<br /> * @param sigPropFile, the properties file for client PKI. Must be on the classpath at runtime<br /> * @return client port<br /> */<br />public static TroubleTicketServiceSoap createClient(URL serviceUrl, String sigPropFile, String keystoreDir) {<br /><br /> JaxWsProxyFactoryBean factory = new JaxWsProxyFactoryBean();<br /> factory.setAddress(serviceUrl.toString());<br /> factory.setServiceClass(TroubleTicketServiceSoap.class);<br /><br /> // request and response traces in the server.log<br /> addLogging(factory);<br /><br /> // SOAP WS-Security header<br /> addSecurityHeaders(factory, sigPropFile);<br /><br /> // create the client<br /> TroubleTicketServiceSoap port = (TroubleTicketServiceSoap) factory.create();<br /> ((BindingProvider) port).getRequestContext().put("schema-validation-enabled", Boolean.TRUE.toString());<br /><br /> // handle trust of server certificates<br /> addCertificateHandling(port, serviceUrl, keystoreDir);<br /><br /> return port;<br />}<br /><br />private static void addLogging(JaxWsProxyFactoryBean factory) {<br /> factory.getOutInterceptors().add(new LoggingOutInterceptor()); // SOAP Request<br />// factory.getInInterceptors().add(new LoggingInInterceptor()); // SOAP Response<br />}<br /><br />private static void addSecurityHeaders(JaxWsProxyFactoryBean factory, String sigPropFile) {<br /><br /> // setup the properties for WS-Security. This is PKI signed. The SIG_PROP_FILE must be on the classpath<br /> Map<string, object=""> wss4jOutProps = new HashMap<string, object="">();<br /> wss4jOutProps.put(WSHandlerConstants.ACTION, WSHandlerConstants.SIGNATURE);<br /> wss4jOutProps.put(WSHandlerConstants.USER, "client");<br /> wss4jOutProps.put(WSHandlerConstants.PW_CALLBACK_CLASS, ClientPasswordCallback.class.getName());<br /> wss4jOutProps.put(WSHandlerConstants.SIG_PROP_FILE, sigPropFile);<br /> wss4jOutProps.put(WSHandlerConstants.SIG_KEY_ID, "DirectReference");<br /><br /> // add required security interceptors<br /> factory.getOutInterceptors().add(new SAAJOutInterceptor());<br /> factory.getOutInterceptors().add(new WSS4JOutInterceptor(wss4jOutProps));<br />}<br /><br />private static void addCertificateHandling(TroubleTicketServiceSoap port, URL serviceUrl, String keystoreDir) {<br /> Client proxy = ClientProxy.getClient(port);<br /> HTTPConduit conduit = (HTTPConduit) proxy.getConduit();<br /> TLSClientParameters tcp = new TLSClientParameters();<br /> tcp.setDisableCNCheck(true);<br /><br /> try {<br /> KeyStore trustStore = KeyStore.getInstance("JKS");<br /> KeyStore keyStore = KeyStore.getInstance("JKS");<br /><br /> // accept all self-signed server certs<br />// tcp.setTrustManagers(CertificateAcceptorCXF.acceptAllCerts());<br /><br /> File truststoreFile = new File(keystoreDir + TRUSTSTORE_FILENAME);<br /> trustStore.load(new FileInputStream(truststoreFile), DEFAULT_PASSWORD.toCharArray());<br /> TrustManagerFactory trustFactory = TrustManagerFactory.getInstance(TrustManagerFactory.getDefaultAlgorithm());<br /> trustFactory.init(trustStore);<br /> TrustManager[] tm = trustFactory.getTrustManagers();<br /> tcp.setTrustManagers(tm);<br /><br /> // enable client certs for authentication<br /> File keystoreFile = new File(keystoreDir + KEYSTORE_FILENAME);<br /> keyStore.load(new FileInputStream(keystoreFile), DEFAULT_PASSWORD.toCharArray());<br /> KeyManagerFactory keyFactory = KeyManagerFactory.getInstance(KeyManagerFactory.getDefaultAlgorithm());<br /> keyFactory.init(keyStore, DEFAULT_PASSWORD.toCharArray());<br /> KeyManager[] km = keyFactory.getKeyManagers();<br /> tcp.setKeyManagers(km);<br /><br /> } catch (KeyStoreException kse) {<br /> System.out.println("Security configuration failed with the following: " + kse.getCause());<br /> } catch (NoSuchAlgorithmException nsa) {<br /> System.out.println("Security configuration failed with the following: " + nsa.getCause());<br /> } catch (FileNotFoundException fnfe) {<br /> System.out.println("Security configuration failed with the following: " + fnfe.getCause());<br /> } catch (UnrecoverableKeyException uke) {<br /> System.out.println("Security configuration failed with the following: " + uke.getCause());<br /> } catch (CertificateException ce) {<br /> System.out.println("Security configuration failed with the following: " + ce.getCause());<br /> } catch (GeneralSecurityException gse) {<br /> System.out.println("Security configuration failed with the following: " + gse.getCause());<br /> } catch (IOException ioe) {<br /> System.out.println("Security configuration failed with the following: " + ioe.getCause());<br /> }<br /><br /> if (serviceUrl.getProtocol().trim().equalsIgnoreCase("https")) {<br /> conduit.setTlsClientParameters(tcp);<br /> LOG.info("GSTv3 Web Service protocol is 'https' so " + "set the TLSClientParameters on the "<br /> + "HTTPConduit. This should cause the" + " request to accept all certificates "<br /> + "and disable the CN Check.");<br /> } else {<br /> LOG.info("GSTv3 Web Service url did was not 'https' so did NOT " + "set the TLSClientParameters "<br /> + "on the HTTPConduit.");<br /> }<br />}<br />}</string,></string,></pre><br /><br /><span style="font-style: italic;">Client client-sign.properties</span><br /><br /><pre class="brush:js"># Do not change these two properties<br />#<br />org.apache.ws.security.crypto.provider=org.apache.ws.security.components.crypto.Merlin<br />org.apache.ws.security.crypto.merlin.keystore.type=jks<br /># Set your keystore location and password here!<br />#<br />org.apache.ws.security.crypto.merlin.file=c:/certs/client-keystore.jks<br />org.apache.ws.security.crypto.merlin.keystore.password=password<br />org.apache.ws.security.crypto.merlin.keystore.alias=codebebeath<br /></pre><br /><span style="font-weight: bold;font-size:130%;" ><br />Conclusion</span><br /><br />I hope that this complete picture of mutual PKI certificate authentication helps. If anything is not clear, let me know and I will try to clarify.Jeffhttp://www.blogger.com/profile/17053090306859709462noreply@blogger.com9tag:blogger.com,1999:blog-831382000244017172.post-91151554784748930312010-06-10T07:39:00.000-07:002010-06-10T07:44:28.351-07:00"Perfectionism is a crime against humanity"I wish I had written this...<br /><br /><blockquote>You could say that perfectionism is a crime against humanity. Adaptability is the characteristic that enables the species to survive-and if there’s one thing perfectionism does, it rigidifies behavior. It constricts people just when the fast-moving world requires more flexibility and comfort with ambiguity than ever. It turns people into success slaves.</blockquote><br /><br />There are several people I work with or deal with in my life that don't realize that the return on investment for their efforts is detrimental to success of the project. It is an ongoing battle and the battle-cry is "Moving on...".<br /><br />Read the rest of the source article here: <a href="http://blogs.techrepublic.com.com/career/?p=2087&tag=results;CR1">Is perfectionism killing your career?</a>Jeffhttp://www.blogger.com/profile/17053090306859709462noreply@blogger.com0tag:blogger.com,1999:blog-831382000244017172.post-24497009551888057322009-12-28T07:42:00.000-08:002009-12-28T07:48:01.973-08:00"There’s no success quite like failure"From <a href="http://www.wired.com/magazine/2009/12/fail_accept_defeat/all/1">Accept Defeat: The Neuroscience of Screwing Up</a>:<br /><br /><blockquote>"This is why other people are so helpful: They shock us out of our cognitive box. “I saw this happen all the time,” Dunbar says. “A scientist would be trying to describe their approach, and they’d be getting a little defensive, and then they’d get this quizzical look on their face. It was like they’d finally understood what was important.”<br /><br />What turned out to be so important, of course, was the unexpected result, the experimental error that felt like a failure. The answer had been there all along — it was just obscured by the imperfect theory, rendered invisible by our small-minded brain. It’s not until we talk to a colleague or translate our idea into an analogy that we glimpse the meaning in our mistake. Bob Dylan, in other words, was right: There’s no success quite like failure."</blockquote>Jeffhttp://www.blogger.com/profile/17053090306859709462noreply@blogger.com0tag:blogger.com,1999:blog-831382000244017172.post-18874083637465458982009-12-04T06:21:00.000-08:002009-12-04T06:33:11.431-08:00Grail Projects in Hudson, Pt 2.I just recently concluded a first sprints worth of work using Grails for the first time. We <a href="http://codebeneath.blogspot.com/2009/09/shipping-is-feature.html">delivered</a>. <br /><br />While cutting the release, I found one additional gotcha in our CI environment. The grails application.properties file holds several key pieces of information including the version number, grails plugin versions and the last build date. This means the file must be checked into source control, but is also modified locally every build by CI (Hudson, nudge, nudge, wink wink). This caused our build to fail while bumping our project version.<br /><br /><ol><li>CI will fail a grails+maven project if you change the version number of the project in the application.properties file.<ul><li>Workaround: 1) Have your CI project perform a clean checkout instead of an update for you build. 2) At the end of your build, add a custom hook to delete the locally modified application.properties file.</li><li>Fix: I think have the build timestamp in this file may be a mistake as Grails grows more mature. As an improvement, it should be removed.<br /></li></ul></li></ol>Jeffhttp://www.blogger.com/profile/17053090306859709462noreply@blogger.com1tag:blogger.com,1999:blog-831382000244017172.post-34943881068596603112009-11-20T15:52:00.000-08:002009-11-20T16:48:38.234-08:00Grail Projects in HudsonHere is a quick reference for adding a Grails project to Hudson. I'm assuming the webtest plugin is installed becuase there is no reason not to use it. It rocks. Also note that for integration into my existing maven build, I'm using the built-in Grails v1.1.1 maven integration. The project structure was created via the maven archetype:<br /><pre><blockquote>mvn org.apache.maven.plugins:maven-archetype-plugin:2.0-alpha-4:generate \<br />-DarchetypeGroupId=org.grails \<br />-DarchetypeArtifactId=grails-maven-archetype \<br />-DarchetypeVersion=1.1 \<br />-DgroupId=com.black -DartifactId=grails-project</blockquote></pre><br />With that, here is what works so far:<br /><br />General sequence (these are also the steps for keeping a development team in synch):<br /><ol><li><pre>svn up</pre></li><li><pre>grails upgrade --non-interactive</pre></li><li><pre>mvn clean install -DnonInteractive=true</pre></li></ol><br />Notes:<br /><ul><li> Step 2 above: This upgrades the grails project to include all core grails plugins and also creates some important files under web-app/WEB-INF (applicationContext.xml, ...). We would rather run <code>mvn grails:exec -Dcommand=upgrade -DnonInteractive</code>, however, this fails with an error: <code>Embedded error: java.lang.reflect.InvocationTargetException<br />/home/black/dev/branches/parent/grails-project/null/src/war not found.</code> Anybody have an idea on this?</li><li>Step 3 above: This step will upgrade all non-core plugins added/modified/removed since the last checkout. Since grails will ask the "are you sure" question, make sure in CI to add the non-interactive switch as above, which will have the effect of cheerfully answering "Y" to all of grails questions.<br /></li></ul><br />Problems you may run into:<br /><ol><br /><li>As part of a big maven multiproject build, grails may consume enough memory to produce an out of memory error.<ul><li> Fix: Set <code>MAVEN_OPTS=-Xmx512m -XX:MaxPermSize=150m</code> </li></ul><br /></li><br /><li>Adding new grails plugins to the build will freeze the CI build due to the "are you sure" prompt. The same will happen when removing a grails plugin.<ul><li> Fix: Add the non-interactive switch to the build (<code>mvn -DnonInteractive=true</code> or <code>grails --non-interactive</code>) </li></ul><br /></li><br /><li>Webtest launches a window during the test run that needs a display, so CI may need to run the webtests headless.<ul><li> Fix: Add non-interactive switch to the build (<code>mvn -DnonInteractive=true</code> or <code>grails --non-interactive</code>)</li></ul><br /></li><br /><li>Webtest starts the jetty server on port 8080 by default which may clash with other services on your CI box using that port.<ul><li> Fix: Use the switch <code>mvn -Dserver.port=8183</code>. Note: The webtest.properties file property <code><code>wt.config.port</code></code> seems to be used if the code under test were already hosted on another running server besides the provided jetty. </li></ul><br /></li><br /><li>A maven multiproject build that includes a grails project with webtests will look for <code>/test/webtest/conf/webtest.properties</code> in the wrong directory. That is, it will not look in ${basedir}, but always in the root project where you invoked maven to start with.<br /><ul><li> Workaround: Create a hudson job that only builds the grails war/webtest project. Create a downstream dependency on your main CI hudson job to trigger this one to build. </li><br /><li> Fix: ?? Anyone have an answer for this one? </li></ul><br /></li><br /></ol><br />I hope this helps. For the few items above that have lingering questions, I would be interested in hearing if you have solved these. Let me know.Jeffhttp://www.blogger.com/profile/17053090306859709462noreply@blogger.com1tag:blogger.com,1999:blog-831382000244017172.post-13601450098946706252009-09-24T07:48:00.000-07:002009-09-24T07:53:01.402-07:00"Shipping is a feature."Another good variation on <a href="http://codebeneath.blogspot.com/2007/08/perfect-is-enemy-of-very-good.html">Perfect is the Enemy of Very Good</a>:<br /><br /><blockquote>"A 50%-good solution that people actually have solves more problems and survives longer than a 99% solution that nobody has because it’s in your lab where you’re endlessly polishing the damn thing. Shipping is a feature. A really important feature. Your product must have it."<br />-- <a href="http://www.joelonsoftware.com/items/2009/09/23.html">http://www.joelonsoftware.com/items/2009/09/23.html</a><br /></blockquote>Jeffhttp://www.blogger.com/profile/17053090306859709462noreply@blogger.com1tag:blogger.com,1999:blog-831382000244017172.post-50059734948023250162009-06-24T21:27:00.000-07:002009-06-24T21:39:36.175-07:00Hudson Gadget For Google Desktop v1.0For all you developers that rely on Hudson for continuous integration build status, I have released version 1.0 of the Hudson Status Gadget for Google Desktop. The code is part of the Hudson project itself, so it is open source.<br /><br /><blockquote>More information can be found on its <a href="http://wiki.hudson-ci.org/display/HUDSON/Hudson+Google+Desktop+Gadget">home page</a>. </blockquote>Release note highlights for version 1.0:<br /><ul><li>In addition to monitoring the multiple main hudson dashboards, any tabbed view can be also be monitored</li><li>A scrollbar shows for a large number of jobs that would take more space then the gadget allows</li><li>Views can be collapsed to a single line with all jobs statuses of that view aggregating their status to the single view line</li><li>A refresh button has been added for manual status refreshes</li><li>Blinking build statuses are now animated</li></ul><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjxkzKJMnGsUe1-1lKDKck_4hrfzkyy86Qq4FADDO_k944ubm_D1KTpBdv1UR_9hGGZa1J_c-JQ_34E3-TTKcYhpy8eBjBU3RY26WPsErvxVWpHo-uJkKXM68LalDRTxZhEGAwN35wOdTgd/s1600-h/screenshot_large.png"><img style="margin: 0pt 10px 10px 0pt; float: left; cursor: pointer; width: 300px; height: 225px;" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjxkzKJMnGsUe1-1lKDKck_4hrfzkyy86Qq4FADDO_k944ubm_D1KTpBdv1UR_9hGGZa1J_c-JQ_34E3-TTKcYhpy8eBjBU3RY26WPsErvxVWpHo-uJkKXM68LalDRTxZhEGAwN35wOdTgd/s320/screenshot_large.png" alt="" id="BLOGGER_PHOTO_ID_5351119146439661810" border="0" /></a><br />Enjoy and let me know how it works out for you.Jeffhttp://www.blogger.com/profile/17053090306859709462noreply@blogger.com0tag:blogger.com,1999:blog-831382000244017172.post-81946244771314997592008-11-23T21:00:00.000-08:002008-11-23T19:02:47.186-08:00The Supply Chain of Continuous Integration<a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgp4OMW7ZaMScIbJ4CuDxC56CuYyeYN0bbbczfE6-QN_2btvOK2wcn2nTxTPXYyT8NZ6H38HUCtYRKntWF9tneJjyc-tuwUulRxB4X7Kr_3os_WCfB9XOA-gtOLiflho_DG_6a0891YYzUe/s1600-h/ci_logo.png"><img style="margin: 0pt 0pt 10px 10px; float: right; cursor: pointer;" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgp4OMW7ZaMScIbJ4CuDxC56CuYyeYN0bbbczfE6-QN_2btvOK2wcn2nTxTPXYyT8NZ6H38HUCtYRKntWF9tneJjyc-tuwUulRxB4X7Kr_3os_WCfB9XOA-gtOLiflho_DG_6a0891YYzUe/s400/ci_logo.png" alt="" id="BLOGGER_PHOTO_ID_5210720429971981410" border="0" /></a>When I was first introduced to Continuous Integration, I viewed it as a black box with a well defined interface. It was the same kind throw-it-over-the-wall mentality that some people have with testing: When the code is "done", give it to the testing shop and mark the checkbox complete. It may or may not return with feedback attached.<br /><br />I got a different perspective of how CI should work, however, while working on a CI team. More than a little of this new perspective is probably due to working in a company that believes in and practices agile development and scrum management. With agile, we are much better at involving testers into day-to-day activities rather than the cold baton hand-off as sprint review time rolls near.<br /><br />What got me thinking about a CI supply chain concept was the story from The Earth is Flat about how UPS has evolved from a package delivery company to an integral part of many companies day-to-day business operations. They started with a core business of picking up and shipping packages. But it turns out, there are inefficiencies to this bolted on approach. In order for the shipping to be efficient for both UPS and the contracting company, the entire supply chain had to be prepped in advance. This includes both pre and post shipment. The visibility into business processes had to be bidirectional.<br /><br />I think the same is true with continuous integration. It is based on the old computer adage, GIGO, Garbage In Garbage Out. So a team that writes plenty of code, but no tests. What is the value of CI for them? The feedback is minimal. To get the most value out of an automated build system, there must be some forethought into what kind of quality feedback you want to see, and then what teams can do to define and integrate the right reporting tools into the build.<br /><br />For example, even after automated tests written and incorporated into the build, you may want to get feedback on some quality measures on your code. In our case, we wanted Checkstyle and PMD reporting on our Java code. We use maven as our build tool, so adding the reporting into our builds was simple. But then the question becomes, what coding standards do we want to compare to? What PMD rulesets represent a sane minimum that teams can deal with and learn from at the same time?<br /><br />So now, a team dedicated to providing CI services is recommending quality reports to both developers and stakeholders AND helping to define the rules that code should be compared against. My first take on this approach, deviating from a traditional CI definition was a sense of invasiveness. Upon further reflection, I have embraced the blurred lines between teams for several reasons. Firstly, it breaks artificial team boundaries and keeps communication lines open. CI is no longer a black box, it is a visible, value contributing component of system development. Secondly, developers won't (at least consistently across an organization) take the time to inject QA into their processes. A CI team, however, does have the time to take the first steps and get the ball rolling. The conversation is more constructive when you have something in place actually working then just a bunch of talking going on about what could be.<br /><br /><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiokLNcSKmj5dlKD5twfxQrhfoUZ1BS-bPGV9DImaxY-2Ju_8pesNfAVHR64jgDip0iAuZKeOH8hIJY6tXVISS1vq0NtwCsb-DbFAHbrdDsdSB0Ruclln4nReOrhxMcrdLNCISN1XSb-Ij5/s1600-h/ci-venn.png"><img style="margin: 0px auto 10px; display: block; text-align: center; cursor: pointer;" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiokLNcSKmj5dlKD5twfxQrhfoUZ1BS-bPGV9DImaxY-2Ju_8pesNfAVHR64jgDip0iAuZKeOH8hIJY6tXVISS1vq0NtwCsb-DbFAHbrdDsdSB0Ruclln4nReOrhxMcrdLNCISN1XSb-Ij5/s320/ci-venn.png" alt="" id="BLOGGER_PHOTO_ID_5211363327212185682" border="0" /></a>I think a simple diagram emphasizes the point. If "A" is a development team, "B" is a CI team, and "C" represents the stakeholders of the solution being developed, then the shaded areas pinpoint a missed opportunity unless you adopt the supply chain analogy. These are not hard control points, but juicy overlap, waiting to be optimized by as AB and BC working together.<br /><br />So far, I've talked about the pre-shipment benefit leading into CI. I also now believe the supply chain post-CI is equally important. That is, where does the feedback go and how do you make is so easy, there is no reason not to use it. In our case, this encompassed two audiences with two different needs.<br /><br />1) Developers. They need the low level test results, test coverage, Checkstyle and PMD reports.<br /><br />2) Stakeholders (Project Managers, Solution Owners, Scrum Masters). They need to know that the developer teams are using the system. What is the source control commit frequency? How long do CI builds stay broken? How long are the automated builds taking? In short, the interest is around are they getting their monies worth on CI investment and knowing if the teams "get it". It provides them with just enough information to start asking the right questions to the right people.<br /><br />So the supply chain from CI into management visibility, in our case, ended up being a enterprise level portal aggregating project CI metrics together into an at-a-glance view of how well their agile teams are performing. This could be stop light charts on current CI conditions or simple graphs of build times or number of tests as plotted over the last 30 days.<br /><br />As always seem to be the case, the benefit of this proposed supply chain approach is inceased communication.<br /><br />Again, this initially seemed to stray from traditional core CI functionality, but in reality it is simply providing the visibility into the process that scrum promises. Warts and all.Jeffhttp://www.blogger.com/profile/17053090306859709462noreply@blogger.com1tag:blogger.com,1999:blog-831382000244017172.post-23313228129747283252008-06-04T10:18:00.000-07:002008-06-04T10:28:55.939-07:00Hudson Status Google Gadget on Linux<a href="http://google-opensource.blogspot.com/2008/06/google-gadgets-for-linux.html">Google </a> has just announced gadget support for the linux version of Google Desktop. This is exciting news for me as that means the Hudson Status gadget is now cross-platform. For all you developers out there that rely on Hudson, this means you don't have to visit your Hudson site or check your email or your feed reader to get the latest job statuses.<br /><br /><blockquote>The dots are right there on your desktop!</blockquote><br />For you existing users out there, make sure you upgrade to the newest gadget version, which has some important bug fixes. More information about the latest release, v0.9.5, can be found on the <a href="http://hudson.gotdns.com/wiki/display/HUDSON/Hudson+Google+Desktop+Gadget">Hudson wiki</a>.Jeffhttp://www.blogger.com/profile/17053090306859709462noreply@blogger.com3tag:blogger.com,1999:blog-831382000244017172.post-729268636740694532008-04-21T12:30:00.000-07:002008-04-21T12:53:13.409-07:00Maven SNAPSHOT TraceabilitySo your scrum team is creating non-unique SNAPSHOT versioned artifacts throughout your sprint. How do trace that SNAPSHOT version back to a baseline that is of any QA relevance (build number, subversion revision, datetime stamp)?<br /><br /><code></code><pre><br /><plugin><br /> <groupId>org.apache.maven.plugins</groupId><br /> <artifactId>maven-antrun-plugin</artifactId><br /> <executions><br /> <execution><br /> <phase>generate-resources</phase><br /> <configuration><br /> <tasks><br /> <tstamp><br /> <format property="now" pattern="MM/dd/yyyy hh:mm" unit="hour" /><br /> </tstamp><br /> <property name="build.version" value="${version} (private-${now}-${user.name})" /><br /> <property name="hudson.build" value="hudson-${BUILD_NUMBER}, subversion-${SVN_REVISION}" /><br /> <br /> <!-- put the version file --><br /> <echo message="The build id is: ${build.version}" /><br /> <mkdir dir="target/${project.build.finalName}/" /><br /> <echo file="target/${project.build.finalName}/version.properties">version=${build.version} ${hudson.build}<br /> </echo><br /> </tasks><br /> </configuration><br /> <goals><br /> <goal>run</goal><br /> </goals><br /> </execution><br /> </executions><br /></plugin><br /><br />...<br /><br /><profiles><br /> <profile><br /> <id>release</id><br /> <properties><br /> <!-- for releases, just use the POM version. --><br /> <build.version>${version}</build.version><br /> <hudson.build></hudson.build><br /> </properties><br /> </profile><br /></profiles><br /></pre><br /><br />This little piece of magic creates a <code>version.properties</code> file that contains some valuable information.<br /><br /><code>version=1.1.0-SNAPSHOT (private-04/21/2008 02:23-jblack), hudson-453, subversion-1124 </code><br /><br />When you do a formal release, specify the <code>-Prelease</code> profile to have this file simply hold the pom version.<br /><br />For the plugin configuration above, this file gets put in a war project root webapp directory, suitable for immediate viewing from your browser!<br /><br />The mad props for this idea go to <span class="ltcaption1"><b><a href="http://weblogs.java.net/blog/kohsuke/">Kohsuke</a>. </b>:)<b><br /><br /></b> </span></code></code>Jeffhttp://www.blogger.com/profile/17053090306859709462noreply@blogger.com1tag:blogger.com,1999:blog-831382000244017172.post-4403031669366889912008-04-09T22:01:00.001-07:002008-04-17T07:15:59.542-07:00Hudson Gadget For Google DesktopI took a first stab at creating a google desktop gadget that can display the job names and statuses of any Hudson instance. Simply configure it to the Hudson server url and it will discover all the jobs and start polling the for blue dots.<br /><br /><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg8RbfcsfkdfVc_igNsuSkYWyo9OhsGoNGttPsKZhR6_hyphenhyphen3BQutC_TcNXZnYM9LiJI0aF8UCSAzXa04TkSeCQlkS9MT6HsUz_kBDtY1pBoc4GeEFru4pxh9EENjNjasPV1CQj2j0o8vgnRn/s1600-h/screenshot_large.png"><img style="margin: 0pt 10px 10px 0pt; float: left; cursor: pointer;" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg8RbfcsfkdfVc_igNsuSkYWyo9OhsGoNGttPsKZhR6_hyphenhyphen3BQutC_TcNXZnYM9LiJI0aF8UCSAzXa04TkSeCQlkS9MT6HsUz_kBDtY1pBoc4GeEFru4pxh9EENjNjasPV1CQj2j0o8vgnRn/s400/screenshot_large.png" alt="" id="BLOGGER_PHOTO_ID_5187477547569337202" border="0" /></a><br /><br /><a href="http://codebeneath.googlepages.com/hudsongadget" target="_top" rel="nofollow">Hudson Gadget, Version 0.9.0.</a><br /><br />Let me know if you have any improvement ideas or other issues.<br /><br /><span style="font-style: italic;">[update] This gadget has been contributed to the open source </span><a style="font-style: italic;" href="http://hudson.gotdns.com/wiki/display/HUDSON/Hudson+Google+Desktop+Gadget">Hudson </a><span style="font-style: italic;">project.</span><b><br /><br /></b>Jeffhttp://www.blogger.com/profile/17053090306859709462noreply@blogger.com2tag:blogger.com,1999:blog-831382000244017172.post-59546137555168315322008-03-31T20:44:00.000-07:002008-03-31T21:05:52.699-07:00Code Reuse Inside Your Own CompanyA short read over on Dr. Dobbs, <a href="http://www.ddj.com/architect/207000711">Where Are the Clients In a SOA?</a> made some good observations about the mindset of creating reusable SOA components, especially when the reuse is likely to occur in your own back yard.<br /><br /><blockquote>"The best "enterprise architecture" plan will never be realized as long as developers assume the client of a piece of code is another piece of code."</blockquote><blockquote>"As software developers we are not very good customers of the work our colleagues do. We are quicker to search the Internet for a utility or example than we are to search our own internal repositories." </blockquote><blockquote>"More importantly, we aren't thinking about other developers when we create the code. We don't create interfaces that make it easy for people to use our code in ways we have not considered. <span style="font-weight: bold; font-style: italic;">We don't provide design-level abstractions that help other programmers and architects fit our solution into theirs.</span> We don't market the code to the rest of the team or to other teams within the organization. We don't do any market research to make sure the nontraditional customers of our code are going to make the most of it. We don't package it in ways that also make it easy to use outside our intended use."<br /></blockquote>The emphasis is mine, although it should be yours, too.Jeffhttp://www.blogger.com/profile/17053090306859709462noreply@blogger.com3tag:blogger.com,1999:blog-831382000244017172.post-59519535438508449522008-03-07T08:23:00.000-08:002008-03-07T12:21:22.648-08:00Jersey... Jetty and Maven style!I recently created a webapp that due to time constraints, I implemented with plain old servlets which I am comfortable with. In reality, the project was a perfect candidate to implement in RESTful style. I had been loosely following the <a href="https://jersey.dev.java.net/">Jersey</a> project, part of the larger java.net <a href="https://metro.dev.java.net/">Metro</a> project, so I decided to give Jersey a try.<br /><br /><blockquote><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhX404gyjX7z9qxqmfzRvwrfJlrCGV1VFffmjrKiE4hbOTU6mcdzBbBOHmASJutXG6YBhT8CWXAGS5P1BNVysXUU6-kOZkSVUqW8wo_fokVDpXsDTde4Y4ESHPmXbULhkv3Wpp7_4Gm4vDk/s1600-h/Jersey_yellow.png"><img style="float: left; cursor: pointer;" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhX404gyjX7z9qxqmfzRvwrfJlrCGV1VFffmjrKiE4hbOTU6mcdzBbBOHmASJutXG6YBhT8CWXAGS5P1BNVysXUU6-kOZkSVUqW8wo_fokVDpXsDTde4Y4ESHPmXbULhkv3Wpp7_4Gm4vDk/s400/Jersey_yellow.png" alt="" id="BLOGGER_PHOTO_ID_5175049971565274002" border="0" /></a>"Jersey is the open source ... JAX-RS (JSR 311) Reference Implementation for building RESTful Web services. But, it is also more than the Reference Implementation. Jersey provides additional APIs and extension points (SPIs) so that developers may extend Jersey to suite their needs."</blockquote><br /><br />My only requirements were really around a quick turn-around dev setup. So I wanted to use maven and the jetty plugin to preview the results quickly.<br /><br /><blockquote><span style="font-weight:bold;">Jersey is still only available as an early access preview (0.5-ea) , so some of the examples I found were "out of date", as the team is still refactoring the implementation.</span></blockquote><br />So, first step: create a maven war project structure and configure the web.xml to look this this:<br /><code></code><pre><br /><?xml version="1.0" encoding="UTF-8"?><br /><!DOCTYPE web-app PUBLIC "-//Sun Microsystems, Inc.//DTD Web Application 2.3//EN" "http://java.sun.com/dtd/web-app_2_3.dtd"><br /><web-app><br /><servlet><br /> <servlet-name>Jersey</servlet-name><br /> <servlet-class>com.sun.ws.rest.spi.container.servlet.ServletContainer</servlet-class><br /> <init-param><br /> <param-name>com.sun.ws.rest.config.property.resourceConfigClass</param-name><br /> <param-value>com.sun.ws.rest.api.core.PackagesResourceConfig</param-value><br /> </init-param><br /> <init-param><br /> <param-name>com.sun.ws.rest.config.property.packages</param-name><br /> <param-value>org.naiade.tutorials.jersey;com.gestalt.gci.rest.resources</param-value><br /> </init-param><br /> <load-on-startup>1</load-on-startup><br /></servlet><br /><servlet-mapping><br /> <servlet-name>Jersey</servlet-name><br /> <url-pattern>/*</url-pattern><br /></servlet-mapping><br /></web-app><br /></pre>The two key things here are:<br /><ul><li>use the ServletContainer servlet (not ServletAdapter)</li><li>use the pluggable PackagesResourceConfig parameter value. This allows Jersey to auto find all REST resources based on the package names. This is configured in the additional parameter value. Here we provide a couple of packages to look in, semicolon separated.<br /></li></ul>For the code, I used the ShoeService example I found <a href="http://objectif-naiade.blogspot.com/2007/11/java-rest-framework-jersey.html">here.</a><br />As I mentioned before, Jersey is still evolving so I ended up making two code changes to still be able to compile under 0.5-ea:<br /><ul><li><code>@HttpMethod()</code> changed to <code>@GET</code> or <code>@POST</code></li><li><code>@UriTemplate()</code> changed to <code>@Path()</code><br /></li></ul>The maven project looks like:<br /><code></code><pre><br /><project xmlns="http://maven.apache.org/POM/4.0.0"<br /> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"<br /> xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd"><br /><modelVersion>4.0.0</modelVersion><br /><groupId>com.shoe</groupId><br /><artifactId>shoeservice</artifactId><br /><version>1.0</version><br /><name>Shoe Service REST</name><br /><packaging>war</packaging><br /><dependencies><br /> <!-- jersery.dev.java.net --><br /> <dependency><br /> <groupId>jersey</groupId><br /> <artifactId>jersey</artifactId><br /> <version>0.5-ea</version><br /> </dependency> <br /></dependencies><br /><build><br /> <plugins><br /> <plugin><br /> <groupId>org.apache.maven.plugins</groupId><br /> <artifactId>maven-compiler-plugin</artifactId><br /> <version>2.0.2</version><br /> <configuration><br /> <source>1.5</source><br /> <target>1.5</target><br /> </configuration><br /> </plugin><br /> <plugin><br /> <groupId>org.mortbay.jetty</groupId><br /> <artifactId>maven-jetty-plugin</artifactId><br /> <version>6.1.8</version><br /> <configuration><br /> <scanIntervalSeconds>10</scanIntervalSeconds><br /> </configuration><br /> <dependencies><br /> <dependency><br /> <groupId>org.apache.geronimo.specs</groupId><br /> <artifactId>geronimo-j2ee_1.4_spec</artifactId><br /> <version>1.0</version><br /> <scope>provided</scope><br /> </dependency><br /> </dependencies><br /> </plugin><br /> </plugins><br /></build><br /></project><br /></pre><br />To bring it all together, it's as simple as<br /><blockquote><code>mvn jetty:run</code></blockquote>Open you browser and hit <code>http://localhost:8080/shoeservice/shoe</code><br /><br /><span style="font-weight:bold;">Easy</span>.<br /><br />I really see Jersey taking off as it is a good project and easy to work with. If you are looking for a good Java solution to creating a RESTful accessible services, give it a try.Jeffhttp://www.blogger.com/profile/17053090306859709462noreply@blogger.com4tag:blogger.com,1999:blog-831382000244017172.post-27359671479370116652008-02-12T12:24:00.000-08:002008-02-12T10:52:07.642-08:00NetBeans 6.0 Maven IntegrationAs a proponent of continuous integration, I naturally don't endorse one IDE over another. The choice should be free and open for developers to choice what works best for them. The important thing is that the project should not be reliant on an IDE to build. That is, there must be an independent build tool that can function outside of any IDE, whether it be ant or maven or whatever.<br /><br />I have used several IDEs over the last couple of year to include Eclipse, IDEA and NetBeans . They all have their strengths and weakness and for the projects I have been working on, I ended up switching around quite a bit. With the recent announcement of the new <a href="http://www.netbeans.org/">Netbeans 6.0</a>, I wanted to revisit it's capabilities and see what was new, particularly with its maven integration.<br /><br />While the Netbeans installer was downloading, I started to search for what plugins I would need to do maven->Netbeans or Netbeans ->maven project conversions. Most of what I found looked outdated and didn't look easy to use. With a little stumbling around I found out that the <a href="http://mevenide.codehaus.org/mevenide-netbeans-project/index.html">Mevenide </a>project <span style="font-weight: bold;">does </span>now support Netbeans 6.0 (this is not apparent on their Netbeans project page). So I installed the Mevenide Maven2 plugin.<br /><br /><blockquote>And was blown away!</blockquote><br />There is no conversion of project types from maven->NetBeans . It just works natively. File > Open Project and browse for your existing maven project. Bingo.<br /><br />I actually started by opening a parent project, <packaging>pom</packaging>, thinking that the Mavenide wouldn't know what to do with a multi module project, but it just worked, too! It showed me the project files, libraries, test libraries and the modules that this parent included. Too cool.<br /><br /><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiZBU9J6GhvMXluRCczqyEh7FJgTRuWYB7lF9fDyUqt34GQ57z2mXrQuGPhR30kvPvUfGbABjSbjRveCw-AfbNJfOiFbB2LGQNM9mt0QAMmdQJLp8BxYS3IRh23wMFZIY03XraWtRXYZxzB/s1600-h/netbeans-maven-1.PNG"><img style="margin: 0pt 10px 10px 0pt; cursor: pointer;" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiZBU9J6GhvMXluRCczqyEh7FJgTRuWYB7lF9fDyUqt34GQ57z2mXrQuGPhR30kvPvUfGbABjSbjRveCw-AfbNJfOiFbB2LGQNM9mt0QAMmdQJLp8BxYS3IRh23wMFZIY03XraWtRXYZxzB/s400/netbeans-maven-1.PNG" alt="" id="BLOGGER_PHOTO_ID_5166160940552877938" border="0" /></a><br /><br />So then I double clicked my war module name hoping for the best and was not disappointed. It opened the war project in the project pane and I was off and running.<br /><br /><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgQeiBJwtP41dz_psgfh3I4KkWWIgV68IQZ57kZoC0GVaNUhpK6IA0VtMqwzv9MJvsBmOgoBZKBVRZP8xOwSJhYD0H1MFIOy0U7jtRpbZDtnp1KOBknOB8DeqKyuyh32TSvT02tdECm-cKS/s1600-h/netbeans-maven-2.PNG"><img style="margin: 0pt 10px 10px 0pt; cursor: pointer;" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgQeiBJwtP41dz_psgfh3I4KkWWIgV68IQZ57kZoC0GVaNUhpK6IA0VtMqwzv9MJvsBmOgoBZKBVRZP8xOwSJhYD0H1MFIOy0U7jtRpbZDtnp1KOBknOB8DeqKyuyh32TSvT02tdECm-cKS/s400/netbeans-maven-2.PNG" alt="" id="BLOGGER_PHOTO_ID_5166161082286798722" border="0" /><br /></a><br />Right click on the project name to run the more common maven goals from right inside NetBeans. To run custom goals, you can create a Custom > Goals... goal that you can use to chain together whatever goals you want for a maven execution. This is the only thing I did that actually created a custom file on my filesystem. Everything else was native.<br /><br /><blockquote>How's that for a slice of fried gold? </blockquote>So, here's a big high five to to the Mevenide team on a job well done! (And update your NetBeans project page to highlight that it works in Netbeans 6.0. This is important news to advertise!).Jeffhttp://www.blogger.com/profile/17053090306859709462noreply@blogger.com9tag:blogger.com,1999:blog-831382000244017172.post-31740840579648956232008-01-15T13:21:00.000-08:002008-02-07T22:11:12.388-08:00Global Collaboraton In the Second Grade<a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEggtw7V_cdbkHJ5G_6PF4MncLCJlJ0-bMrYcqZ18agHmyedopPGyVtLNaNug6UhqsFU9VywqOVvdby6oSsoAOfdooduKJ40KQNPU_m9JrgoyJDqPKnWdDXsFnwRS1fFPlqa8bIKQFEIkLGa/s1600-h/world.gif"><img style="margin: 0pt 0pt 10px 10px; float: right; cursor: pointer; width: 244px; height: 156px;" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEggtw7V_cdbkHJ5G_6PF4MncLCJlJ0-bMrYcqZ18agHmyedopPGyVtLNaNug6UhqsFU9VywqOVvdby6oSsoAOfdooduKJ40KQNPU_m9JrgoyJDqPKnWdDXsFnwRS1fFPlqa8bIKQFEIkLGa/s320/world.gif" alt="" id="BLOGGER_PHOTO_ID_5164480189886365218" border="0" /></a><br /><br />I just finished reading <a href="http://en.wikipedia.org/wiki/The_World_is_Flat">The World is Flat</a> by Thomas L. Friedman. He had some good insights and specific example about how the world is increasingly connected and how it functions as a global marketplace. The most interesting take away I had was that simply providing the cabling into countries that had not been previously connected was enough of a spark to allow individuals with curiosity and an innovative spirit to engage a global community.<br /><br />Those of us in the software business, take this interconnectedness for granted now. I find it fascinating, being a father of two, that my children will know no different; the world was always connected. In the span of a single generation the world got very small indeed. I have friends who grew up their entire life in single small town. Any road trip of more than 100 miles was a big deal. But my kids won't have to "go" anywhere to instantly know what is going on on the other side of the planet or to actively collaborate with people on other continents.<br /><br /><br /><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg-zLbgsBjnfm9iCBJFod4iH9ABz73onH7zWJE2SXBPyCsPcTgceZvavHBPwZLuDqrG0_pPdvwzZPlQm5v7cp2XxtLLYAqFyxEuO_HVZut4JhfeHUW4wa6gxFrEyFdGjl9OSZVlm50OTSLb/s1600-h/line_of_kids.jpg"><img style="margin: 0pt 0pt 10px 10px; float: right; cursor: pointer; width: 238px; height: 178px;" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg-zLbgsBjnfm9iCBJFod4iH9ABz73onH7zWJE2SXBPyCsPcTgceZvavHBPwZLuDqrG0_pPdvwzZPlQm5v7cp2XxtLLYAqFyxEuO_HVZut4JhfeHUW4wa6gxFrEyFdGjl9OSZVlm50OTSLb/s320/line_of_kids.jpg" alt="" id="BLOGGER_PHOTO_ID_5164481920758185522" border="0" /></a>So with this in mind, I had an idea for an education software project. My second-grader already has computer lab class once a week where he and his classmates learn and solve problems on a standalone system. What better time to learn about teamwork, on a global scale, then now, while he is in school.<br /><br />My idea is a collaborative system that lets kids work together to solve and reinforce basic school (and life) skills. As math is a universal language, it would be a good first subject.<br /><br />Picture this:<br /><br /><ul><li>The system is browser-based and has a easy to use, friendly RIA interface.</li><li>One student from the USA joins the "Collabmath" course; One student from India joins the same course and they are paired together to solve 5 math problems.</li><li>Each student will have a friendly avatar with their county's flag next to it. Each student's name can be clicked on to hear it pronounced in the native language. Google maps can show each student's school marked on a world map.</li><li>The math problem will be presented in a large left hand workspace, and a chat window could be in a right-hand pane.</li><li>The chat window can be used by the students to provide encouragement or ask for help. The chat would be I18N capable and feature emoticons that would make the session more personable.<br /></li></ul><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj4IOV6SmpBdDr8dIRxNXevETWGnEYplN36Vllqlz5SmcF7yhKhTkMwRH9MSUsCL9izzdl1XYwOXNHYB4qz07apRXkMyVl4IocFrKFsRgrmVHbjB4Nm24yGyjmuT-tupEVhtMPWkWShW7zz/s1600-h/emots.jpg"><img style="margin: 0pt 0pt 10px 10px; cursor: pointer; width: 298px; height: 70px;" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj4IOV6SmpBdDr8dIRxNXevETWGnEYplN36Vllqlz5SmcF7yhKhTkMwRH9MSUsCL9izzdl1XYwOXNHYB4qz07apRXkMyVl4IocFrKFsRgrmVHbjB4Nm24yGyjmuT-tupEVhtMPWkWShW7zz/s400/emots.jpg" alt="" id="BLOGGER_PHOTO_ID_5164473184794705426" border="0" /></a><br /><ul><li>Each student takes turns solving a problem a step at a time. When each step is complete, control transfers to the other student, and so on. So, for a simple example, if the problem is to draw an octagon, the first student will draw the first line segment, then the second student will draw a complementary line segment connected to the first. When the octagon is complete, the students will have worked together to solve the problem.</li><li>At the conclusion of each problem, a practical real world example of the use of that problem in each culture could be presented. In my example, the students would learn that in America, the octagon is the shape of a traffic stop sign; a related symbolic example could be presented that would be relevant in the country of India.</li></ul><br />A wide range subjects and grade levels could be taught, of course, but for starters, this could get the ball rolling. I envision my kids being excited one day, coming home from school, "Dad! Guess what? I learned division and fractions with a kid from <span style="font-weight: bold;">China </span>today! Her name was Ming Wu! It was <span style="font-weight: bold;">so cool</span>!"<br /><br /><blockquote>I can see it. Can you?<br /><br /><br /></blockquote>Jeffhttp://www.blogger.com/profile/17053090306859709462noreply@blogger.com3tag:blogger.com,1999:blog-831382000244017172.post-83121194171266893202007-12-06T07:08:00.000-08:002007-12-06T07:15:33.605-08:00Continuous Integration Strategies (Part IV)Bonus edition.<br /><br /><blockquote>Check out <a href="http://www.ibm.com/developerworks/java/library/j-ap11297/index.html">Continuous Integration Anti-Patterns.<br /></a></blockquote>Jeffhttp://www.blogger.com/profile/17053090306859709462noreply@blogger.com0tag:blogger.com,1999:blog-831382000244017172.post-48760428308321299562007-12-04T19:24:00.000-08:002007-12-04T21:59:11.449-08:00Continuous Integration Strategies (Part III)How do you get your code to talk to you? Continuous integration is all about automated feedback. Beyond test reports and self-describing-code there are many techniques and tools that you can use to find out if you are producing "quality" code.<br /><br />When you pick the right reports and integration them into the software build, the level of effort required to use these tools becomes very low (and we all know that developers are legendarily lazy).<br /><br />Ideally, you should not have to remember to ask, the code should tell.<br /><br />It's like being the parent of a teenager. At the dinner table you ask,<br /><blockquote><br />"So, how was school today?"<br />"I dunno."<br /><br />"What did you learn?"<br />"Nothin'"<br /><br />"Did anything interesting happen?"<br />"I dunno."<br /><br />"How did you do on your history test"<br />(shrug)<br /></blockquote><br />You would rather he came home with the enthusiasm of a kindergartener eager to tell you about his day as soon as he gets home. No effort on your part to ask, he just gushes unprompted,<br /><blockquote><br />"Dad!! Guess what happened today at school?? It was so cool, I got an A+ on my spelling test!!"<br /></blockquote>That's what continuous integration can do for you.<br /><br />So, recalling the computer adage, Garbage In, Garbage Out, we have carefully picked a select few reports that we feel give us a good measure of quality and integrated them into our build so that the feedback from CI is meaningful. Although several CI engines (we currently use Hudson) do have plugins to generate quality reports such as unit test results and test coverage, we feel it is important that developers have full access to run all the same reports that CI will run. Since we run maven, it is easy to plugin the reports of interest into the <code><reporting></code> section of the pom. That keeps the report configurations in source control along with the code.<br /><br />So which reports do we run? Here's the rundown:<br /><br /><ul><li><b>Checkstyle</b>. Validates source code against coding standards and reports any violations. We customized the ruleset, packaged it in a versioned jar and deployed it to our maven repository. Additionally, we forced it to run as part of every build in the <code>validate</code> phase, and configured it to fail the build if violations are found. It's possible for an individual project to override the custom ruleset, but we don't encourage that.</li><li><b>PMD</b>. Performs design time analysis of the source code against a standard ruleset. We customized the ruleset, packaged it in a versioned jar and deployed it to our maven repository. Additionally, we forced it to run as part of every build in the <code>validate</code> phase, and configured it to fail the build if violations are found.</li><li><b>Test Results </b>(surefire). This is a no-brainer. You want to see the results of your tests.</li><li><b>Code Coverage </b>(cobertura). Shows branch and line coverage for your source files. I think the key to using this metric is to not to set a percentage that the project team must meet, but to be smart about interpreting the trends. "Metrics are meant to help you think, not to do the thinking for you."</li><li><b>Javadoc</b>. Creates the API documentation for your project.</li><li><b>Dashboard</b>. Aggregates maven multiproject build reports into a single report page. This is critical so that developers and stakeholders don't have to hunt and drill down page after page to find the meaningful metrics you worked so hard to set up.</li></ul><br /><br />In addition, make full use of Maven's pom to declare all the sections that feed the default generated site, including:<br /><br /><ul><li><scm> Source control management. New team members, for example will need to know the subversion URL for checkout.</li><li><developers> Identifies subject matter experts and feeds developer activity reports.</li><li><ciManagement> Identifies the CI engine being used and the URL to see the live status and force new builds.</li><li><issueManagement> Identifies the issue tracking system. I think there are maven plugins that will map issues to source control commits, providing bi-traceability of code to requirements.</li></ul><br /><br />Also, don't forget for maven multiproject builds to review the Dependency Convergence report. It will show all dependencies for all projects along with the versions of those dependencies. This will help you find dependencies your using inadvertently using multiple versions of.<br /><br />Once you have these reports baked in, make the results easy to find. Use your CI engine to generate the maven site on a nightly basis and publish the results to a web server where developers or project stakeholders can find them.<br /><br />After you spend the time and effort to identify which reports you want and get them configured and working correctly, make it repeatable for new projects by creating an archetype template project. This sets up a model pom.xml and project directory structure right off the bat for new projects. When it's there from the start, with no effort on the team's part, good things happen.<br /><br />With a little up front effort, your code (with a little help from continuous integration) can talk to you. What are your strategies? How do you use CI to reveal code quality? I would be interested in hearing your strategies.Jeffhttp://www.blogger.com/profile/17053090306859709462noreply@blogger.com0