The key is really a passphrase that is then generated into a key and IV for use with the underlying CBC/cipher. These modules are by default compatible with OpenSSL. I thought that since this is password-based encryption, that I could use Jasypt, one of my favorite libraries. Unfortunately, Jasypt only supports the PBE* cipher algorithms and none of them are the OpenSSL standards. So then I thought that I could at least get Jasypt to support Blowfish. No luck…the algorithm is just hard-coded to PBE-based encryption. Even the IV work would be impossible.

So for the project, I created my own mini-framework that includes converters (hex/base64 string to byte[] or String to byte[] based on a character-set) and ciphers defined via generics with the ability to create a string-to-string “encryptor” by combining a String-to-byte[] (utf-8) converter with a byte[]-to-byte[] cipher with a byte[]-to-String (hex) converter. This is sort of what Jasypt does but its not very pluggable in that fashion.

Then to write the byte[]-to-byte[] cipher, I started with a generalized algorithm that works for both the PBE* algorithms but also for AES and Blowfish with the key and IV generation handled in the process. Plug in BouncyCastle’s OpenSSLPBEParametersGenerator for key/IV generation and write my own decorator for dealing with sharing the salt as “Salted__XXXXXXXX” in front of the ciphertext and voila! perl-java encryption interoperability based on passwords and random salts!!

That project has ended and I’m now in-between gigs so I worked that code into Jasypt – just added a feature request (with a patch) to Jasypt. Not specifically the perl stuff but the generalized algorithm. That allows users to finally extend Jasypt – still for password-based encryption but not limited to the PBE* algorithms. Support is finally in there for AES and Blowfish with key and IV generation based on PBKDF2 or whatever else you want to add. Changes to Jasypt to support the configuration of the whole “pipeline” is not in there – that would require some serious changes to Jasypt.

As for the algorithms – they look like this:

For encryption,

EncryptionData data = buildEncryptionData();
data.setMethodInput(message);
dataProcessor.preProcess(Cipher.ENCRYPT_MODE, data);
SecretKey key = keyGenerator.generateSecretKey(data);
AlgorithmParameterSpec parameterSpec = paramGenerator.generateParameterSpec(data);
synchronized (encryptCipher) {
encryptCipher.init(Cipher.ENCRYPT_MODE, key, parameterSpec);
data.setCipherOutput(encryptCipher.doFinal(data.getCipherInput()));
}
dataProcessor.postProcess(Cipher.ENCRYPT_MODE, data);
return data.getMethodOutput();


For decryption,

EncryptionData data = buildEncryptionData();
data.setMethodInput(encryptedMessage);
dataProcessor.preProcess(Cipher.DECRYPT_MODE, data);
SecretKey key = keyGenerator.generateSecretKey(data);
AlgorithmParameterSpec parameterSpec = paramGenerator.generateParameterSpec(data);
synchronized (decryptCipher) {
this.decryptCipher.init(Cipher.DECRYPT_MODE, key, parameterSpec);
data.setCipherOutput(decryptCipher.doFinal(data.getCipherInput()));
}
dataProcessor.postProcess(Cipher.DECRYPT_MODE, data);
return data.getMethodOutput();


And all the real work is done in the SecretKeyGenerator (which actually generates more than just the key – it just returns the key), the AlgorithmParamsGenerator and the EncryptionDataProcessor – all of which are just interfaces. All the transient data for the method is kept in the EncryptionData class or subclass. So that is the patch just submitted.

And perl interoperability could be added with an OpenSSLSecretKeyGenerator and an OpenSSLEncryptionDataProcessor to handle the “Salted__XXXXXXX” format – the rest is all in there (once the patch is approved, committed and released). The OpenSSLSecretKeyGenerator would work like the PBKDF2SecretKeyGenerator in that it would produce the key and IV based on a password and fixed or random salt. Its just that OpenSSL does a funky key and IV generation mechanism that I’m not sure is in the default JCE providers. And the OpenSSLEncryptionDataProcessor is just an extension of the existing one with the hardcoded “Salted__” thrown in for good measure.

Here’s hoping that gets added by the next project. Jasypt team – I’m more than willing to help!

In the end, I really think that this is the year that Agile testing via executable requirements will take off and I do think that JBehave can be a part of that. There are some key things to work out however (library bundling issues, integration with JUnit for Spring) that need to be looked at before its really ready for prime time. And getting the Pico Ajax Email / Selenium example working was painful. Please don’t release versioned software that depends on snapshot releases of other code!

So, it would be useable on projects if you are willing to spend some time on getting your base class / infrastructure stuff setup. Personally, I like it more than FIT/Fitnesse.

If all the team members were located in their own location – how would I set this up? The kicker here is the mess of timezones that might be in the mix. Obviously, I’d have a wiki and perhaps an agile PM (kanban/scrum) web app running somewhere that we could all access in our timezone. When folks are distributed and have an overlapping time, then can use VNC (or other free solutions like that) to “pair-up” as needed. Likewise, VoIP/IM conferences or just VoIP/IM for issues and/or questions.

But how to do “standups” when there isn’t a time that everyone can standup? How to let someone know you are stuck on something and how to hand off a potential solution to someone that will get it hours later. My insight was that a team could host an internal blog/Twitter to share what they did yesterday, what they are doing today and what issues are blocking them. Status updates (“working on X” or “can’t figure out Y”) can then really be done at anytime and those folks that are online can help step in. Some IM systems have a status but I’m not sure that that is very visible. Add an RSS feed on top of the teams blogs (like twitter) and you’ll start to see team collaboration. Start your day by reading all the updates from folks since you were last on. The whole project life could be read if you really wanted too – like emailing the IM discussion. Still doesn’t help with the I have an issue with X and what a potential solution might be (hours later). I could see the wiki or issue tracking system kind of work in that space. In this scenario, the world of “standup” starts to look like “status” – but that might be ok just given the realities of multiple people in multiple timezones. I thought that it would be an interesting way to run a project without standups.

Iteration and release planning would be difficult in this situation – that might just have to be done together.

I’ve not had this situation – have you? What worked or didn’t work?

Now, Ant 1.8 has been released and with it some new features that could potentially make it possible to have very modular Ant builds that would be even better than Maven2. One of the main concepts within Maven2 is the various lifecycles (clean, build/default and site) and that build tasks from plugins are bound to various parts of the lifecycle. Ant 1.8 introduced the notion of extension-points and extensionOf as well as imports and local properties – these could all be used to both create plugins (macrodefs) and our own lifecycles (sets of extension-points) and then bind them all up together in a build.xml and just import what you need – potentially from an http URL.

Well, that was the thought…

Turns out that imports are processed after the build.xml is parsed. That’s all well and good, but when an extensionOf attribute is parsed, Ant looks for the target(s) named in the extensionOf value in order to add the current target as a dependency. That requires that the target has to exist in the project and if that target is part of an import (as the documentation seems to suggest), then the target doesn’t exist (yet) at the time of parsing and you get a nice error message to that effect.

I think that this is a design flaw in how extension-point / extensionOf is supposed to work and contradicts the example cited in the documentation – which doesn’t work.

Its too bad because with these features, I could define my own lifecycles or even change/modify the existing ones from Maven2 to do things related to database SQL modules (create the database from all the SQL scripts and some data files) or be able to mix the SQL and java files together in the same module and add phases to the lifecycle related to database setup. This has always been something that I have to hack up the pom for anyway – which is part of why I like going back to Ant – I can change it easier when I need to.

Work-arounds? Change the ProjectHelper/TargetHelper to deal with extensionOf attributes after the import stack is popped (and all the targets are resolved) or import the extensionOfs (the bindings or which macros get called for each step) after the extension-points are imported. I’m not a fan of the latter as I really think that the bindings are the build – execute these steps for these lifecycle stages – but if my build is just a bunch of imports, that’s not the worst of it. Or screw the use of extension-points/extensionOf and just use imports with empty targets (which is kind of what extension-points are – except that I could then create a target that gets bound to multiple extension-points with extensionOf=”target1,target2″).

It does sadden me that the example cited doesn’t even work however. If I get this working, I’ll post the example.

Backlog grooming would actively work the story prep board.  The team could see what stories are getting ready for planning as well as a release team (or management team) seeing what is getting ready for release to production.  I think that it would actually engage those team members that are at the daily standup but are not developers or testers – they can point to what they are working on – its just on the story prep kanban board.  I think that it could make for a good information radiator for a wider “team”. 

The other way to look at this, from a metrics standpoint, is to see that the whole Scrum board is just one column of a larger Kanban board and that you could measure and reduce the throughput time of a story from backlog to released to production on that larger Kanban board.

Has anyone ever done anything like this?  Did it work? Things to improve about it?

Putting this post together, I just noticed Henrik’s Kanban and Scrum – Making the Most of Both – something new to read!

My original use of pink cards was for a Scrum board for developers to fix a critical or blocker bug.  This was for a team that was just developers – the testers were a completely different team.  And the pink card was basically a request for the development team to help un-block the testing team (and dev team would estimate it and decide if they would need to take something else off the board).

I’ve also used pink post-its on Scrum boards to report a blocking issue on a task – just as a way to remind people working on the issue that it is important and to bring it up in standup until the issue is resolved.

On another team that I worked on we sort-of had a Kanban board for release or operations tasks related to the program (multiple projects – one operations team) and if the implementation team had a request to make of them (e.g. database to setup), then we would create a pink card for their board.  Basically a pink card is a Please Do This ASAP request.  Maybe pink should stand for Please Implement Now, Kind (Sir/Madam).

What I realized is that one way to connect these teams, with their own boards and tasks and stories is that the issue (post-it) is tied to request(s) to resolve the issue (cards) and you could track and connect those issues/tasks that way. So basically, for that first scenario (separate dev and test teams), if the test team had had a board, their pink post-it (the blocking issue) was tied to the pink card for the development team (the issue resolvers).  And a board with a lot of pink is a conversation waiting to happen.

Simple and easy way to handle and track issues that need to get done now that I think helps build an agile enterprise.

This misses alot of the work that needs to be done in order to make teams effective. I like to have regular backlog grooming meetings with part of the team and ask the question – is this story prepared?.  What is needed in order to bring this story to iteration planning? That needed work might involve QA (quality assurance) for acceptance tests or functional tests.  That might involve UX (user experience) for wireframes or drawings.  That might involve some TA (technical architecture) work or IA (information architecture, or domain modeling) work depending on the story.  It might require a BA to work out the business value of this story or how to break it up into what needs to done now versus later (breaking up stories into smaller, potentially optional pieces).  Its only when a story is prepared that it should be brought to iteration planning.

I also use this picture to help explain why some work is “on the board” for the iteration (meaning we are tracking velocity and burndown charts – its the developer/tester circle) vs work that needs to get done but we aren’t measuring velocity for it.  The first is working towards completing the story.  The latter is working towards getting the story prepared.

It also helps explain the roles of the non-customer, non-developer and non-tester folks…though I’m pretty careful to explain that that 2nd circle is optional work (story by story) and that that work can be done by anyone with those skills.  Its really about what would make the communication of this story effective and doing that in as lightweight of a fashion as you need.

The last part of this drawing is that the story doesn’t stop because development is done.  Its really done when its deployed (some would say deployed to production) and supported.  This means that the story needs to be shared with operations and support teams.  I’ve seen this done as part of a release process and actually made it the responsibility of the whole team to figure how to to communicate the stories that are being released.  Really each circle needs to figure out when its done with the story and how to communicate to the next circle (and then there are feedback loops!).

Its really about effective communication and community.  I didn’t get this last part until attending a session with David Hussman who talks about building community around a story.

There is a skill to chosing which architectural requirements need to be addressed when.  On one project that I was on, I put off deciding on how to handle exceptions through the various layers of theh architecture.  We eventually tackled the issue (with ingenious input from the team) but by then we had lots of code to change and refactor.  That lesson painfully showed the cost of “technical debt” – we borrowed that time from the future of the project and had to pay it back with interest.  Lesson learned – I would not put that concern/requirement off that late again.

I think that html style guides (what css classes are we using and for what purpose) on web-based projects are another common concern that gets put off and the price is paid later with interest.  I’ve done whole iterations of nothing but styling.

The trick is to make those architectural choices at the last responsible moment – but you never really know when that is.  Its like knowing how to play an instrument – practice, practice, practice.

Are there other that you like to see addressed earlier than when you’ve actually done them?