MySQL useCursorFetch property and Grails 2.2.4

Just hit a problem with using the useCursorFetch property in Grails 2.2.4. It may be present in earlier versions of Grails 2, but this was working up through Grails 1.3.9. I'm upgrading a Grails 1.3.9 application to Grails 2.2.4 and hit the following runtime exception when the JDBC URL has the useCursorFetch property set to true.

---

| Error 2013-08-19 09:05:50,365 [localhost-startStop-1] ERROR context.GrailsContextLoader  - Error initializing the application: java.lang.LinkageError: Illegal class file encountered. Try running with -Xverify:all in method executeBatchSerially
Message: java.lang.LinkageError: Illegal class file encountered. Try running with -Xverify:all in method executeBatchSerially
   Line | Method
->> 308 | evaluateEnvironmentSpecificBlock in grails.util.Environment
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 
|   301 | executeForEnvironment            in     ''
|   277 | executeForCurrentEnvironment . . in     ''
|   303 | innerRun                         in java.util.concurrent.FutureTask$Sync
|   138 | run . . . . . . . . . . . . . .  in java.util.concurrent.FutureTask
|   895 | runTask                          in java.util.concurrent.ThreadPoolExecutor$Worker
|   918 | run . . . . . . . . . . . . . .  in     ''
^   680 | run                              in java.lang.Thread

Caused by LinkageError: Illegal class file encountered. Try running with -Xverify:all in method executeBatchSerially
->> 4566 | prepareStatement                 in com.mysql.jdbc.ConnectionImpl
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 
|   4479 | prepareStatement                 in     ''
|   281 | prepareStatement . . . . . . . . in org.apache.commons.dbcp.DelegatingConnection
|   313 | prepareStatement                 in org.apache.commons.dbcp.PoolingDataSource$PoolGuardConnectionWrapper
|   111 | methodMissing . . . . . . . . .  in org.grails.datastore.gorm.GormStaticApi
|    56 | loadUsersAndRoles                in com.vocabra.eventlogging.services.DefaultDataLoaderService
|    22 | load . . . . . . . . . . . . . . in     ''
|   165 | doCall                           in GrailsMelodyGrailsPlugin$_closure4_closure15_closure16
|    18 | doCall . . . . . . . . . . . . . in BootStrap$_closure1
|   308 | evaluateEnvironmentSpecificBlock in grails.util.Environment
|   301 | executeForEnvironment . . . . .  in     ''
|   277 | executeForCurrentEnvironment     in     ''
|   303 | innerRun . . . . . . . . . . . . in java.util.concurrent.FutureTask$Sync
|   138 | run                              in java.util.concurrent.FutureTask
|   895 | runTask . . . . . . . . . . . .  in java.util.concurrent.ThreadPoolExecutor$Worker
|   918 | run                              in     ''
^   680 | run . . . . . . . . . . . . . .  in java.lang.Thread

---

Removing the property from the JDBC URL resolves the issue. Need to do more research into why this is happening.

Potential issue when mocking in Groovy

The groovy.mock.interceptor.MockFor and grails.test.GrailsMock allow for mock objects in Groovy and Grails, respectively. I've been using both of these classes with good success for a long time. But recently, a small refactoring involving the the removal of a parameter from a method signature has caused me to re-evaluate the mock object usage in a dynamic language like Groovy.

As I said, there was a refactoring done on the public contract of a service, where the method name stayed the same, but a parameter was removed from the method signature. The contract unit tests for this service were changed to drive the refactoring ("test-driving the refactoring"). However, the collaboration unit tests, where this service is now acting a dependency, were not changed and they continued to pass successful. I tried cleaning the old .class files and compiling the Groovy tests, but to no avail, the unit tests which mocked this service continued to pass successfully, even though the method signature no longer existed on the real service implementation. After perusing the javadoc documentation, there does not seem to be any functionality in either of these classes to verify that the type that is being mocked has a method signature that matches the method signature being mocked. Therefore, these classes can mock methods which are non-existent on the real dependency implementations. Fixing the issue involved finding the method signature using a text search.

The whole episode was a bit unsettling; we have a lot of unit tests and we may be testing scenarios which are not representative of the real world. In my case, the real world scenario manifested itself as a runtime exception stating that the method was missing. In the case of Java and Mockito, the method signature change would result in a compilation error where the changed method signature was mocked in unit tests. My takeaway was to be more diligent with my refactoring and really ensure that I have changed all places in the code where a particular method is referenced.

Allocating business logic in Grails

The dilemma

I've been on a couple of larger Grails projects in the past year and half and I'm witnessing a disturbing phenomenon. The allocation of business logic responsibility across the abstractions that Grails provides today is causing considerable pain. Grails provides controllers, services and domain objects where business logic can reside. I'll contend in this blog entry that these abstraction categories work well for small- to medium-sized Grails projects, but thing quickly start to unravel once your application gets to be large.

Controllers

I see a lot of business logic code in controllers these days. Controllers shouldn't contain any business logic whatsoever. The controller's responsibility is to handle the web request and response. Anything else should be delegated to a collaborator. Don't do it!

Domain objects

The next logical place to put business logic is in the domain class. Allocating responsibility here works to a point, but you will quickly encounter issues when you need business logic that resides in services. I'm not a fan of injecting Grails services into domain classes. This situation quickly spirals out of control and makes unit testing very difficult to perform. For simple per-domain business logic, free free to allocate to the domain class. Anything more, and it belongs in a service (or something else, which we'll discuss in a bit).

Services

So most business logic seems to end up in Grails services these days. That's what the creators of Grails intended. I have no qualms about that. The beef I have with services is that a method is a crappy abstraction for representing business logic in the system. On larger Grails projects, the service approach seems to break down, as services seem to take on more and more responsibility, making them more difficult to test. I'm also witnessing a lot of code duplication; in it's current incarnation, there is no delineation of public API services versus private services which the public API services compose larger sets of business logic with. What we end up with is large, hard-to-test service methods that collaborate with too many dependencies and do too much.

The desire

I want an abstraction in Grails that promotes proper factoring of business logic into unit-testable abstractions. These abstractions can be composed into larger abstractions to provide the necessary logic to fulfill the requirements of the system. The chain of responsibility design pattern may offer some value here. Individual commands that have a singular responsibility can be created, unit tested, and finally composed into "chains" of commands that provide the necessary functionality of the system. The command chains can be integration tested to ensure that the individual commands composition provides the functionality required by the customer/business. When new functionality is needed, a new command chain is created, reusing existing commands where appropriate and creating new commands where functionality does not exist. Spring Batch has a similar concept that is core to its design.

Conclusion

I hope to blog a bit more around this in the coming weeks. I really like Grails and would love to see its usage increase in the coming months and years. I think it has some really cool features that allow you to get up and running very quickly. The plugin system alone is a huge advantage to using Grails, because features like a Chain of Responsibility executor can easily be added to the core Grails system.

Spring Security Grails plugin 1.2.4 and transparent password encoding in the User domain object

Just spent some significant time today integrating Spring Security Grails plugin 1.2.4. It should have been a quick integration but I got stuck on some new functionality that has been added to the generated User domain object that this plugin generates when using the quick start script. You should now set the password on this domain object using the cleartext string. Look at the User domain object (you may have it named something else, but it's the domain object that represents the user). This domain class is now handling the encoding of the password transparently. I copied some code from another Grails app that was doing the encoding of the password explicitly. You should no longer do this--it will cause problems when attempting to authenticate. Basically I was doubly encoding the password, the plugin definitely does not like that. Here's what the plugin generates for a user domain object in 1.2.4:

 1 class User {
 2 
 3     transient springSecurityService
 4 
 5     String username
 6     String password
 7     boolean enabled
 8     boolean accountExpired
 9     boolean accountLocked
10     boolean passwordExpired
11 
12     static constraints = {
13         username blank: false, unique: true
14         password blank: false
15     }
16 
17     static mapping = {
18         password column: '`password`'
19     }
20 
21     Set<Role> getAuthorities() {
22         UserRole.findAllByUser(this).collect { it.role } as Set
23     }
24 
25     def beforeInsert() {
26         encodePassword()
27     }
28 
29     def beforeUpdate() {
30         if (isDirty('password')) {
31             encodePassword()
32         }
33     }
34 
35     protected void encodePassword() {
36         password = springSecurityService.encodePassword(password)
37     }
38 }

Notice that the domain object now has a springSecurityService injected into it. There's also some GORM callbacks that will be called before the state of the domain object is saved and updated in the database. This is where the encoding now occurs--you should not be doing the encoding explicitly yourself.

Lesson learned!

Grails GSPs consume Underscore.js ERB-style interpolation delimiters

If you are planning to do any work with Grails and Underscore.js (or Backbone.js) and you have view templates maintained in Groovy Server Pages (GSP) with Underscore.js ERB-style interpolation markup, watch out! GSP will eat up the Underscore.js interpolation markup (<%= %>) before Underscore ever gets a chance to use it. I assume evaluations would also be consumed by GSP. Took me a while to figure this out, so I thought I'd raise it up as a concern. Underscore.js does give you a way to change the delimiters using regexes.

Using JDBC URLs containing LDAP URI to connect to Oracle databases within Grails

I'm working on a Grails application that needs to connect to a Oracle database using a LDAP context.  The URL format is something like the following:

jdbc:oracle:thin:@ldap://tns.mycompany.com:389/marketing,cn=OracleContext,dc=com,dc=marketing

I'm also not using the Grails DataSource.groovy configuration for this.  I'm managing a separate DataSource in the resources.groovy using Spring DSL.  I'm using the org.springframework.jdbc.datasource.DriverManagerDataSource.  I have not tried this with the standard DataSource.groovy stuff.  When I first tried using this, I would get an exception with the following text: "javax.naming.NotContextException Not an instance of DirContext".  There seems to be a bug with the Spring LDAP and the SimpleNamingContextBuilder class.  Basically the SimpleNamingContextBuilder returns a Context implementation, not a DirContext implementation.  You can work around this in Grails by adding the following to the Config.groovy file:

grails.naming.entries = null

Problem solved.  The DataSource now bootstraps correctly and I can go on my merry way.  Kudos to Luke Daley for bringing this to my attention.

 

Tracing and profiling SQL in Grails using log4jdbc

I spent some time today tracing and profiling SQL in one of the Grails applications that I support.  I was looking around for proxy JDBC driver and happened on log4jdbc.  It's similar to p6spy, but it seems to be actively developed and supported.  Downloaded the driver, dropped it in my lib directory, and changed the logging and datasource configurations a bit in Grails and I was up and running.  Very handy.  I made copious use of the SQL timings profiling today.  There are many other options for tracing and profile with this tool.  Here are my changes to Config.groovy for enabling SQL timings to all SQL statements:

Config.groovy change to enable logging of SQL information from log4jdbc:

log4j = {
info 'org.codehaus.groovy.grails.web.servlet',
...
'grails.app',
'jdbc.sqltiming'
}

 

DataSource.groovy changes to enable log4jdbc:

development {       
    dataSource {
        driverClassName = "net.sf.log4jdbc.DriverSpy"
        url = "jdbc:log4jdbc:mysql://localhost/mydb-DEV?useUnicode=true&characterEncoding=UTF-8&useCursorFetch=true&autoReconnect=true"
    }   
}

 

Can't say enough good things about this tool.  Really helped me zero in on some queries that were performing poorly with large data sets.

Grails JAR dependencies with classifiers

Quick post on specifying Grails dependencies in BuildConfig.groovy.  The recommended way to suck in JAR dependencies in Grails is to use the dependencies DSL maintained in BuildConfig.groovy.  I had a need to bring down a dependency that has a classifier attribute on it.  Didn't really find anything definitive on how to do it, but it seemed like following a convention might do the trick.  Here's how I solved the issue:

repositories {

  grailsPlugins()

  grailsHome()

  grailsCentral()

  mavenCentral()

  ebr() // SpringSource Enterprise Bundle Repository

}

dependencies {

  runtime group:'net.sf.json-lib', name:'json-lib', version:'2.4', classifier:'jdk15'

}

DevJam course offerings for February-March 2010

David Hussman recently posted the new DevJam course offerings for February and March 2010. I'll be leading the Introduction to Groovy, Introduction to Grails and Test Driven courses. The Test Driven course is open to Java and .NET developers (.NET examples in C#). Detailed information can be found at http://www.devjam.com/courses-and-training/index.php. Seating is limited to about 12 participants per course.

Second Groovy and Grails training in the bag

Completed my second Groovy and Grails training for DevJam this past Saturday. This training session was much smoother than the first training. Still too much material and exercises to be taught in just one day. Received a lot of good feedback from the participants and hopefully this feedback helps solidify the training. I'm always amazed at how much stuff I could focus in on for the Groovy training. Really need to beef up the meata-object protocol stuff in the Groovy training.

First Groovy and Grails training is in the bag

I made it through my first test run of Groovy and Grails training for DevJam this past Saturday. Had a couple of snafus occur. Nothing major and easily fixed. Did learn a couple of things though:

Don't put developers on an operating system that they don't know for training

The DevJam training has Mac minis which boot either Mac OS X Snow Leopard or Windows Vista (via Bootcamp). I had the systems booted to Mac OS X for the training. Unfortunately all of the developers that came to this training were unfamiliar with Mac OS X, but willing to try it. Bad move on my part. I ended up answering far too many questions on the operating system and which tools were were going to use. Oracle SQL Developer also gave me problems in the Mac OS X environment when trying to update the tool with the MySQL drivers through its software updating system.

If you think you have enough code examples, you don't!

I had about 10-12 Groovy code examples to demonstrate various features of the language. Far too few for the questions that cropped up. Luckily, it was Groovy and writing new code examples or changing existing code examples was pretty straightforward. Kudos to Groovy for being very easy to explore and play with. The participants thought very highly of the interactiveness of the coding during the session.

Don't try to do both Groovy and Grails in a single day.

I knew going in that doing both was going to be very difficult. I just didn't realize how difficult it would be. Again, due to operating system and tool snafus, I didn't finish up the Groovy stuff until well into the afternoon. Not much time for Grails. I was looking forward to the exercise in Grails and we didn't get very far with it.

Automate the packaging of the student materials in electronic format.

We decided to put all the training handouts, examples, and anything else helpful for the students on 4 GB flash drives and give the flash drives to the students to keep. That's good. What's not good is missing some things on the flash drive and updating flash drives during the course. Next time I'll use an Ant build script to build a distribution and clean out any Subversion metadata from the student materials.

The Groovy Eclipse plugin seems to be making headway.

One of the participants in the group, Nick Spilman, had his laptop along and was using Eclipse Galileo and the new Groovy Eclipse plugin during the Groovy portion of the training. He thought it worked well with Groovy. I used IntelliJ 9.0 EAP (Maia) and that also works well. Looks like SpringSource (or shall I say VMware now) is getting serious on the tooling for Groovy and Grails.

Need to spend more time on understanding Groovy's meta-programming facilities.

It's one thing to use Groovy and use its meta-programming faclities (aka MOP) successfully in your own work. It is a far different thing to try and teach others about Groovy's meta-programming facilities. Teaching a concept, especially a concept as complicated as meta-programming, is extremely difficult.

Performing test runs of presentations and trainings is essential.

I'm very pleased that I was afforded the opportunity to be able to offer a couple of test runs of this training to some developers before offering it to the public. Like anything else, it takes practice and feedback to get good at something. I have another test run of this training later this month and I'm sure it will be much better than it was the first time out.

Grails domain object modeling with Groovy enums

I had a need to use enumerations within my object model in my Grails app today. I wrote up a Google Sites page about my solution. I am using Grails 1.1 for this solution.

Having another look at Grails

It's been a while since I've really worked with Grails. I used it in 2007 for about 7 months, but we really didn't use it in a typical fashion, as the front end was done in Adobe Flex. 2007 was also fairly early in the life of Grails; I remember using version 0.5.6 when I started at the client.

I'm now building a sample application in Grails 1.0.4 and doing all the AJAX stuff in jQuery. I'm really impressed with Grails now! It's so much more productive than I remember it. Also, I have a couple of books that have been absolutely essential to getting me going with Grails again: The Definitive Guide to Grails, Second Edition and Groovy and Grails Recipes. I'm using IntelliJ IDEA 8.1 for all my development and its Groovy and Grails support is much better than I remember it. JetBrains really makes it easy to give them money every year for a personal license upgrade for IntelliJ.

I haven't been super pleased with the Grails documentation on the Grails site and hopefully SpringSource has a positive effect on this issue in the near future. My development has been plowing ahead very nicely. I really like that I don't have to restart the application server for every little change. That saves a ton of time and has a positive effect on your ability to stay focused on job at hand. Very impressed with Grails this time around.