MySQL Communications Link Failure

I'm building a MySQL Driven web app using MySQL JPA Hibernate JConnector. I've hit an exception:

HTTP Status 500 - exception
javax.persistence.PersistenceException: org.hibernate.exception.JDBCConnectionException: could not load an entity: [net.bmo.model.BmoCalendar#2]

The root cause of this error is:

root cause:java.io.EOFException: Can not read response from server. Expected to read 4 bytes, read 0 bytes before connection was unexpectedly lost.

Basically this means that the DB connection times out and when the app. went back to the DB to retrieve some data the connection had been lost.

Solution? Add connection pooling. Something I'd been meaning to get round to but hadn't had a chance. So I added C3P0 Connection pooling. I'm using Hibernate so I added the following dependency to the pom.xml:

  <dependency>
    <groupId>org.hibernate</groupId>
    <artifactId>hibernate-c3p0</artifactId>
    <version>3.3.2.GA</version>
  </dependency>

This drops in the required jar(s) hibernate-core-3.3.8.GA.jar and c3p0-0.9.1.jar add them to the class path if you don't already have them.

Next I added the c3p0 properties to the persistence.xml file. The amendments are highlighted in blue.

<?xml version="1.0" encoding="UTF-8"?>
<persistence xmlns="http://java.sun.com/xml/ns/persistence"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://java.sun.com/xml/ns/persistence
http://java.sun.com/xml/ns/persistence/persistence_1_0.xsd" version="1.0">
  <persistence-unit name="myPersistence"
      transaction-type="RESOURCE_LOCAL">
   <provider>org.hibernate.ejb.HibernatePersistence
   <properties>
     <property name = "hibernate.connection.driver_class"
          value = "com.mysql.jdbc.Driver"/>
     <property name = "hibernate.connection.url"
          value = "jdbc:mysql://localhost:3306/myDB"/>
     <property name = "hibernate.connection.username" value = "uid"/>
     <property name = "hibernate.connection.password" value = "pwd"/>
     <property name = "hibernate.hbm2ddl.auto" value = "validate"/>
     <property name="hibernate.connection.provider_class"
          value="org.hibernate.connection.C3P0ConnectionProvider" />
     <property name="hibernate.c3p0.max_size" value="100" />
     <property name="hibernate.c3p0.min_size" value="0" />
     <property name="hibernate.c3p0.acquire_increment" value="1" />
     <property name="hibernate.c3p0.idle_test_period" value="300" />
     <property name="hibernate.c3p0.max_statements" value="0" />
     <property name="hibernate.c3p0.timeout" value="100" />
   </properties>
  </persistence-unit>
</persistence>


The effect of the lines above is to force the test of the connections in the pool at regular intervals and refresh the connection if required. This prevented the above error for my deployment. I did this without making any changes in the MySQL Database configuration

...I know a bloke who does a bit of that....

I met a guy the other day who was having a bit of difficulty with his company's IT infrastructure. If I had a pound for every time I have this conversation.

"We've got xxx application running on a server in our office and it's broken/not doing what we want any more. Can you fix it?"

Well, yes, probably. But how do companies with no IT expertise end up with huge systems flogged to them by some bloke they knew then he left/went bust/went elsewhere and now we're high and dry with no IT resources.

Also, why does someone recommend MS Small Business Server and MS Exchange to an organisation that clearly don't have the resources to maintain it? At some point the underlying SQL Server will run out of space/run into problems emails will get inevitably get lost and the whole thing will fall apart. It happens at big companies (I've seen it!) and can cause all sorts of issues, but it can cripple a small organisation.

These are the same organisations that don't get a consultancy in to give them a realistic breakdown of the the options and costs. I'd want to see 3 or 4 different options and cost/benefits for each and make my mind up from there. There's no doubt that there are a lot of clever systems out there with lots of features but they all come at a price and maintenance is no small part of that price.

I'm not saying that consultancy is a solution in itself, far from it. In the UK Business Link can be a good source of advice as well as numerous other organisations. There's a range of issues to consider when buying an IT system. Hardware failure support, software training support, procurement, installation & deployment, maintenance & disposal. Having seen a few RFP's (Request For Proposal) usually issued by very large companies to invite bids from suppliers. This seems a good way for small businesses to think too. If nothing else it forces the purchaser to consider the requirements as a whole rather then just as buying a shiny sparkly new computer that looked great in the add. A little bit of light research will show up RFP examples and guidance

This company I spoke to could have achieved all they wanted (email, shared docs & calendars, holiday forms linked to storage and basic work flow) from Google Apps for example or any number of other systems and it wouldn't have cost him the best part of 6k GBP and he wouldn't be high and dry when it went wrong.

In this case SAAS(Software As A Service) services (or ASP - Application Service Provision - as it used to be known) could have a real benefit. YOU don't have to have a DBA and a Sys Admin on your books. There's plenty of time and scope to bring that in house as and when you're big enough. But please don't "get a bloke you know" to do it for you unless he's prepared to enter into an SLA(Service Level Agreement) and can provide you with a DR strategy for when he gets hit by a bus.

OutOfMemoryError: PermGen space in Tomcat 6. The Solution:

I'm blogging this as much for my own use as help for other's.

Every time I set up a new Tomcat server I forget these options! Once you find them keep them safe.

set JAVA_OPTS=-Xms256m -Xmx1024m -Xss96k -XX:MaxPermSize=512m -server -XX:+CMSClassUnloadingEnabled -XX:+CMSPermGenSweepingEnabled

It's the CMSClassUnloadingEnabled that prevents the OOM error. The memory options depend on your hardware.

To set these options for the Sand Box Server in eclipse goto window -> preferences. In the left hand view select servers click on the tomcat server and add the above to the startup options. I'd post a screen shot but I'm using MyEclipse which is a bit niche for most developers.

Cross Domain AJAX Support with JSON-P

I was asked to get an online payment system working recenlty but the site was hosted away from the customer's site. Payment confirmation was on the customer's site and payments were run from WOLRDPAY. So my problem was how do you get the page to not accept payment twice for the same invoice? Answer a payment check service using ajax returning a simple paid/not paid response to the invoice id in the request. But in AJAX you can't do that because you're stepping off your domain.

Eventually I found JSON-P according to the browser implementation of XMLHTTPRequest the URL needs to be relative ie /ajax/CheckInvoice?invId=12345
I wanted http://some3rdpartysite/CheckInvoice?invId=12345

JSON is a way of encoding the XML request into key-value pairs but JSON-P adds a call back function that is created locally the AJAX request then looks at the local call, the local call wraps a function that calls the remote site. The Browser is not then aware of the cross domain aspect of the call and doesn't throw an exception. Job done, customer happy.

I'm not, though. Unfortunately there is some disagreement around whether or not this is a legit process. Many developers think that because JSON-P takes advantage of the fact that <script> tags are not evaluated, this is in fact a security flaw and should be plugged. So will my solution be broken by a future browser release? There is another school of thought that this is, in fact a legitimate process - certainly in this case, that should be handled appropriately and that until browser technology catches up JSON-P is a legit solution.

I agree with the latter. In my case the sites are all legitimate and it is simply a case of integrating two SAAS (Software As A Service) Services and that's becoming ever more important. The fact that they haven't been built to provide these services formally is more to do with the implementation rather than the principle.

Microgen Aptitude^®

Having worked with Microgen Aptitude^® development tool whilst employed by Microgen^® I naturally included it on my CV. However, every time I go to an interview for a new contract I get asked “What is Microgen Aptitude?” and have to explain exactly what it is and why it's relevant to my work as a developer.

Microgen^® is a Software Consultancy & Supply organisation focussing mainly on, but not limited to, the financial sector.

Aptitude^® is described as "The Enterprise Class Business Process Management Suite" essentially a piece of software that allows you to define the data in your business and your organisations rules for processing that data. You can use the rules you create to generate business processes based on those rules and run those processes in an ordered manner. Including all the tools & facilities you've come to expect from an enterprise level development environment.

Data exists within an organisation in a number of forms Databases, files, message feeds etc. Aptitude^® caters for all the common formats and has facilities for transforming these sources to Data Objects which can then be used to create rules.

This is interesting but nothing new. I can model my business objects in Java and map those objects to a Database schema using a ORM (Object Relational Mapping) tool like Hibernate Aptitude^®, however, does this using a graphical development environment, which allows you to drag and drop entities around the development environment.

fig. 1 An example rule

fig. 2 An example of a business process designed with a source, rule, and target entities.

Graphically representing the elements of a business process gives a visual representation of the rules involved in a business process which, in turn, contributes to a better understanding of the rules and processes required to make business decisions. Not only this but there is no need to learn a complex 3rd generation programming language. Microgen^® suggest that this tool can be used by a Business Analyst, however I, personally wouldn't go that far. You still need to have a strong technical understanding of software development to use Aptitude^® efficiently. Not least because you'll more than likely be using it in conjunction with a database of some sort and that means an understanding of the Relational Model at least. Then there's understanding XML, messaging, SOA, Transactions processing, file formats.

The next most significant part of the Aptitude^® suite is the ability to then use the it's built in Web interface designer. I'm not going into much detail in this article, but at a high level you can design a web page and deploy it to IIS or Tomcat in the more recent incarnations.
Looking a little like a Visual Basic style form designer, the design interface lets you drag and drop web widgets onto a 'page' and connect them to your Business Processes or Reference Objects. Once deployed you have a working web interface that can deliver functionality around your organisation based on a common set of business rules.

Pros: Graphical development environment, wizards to rapidly model existing objects from table/file specification, security model, test/debug environment,
Cons: Single vendor, fixed device interfaces, limited after market consultant base, slow costly development cycle, maturity.


Conclusion
If you've got the money, Aptitude^® is a nice tool. Out of reach to all but the most well off organisations, however, as technologies like this become more popular and competition in the market place increases this may change.

B49

Attended a new networking organisation B49 (before Nine!) run by Nick Lofthouse keyfins.co.uk and his business partner. The group is a support and referral organisation and I've already benefited from help with invoicing, office space and contacts.

The group is located at Welwyn Garden City Golf Club and meets on a Tuesday from Seven Am until Nine

I met Matt from 3internet, great chap, even if he is a .NET guy ;)

They have a nice site, and do all sorts of MS based Web Applications including design and content management.

Back to the Blog

Wow it's been ages since I updated the Blog. I've been working hard for Microgen on a new Billing portal for Virgin Media (ntl:Telewest Business). It was an awesome project. In Four months we re-wrote the online billing application that they'd inherited through an acquisition.

Now it's leaner and meaner. User admin functionality, self sign-up, still with full integration with Microgen's eBilling architecture only now the load process is more maintainable and leveraging JSF, Spring, JPA & Hibernate, we managed to produce a lovely RIA. Only draw back is the configuration nightmare that was the project inception. So what next?

There's a phase II that I may or may not be involved in because I've got other projects to work on, although I'd love it if I could be part of it but Microgen is back up to strength in that department so maybe consultancy only.

I've been investigating other alternatives in the RIA market place. Currently I'm messing with Google's App Engine which I love, with GWT. I played with flex but there are issues there with page ranking, and forcing customers to use Flash - not a bad thing in itself but some people registered protests about moving away from vanilla web (silverlight? JavaFX?)