MockServer in Webservice

It is normal sometimes we will face such situation: your implementation is ready however your downstream system is still unknown i.e. it has not been deployed, it has not been developed whatever in one word not-ready! What can you do to do an integration test? The answer is mock it.
We use mock server as a mock tool. A maven configuration is as follows:
<plugin>
                <groupId>org.mock-server</groupId>
                <artifactId>mockserver-maven-plugin</artifactId>
                <version>3.10.4</version>
                <configuration>
                    <skip>false</skip>
                    <serverPort>57333</serverPort>
                    <logLevel>ERROR</logLevel>
                    <pipeLogToConsole>true</pipeLogToConsole>
                </configuration>
                <executions>
                    <execution>
                        <iFold>start-mork-server</id>
                        <phase>clean</phase>
                        <goals>
                            <goal>start</goal>
                        </goals>
                    </execution>
                    <execution>
                        <id>stop-mock-server</id>
                        <phase>test</phase>
                        <goals>
                            <goal>stop</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>

Configuration:

Folder Structure:

The configuration is per each downstream system/service. It is a folder with one “service.json” file and resource files (in subfolders).
All paths in the “service.json” are relative to the parent folder of the “service.json”.

folder structure.png

Service configuration file:

The service configuration contains “provider” name and a list of “Interactions”.
Each “interaction” has

  1. “request” where some values are variables
  2. “response” with default response parameters
  3. “switch” – list of variables (from request) – OPTIONAL
  4. “responses” – a map, where the keys are made from the variables and expected variable values, while the map values are “response” objects, whose values override the default response properties. – OPTIONAL

The order is important. It allows to define a default response by setting the most generic request matching conditions at the end.

Schema:

{
  “title”: “Service”,
  “description”: “Mock service configuration”,
  “required”: [
    “provider”,
    “interactions”
  ],
  “additionalProperties”: false,
  “properties”: {
    “provider”: {
      “type”: “string”
    },
    “interactions”: {
      “type”: “array”,
      “items”: {
        “$ref”: “#/definitions/interaction”
      }
    }
  },
  “definitions”: {
    “props”: {
      “type”: “object”,
      “patternProperties”: {
        “.+”: {
          “type”: “string”,
          “pattern”: “(^\\$\\{[^/]+\\}$)|(^[^\\$]{1}.*$)”
        }
      },
      “additionalProperties”: false
    },
    “response”: {
      “type”: “object”,
      “properties”: {
        “status”: {
          “type”: “integer”
        },
        “headers”: {
          “$ref”: “#/definitions/props”
        },
        “body”: {
          “type”: “string”
        },
        “latency”: {
          “type”: “integer”
        }
      }
    },
    “interaction”: {
      “type”: “object”,
      “properties”: {
        “request”: {
          “type”: “object”,
          “required”: true,
          “properties”: {
            “method”: {
              “type”: “array”,
              “items”: {
                “type”: “string”
              }
            },
            “path”: {
              “type”: “string”
            },
            “query”: {
              “$ref”: “#/definitions/props”
            },
            “cookies”: {
              “$ref”: “#/definitions/props”
            },
            “headers”: {
              “$ref”: “#/definitions/props”
            },
            “body”: {
              “type”: “object”,
              “properties”: {
                “regex”: {
                  “type”: “string”,
                  “pattern”: “(^\\$\\{[^/]+\\}$)|(^[^\\$]{1}.*$)”
                },
                “json”: {}
              }
            }
          },
          “required”: [
            “method”,
            “path”
          ]
        },
        “response”: {
          “$ref”: “#/definitions/response”
        },
        “switch”: {
          “type”: “array”,
          “items”: {
            “type”: “string”
          }
        },
        “responses”: {
          “type”: “object”,
          “additionalProperties”: {
            “$ref”: “#/definitions/response”
          }
        }
      },
      “additionalProperties”: false
    }
  }
}

Switch:

The purpose of these feature is to make the configuration more compact and easier to read. Without switch the configuration would look like a request –> response map, where the request and response are concrete values.
With switch we can mark some parts of the request as variables and move the corresponding list of values (n-tuple) to a switch key.

switch.png

Sample:

{
“request”: {
“method”: [
“POST”
],
“path”: “/myserice/mypath1/mypath2”,
“body”: {
“regex”:”myId\”:\”[0-9]{10,}”
}
},
“response”: {
“status”: 422,
“latency”:500,
“headers”:{
“Content-type”:”application/json;charset=UTF-8″
},
“body”: “data/response_422.json”
}
}

This tells you when it goes to URL Path/myserice/mypath1/mypath2 with any TEN digits myId  then it will give you a response with content as data/response_422.json

Statement Handling Literal string with Single Quote (‘) in where clause criteria

In simple query using JDBC Statement, NOT PreparedStatement, you will get a SQL exception in statement like this one

select customer.* from t_customer_table customer
where customer.last_name = ‘O’Reilly’;

You can workaround this by replace the single quote inside the criteria value by two consecutive single quotes (”). Not this is no a double quote, but 2 single quotes.

select customer.* from t_customer_table customer
where customer.last_nam = ‘O”Reilly’;

Please keep in mind it is always highly recommenced to use prepared statement with parameters, then you would need this workaround. But in case you don’t don’t want to that, this solution comes in handy.

jar sequence in pom.xml

I am working in a very complex env.
The file I needed, ServiceInstanceServiceTypeEnum.class is unfortunately included in THREE different jars.
That file is changed by the related team. However the change is only reflected in one of the three jars (let’s say they are called A.jar, B.jar and C.jar). The changed class is only included in C.jar.
Unfortunately my code only pick up the ServiceInstanceServiceTypeEnum class in A.jar which does not contain the changes.
I do not think it is a good design to include a file in different jars. But I am not the designer.
Actually if the related team can help me to include their changes in A.jar and B.jar too, it will not be an issue. But for some reason they can not do that.
Solution:
Change the sequence of those THREE jars in pom.xml and make C.jar the first one.
This may lead to some other unknown issue, but I think this is the only thing I can do at this stage.

Exceptions during installation of SoapUi

When I try to install SoapUI for Linux it complains:

Unpacking JRE …
Preparing JRE …
Starting Installer …
java.lang.NoClassDefFoundError: java.awt.Container
at com.install4j.runtime.installer.frontend.headless.AbstractHeadlessScreenExecutor.init(Unknown Source)
at com.install4j.runtime.installer.frontend.headless.ConsoleScreenExecutor.<init>(Unknown Source)

It appears that the installer can’t find a class in the WT library. I can’t see how how that can be possible. Does anyone know what is causing this, and how I may fix it?

[System Information]

java version “1.6.0_45”
Java(TM) SE Runtime Environment (build 1.6.0_45-b06)
Java HotSpot(TM) 64-Bit Server VM (build 20.45-b01, mixed mode)

Solution:

This seemed to fix the problem:

sudo apt-get install ia32-libs

The ia32-libs package is longer present in Ubuntu 12.04 repositories for a 64bit system. The ia32-libs package was a hack to get 32-bit packages installed on a 64-bit installation. Since Ubuntu version 11.10 (Oneiric), Multi Arch has been added. One of the objectives for it is removing the ia32-libs package. You can install the 32-bit libraries of a package with:

sudo apt-get install ia32-libs

OutOfMemoryError in Jenkins

Lars Vogel has given us an introduction of Jenkins in his blog http://www.vogella.com/articles/Jenkins/article.html

However I suffered an OutOfMemory Exception after following the steps in his guide. The Jenkins’s web site (https://wiki.jenkins-ci.org/display/JENKINS/I’m+getting+OutOfMemoryError) does not tell you how to fix this in detail. It is not good for Jenkins newbee like me.

After searching the web, I found the solution is to increase the perm size from default to 512m in the catalina.bat file (I am using Tomcat Windows as the host for Jenkins) as follows:

JAVA_OPTS=”-Djava.awt.headless=true -Dfile.encoding=UTF-8 -server -Xms512m -Xmx512m -XX:NewSize=512m -XX:MaxNewSize=512m -XX:PermSize=512m -XX:MaxPermSize=512m -XX:+DisableExplicitGC”

The JAVA_OPTS variable will be picked up by Tomcat automatically when it is started up.

After changing the perm size , the Out of memory error problem disappeared.

Stored procedures vs HSQL

Tags

,

In this post let us discuss whether stored procedures should be used instead of Hibernate based queries i.e. HSQL

If you google it, you may find there are many myths and arguments associated with the stored procedure vs HSQL. Here are a few (I’m sure that there are more):

1. sp is pre-compiled by the server, this will increase the speed.

In some case it is true especially when the SQL is quite complicated. DBAs can help us to optimise SQls, we can take advantages of Database vendor’s pre-defined functions etc. Those are the advantages, however , the case of stored procedures for update routines is an another story.

An application can easily generate optimal update SQL for a single field involving a single parameter. In contrast, a stored procedure implementation will normally end up having to provide an “all fields” update mechanism with nullable parameters. This in turn leads to null checking logic within the stored procedure that is technically unnecessary.

2. Hibernate is slow because it makes unnecessary calls.

Hibernate can be made slow through the inappropriate use of eager fetching strategies where unnecessary associations are explored.  Consider the situation of a 3 way relationship between User, Role and Authority. The requirement is to fetch all Users with a first name of “Bob” (a set of 10 Users), along with all their Roles and Authorities. Hibernate can build this object graph in 3 queries: one for each entity followed by application side merging. An sp will require a routine to filter on the User table to get the ones with the correct first name. Then the Role fetch routine will have to be called to get the Roles for each of the Users, then the Authorities fetch routine will have to be called for each of the Roles. This is more complicated than the HSQL approach and makes a bunch of unnecessary calls as well.

3. Stored procedures make it obvious where the SQL is.

In the application, the actual SQL being run is buried in the depths of the mapping framework and usually only appears in the debug logs. The location in the application code where this query is constructed is not always clear – it should be in the DAOs but can be hidden in annotations on DTOs. Consequently, it takes some time and skill to identify where the query is being constructed, and how to alter the structure to make it more efficient. However, by following well known design patterns for entity mapping, these inefficiencies can be ironed out early and permanently.

So what should we choose?

There is no silver bullet in the world. In my opinion, if you have a really complicated SQL or your DBA can handle all the biz logic, then hide it in sp, anyway, based on my knowledge to call a sp is easier than write a bunch of HSQL. 

HSQL is a good candidate for update, or delete.

How to init HSQL in Java Configuration Spring MVC

Tags

, , ,

These days I was asked to write a web based application using  Java Configuration Spring MVC. The database is HSQL. Based on my knowledge all the tables in HSQL are created in the server’s memory, so the problem is how to init my database?

(If the database is MySQL I think I can create the database structure in advance. But for HSQL there is no platform for me to do that  before coding. Thus my solution is to create the structure every time the application is launched.)

The solution is:

1. create two SQL files one called initDB.sql the other one is called populateDB.sql and put them under the resources/db folder.

2. the content of initDB.sql looks like:

CREATE TABLE AIRPORT (
CODE VARCHAR(30) PRIMARY KEY,
NAME VARCHAR(30),
CITY VARCHAR(80)
);

3. the content of populateDB.sql looks like:

insert into AIRPORT(CODE,NAME,CITY) values(‘SYD’, ‘Sydney International’, ‘Sydney’);

4. in the DAO config file add the @EnableJpaRepositories annotation and a EmbeddedDatabaseBuilder class so that it looks like:

@Configuration
@EnableTransactionManagement
@ComponentScan(“com.foo.flight.model”)
@EnableJpaRepositories(“com.foo.flight.dao.jpa”)
public class JpaDefaultDaoConfig {
@Bean
public DataSource dataSource() {
return new EmbeddedDatabaseBuilder().setType(EmbeddedDatabaseType.HSQL)
.addScript(“classpath:db/initDB.sql”)
.addScript(“classpath:db/populateDB.sql”)
.build();
}

@Bean
public LocalContainerEntityManagerFactoryBean entityManagerFactory() {

entityManagerFactoryBean.setDataSource(dataSource());

return entityManagerFactoryBean;
}

}

All done.

Trouble shooting NoSuchMethodError: org.slf4j.impl.StaticLoggerBinder.getSingleton()

Tags

These days I was trying to create a Spring MVC jpa application. However when I try to run the application under tomcat 6.0 I got:

java.lang.NoSuchMethodError: org.slf4j.impl.StaticLoggerBinder.getSingleton()Lorg/slf4j/impl/StaticLoggerBinder;
at org.slf4j.LoggerFactory.bind(LoggerFactory.java:128)

After searching the website, I found the problem is with the version of slf4j-log4j12 jar package.

When I changed from version 1.5.6 to version 1.7.5. i.e. change from

<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-log4j12</artifactId>
<version>1.5.6</version>
</dependency>

To

<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-log4j12</artifactId>
<version>1.7.5</version>
</dependency>

This can be resolved.