Tag Archives: java spring

Batch inserts with Spring Data and MySQL

Posted on by | 6 Replies

It’s been ages since I have posted some sample code. It’s mainly because I don’t have time to collect and post sample code anymore. This once was a bit more challenging and googling wasn’t help much, so now that I have some time I though I would post some sample code that achieves batch inserts with spring data.

For example this link:
http://forum.spring.io/forum/spring-projects/data/118203-bulk-insert-with-crudrepository indicated that I had to manually get the session and iterate/flush (which was true when using Spring/Hibernate/JPA). But when using the CRUDRepository it appears it’s much simpler.

FULL CODE

Full code sample (maven project) can be found on github: https://github.com/cyrus13/anastasakis-net-sample-code/tree/master/spring-data-batch

You basically need to have the following elements:

  • Add: ?rewriteBatchedStatements=true to the end of the connectionstring.
  • Make sure you use a generator that supports batching in your entity. E.g.
@Id
@GeneratedValue(generator = "generator")
@GenericGenerator(name = "generator", strategy = "increment")
  • Use the: save(Iterable<S> paramIterable); method of the JpaRepository to save the data.
  • Use the: hibernate.jdbc.batch_size configuration.

RESULT

So enabling the query log in MySQL:

SET global general_log = 1;
SET global log_output = 'table';

we can see the following mysql code is executed:

SET autocommit=0;
select max(id) from ExampleEntity;
SHOW WARNINGS;
select @@session.tx_read_only;
insert into ExampleEntity (exampleText, id) values 
('de32bec8-1cf9-4f14-b816-0ab7a00b1539', 4),
('c0c85b32-eb2d-4a69-ade4-ac70ea94241c', 5);
commit;
SET autocommit=1;

Note: Don’t forget to stop logging statements into MySQL general log!

SET global general_log = 0;

Spring Async Method Qualifier to Specify Executor Issue

Posted on by | 2 Replies

Since Spring 3.2 it should be possible to use a qualifier in the “Async” annotation of a method, to indicate which specific executor to use. For example, I have the following class, that is supposed to collect the HTML from a website asynchronously:

HTMLFetcher Interface

import java.io.IOException;
import java.net.MalformedURLException;
import java.util.Date;
import java.util.concurrent.Future;

import org.springframework.scheduling.annotation.Async;

public interface HTMLFetcher {

Future&lt;HTMLFetcher.HTMLFetcherResult&gt; getHTML(String baseUrl,Date date);

interface HTMLFetcherResult {

String getHTMLResult();

Date getDate();
}
}

TestHTMLFetcher Class

import java.io.IOException;
import java.net.MalformedURLException;
import java.util.Date;
import java.util.concurrent.Future;

import org.apache.log4j.Logger;
import org.springframework.scheduling.annotation.Async;
import org.springframework.scheduling.annotation.AsyncResult;
import org.springframework.stereotype.Component;

import agonesgr.html.HTMLFetcher;

@Component
public class TestHTMLFetcher implements HTMLFetcher{

@Async(value="htmlFetcherExecutor")
public Future&lt;HTMLFetcher.HTMLFetcherResult&gt; getHTML(String baseUrl, Date date) {
try {
System.out.println("Before execute!!");
Thread.sleep(100000);
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
HTMLFetcher.HTMLFetcherResult r = new TestHTMLFetcher.HTMLFetcherResultImpl("",new Date());

return new AsyncResult&lt;HTMLFetcherResult&gt;(r);
}

private static class HTMLFetcherResultImpl implements HTMLFetcherResult{

private String htmlResult;
private Date date;

public HTMLFetcherResultImpl(String htmlResult, Date date) {
super();
this.htmlResult = htmlResult;
this.date = date;
}

public String getHTMLResult() {
return htmlResult;
}

public Date getDate() {
return date;
}
}
}

And the following excerpt from the context.xml file:



This didn’t work! I submitted 10 tasks to the executor and 10 thread, instead of 1 as I had instructed it had been created. I digged into the Spring code a bit and found the AnnotationAsyncExecutionInterceptor class that was doing tbe job of assigning tasks from methods annotated as async to executors. Putting  a breakpoint on its getExecutorQualifier method it became evident why it doesn’t work.


You need to annotate the interface method rather than the class with the:


@Async(value="htmlFetcherExecutor")


annotation. In my example that is the getHTML method of the HTMLFetcher interface. It now works. I am not sure if it was done on purpose (i.e. if it’s part of the specification). I don’t have time to read the related documentation or search the Spring Jira. However, I would assume that the right place to put the annotation is the implementation. I may want to have two implementations of the same method, one decorated with “Async” that will be asynchronously executed and another without any annotation that will be synchronously executed.