It’s been ages since I have posted some sample code. It’s mainly because I don’t have time to collect and post sample code anymore. This once was a bit more challenging and googling wasn’t help much, so now that I have some time I though I would post some sample code that achieves batch inserts with spring data.

For example this link:
http://forum.spring.io/forum/spring-projects/data/118203-bulk-insert-with-crudrepository indicated that I had to manually get the session and iterate/flush (which was true when using Spring/Hibernate/JPA). But when using the CRUDRepository it appears it’s much simpler.

FULL CODE

Full code sample (maven project) can be found on github: https://github.com/cyrus13/anastasakis-net-sample-code/tree/master/spring-data-batch

You basically need to have the following elements:

  • Add: ?rewriteBatchedStatements=true to the end of the connectionstring.
  • Make sure you use a generator that supports batching in your entity. E.g.

  • Use the: save(Iterable<S> paramIterable); method of the JpaRepository to save the data.
  • Use the: hibernate.jdbc.batch_size configuration.

RESULT

So enabling the query log in MySQL:

we can see the following mysql code is executed:

Note: Don’t forget to stop logging statements into MySQL general log!

SET global general_log = 0;

I wanted to add the suffix of the day in Java (i.e. 1st, 2nd, 3rd, etc.). This is a small method that does the Job

I am using Logback to do most of my logging. I am using a third party library for some computation. Luckily the library uses slf4j, meaning it’s agnostic of the logging implementation.

The library caches two collections of items and depending on the circumstances the item I am looking for may be in any of the two collections. This is not important. What is important, is that the library for some reason logs an error if an item cannot be found in the first collection.

I wanted to get rid of this message because it was cluttering my logs. But I didn’t want to disable all logging from this specific class, because it may print some useful logs. I just wanted to get rid of this specific log. There is a way to do it Continue reading

Since Spring 3.2 it should be possible to use a qualifier in the “Async” annotation of a method, to indicate which specific executor to use. For example, I have the following class, that is supposed to collect the HTML from a website asynchronously:

HTMLFetcher Interface

TestHTMLFetcher Class

And the following excerpt from the context.xml file:

This didn’t work! I submitted 10 tasks to the executor and 10 thread, instead of 1 as I had instructed it had been created. I digged into the Spring code a bit and found the AnnotationAsyncExecutionInterceptor class that was doing tbe job of assigning tasks from methods annotated as async to executors. Putting  a breakpoint on its getExecutorQualifier method it became evident why it doesn’t work.

You need to annotate the interface method rather than the class with the:

annotation. In my example that is the getHTML method of the HTMLFetcher interface. It now works. I am not sure if it was done on purpose (i.e. if it’s part of the specification). I don’t have time to read the related documentation or search the Spring Jira. However, I would assume that the right place to put the annotation is the implementation. I may want to have two implementations of the same method, one decorated with “Async” that will be asynchronously executed and another without any annotation that will be synchronously executed.

By default on Windows Eclipse autocomplete shortcut is: Ctrl+Space. In my Mac laptop I use this combination for the operating system’s spotlight quick search. As a result, I wanted to change the default autocomplete shortcut. What you need to do is:

1. Go to Window->Preferences (Eclipse -> Preferences on Mac).

2. In the new window that pops up, go to “General -> Keys”

3. Find the “Content Assist” command and change it to whatever you like (in my case I chose Cmd+Space).

4. Don’t forget to click on “Save” on this pop up window!

Recently I’ve been playing around with a webbrowser control to automate my interaction with a website for testing purposes. I am using C# and DOT NET.

I found it a bit difficult to change the value of an HTML combobox (i.e. dropdown), but as it turns out it’s rather easy.I used the following code to change the value of a combobox named: “test” to the value 12.

This code was used for the following HTML:

Finally the following code was required to press the “Submit” button.
 

I have developed a component in Java that requires an HTML parser. The component goes through around 2000 webpages and gets some data.

It was quite easy to implement it using the org.htmlParser (http://htmlparser.sourceforge.net/). Even though some of the webpages are quite big (some of a size of up to a few hunders of MBs) the memory of the component seemed to grow inexplicably leading to a Java heap out of memory error. I spent a good deal of time trying to figure out the source of the leak thinking it was my code. After a few attempts to identify the problem, I used the IMB Support Assistant workbench and took a heap dump using the command:

jmap -dump:format=b,file=heap.bin processID

I was able to identify a lot of Finalizer objects referencing the org.htmlParser.lexer. This looks like a memory leak, where the garbage collector can’t collect the objects properly?

Well.. the fact of the matter is I haven’t spent an enormous amount of time reading the documentation and/or source code of the project.  It seems there is a close() method that can be called on the Page reference of the lexer and I haven’t used it. So, at the end of my method that does the parsing I added:

parser.getLexer().getPage().close();
parser.setInputHTML("");

The first statement closes the Page object. I added the second statement just to be on the safe side, even though it’s probably redundant.

And the “Memory Leak” seems to have vanished…

I have received an Email with the top 50 programming quotes. Here are my favourites:

  • “Programming today is a race between software engineers striving to build bigger and better idiot-proof programs, and the universe trying to build bigger and better idiots. So far, the universe is winning.” – Rick Cook
  • “Walking on water and developing software from a specification are easy if both are frozen.” – Edward V. Berard
  • “They don’t make bugs like Bunny anymore.” – Olav Mjelde
  • “A C program is like a fast dance on a newly waxed dance floor by people carrying razors.” – Waldi Ravens
  • I have always wished for my computer to be as easy to use as my telephone; my wish has come true because I can no longer figure out how to use my telephone.” – Bjarne Stroustrup
  • “In the one and only true way, the object-oriented version of ‘Spaghetti code’ is, of course, ‘Lasagna code’ (too many layers).” – Roberto Waltman
  • “For a long time it puzzled me how something so expensive, so leading edge, could be so useless. And then it occurred to me that a computer is a stupid machine with the ability to do incredibly smart things, while computer programmers are smart people with the ability to do incredibly stupid things. They are, in short, a perfect match.” – Bill Bryson
  • “Good design adds value faster than it adds cost.” – Thomas C. Gale
  • “Perfection [in design] is achieved, not when there is nothing more to add, but when there is nothing left to take away.” – Antoine de Saint-Exupry
  • “The best programmers are not marginally better than merely good ones. They are an order-of-magnitude better, measured by whatever standard: conceptual creativity, speed, ingenuity of design, or problem-solving ability.” – Randall E. Stross
  • “To iterate is human, to recurse divine.” – L. Peter Deutsch
  • “The trouble with programmers is that you can never tell what a programmer is doing until it’s too late.” – Seymour Cray
  • “Most good programmers do programming not because they expect to get paid or get adulation by the public, but because it is fun to program.” -Linus Torvalds

And my favourites:

  • “You can’t have great software without a great team, and most software teams behave like dysfunctional families.” – Jim McCarthy
  • “Beware of bugs in the above code; I have only proved it correct, not tried it.” – Donald E. Knuth
  • “Sometimes it pays to stay in bed on Monday, rather than spending the rest of the week debugging Monday’s code.” – Christopher Thompson
  • “First learn computer science and all the theory. Next develop a programming style. Then forget all that and just hack.” – George Carrette
  • “Most software today is very much like an Egyptian pyramid with millions of bricks piled on top of each other, with no structural integrity, but just done by brute force and thousands of slaves.” – Alan Kay
  • “Always code as if the guy who ends up maintaining your code will be a violent psychopath who knows where you live.” – Martin Golding
  • “There are two ways of constructing a software design. One way is to make it so simple that there are obviously no deficiencies. And the other way is to make it so complicated that there are no obvious deficiencies.” – C.A.R. Hoare