I recently had a discussion with another developer who claimed to me that JOINs (SQL) are useless. This is technically true but he added that using joins is less efficient than making several requests and link tables in the code (C# or Java).

For him joins are for lazy people that don't care about performance. Is this true? Should we avoid using joins?

No, we should avoid developers who hold such incredibly wrong opinions.

In many cases, a database join is several orders of magnitude faster than anything done via the client, because it avoids DB roundtrips, and the DB can use indexes to perform the join.

Off the top of my head, I can't even imagine a single scenario where a correctly used join would be slower than the equivalent client-side operation.

Edit: There are some rare cases where custom client code can do things more efficiently than a straightforward DB join (see comment by meriton). But this is very much the exception.

I'm in a "special" situation about effeciency of my program. Now I'm at a phase where I want to maximize my application; being top notch and reducing battery consumption.

Before the question:

• First, my application work. It runs fine - no errors whatsoever.
• Second, I have read Designing for Performance on Android developers website and I have optimized small things they're mentioning. No problems whatsoever.

Now, I'm curious to hear about developers "special" tricks (fixes) that optimized their application(s). Stuff that users never gonna recognize or pay attention to; but instead increased the battery life or effectively made your application easier to maintain.

So, what's your unique optimizing trick(s)? (e.g. Object Pools? other patterns? unknown things? things that the website forgot to mention?) It doesn't have to be Android specific.

I'm in a particular situation where I'm really looking for knowledge and I think this will be a great opportunity to share developers knowledge about a situation they've all been in.

Please, vote up great answers as that will encourage great developers to share their knowledge.

At some point you are going to get to the point where using known tricks will hit their limits. The best thing to do at this point is profile your code and see what areas are the bottle-necks based on your specific requirements.

Profiling with Traceview and dmtracedump: an article on how to use the tools to profile your application.

Possible Duplicate:
Do you prefix your instance variable with 'this' in java ?

Simple question.

Is there a real difference between (generally, not only in constructors):

class ThisClass{
    private int a;
    public ThisClass(int a)
    {
        this.a = a;
    }
}

and

class ThisClass{
    private int _a;
    public ThisClass(int a)
    {
        _a = a;
    }
}

Other than coding convention?

Clarification - I didn't mean to the use of "this" as a reference to the object (such as passing it in a method) only to the use of it when assigning values in methods.

Technically, most languages don't care what the variable is called. This just changes the symbol lookup rules when compiling. The code to access this.a and _a in your examples should be the same, although the symbol tables will be different.

Locating to this.a in the symbol table may be slightly faster than locating _a during compilation. Any gain in lookup speed may be used in parsing this. so I wouldn't consider it an optimization. The optimization would only apply during compilation. a may be slightly more space efficient than _a in the symbol table.

When not being used for disambiguation this is likely a required syntax element. There are a number of other disambiguation techniques. IMHO none of the techniques is a clearly better than any others.

I have recently written my first Android app which was roughly 8,000-10,000 lines of code. One thing that continuously hindered my use of normal design patterns was android's heavy use of asynchronous calls (opening dialogs, activities, etc). Due to this, my code very quickly began looking "spaghetti" like, and I eventually started to dislike looking at certain classes.

Are there specific design patterns or programming methodologies which are for systems such as these that anyone would recommend? Are there any suggestions for writing manageable asynchronous code?

• Use global variables

If you do not want to mess up your code with simple Intent.putExtra() and manage this things for each unique Activity you'll have to use global variables within the application. Extend Application and store data that you need as long your application is alive. To actually implement it, use this excellent answer. This will make dependecies between activities go away. For example, say that you need a "username" for your application during the application's life cycle - this is an excellent tool for just that. No need for dirty Intent.putExtra() calls.

• Use styles

One common mistake when making the first Android application is that you just start writing your XML views. The XML files will (without problem and very fast) go up to very many lines of code. Although, people just "copy" the behaviour between the different views. Here you can have a solution where you'll just use the style attribute to implement a specific behaviour. For example, consider this piece of code:

values/styles.xml:

<style name="TitleText">
    <item name="android:layout_height">wrap_content</item>
    <item name="android:layout_width">wrap_content</item>
    <item name="android:textSize">18sp</item>
    <item name="android:textColor">#000</item>
    <item name="android:textStyle">bold</item>   
</style>

layout/main.xml:

Now, if you're having, let's say, two TextViews and both of them should have the same behaviour, make them use the TitleText style. Sample code:

<!--- ... -->
<TextView
   android:id="@+id/textview_one"
   style="@style/TitleText" 
/>

<TextView
   android:id="@+id/textview_two" 
   style="@style/TitleText" 
/>
<!--- ... -->

Simple and you don't need to duplicate code. If you really want to look further on these particular subject, please look at Layout Tricks: Creating Reusable UI Components from Android developers website.

• Use strings

This point is short but I think it is important to mention it. Another mistake that developers might do is to skip the strings.xml and just write UI messages (and attribute names) inside the code (where he will need it). To make your application easier to maintain; just define messages and attributes in the strings.xml file. This will make you win in the long run.

• Create and use an abstract tool class

When I wrote my first application I was just writing (and duplicating) methods where I needed it. The result? A lot of methods that had the same behaviour between various activities. What I have learned is to make a tool class. For example, let's say you have to make web requests in all of your activities. In that case, skip defining them inside the actual Activity and make a static method for it. Sample code:

public abstract class Tools {

    public static final void sendData(String url, 
              String user, String pass) {
        // URLConnections, HttpClients, etc...
    }

}

Now, you can just use this code below in your Activity that needs to send data towards a server:

Tools.sendData("www.www.www", "user", "pass");

However, you get the point. Use this "pattern" where you need it, it will keep you from messing up your code.

• Let custom classes define the behaviour where the user needs to interact with your application

This is probably the most useful point. To just define "where the user needs to interact with your application" let's say you have a Menu, which behaviour is very long in terms of lines, why do we keep the Menu's calculations in the same class? Every little item will make your Activity class a painful piece of code longer - your code look like "spaghetti". For example, instead of having something like this:

@Override
public boolean onPrepareOptionsMenu(Menu menu) {
    MenuItem item;
    item = menu.findItem(R.id.menu_id_one);
    if(aBooleanVariable) {
        item.setEnabled(true);
    } else {
        item.setEnabled(false);
    }
    // More code...
    return super.onPrepareOptionsMenu(menu);
}

@Override
public boolean onOptionsItemSelected(MenuItem i) {
    // Code, calculations...
    // ...
    // ...
    return super.onOptionsItemSelected(i);
}

redesign it to something like this:

private MyCustomMenuInstance mMenuInstance;

@Override
public void onCreate(Bundle savedInstanceState) {
    super.onCreate(savedInstanceState);        
    setContentView(R.layout.main);

    mMenuInstance = new MyCustomMenuInstance();
}  

@Override
public boolean onPrepareOptionsMenu(Menu menu) {
    mMenuInstance.onPrepareOptionsMenu(menu);
    return super.onPrepareOptionsMenu(menu);
}

@Override
public boolean onOptionsItemSelected(MenuItem i) {
    mMenuInstance.onOptionsItemSelected(i);
    return super.onOptionsItemSelected(i);
}

For example, MyCustomMenuInstance:

public class MyCustomMenuInstance { 

    // Member fields..

    public MyCustomMenuInstance() {
        // Init stuff.
    }

    public void onPrepareOptionsMenu(Menu menu) {
        // Do things..
        // Maybe you want to modify a variable in the Activity 
        // class? Well, pass an instance as an argument and create
        // a method for it in your Activity class.
    }

    public void onOptionsItemSelected(MenuItem i) {
        // Do things..
        // Maybe you want to modify a variable in the Activity 
        // class? Well, pass an instance as an argument and create
        // a method for it in your Activity class.
    }

}

You'll see where this is going. You can apply this to many things, e.g. onClick, onClickListener, onCreateOptionsMenu, the list is long. To learn more "best practices" you can see some sample applications from Google here. Look for how they've implemented things in a nice and correct way.

Last word; keep your code clean, name your variables and methods in a logical manner and especially in a correct way. Always, always understand where you are in your code - that is very important.

Hi guys! My name is Jevgeni Kabanov and I’m a computer scientist/hackerpreneur. Recently I’ve been trying to figure out what is happening in the world of Java EE production deployment and frankly it seems pretty scary. After speaking to over a 100 people these are my hypotheses:

1. Nobody uses redeployment in production (as in the actual button that does in-server update). It just isn’t reliable enough due to OutOfMemoryError-s and other failures.
2. The common way to update an application is to:
   1. Take all servers down at 2am and hope no one is using it.
   2. Take servers down one at a time, upgrade them and either drop or migrate the user sessions.
   3. Use weird hacks like copying one file at a time.

I’m also trying to find out how the update process happens, how hard it is and what does it cost in human measure (hours) and in soulless business measure (dollars).

I ask you to help me out and provide me with some semi-solid data I can use to better understand what’s going on in reality. Hopefully you’ll prove me wrong. I'm looking forward to you answers, but would also ask you to answer this survey to get some data: http://0t.ee/appupdq11

I know this is not the typical StackOverflow question, as there isn't a right answer here, but by filling in the survey you'll help me a great deal and if you provide a longer answer here it will be just awesome.

Take a look at liverebel http://www.zeroturnaround.com/liverebel/ - this doesn't suffer from dreaded OOM and has a rollback feature. Liverebel does make JEE hot deployment a reality (nothing else does IMO).

For some of our products we're using OSGi so we can do live updates relatively painlessly - that said we still have some issues redeploying 3rd party libs that seem to hold tightly to classloaders etc.

Database changes are always painful in with respect to deployment, in terms of managing horrible DML/DLL changelogs have a look at http://www.liquibase.org/

I'm currently just working my way through some recursion problems, and I am currently stuck on one.

The problem is to recursivly insert spaces into a string, into every single possible location, such that the output looks something like:

Input: ABCD
Out:
       ABCD
       A BCD
       A B CD
       A B C D
       A BC D
       AB CD
       AB C D
       ABC D

I have currently worked on the problem, and got to a point much like:

Input: ABCD
Out:
       ABCD
       A BCD
       A B CD
       A B C D

My code for the problem so far:

import java.util.Scanner;



public class Words 
{
    static int counter = 0;
    static String fString = "";
    static String fString2 = "";
    static String previous = "";
    static String input = "";
    static String other = "";

    public static String segment(String inputPrefix, String restOfString)
{
    if(restOfString.length() != 0)
    {   
        if(inputPrefix.equals(""))
        {
            fString += restOfString + "\n";
            segment(restOfString.substring(0,1), restOfString.substring(1));
        }
        else
        {
            previous += inputPrefix + " ";
            fString += previous + restOfString + "\n";
            fString2 = previous + restOfString;
            segment(restOfString.substring(0,1)
                            , restOfString.substring(1));
        }
    }
    /*else
    {
        counter++;
        other = fString2.replaceAll(" ", "");
        System.out.println(other);
        if((counter + 1) < other.length())
        {
            System.out.println("Other: " + other);
            input = other.substring(0, counter + 1);
            other = other.substring(counter + 1);
            System.out.println(counter);
            System.out.println("input: " + input);
            System.out.print("other: " + other);

            segment(input, other);
        }
        else
            return fString;
    }*/

    return fString;

}

public static void main (String[] args) 
{
    Scanner scan = new Scanner(System.in);
    System.out.print("Enter a string: ");
    String input = scan.next();
    System.out.println();
    System.out.println(segment("", input));

}
}

That second else clause is where I am having my most trouble, because every time I run it un-commented it goes into an infinite loop. I even put int trace statements (the println statements) and it still isn't helping.

I've read through it many times and it just doesn't make sense to me why it doesn't work.

It looks like you've been able to do the first 'grouping' correctly, but unable to get the next groupings.

The groupings are: 'A BCD', 'AB CD', and 'ABC D'. You need to apply your algorithm to each of these groupings. You've applied it to the first. How do you get the rest of them?

Has enough time passed? I wrote up a python solution just to see what it'd look like compared to Java.

def segment(input, separator=' ', start_from=0):
    print input
    # add spaces after each letter starting from start_from index, terminating at last letter-1
    for i in range(start_from, len(input)-1):
        # if the next letter is already a space, or this letter is a space, move on
        if separator in (input[i+1], input[i]): continue
        # whatever index we're on, do the next one recursively
        segment(input[:i] + input[i] + separator + input[i+1:], separator=separator, start_from=i+1)

segment('ABCD')

I was asked this question in interview: "How to detect the loop in linked list?", I solved this but immediately the interviewer asked me how do I remove the loop in a linked list. I fumbled.

So any pointers on how to solve this, may be pseudo code, or method definition?

I'm comfortable with Java so I have tagged this question under java.

For Instance this linked list has a loop

 0--->1---->2---->3---->4---->5---->6
                  ▲                 |
                  |                 ▼
                 11<—-22<—-12<—-9<—-8

There are two parts to this problem:

1. Detect if there is a loop in the list
2. Identify the start of the loop

Once you know where the loop starts, it's easy to identify the last element in the list since it's the element in the list following the start of the loop that ends up pointing back to the start of the loop. It is then trivial to set the next pointer/reference of this element to null to correct the cyclic link list (not circular linked list which is where the last elements points back to the first - this would be a specific instance of cyclic lists).

1. Floyd's cycle detect algorithm, also called the tortoise and hare algorithm as it involves using two pointers/references that move at different speeds, is one way of detecting the cycle. If there is a cycle, the two pointers (say p1 and p2) will end up pointing to the same element after a finite number of steps. Interestingly, it can be proved that the element at which they meet will be the same distance to the start of the loop (continuing to traverse the list in the same, forward direction) as the start of the loop is to the head of the list. That is, if the linear part of the list has k elements, the two pointers will meet inside the loop of length m at a point m-k from the start of the loop or k elements to the 'end' of the loop (of course, it's a loop so it has no 'end' - it's just the 'start' once again). And that gives us a way to find the start of the loop:

2. Once a cycle has been detected, let p2 remain pointing to the element where the loop for the step above terminated but reset p1 so that it's pointing back to the head of the list. Now, move each pointer one element at a time. Since p2 began inside the loop, it will continue looping. After k steps (equal to the distance of the start of the loop from the head of the list), p1 and p2 will meet again. This will give you a reference to the start of the loop.

3. It is now easy to set p1 (or p2) to point to the element starting the loop and traverse the loop until p1 ends up pointing back to the starting element. At this point p1 is referencing the 'last' element list and it's next pointer can be set to null.

---

Here's some quick and dirty Java code assuming a linked list of Nodes where a Node has a next reference. This could be optimized but it should give you the basic idea:

Node slow, fast, start;
fast = slow = head;

//PART I - Detect if a loop exists
while (true)
{
    // fast will always fall off the end of the list if it is linear
    if (fast == null || fast.next == null)
    {
        // no loop
        return;
    }
    else if (fast == slow || fast.next == slow)
    {
        // detected a loop
        break;
    }
    else
    {
        fast = fast.next.next; // move 2 nodes at at time
        slow = slow.next; // move 1 node at a time
    }
}

//PART II - Identify the node that is the start of the loop
fast = head; //reset one of the references to head of list

//until both the references are one short of the common element which is the start of the loop
while(fast.next != slow.next) 
{
    fast = fast.next;
    slow = slow.next;
}

start = fast.next;

//PART III - Eliminate the loop by setting the 'next' pointer 
//of the last element to null
fast = start;
while(fast.next != start)
{
    fast = fast.next;
}

fast.next = null; //break the loop

---

This explanation might help the why behind Part II:

Assume the length of the cycle is M, and the length of the rest of the linked list is L. Let's figure out what is the position in the cycle when t1/t2 first meet?

Define the first node in the cycle is position 0, following the links we have position 1, 2,..., up to M-1. ( when we walk in the cycle, our current position is (walk_length) mod M, right?) Suppose t1/t2 first meet at position p, then their travel time are the same, (L+k1*M+p)/v = (L+k2*M+p)/2v for some k1

So it concludes that if t1 start from p, t2 start from head and move at the same speed, then will grantee to meet at position 0, the first node of the cycle. QED.

More references:

• http://www.quora.com/How-does-Floyds-cycle-finding-algorithm-work
• Explain how finding cycle start node in cycle linked list work?
• Proof of detecting the start of cycle in linked list
• Hristo's answer to this question on this page also quotes an explanation from an interview book

I just added a standard "Open file" dialog to a small desktop app I'm writing, based on the JFileChooser entry of the Swing Tutorial. It's generating a window that looks like this:

but I would prefer to have a window that looks like this:

In other words, I want my file chooser to have Windows Vista/Windows 7's style, not Windows XP's. Is this possible in Swing? If so, how is it done? (For the purposes of this question, assume that the code will be running exclusively on Windows 7 computers.)

It does not appear this is supported in Swing in Java 6.

Currently, the simplest way I can find to open this dialog is through SWT, not Swing. SWT's FileDialog (javadoc) brings up this dialog. The following is a modification of SWT's FileDialog snippet to use an open instead of save dialog. I know this isn't exactly what you're looking for, but you could isolate this to a utility class and add swt.jar to your classpath for this functionality.

import org.eclipse.swt.*;
import org.eclipse.swt.widgets.*;

public class SWTFileOpenSnippet {
    public static void main (String [] args) {
        Display display = new Display ();
        Shell shell = new Shell (display);
        // Don't show the shell.
        //shell.open ();  
        FileDialog dialog = new FileDialog (shell, SWT.OPEN | SWT.MULTI);
        String [] filterNames = new String [] {"All Files (*)"};
        String [] filterExtensions = new String [] {"*"};
        String filterPath = "c:\\";
        dialog.setFilterNames (filterNames);
        dialog.setFilterExtensions (filterExtensions);
        dialog.setFilterPath (filterPath);
        dialog.open();
        System.out.println ("Selected files: ");
        String[] selectedFileNames = dialog.getFileNames();
        for(String fileName : selectedFileNames) {
            System.out.println("  " + fileName);
        }
        shell.close();
        while (!shell.isDisposed ()) {
            if (!display.readAndDispatch ()) display.sleep ();
        }
        display.dispose ();
    }
} 

One of the legal modifiers you can use with method local inner classes is abstract.

For example:

public class Outer {
    public void method(){
        abstract class Inner{
        }
    }
}

Is there any situation where you would actually use this?

You have to know this for the SCJP exam.

The are some invalid assumptions in the original question. That something is legal/valid Java doesn't mean that it is something that you need to use, or need to know.

I can't recall that the SCJP contains odd corner case questions.

I tried to come up with a case where I would have used an abstract class declared in a method, but everything looks very odd, and reeks of bad design. Here's however a code example that I came up with (still bad code design IMHO)

public class BatchExecutor {

    public static enum ResultNotification {
        JMS,
        MAIL
    };

    public Runnable createRunnable(ResultNotification type) {
        abstract class Prototype implements Runnable {
            public void run() {
                performBusinessLogic();
                publishResult();
            }

            abstract void publishResult();
        }

        switch (type) {
            case JMS: {
                return new Prototype() {
                    void publishResult() {
                        //Post result to JMS
                    }
                };
            }
            case MAIL: {
                return new Prototype() {
                    void publishResult() {
                        //Post result to MAIL
                    }
                };
            }
        }
        return null;
    }

    private void performBusinessLogic() {
        //Some business logic
    }

}

When playing around with multithreading, I could observe some unexpected but serious performance issues related to AtomicLong (and classes using it, such as java.util.Random), for which I currently have no explanation. However, I created a minimalistic example, which basically consists of two classes: a class "Container", which keeps a reference to a volatile variable, and a class "DemoThread", which operates on an instance of "Container" during thread execution. Note that the references to "Container" and the volatile long are private, and never shared between threads (I know that there's no need to use volatile here, it's just for demonstration purposes) - thus, multiple instances of "DemoThread" should run perfectly parallel on a multiprocessor machine, but for some reason, they do not (Complete example is at the bottom of this post).

private static class Container  {

    private volatile long value;

    public long getValue() {
        return value;
    }

    public final void set(long newValue) {
        value = newValue;
    }
}

private static class DemoThread extends Thread {

    private Container variable;

    public void prepare() {
        this.variable = new Container();
    }

    public void run() {
        for(int j = 0; j < 10000000; j++) {
            variable.set(variable.getValue() + System.nanoTime());
        }
    }
}

During my test, I repeatedly create 4 DemoThreads, which are then started and joined. The only difference in each loop is the time when "prepare()" gets called (which is obviously required for the thread to run, as it otherwise would result in a NullPointerException):

DemoThread[] threads = new DemoThread[numberOfThreads];
    for(int j = 0; j < 100; j++) {
        boolean prepareAfterConstructor = j % 2 == 0;
        for(int i = 0; i < threads.length; i++) {
            threads[i] = new DemoThread();
            if(prepareAfterConstructor) threads[i].prepare();
        }

        for(int i = 0; i < threads.length; i++) {
            if(!prepareAfterConstructor) threads[i].prepare();
            threads[i].start();
        }
        joinThreads(threads);
    }

For some reason, if prepare() is executed immediately before starting the thread, it will take twice as more time to finish, and even without the "volatile" keyword, the performance differences were significant, at least on two of the machines and OS'es I tested the code. Here's a short summary:

---

Mac OS Summary:

Java Version: 1.6.0_24
Java Class Version: 50.0
VM Vendor: Sun Microsystems Inc.
VM Version: 19.1-b02-334
VM Name: Java HotSpot(TM) 64-Bit Server VM
OS Name: Mac OS X
OS Arch: x86_64
OS Version: 10.6.5
Processors/Cores: 8

With volatile keyword:
Final results:
31979 ms. when prepare() was called after instantiation.
96482 ms. when prepare() was called before execution.

Without volatile keyword:
Final results:
26009 ms. when prepare() was called after instantiation.
35196 ms. when prepare() was called before execution.

---

Windows Summary:

Java Version: 1.6.0_24
Java Class Version: 50.0
VM Vendor: Sun Microsystems Inc.
VM Version: 19.1-b02
VM Name: Java HotSpot(TM) 64-Bit Server VM
OS Name: Windows 7
OS Arch: amd64
OS Version: 6.1
Processors/Cores: 4

With volatile keyword:
Final results:
18120 ms. when prepare() was called after instantiation.
36089 ms. when prepare() was called before execution.

Without volatile keyword:
Final results:
10115 ms. when prepare() was called after instantiation.
10039 ms. when prepare() was called before execution.

---

Linux Summary:

Java Version: 1.6.0_20
Java Class Version: 50.0
VM Vendor: Sun Microsystems Inc.
VM Version: 19.0-b09
VM Name: OpenJDK 64-Bit Server VM
OS Name: Linux
OS Arch: amd64
OS Version: 2.6.32-28-generic
Processors/Cores: 4

With volatile keyword:
Final results:
45848 ms. when prepare() was called after instantiation.
110754 ms. when prepare() was called before execution.

Without volatile keyword:
Final results:
37862 ms. when prepare() was called after instantiation.
39357 ms. when prepare() was called before execution.

---

Mac OS Details (volatile):

Test 1, 4 threads, setting variable in creation loop
Thread-2 completed after 653 ms.
Thread-3 completed after 653 ms.
Thread-4 completed after 653 ms.
Thread-5 completed after 653 ms.
Overall time: 654 ms.

Test 2, 4 threads, setting variable in start loop
Thread-7 completed after 1588 ms.
Thread-6 completed after 1589 ms.
Thread-8 completed after 1593 ms.
Thread-9 completed after 1593 ms.
Overall time: 1594 ms.

Test 3, 4 threads, setting variable in creation loop
Thread-10 completed after 648 ms.
Thread-12 completed after 648 ms.
Thread-13 completed after 648 ms.
Thread-11 completed after 648 ms.
Overall time: 648 ms.

Test 4, 4 threads, setting variable in start loop
Thread-17 completed after 1353 ms.
Thread-16 completed after 1957 ms.
Thread-14 completed after 2170 ms.
Thread-15 completed after 2169 ms.
Overall time: 2172 ms.

(and so on, sometimes one or two of the threads in the 'slow' loop finish as expected, but most times they don't).

The given example looks theoretically, as it is of no use, and 'volatile' is not needed here - however, if you'd use a 'java.util.Random'-Instance instead of the 'Container'-Class and call, for instance, nextInt() multiple times, the same effects will occur: The thread will be executed fast if you create the object in the Thread's constructor, but slow if you create it within the run()-method. I believe that the performance issues described in Java Random Slowdowns on Mac OS more than a year ago are related to this effect, but I have no idea why it is as it is - besides that I'm sure that it shouldn't be like that, as it would mean that it's always dangerous to create a new object within the run-method of a thread, unless you know that no volatile variables will get involved within the object graph. Profiling doesn't help, as the problem disappears in this case (same observation as in Java Random Slowdowns on Mac OS cont'd), and it also does not happen on a single-core-PC - so I'd guess that it's kind of a thread synchronization problem... however, the strange thing is that there's actually nothing to synchronize, as all variables are thread-local.

Really looking forward for any hints - and just in case you want to confirm or falsify the problem, see the test case below.

Thanks,

Stephan

public class UnexpectedPerformanceIssue {

private static class Container  {

    // Remove the volatile keyword, and the problem disappears (on windows)
    // or gets smaller (on mac os)
    private volatile long value;

    public long getValue() {
        return value;
    }

    public final void set(long newValue) {
        value = newValue;
    }
}

private static class DemoThread extends Thread {

    private Container variable;

    public void prepare() {
        this.variable = new Container();
    }

    @Override
    public void run() {
        long start = System.nanoTime();
        for(int j = 0; j < 10000000; j++) {
            variable.set(variable.getValue() + System.nanoTime());
        }
        long end = System.nanoTime();
        System.out.println(this.getName() + " completed after "
                +  ((end - start)/1000000) + " ms.");
    }
}

public static void main(String[] args) {
    System.out.println("Java Version: " + System.getProperty("java.version"));
    System.out.println("Java Class Version: " + System.getProperty("java.class.version"));

    System.out.println("VM Vendor: " + System.getProperty("java.vm.specification.vendor"));
    System.out.println("VM Version: " + System.getProperty("java.vm.version"));
    System.out.println("VM Name: " + System.getProperty("java.vm.name"));

    System.out.println("OS Name: " + System.getProperty("os.name"));
    System.out.println("OS Arch: " + System.getProperty("os.arch"));
    System.out.println("OS Version: " + System.getProperty("os.version"));
    System.out.println("Processors/Cores: " + Runtime.getRuntime().availableProcessors());

    System.out.println();
    int numberOfThreads = 4;

    System.out.println("\nReference Test (single thread):");
    DemoThread t = new DemoThread();
    t.prepare();
    t.run();

    DemoThread[] threads = new DemoThread[numberOfThreads];
    long createTime = 0, startTime = 0;
    for(int j = 0; j < 100; j++) {
        boolean prepareAfterConstructor = j % 2 == 0;
        long overallStart = System.nanoTime();
        if(prepareAfterConstructor) {
            System.out.println("\nTest " + (j+1) + ", " + numberOfThreads + " threads, setting variable in creation loop");             
        } else {
            System.out.println("\nTest " + (j+1) + ", " + numberOfThreads + " threads, setting variable in start loop");
        }

        for(int i = 0; i < threads.length; i++) {
            threads[i] = new DemoThread();
            // Either call DemoThread.prepare() here (in odd loops)...
            if(prepareAfterConstructor) threads[i].prepare();
        }

        for(int i = 0; i < threads.length; i++) {
            // or here (in even loops). Should make no difference, but does!
            if(!prepareAfterConstructor) threads[i].prepare();
            threads[i].start();
        }
        joinThreads(threads);
        long overallEnd = System.nanoTime();
        long overallTime = (overallEnd - overallStart);
        if(prepareAfterConstructor) {
            createTime += overallTime;
        } else {
            startTime += overallTime;
        }
        System.out.println("Overall time: " + (overallTime)/1000000 + " ms.");
    }
    System.out.println("Final results:");
    System.out.println(createTime/1000000 + " ms. when prepare() was called after instantiation.");
    System.out.println(startTime/1000000 + " ms. when prepare() was called before execution.");
}

private static void joinThreads(Thread[] threads) {
    for(int i = 0; i < threads.length; i++) {
        try {
            threads[i].join();
        } catch (InterruptedException e) {
            e.printStackTrace();
        }
    }
}

}

It's likely that two volatile variables a and b are too close to each other, they fall in the same cache line; although CPU A only reads/writes variable a, and CPU B only reads/writes variable b, they are still coupled to each other through the same cache line.

In your example, we have two allocation schemes:

new Thread                               new Thread
new Container               vs           new Thread
new Thread                               ....
new Container                            new Container
....                                     new Container

In the first scheme, it's very unlikely that two volatile variables are close to each other. In the 2nd scheme, it's almost certainly the case.

CPU caches don't work with individual words; instead, they deal with cache lines. A cache line is a continuous chunk of memory, say 64 neighboring bytes. Usually this is nice - if a CPU accessed a cell, it's very likely that it will access the neighboring cells too. Except in your example, that assumption is not only invalid, but detrimental.

Suppose a and b fall in the same cache line L. When CPU A updates a, it notifies other CPUs that L is dirty. Since B caches L too, because it's working on b, B must drop its cached L. So next time B needs to read b, it must reload L, which is costly.

If B must access main memory to reload, that is extremely costly, it's usually 100X slower.

Fortunately, A and B can communicate directly about the new values without going through main memory. Nevertheless it takes extra time.

To verify this theory, you can stuff extra 128 bytes in Container, so that two volatile variable of two Container will not fall in the same cache line; then you should observe that the two schemes take about the same time to execute.

Lession learned: usually CPUs assume that adjecent variables are related. If we want independent variables, we better place them far away from each other.

I would like to be able to wrap a long String to a fixed length. Is there a way to do that in Guava?

Apache Commons / Lang has the method WordUtils.wrap(String, length) that does exactly what I need. Does Guava have a simple means to accomplish this?

I know I can do a hard wrap using Splitter.fixedLength(int), but I would like a soft wrap.

---

UPDATE: There is now a bounty for this question.

Obviously this functionality isn't available in Guava out of the Box, so the bounty goes to the most concise (or most complete) and Guava-like answer that uses what's there in Guava. No libs except Guava allowed.

I did this for fun just to do as much in guava as possible. javanna's answer is better though,

import java.util.Iterator;

import com.google.common.base.Joiner;
import com.google.common.base.Splitter;
import com.google.common.collect.Iterators;
import com.google.common.collect.PeekingIterator;


public class SoftSplit {

    public static String softSplit(String string, int length) {
        //break up into words
        Iterable<String> words = Splitter.on(' ').split(string);

        //an iterator that will return the words with appropriate
        //white space added
        final SoftSplitIterator softIter = new SoftSplitIterator(words, length);
        return Joiner.on("").join(new Iterable<String>() {
            @Override
            public Iterator<String> iterator() {
                return softIter;
            }
        });
    }

    static class SoftSplitIterator implements Iterator<String> {
        private final int maxLength;
        private final PeekingIterator<String> words;
        private int currentLineLength;

        SoftSplitIterator(Iterable<String> words, int maxLength) {
            this.words = Iterators.peekingIterator(words.iterator());
            this.maxLength = maxLength;
        }

        @Override
        public boolean hasNext() {
            return words.hasNext();
        }

        @Override
        public String next() {
            String current = words.next();

            //strip leading spaces at the start of a line
            if(current.length() == 0 && currentLineLength == 0) {
                return "";
            }
            //nothing left after us
            if(!words.hasNext()) {
                return current;
            }
            String next = words.peek();

            if(currentLineLength + current.length() + next.length() < maxLength) {
                //this word and the next one won't put us over limit
                currentLineLength += current.length();
                return current + " ";
            } else {
                //the next word will put us over the limit 
                //add a line break
                currentLineLength = 0;
                return current + "\n";
            }
        }

        @Override
        public void remove() {
            throw new UnsupportedOperationException();
        }
    }

    public static void main(String[] args) {
        String text = 
            "Lorem ipsum dolor sit amet, consectetur adipiscing elit. " +
            "Maecenas porttitor risus vitae urna hendrerit ac condimentum " +
            "odio tincidunt. Donec porttitor felis quis nulla aliquet " +
            "lobortis. Suspendisse mattis sapien ut metus congue tincidunt. " +
            "Quisque gravida, augue sed congue tempor, tortor augue rhoncus " +
            "leo, eget luctus nisl risus id erat. Nunc tempor pretium gravida.";
        System.out.println(softSplit(text, 60));
    }
}

I'm looking for an API for rendering music scores in Java.

The only library I've been able to find (which I'll mention in an answer) is far from feature-complete.

Several people online have mentioned JFugue, but as far as I can tell, JFugue can play music, but rendering sheet music is only part of his wish list.

So, does anyone know of any others? Plus big points for being free, and plus tiny points for having a non-viral license.

The only library I've been able to find is ABC4j, which takes an ABC string as input, and outputs some pretty sheet music.

However, it is still in alpha, and seems to be missing some pretty basic features:

• Dynamics
• Grand Staff (for writing piano music)
• Dotted notes (???)

---

ABC4j is licensed under LGPL (non-viral)

I've recently been more and more frustrated with a problem I see emerging in my projects code-base.

I'm working on a large scale java project that has >1M lines of code. The interfaces and class structure are designed very well and the engineers writing the code are very proficient. The problem is that in an attempt to make the code cleaner people write Utility classes whenever they need to reuse some functionality, as a result over time and as the project grows more and more utility methods crop up. However, when the next engineer comes across the need for the same functionality he has no way of knowing that someone had already implemented a utility class (or method) somewhere in the code and implements another copy of the functionality in a different class. The result is a lot of code duplication and too many utility classes with overlapping functionality.

Are there any tools or any design principles which we as a team can implement in order to prevent the duplication and low visibility of the utility classes?

Example: engineer A has 3 places he needs to transform XML to String so he writes a utility class called XMLUtil and places a static toString(Document) method in it. Engineer B has several places where he serializes Documents into various formats including String, so he writes a utility class called SerializationUtil and has a static method called serialize(Document) which returns a String.

Note that this is more than just code-duplication as it is quite possible that the 2 implementations of the above example are different (say one uses transformer API and the other uses Xerces2-J) so this can be seen as a "best-practices" problem as well...

Update: I guess I better describe the current environment we develop in. We use Hudson for CI, Clover for code coverage and Checkstyle for static code analysis. We use agile development including daily talks and (perhaps insufficient) code reviews. We define all our utility classes in a .util which due to it's size now has 13 sub-packages and about 60 classes under the root (.util) class. We also use 3rd party libraries such as most of the apache commons jars and some of the jars that make up Guava.

I'm positive that we can reduce the amount of utilities by half if we put someone on the task of refactoring that entire package, I was wondering if there are any tools which can make that operation less costly, and if there are any methodologies which can delay as much as possible the problem from recurring.

Your problem is a very common one. And a real problem too, because there is no good solution.

We are in the same situation here, well I'd say worse with 13 millions line of code, turnover and more than 800 developpers working on the code. We often discuss about the very same thing as your problem.

The first idea - that your developpers have already used - is to refactor common code in some utility classes. Our problem with that solution, even with pair programming, mentoring and discussion, we are simply too many for this to be effective. In fact we grow in subteams, with people sharing knowledge in their subteam, but the knowledge doesn't transit between subteams. Maybe we are wrong but i think that even pair programming and talks alone can't help in this case.

Then, we also have an architecture team. This team is responsible to deal with design and architecture concerns and to mke common utilities that we might need. This team in fact produce something we could call a corporate framework. Yes it is framework, and sometime it work well. This team is also responsible to push best pratice and to raise awareness of what should be done or not, was is available or not.

Good core Java API design is one of the reason of java sucess. Good third party open sources libraries count a lot too. Even a small well crafted api allow to make really usefull abstration and can help reduce code size a lot. But you know making framwork and public API is not the same thing at all as just coding an utility class in 2 hours. It has a really hight cost. An utility class cost 2 hour for the initial coding, maybe 2 days with debugging and unit tests. When you start sharing common code on bigs projects/teams, you really make an API. You must ensure perfect documentation then, really readable and maintenable code. When you release new version of this code, you must stay backward compatible. You have to promote it company wide (or at least team wide). From 2 day for you small utility class, you grow for 10 days, 20 days or even 50 days.

And your API design may not be so great. Well it is not that your engineers are not bright. Yes they are. But do you let them work 50 day on a small utility class that just help parsing number in a consistent way for the UI ? Do you let them redisgn the whole thing when you start using a mobile UI with totally different needs ? Also have you noticed how the brigtest engineers in the word make API that will never be popular or will fade slowly ? You see the first web project used only internal framworks or no framework at all. We add then PHP/JSP/ASP. Then in java we added struts. Now JSF is the standard. Now spring webflow, vaadin or lift...

All I want to say, is that there is no good solution, the overhead grow exponentially with code size and team size. Sharing a big codebase restrict your agility and responsiveness. Any change must be done carrefully, with thinking of all potential integration problem and everybody must be trained of the new specifities and features.

But the main productivity point in a software company is not to gain 10 or even 50 lines of code when parsing XML. A generic code to do this will grow to thousand line of code anyway and recreate a complex API that will be layered by utilities classes. When the guy make an utility class for parsing XML, it is good abstraction. He give a name to one dozen or even one hundred line of specialized code. This code is usefull because it is specialized. The common API allow to work on streams, URL, strings, whatever. It has a factory so you can choose you parser implementation. The utility class is good because it work only with this parser and with strings. And because you need one line of code to call it.

But of course, this utility code is of limited use. It work well for this mobile application, or for loading XML configuration. And that why the developper added the utility class for it in the first place.

In conclusion, What i would consider instead of try to consolidate the code for the whole codebase is to split code responsability as the teams grow. Transform your big team that work on one big project into small teams that work on several subprojects.

Ensure that interfacing is good to minimize integration problem, but let team have their own code.

Inside theses teams and corresponding code base, ensure you have the best pratices. No duplicated code, good abstractions. Use existing proved API from the commmunity. Use pair programming, strong API documentation, wiki... But you should really let differents teams make their choice, make their own code, even if it mean duplicated code accross teams or different design decision. You know if the design decisions are differents, is may be because the needs are differents.

What you really manage is complexity. In the end if you make one monolitic codebase, very generic and advenced one, you increase the time for newcommers to ramp up, you increase the risk that developpers will not use your common code at all, and you slow down everybody because any change has far greater chances to break existing functionnality.

I am trying to convince myself that actions taken in the finally clause happen-before the function return (in the memory consistency sense). From the JVM spec, it is clear that within a thread, program order is supposed to drive the happens-before relationship -- if a happens b in the program order then a happens-before b.

However I have not seen anything explicitly stating that finally happens-before return, so does it?. Or, is there some way that the compiler could reorder the finally clause since it is simply logging.

Motivating example: I have one thread fetching objects out of a database, and putting them into an ArrayBlockingQueue and another thread taking them out. I have some try finally blocks for event timing and I am seeing after affects of the return before the log statement

Thread 1:
public Batch fetch() {
 try {
  log("fetch()+");
  return queryDatabase();
 }
 finally { log("fetch()-"); }
...
workQueue.put(fetch());

Thread 2:
log("take()+");
Batch b = workQueue.take();
log("take()-");

To my great surprise this prints out in an unexpected order. While, yes, logging statements in different threads can appear out of order, there is a time difference of at least 20 ms.

124 ms : take()+
224 ms : fetch()+
244 ms : take()-
254 ms : fetch()-

Note this is not exactly the same question as does finally trump return, I'm not asking what will be returned, but instead about memory consistency and execution order.

@David Heffernan has the correct answer. The JLS specification talks about the behavior of the return statement(including how it interacts with finally blocks) in section 14.17. Copying from there (emphasis mine):

A return statement with an Expression attempts to transfer control to the invoker of the method that contains it; the value of the Expression becomes the value of the method invocation. More precisely, execution of such a return statement first evaluates the Expression. If the evaluation of the Expression completes abruptly for some reason, then the return statement completes abruptly for that reason. If evaluation of the Expression completes normally, producing a value V, then the return statement completes abruptly, the reason being a return with value V. If the expression is of type float and is not FP-strict (§15.4), then the value may be an element of either the float value set or the float-extended-exponent value set (§4.2.3). If the expression is of type double and is not FP-strict, then the value may be an element of either the double value set or the double-extended-exponent value set.

It can be seen, then, that a return statement always completes abruptly.

The preceding descriptions say "attempts to transfer control" rather than just "transfers control" because if there are any try statements (§14.20) within the method or constructor whose try blocks contain the return statement, then any finally clauses of those try statements will be executed, in order, innermost to outermost, before control is transferred to the invoker of the method or constructor. Abrupt completion of a finally clause can disrupt the transfer of control initiated by a return statement.

My goal is to authenticate to the database using a JDBC/Hibernate in a secure manner, without storing passwords in plain text. Code examples appreciated. I'm already using waffle to authenticate the user so if there was some way to use the credentials that waffle obtained from the user, and forward those to the DB, that would be good.

Two questions:

1. What is the recommended way to do multi hop authentication (the client, web server, and database are all different machines) with tomcat/hibernate/spring on web server, a sql database, and obviously client browser?
2. I would also settle for a way to use a single user account to do authentication, as long as that user account's information was not stored in plain text anywhere. The user account will need both read/write privileges on the DB.

I found some useful information about connecting to SQL Server in this thread. However, I'm expecting that Tomcat will be running under the default account which is like, Local System or something. As far as I know, that account cannot be used to do windows authentication to the database.

My solution:

I did end up using the approach mentioned in the above thread. Instead of running the Tomcat service as Local System it is now running as a user. That user has permission to access the database. My hibernate configuration file is configured as follows:

    <property name="hibernate.connection.url">
jdbc:sqlserver://system:port;databaseName=myDb;integratedSecurity=true;
</property>

To those who provided responses

I appreciate everyone's help and I will try out some of the techniques mentioned in the thread. My issue with some of the responses is that they require symmetric encryption which requires a secret key. Keeping the key secret is almost the exact same problem as storing the password in plain text.

Okay, let's take a look at the problem. You want to have the authentication information made available but not hardcoded anywhere in code or in file system. What I would suggest:

• require the administrator of the application to specify the authentication information upon application startup either via jmx or via a webpage that does not require any database connection.
• Add a servlet filter to limit access until database authentication information is entered.

This solution does require some extending spring context loading so that it waits until the authentication information is specified (via entry page).

Hello.

Anyone knows some good SQL builder library for Java like Squiggle (not maintained anymore it seems). Preferably, a project in active development.

Preferably with syntax like Zend_Db_Select, something that will allow to make a query like

String query = db.select().from('products').order('product_id');

Querydsl and jooq are two popular choices.

Note: I work for the company behind Querydsl.

I am using a regular expression to match whether or not a pattern matches, but I also want to know when it fails.

For example, say I have a pattern of "N{1,3}Y". I match it against string "NNNNY". I would like to know that it failed because there were too many Ns. Or if I match it against string "XNNY", I would like to know that it failed because an invalid character "X" was in the string.

From looking at the Java regular expression package API (java.util.regex), additional information only seems to be available from the Matcher class when the match succeeds.

Is there a way to resolve this issue? Or is regular expression even an option in this scenario?

I guess you should use a parser, rather than simple regular expressions.

Regular Expressions are good providing matches for string, but not quite so in providing NON-matches, let alone explaining why a match failed.

Do the "control characters" used in regular expressions differ a lot among different implementations of regex parsers (eg. regex in Ruby, Java, C#, sed etc.).

For example, in Ruby, the \D means not a digit; does it mean the same in Java, C# and sed? I guess what I'm asking is, is there a "standard" for regex'es that all regex parsers support?

If not, is there some common subset that should be learned and mastered (and then learn the parser-specific ones as they're encountered) ?

See the list of basic syntax on regular-expressions.info.

And a comparison of the different "flavors".

I have a project whose core domain is dramatically changing. It's possible to use 50% of the core functionality from this site and just add the 50% new functionality, but I am starting to consider that maybe it might be faster to simply redo the product in Rails. Development speed is very important.

There are some things I really like about java - the performance and scalability are very good. I am not a crappy Java developer, so my apps tend to run very well - better than the Rails sites I've seen. I've always accepted the idea that people probably just throw a little more money at the problem when it comes to using Rails, which probably works itself out in the end because of the insane productivity benefits.

I am actually quite agile with Java. I know it will still take me longer to add a basic entity to the system, but I am quick at it and I don't mind it that much. At least it's easy and straight-forward to do.

What I do mind is:

• having to start/stop the server just to fix a route, lazy load exception, controller is going to wrong view, etc.
• putting up with the fact that unit/integration tests sometimes have different results than the production environment (because annotations on controllers can't be tested, or lazy-loading exceptions occur during asynchronous service calls, or things like that). Knowing if your Jackson is marshaling your data properly is another Tomcat-only thing because it's handled by Spring. There are lots of things that go wrong after you have tested all that you can, and this frankly annoys the crap out of me.
• putting up with the occasional maven/classloader problem that doesn't rear its ugly head until you deploy into tomcat. It gives the false impression that everything is "a-okay" when you are in your IDE.
• having to put more effort to do database migrations than the ruby people ever have to.
• putting up with framework bugs in Spring that block (it's happened about 5 times on this project since 2009) or Hibernate. I also don't like upgrading Spring Security and having them constantly change the configuration, apis and tag libraries over and over again. This is annoying.
• wasting so much time uploading 58 MB war files to the server! These take me 12 minutes to upload whenever I need to deploy changes. If I forgot to do 'mvn clean' before I upload, Spring might complain that 2 beans exist with the same name because I moved one to a new package... and then I have to re-upload the whole stupid war file again. Why isn't "clean" run by default whenever you do 'mvn package' for?!?! Sometimes these frameworks and tools use the stupidest default settings. This is just so common in the Java world.
• Having to spend hour(s) to figure out where a framework wants to plug-in your own custom implementation for something. This is very annoying. You can spend 2 hours sifting through Google and crappy documentation trying to figure out how to override Spring Security's authentication mechanism for example... and then spend only 5 minutes writing the actual implementation. Of course, they wrote paragraphs upon paragraphs explaining the architecture and how awesome it is, but nobody cares. For something so common, why not just give example source code and be done with it?
• Waiting 10-15 seconds for Spring to start up whenever you want to run your integration tests. This is a drag.

There are a few things I like about Java though. Role-based access is very easy to do with Spring Security. Authentication is never that big of a gain, but I like the implementation inside of Spring.

I also like Spring's form-backing objects and @ModelAttribute. These are huge wins when it comes to controllers, and I don't know if Rails can do these things. I honestly never liked passing request parameters around in every action - Spring MVC is actually a lot easier to use when it comes to this common bloat.

Being able to cache really massive structures in memory and have them stay in memory when you start the application is also highly desirable, especially for this application actually. I have an in-memory thesaurus and grammar checker that needs to get called hundreds of times per request, so in memory is pretty much the fastest option for me.

Even still, I think I could rebuild what I have in 2-3 weeks, and then add all of the new features in a few weeks using rails.

On the bright side, all of the really well-designed css, html and javascript could be ported over with very little problems.

I'd appreciate some advice on the subject before I continue.

PS: I could also go to Spring-ROO... but that would also be a considerable rework. I was never using JPA - I was using Hibernate directly. I am also not using JSP's - I am using Freemarker.

It takes more time to get good at Ruby, and Rails. I worked as an independent contractor as Spring and Hibernate expert myself, but I felt strangled by java and it's web frameworks so I decided to learn Ruby on Rails.

I would advice you to learn Ruby, from what I read you would probably master it, although get pretty frustrated with the very different way the use the ORM. I had issues with it, used to working on aggregate roots in Hibernate to the ActiveRecord one class one table kind of pattern. But hey, you could easily try out MongoDB to have some real fun.

Ruby is

• less code
• it's fast and scalable (slower than java on the specific tasks, but you get rid of stacks of layers.)
• the problems are more often; which gem should I use. Luxorious!
• a unique, big, sharing and caring open source community
• nice frameworks, as Rails and Sinatra
• powerful.
• fun!

Would I advice you to do the project you describe in Ruby.

NO.

Not if speed of development matters. You will be slower, trust me. There's a lot to learn, it's conventions are not familiar to a java programmer and when you get stuck, lots of hours fly by.

The best option would be to hire a senior ruby developer to pair up with you and teach you. Be a good apprentice and you'll learn fast. Faster than me, I had to learn most by myself, which is really inefficient.

Good luck!

I am currently testing a few different relational database management systems. (MySQL, PostgreSQL, Oracle and Firebird SQL) using a Java application to do so.

I was debating what tests I should run to distinguish the performances of each.

The obvious ones would be insert, select, delete and drop.

I would love to hear your opinions and to make this apply to the question answer format I shall ask for the 5 most appropriate tests to indicate performace differences. In an ideal world I would like to mimic real word use.

Thanks to all who answer.

I think that any of them would probably be fine. However, your configuration of the different databases for what you are trying to do would be different based on your application.

Suggested place to start: look for apps similar to yours. See what they are using, if you can. Then start testing the different databases with similar configurations and see what works for you.

Personally I've used Oracle, MySql, and Postgres over the last 11 years and they've all worked well. It's really all in your configuration, which is where a good DBA comes in handy.